@ Sebastian
RE: When a new user decides to register as a member (currently using WP users) will a new Connections entry be created in the directory for them?
No, it will not be created for them. This is because the WP registration only requires a username and password. Connections requires a first and last name or org name (based on entry type). So, there is no enough info available to auto create a Connections entry for the WP user on registration.
RE: Or will I have to create these new entries manually (which would then link to the user and let them modify as you explained above)?
You could do it that way, or, you could opt to let the user to choose add their own entry (assuming you have the Link extension which is require to allow a user to add/edit and maintain their own entry.)
Not sure if my answer helps clear this up for you but I hope it does.