I completely agree with you and started a solution for this a couple of month ago. It is called NoseRub and is a protocol for distributed social networks. Right now, we are working on enhancing the protocol, but on http://identoo.com you can see an example implementation of this.The goal is, that you can install NoseRub on your own server and there manage your profile and contacts. And of course you should be able to move your contact and profile from one server to another.

This all is built on top of XFN, FOAF, RSS and OpenID, because we don’t want to invent any things new.