I have my bin folder in ~/usr/bin. The structure of ~/usr matches /usr and /usr/local, so I can build stuff into it with ./configure --prefix=$HOME/usr or similar. (You could of course do ./configure --prefix=$HOME, but then you run the risk of ending up with ~/share and ~/include and so on, something I don't want.)