I have almost completely moved away from installing applications on my computer. This way I can move freely from PC to PC and still have my email, documents, development tools and utilities quickly accessible. USB application do not have to loaded on a USB drive however, you can just as easily copy them to a folder on your local computer and run them from there. That is the great thing about portable applications, they are just that, portable. I have also setup applications on my cell phone microSD card. When I connect my phone to the computer via a USB cable I have access to these programs, and I seldom am anywhere without my cell phone.
Some of the applications are definetly better that others, it took several tries to find a good menu program that was simple and effective. Other programs I just have to have like putty, winSCP, Foxit reader and To Do List. You just have to try each application that you are interested in. Be sure to have a good spyware / anti-virus program running just in case you hit an odd application out there. I have yet to find any problem with applications from the two web sites listes above. Have fun going portable!!