You can get just about any OS for free. WinVista Win7, even Win8. And even though it's been said I will say it again. There are ALOT of different things that make you anonymous, don't read any one tutorial and think you are good. Sure there's using a proxy, but there is also encryption, caches and logs, there is browser security such as User Agent and referrer url, there is local net security such as MAC address and DHCP properties like your hostname and vendor class id. Just because you use a proxy or VPN and encrypt your home folder doesn't make you safe at all. Also, openVPN is based in California. If you live in the US don't rely on anything that is in the same country as you for obvious reasons.
By the way, don't be discouraged. Just about everything I mentioned up there is unbelievably easy to fix/hide/deal with, at least in linux. For example, changing your broadcasted vendor class id and hostname consists of altering a few lines in a config file. User Agent and referrer url consists of changing a line or two in the about:config of firefox. MAC address is just a command away. A virtual machine is a decent idea, especially if you have it set up inside an external encrypted drive. Maybe with a few keyfiles to unlock it. A better idea is to have a liveUSB with no persistence. Nothing is saved on the actual computer and every time you boot up the USB it's as if it's a fresh install more or less, assuming you have no persistence set up.
Your name is 4ntis3c, I'm sure you'd be interested in things like Liberte or Tails. One more thing though, DON'T RELY ON TOR FOR ABSOLUTE ANONYMITY. It can help, assuming you use it right which, most people don't. On that note, don't rely on anything for complete anonymity.
Good luck.