I don't click on a lot of ads, but when I see an ad that says Ubuntu Servers starting at $5.00 per month. I feel compelled to hear them out. The service is called DigitalOcean and it is awesome. So awesome in fact, I've just migrated jasonsnider.com over from Rackspace.
After nearly 24 hours on DigitalOcean, I have to say I am quite pleased. At the cloud server level, this is a far better value than Rackspace. For just $5.00 a month you get the entry level droplet (that's DigitalOcean speak for an instance). The entry level droplet provides 512mb of RAM, 1tb of transfer and 20gb of storage. This is less than a third of what I was paying at Rackspace for a similar set up, before the bandwidth charges.
On the downside there offerings are limited, they only have the cloud server offering. When I'm needing a robust infrastucture complete with load balnacers and centralized storage, then Rackspace is still my go to provider. If however, all you need is a single server for a single server installs such as this, then I suggest giving DigitalOcean a try.