I'd normally look for in below order:
- support (technical support is very important),
- value for price (you will get what you pay for),
- uptime and reliability.
The folks at CentoServer.com are so on the ball it's amazing. I'm happy with the performance and reliability of the service.