I wear contact lenses and have done for many years. I've never even heard of anyone keeping them in the fridge. As long as they are stored in the correct solution they will be fine at room temperature.
If you are worried about it you could always get daily disposables which don't need storing at all. I wear these and find them so easy, a new pair every day.