Loop for 20ms? NOOOO! 
I just service the debounce counter in the timer routine, you save the state and the count. As long as the state stays the same, you decrement the debounce counter. When it's at zero, you stop updating it, and the fact that it's zero tells you the switch state stored is debounced. If the switch state is ever detected as changed, you reload the debounce timer and start decrementing it again. All you have to do to know the switch state is check if the debounce counter is zero and read the switch state. As for how large a value to use for debounce, as you say, that's dependent on what type of contact you're servicing.
