Probably tpl_index_categories.php would be the place to start. You would need a counter to tell how many categories had been output, and stop at 20. That is the easy part - next you would need a link/switch that would set a variable to disable the cutoff when clicked. It would need to go through the URL, so you would probably need a form submit...

There is a mod for users to choose how many products to display per page; that code could guide you.