Yes, you can create the buttons with HTML, though there are Zen Cart functions and PHP constants that will make the buttons more bulletproof (so they will still work when you transfer the site between a development server and a live server, for example).

Button code for the header area would go in (surprise!) /includes/templates/your_template/common/tpl_header.php. You can look at the "navigation display" block for examples of how to work in links effectively.