This is presented where 'custom' is your template name, replace with the name of your template and insure that the folders exist and place your edited files in them. What you want is not straight away in the tpl_header.php that you referenced.
In your includes/languages/CUSTOM/english.php find this line of code:
define('BOX_HEADING_CATEGORIES', 'Categories');
To use an image for this, can change it as follows:
define('BOX_HEADING_CATEGORIES', 'your_graphic.jpg');
Save your graphic to includes/templates/CUSTOM/images.
in includes/modules/sideboxes/CUSTOM/categories.php Find this:
$title = BOX_HEADING_CATEGORIES;
and change it to:
$title = zen_image($template->get_template_dir(BOX_HEADING_CATEGORIES, DIR_WS_TEMPLATE, $current_page_base,'images'). '/' . BOX_HEADING_CATEGORIES, HEADER_ALT_TEXT);
Note: You would do this for each sidebox that you are changing and assumes that your graphic has the box title as part of the graphic as you want it displayed.