Yes this attainable.
Check the image for the sidebox headings that you have for the width in pixels.
Insure that you set the box and column width to match in your admin.
There are at least two types or ways to do this.
One is using a styled sidebox header image(over the default rectangle with a solid color) and this would use the normal method for the box description.
Another is to have a complete image with the description as part of the image. I will assume that this is what you have.
Collect the default box titles and the image that you want to replace for these.
In your english.php file open it in a text editor. find the titles that you need to change. For illustration the categories is represented:
Find:
Code:
define('BOX_HEADING_CATEGORIES', 'Categories');
The red highlighted text would be replaced with:
Code:
define('BOX_HEADING_CATEGORIES', 'your_graphic.jpg');
You also need to alter the sidebox file for each that you want to change: find includes/modules/sideboxes/categories.php file and open this in your text editor. Find this entry:
Code:
$title = BOX_HEADING_CATEGORIES;
and change it to:
Code:
$title = zen_image($template->get_template_dir(BOX_HEADING_CATEGORIES, DIR_WS_TEMPLATE, $current_page_base,'images'). '/' . BOX_HEADING_CATEGORIES, HEADER_ALT_TEXT);
Repeat for each that you will alter and save them to your override folder for your template in includes/modules/sideboxes/your_template/categories.php
Place the images files in: includes/templates/your_template/images/
folder.