New Products [more] sidebox customization

Basically what I want to do is change this title from "New Products [more]" to something else such as "New Products by Date" and include the drop down from the New Products page.

Is that possible?

I have checked tpl_whats_new.php, whats_new.php, and english.php and I am not sure what to do.

I can help you with first problem.
In english.php find next lines:

define('BOX_HEADING_WHATS_NEW', 'New Products');
define('CATEGORIES_BOX_HEADING_WHATS_NEW', 'New Products ...');

and change blue marked lines in what you want.
To change or even delete [more] you must alter next line:

define('BOX_HEADING_LINKS', '  [more]');

Good luck!