Are they actually physically separate "servers"? Or just different domains on the "same server" hardware?

Apart from security, the main concern is speed of database queries:
Fastest: both databases on the same physical server
Next fastest: both databases on servers in the same network, with proper firewall rules allowing both servers to talk to each other
Slowest: servers on different networks

Your hosting company should be able to set up the database firewall rules on the new server to allow only the IP address of the old server to "use" the new server's database.
Then in your old site's configure.php files you'd put all the DB_XXXXXX credentials copied from the new server's configure.php. Admin and non-admin.