What is lost is that none of your communications with your Admin would be encrypted, in particular login, so you would risk somebody sneaking a look at your Admin password and gaining access to your customer data.

And links embedded in some customer emails, in particular those involved in the order processing process, would encourage customers to access their accounts in a non-secure way.

I'm not sure that anything is gained, except perhaps an almost inperceptible increase in the site's speed.