Currency which is not installed is displayed

I use Zen Cart 2.1.0 with ZCA Bootstrap-4 (3.7.8) as template.
I got two languages in the shop (Japanese and English), but only one currency (Japanese Yen). For some reasons customers who go to the English page got US$ displayed as currency. When changing the language to Japanese and then back to English, the currency is displayed correctly as Yen.
I want to avoid that the currency is displayed in US$. It shouldn't anyways, because this currency is not installed.
Anybody got a solution for this?

I forgot to add.
The currency is not changed by exchange rate. What is 15000 JPY becomes 15000 US$.

Is your setting in My Configuration::Store -> Switch To Default Language Currency set to false?

It wasn't.
Thank you very much. This probably solved the problem. I write "probably" because sometimes even when I go to English, the currency is displayed correctly in Yen; sometimes not.

It might be your browser cache. If it is the case, the problem should disappear soon.
There might be a bug too in ZC with the option Switch To Default Language Currency trying to change currency to something that does not exist. As ZC is English based, when there is a problem, it defaults to English for language and USD for currency.