I wonder if ZenCart team has plan to implement it in the core code in a new feature, and should I wait for it?
I would not hold my breath. Cleaner URLs will be implemented in the core at some future point, but it is not on the immediate list.

Or should I just go ahead and install a mod like Ultimate seo url? Are there any known problem/bug with this famous mod?
Read the threads on it for more information.