I have set in my admin "Show Category Counts" to false, but now there are still arrows next to all categories (without folowing numbers). How can I erase those arrows?

You can control those settings here:

Admin->Configuration->Layout Settings
- Categories Separator between the Category Name and Count
- Categories Separator between the Category Name and Sub Categories
- Categories Count Prefix
- Categories Count Suffix