Okay I have managed to fix the problem.
For those of you experiencing a similar problem. this is what
you do:
Look under Zone Definitions.
There you will have one column for country and another for zones.
Under the Zone column, look to see if there is a zone described as
"all zones", if so, then tax/vat will be applied twice during check out.

This is what I had:
Country------------Zone
South Africa ------Gauteng
South Africa ------Eastern Cape
South Africa-------Free State
South Africa-------KwaZulu-Natal
South Africa-------Limpopo
South Africa-------Western Cape
South Africa-------Northern Cape
South Africa-------Mpumalanga
South Africa-------All Zones

By removing the "all zones" above in red, Vat is now once again
being applied correctly.