This popular South African holiday destination is panicking as local tourists stay away