Superstars in long trousers threaten the Olympics and golf's unfairer sex