Brexit backing Labour MP 'defects to Remain amid concerns about racism'