The Tories may soon be shaken out of their complacency on immigration