Interest rate hikes will start to hurt the key swing voters who gave Boris Johnson his majority