PM refuses to change phone number amid concerns over continual lobbying