Speaking to the BBC just before Prime Minister David Cameron announced he would step down in the next three months, the minister for small business said her experiences campaigning locally took her back to harsher times.
"I am an East Midlands girl through and through and I have seen stuff, I have heard stuff that I have not heard since I was a student in Dalston back in the late 1970s," she said.
Read more: How the night of the EU referendum unfolded
"I have seen people thinking that it's acceptable not only to shout traitor at you, but to stand and say I am voting out because I want these immigrants out."
Ms Soubry said she thinks "we have unleashed something" through the EU referendum and that the "overlying tolerance has been removed from Britain."
But the Conservative pointed the finger at former Labour strongholds for a large portion of the Leave vote, and at politicians for leading them down that path.
Read more: What do YOU think about Britain leaving the EU?
"If you look at the vote, I am afraid it has to be said there has been a vote from white working class Labour supporters," she said. "They are not going to listen to a liberal Tory like me and they have voted in the face of the fact that they have probably never even seen a migrant and it's the fault of politicians.
"You cannot for decades say that the EU is partly rubbish, the route of many of our concerns, and say that migrant workers need controlling as opposed to say how much they contribute, and then say to people actually, in four and a half months' time not only ignore what we have been telling you, but actually positively go out and vote for it. We cannot be surprised at this."