The shadow Health Minister claimed her leader was shy of publicly standing up to Tory calls for tighter restrictions on the number of newcomers arriving in Britain.
She also claimed Mr Miliband spent too much time worrying about opinion polls which showed that promises to cut immigration were popular with voters.
Her outburst in an interview with a Labour-supporting website intensified the party’s row that has dominated the Parliamentary summer break.
And it provoked fears that many in Labour would reverse the Government’s to cut annual net immigration to “tens of thousands” after a mass influx of migrants under Tony Blair and Gordon Brown.
Ms Abbott, a former Labour leadership candidate, told the Progress website: “I’ve spoken to Ed Miliband about this quite a bit.
“I know that, on this issue, Ed’s heart is in the right place and he’s trying hard to position us correctly.
“What weighs with him is the polling, which shows what the Tories are doing on immigration is popular. And it certainly weighs on other members of the shadow .”
Ms Abbott claimed there was a lack of people at the top of the party willing to make the case for more immigration to the UK.
The MP for Hackney North and Stoke Newington said: “Not many members of the shadow cabinet represent a diverse London constituency. Views about race and immigration are different once you step outside London.
“So I think Ed himself means well and is doing his best but as a party we should do more because it’s a downward spiral.”
Ms Abbott rubbished Government plans to introduce charges for immigrants who seek NHS treatment.
“The mechanisms already exist to charge people who are not entitled to free health care,” she said.
“Doctors themselves do not want to be immigration officers.”
She added: “There is a public health issue about checking people’s passports before they can see the GP.
“That is a problem for a lot of conditions, like TB.”
What weighs with him is the polling, which shows what the Tories are doing on immigration is popular. And it certainly weighs on other members of the shadow cabinet
UK Independence Party leader Nigel Farage responded: “Mr Miliband’s recent conversion to some form of immigration control is entirely a response to the opinion polls. It is transparent that neither his nor his party’s heart is in it.”
Tory MP Nadhim Zahawi said: “Diane Abbott should apologise. Her comments are a reminder of the immigration chaos we suffered under Labour.”
A senior Labour source said: “Ed understands the positive contribution immigration made to this country.
“He also recognises it can affect public services and employment. We have to address those concerns.”
The party’s leadership woe deepened last night as a former Labour minister urged Mr Miliband to raise his game.
Brian Wilson, who quit as an MP in 2005, said: “I have never been quite clear about why Ed Miliband wanted to be leader, or why he thought he should be.
“His poll ratings are somewhere between dire and disappointing.”