Mangosuthu Buthelezi was heckled by a rowdy section of the crowd.
Dozens of men marched through the area calling for foreigners to leave.
At least 10 people, two of them foreign nationals, have been killed in the violence over the last week.
By Thursday police said they had arrested more than 420 people and the unrest had subsided.
But on Sunday groups of men carrying clubs marched down the streets of Johannesburg singing “foreigners must go back to where they came from”, Soweten Live reports.
The crowd headed to Jules Park in Johannesburg where Mr Buthelezi, a Zulu leader during apartheid and former leader of South Africa’s opposition Inkatha Freedom Party, was due to address them about the recent attacks.
He told the crowd he had come as a mediator and said he felt ashamed about the recent violence which he said was tarnishing the name of South Africa across the continent.
But he was heckled throughout and video shared on Twitter shows crowds walking out of the meeting.
Mobs began looting foreign-owned shops and torching foreigners’ lorries on Monday.
The attacks started after South African lorry drivers staged a nationwide strike to protest against the employment of foreign drivers.
The country has become a magnet for migrants from other parts of Africa because it has one of the continent’s biggest and most developed economies.
But there is also high unemployment in South Africa and some people feel foreigners are taking their jobs.
As the week progressed, people across the continent shared videos on WhatsApp which appeared to show violent attacks on Nigerians.
Some of the videos turned out to be misleading – old, or even from other countries.
Nonetheless, a diplomatic row ensued with Nigeria’s government being particularly outspoken in its condemnation of the violence.
The Nigerian President Muhammadu Buhari sent an envoy to South Africa to “express Nigeria’s displeasure over the treatment of her citizens”.