Russian forces pummelled the port city for nearly two months, turning Mariupol into a wasteland with an unknown death toll and thousands trying to survive without water, sanitation or food.
The city is under Russian control but some fighters and civilians have sheltered underground in the Azovstal works - a vast Soviet-era plant founded under Josef Stalin and designed with a labyrinth of bunkers and tunnels to withstand attack.
A Reuters photographer saw dozens of civilians arriving on Sunday at a temporary accommodation centre. The United Nations later said that an operation to evacuate people from the steel works had been under way since Friday.
The "UN confirms that a safe passage operation is ongoing in Azovstal steel plant, in coordination with the ICRC (International Committee of the Red Cross) and the parties to the conflict," UN spokesperson Saviano Abreu said.
"At this point, and as the operations are underway, we will not share further details, as it could jeopardise the safety of the civilians and the convoy," he said.
The convoy had travelled 230 kilometres to reach the steel works, the ICRC said.
The Reuters photographer saw civilians arriving in the village of Bezimenne in an area of Donetsk under the control of Russia-backed separatists around 30km east of Mariupol.
They were receiving refreshments and care after weeks of suffering.
Young children were among those evacuated from the plant - where people cowered underground, huddling together under blankets in the plant's bunkers and tunnels as the shelling tore their city apart.
A first group of 100 evacuees will arrive in the Ukrainian-controlled city of Zaporizhzhia on Monday President Volodymyr Zelenskiy tweeted.
Ukraine was also working with the United Nations to evacuate other civilians from Azovstal, he said.
"The operation is (still) ongoing," Deputy Prime Minister Iryna Vereshchuk said in a video address.
Zelenskiy's Chief of Staff Andriy Yermak suggested the evacuations could go further than just the civilians holed up in the steel works.
"This is just the first step, and we will continue to take our civilians and troops out of Mariupol," he wrote on Telegram.
Outside blue tents in Bezimenne, two children sat looking pensive as they waited, the boy playing with a lighter and heavily armed men looking on. One woman clasped her hands to her face in emotion. A young woman reached out to stroke a cat.
The civilians Reuters saw had been evacuated in a convoy with Russian forces and vehicles with United Nations symbols.
UN Secretary General Antonio Guterres said after meeting Zelenskiy in Kyiv on Thursday that intense discussions were under way to enable the evacuation of Azovstal.
Pope Francis on Sunday described Mariupol as "barbarously bombarded and destroyed", saying the war in Ukraine makes him "suffer and cry" and calling for humanitarian corridors to evacuate people trapped in the Mariupol steelworks.
Russian President Vladimir Putin calls the invasion launched on February 24 a "special military operation" that was necessary because the United States was using Ukraine to threaten Russia and Moscow had to defend Russian-speakers against persecution.
Ukraine and the West have dismissed these arguments as a baseless pretext for war.