President Volodymyr Zelenskiy demanded a tough international response against Russia over the attack, which came with US President Donald Trump's push to rapidly end the war struggling to make a breakthrough.
"Our response will be harsh," the Ukrainian General Staff said in a post on social media.
Dead bodies were strewn on the ground in the middle of a city street near a destroyed bus and burnt-out cars in a video posted online by Zelenskiy.
"Only scoundrels can act like this, taking the lives of ordinary people," he said, noting that the attack had come on Palm Sunday when some people were going to church.
"You know, the people who are fighting against us always say that they are Orthodox (Christian) believers, that they believe in God, but we have experienced first-hand terrorism today. I have no words," said 27-year old PhD student Yevhen, a local resident who declined to give his surname.
The leaders of the United Kingdom, Germany and Italy condemned the attack.
"These attacks show just what Russia's supposed readiness for peace is worth," German Chancellor Olaf Scholz wrote on social media.
US Secretary of State Marco Rubio, in a statement, expressed condolences for the victims and said the attack was a "tragic reminder of why President Trump and his administration are putting so much time and effort into trying to end this war".
Zelenskiy, in an interview with CBS set to air later on Sunday, urged Trump to visit Ukraine.
"Please come to see people, civilians, warriors, hospitals, churches, children, destroyed or dead," Zelenskiy said in a video clip the US broadcaster posted on social media.
Under Trump's administration, US officials have held separate rounds of talks with Russian and Ukrainian officials to try to move toward a cessation of hostilities in Ukraine.
Russian authorities did not immediately respond to a Reuters request for comment.
Russia denies targeting civilians, but thousands have been killed and injured in its invasion of Ukraine.
Sumy, with a population of about 250,000 and located about 25km from the Russian border, became a garrison city when Ukrainian forces launched an incursion into Russia in August that has since been largely repelled.
Sumy's acting mayor, Artem Kobzar, announced three days of mourning for the victims starting from Monday.
The people who were caught in Sunday's strike were out on the street or inside cars, public transport and buildings when the missiles hit, Interior Minister Ihor Klymenko said.
"Deliberate destruction of civilians on an important church feast day," he wrote.
Andriy Yermak, Zelenskiy's chief of staff, said the missiles contained cluster munitions.
"The Russians are doing this to kill as many civilians as possible," he said.
Maryana Bezuhla, an outspoken Ukrainian MP known for her sharp public criticism of military commanders, suggested on the Telegram app that the attack had taken place due to information about a gathering of soldiers leaking out.
Reuters was not able to verify that information, and Bezuhla did not post evidence.
Russia launched a full-scale invasion of Ukraine in February 2022 and holds nearly 20 per cent of the neighbouring country's territory in the east and south.
Russian forces have been slowly advancing in the east.
Foreign Minister Andrii Sybiha said Ukraine was "sharing detailed information about this war crime with all of our partners and international institutions".
In the aftermath of Sunday's Sumy strike, Zelenskiy called on the United States and European allies to respond robustly to what he described as Russian terrorism.
"Russia wants exactly this kind of terror and is dragging out this war. Without pressure on the aggressor, peace is impossible. Talks have never stopped ballistic missiles and aerial bombs," he wrote.
Russia's defence ministry accused Ukraine on Saturday of having carried out five attacks on Russian energy infrastructure over the previous day in what it called a violation of the US-brokered moratorium on such strikes.
With AAP.
