There were also rallies in Macedonia in spring, demanding the country's name to be left in place.
The accord still requires ratification by the two national parliaments and a referendum in Macedonia, a tough test for the leaders in both countries.
After splitting from the former Yugoslavia, Macedonia was admitted into the United Nations in 1993 as the former Yugoslav Republic of Macedonia (FYROM).
However, both prime ministers faced dissent at home.
But under the Romans, the province of Macedonia was expanded to include territory in modern-day Greece, Macedonia, Bulgaria and Albania. He added that Macedonia "cannot and will not be able in the future to claim any connection with the ancient Greek civilization of Macedonia".
"This is a bad agreement that is in conflict with the majority of the Greek people", he said.
Greece will then back invitations for Macedonia to join North Atlantic Treaty Organisation and start negotiations on joining the EU.
The dispute between the two neighbours had been an obstacle to Macedonia joining either the North Atlantic Treaty Organisation military alliance or the European Union.
The deal was welcomed by EU officials, with European Council president Donald Tusk tweeting his "sincere congratulations" to Mr Tsipras and Mr Zaev.
By solving the dispute which has lingered for more than 25 years, Macedonia will remove the biggest obstacle on its path towards European Union and North Atlantic Treaty Organisation accession. "And it will help to consolidate peace and stability across the wider Western Balkans", he added. "Thanks to you, the impossible is becoming possible", he said.
"We wholeheartedly congratulate Prime Ministers Alexis Tsipras and Zoran Zaev for their determination and leadership in reaching this historic agreement between their countries, which contributes to the transformation of the entire region of South-East Europe", Hahn and Mogherini said in a joint statement.
"Macedonia will be called the Republic of Northern Macedonia [Severna Makedonija]", Zoran Zaev, the country's prime minister, announced Tuesday. But the new deal will also emphasize the fact that Macedonians to the north speak a Slavic language and don't have a linguistic kinship with the Macedonians of Greece.
"Maybe what has the most historic gravity and value for Greece (is that) according to this accord. our northern neighbours don't have, and can not assert, any link to the ancient Greek culture of Macedonia", Tsipras said in a televised address on Tuesday evening.
In a televised address, Tsipras said the 140 countries which had recognized the Balkan state simply as Macedonia would now recognize it as Republic of Northern Macedonia.
On the timeline of the deal, Tsipras said that it would be first signed by the two countries' foreign ministers and then ratified by Macedonia's parliament. He did not, however, spell out what the name will be.
The Greek government has spent 27 years fighting with Macedonia over the use of the name, which references ancient Macedonia and by association its famous leader, Alexander the Great.
The row between Athens and Skopje has been ongoing since 1991, when Macedonia seceded from Yugoslavia and declared its independence.
Following the latest round of discussions, Macedonia is now set to be known as the Republic of Northern Macedonia.
Defense Minister Panos Kammenos, whose right-wing Independent Greeks party is Tsipras' government partner, said Tuesday he would oppose an agreement in a parliamentary vote.