Downing Street believed it had assurances from the Greek government that their prime minister would not raise the subject of the Parthenon Sculptures on his visit to the UK.
Greek PM Kyriakos Mitsotakis told the BBC on Sunday that having some of the treasures, also known as the Elgin Marbles, in London and others in Athens was like cutting the Mona Lisa in half.
No 10 then cancelled PM Rishi Sunak’s meeting with Mr Mitsotakis.
Labour has branded the row “pathetic”.
The sculptures are a collection of ancient Greek treasures from the Parthenon which were taken and brought to the UK by British diplomat Lord Elgin in the early 19th Century. They are now in the British Museum.
Both Greece and the UK have long-standing positions on the sculptures, but diplomatic talks were expected to focus on other topics.
Transport Secretary Mark Harper told BBC Breakfast on Tuesday that it was a “matter of regret” that no meeting would take place between the two countries after Mr Mitsotakis declined a secondary offer to meet Deputy Prime Minister Oliver Dowden instead.
Asked whether the government’s treatment of the Greek leader was rude, Mr Harper said the Greeks had been offered a senior-level meeting but were entitled to take their own view.
Mr Sunak is keen to be seen as a defender of the marbles’ place in London. A senior Conservative source said: “Our position is clear – the Elgin Marbles are part of the permanent collection of the British Museum and belong here.”
But Mr Mitsotakis told reporters on Monday evening he was “deeply disappointed by the abrupt cancellation” of the meeting, which had been planned for Tuesday lunchtime.
“Those who firmly believe in the correctness and justice of their positions are never hesitant to engage in constructive argumentation and debate,” he said.
Sources with knowledge of the mood in the Greek government have suggested he was both “baffled” and “annoyed”.
This video can not be played
To play this video you need to enable JavaScript in your browser.
In his interview on BBC’s Sunday with Laura Kuenssberg the day before, the Greek PM had called for the sculptures to be returned.
He said the sculptures were “essentially stolen” but called for a partnership with the British Museum so people could “appreciate” the works “in their original setting”.
During his visit to the UK, Mr Mitsotakis has meet Labour leader Sir Keir Starmer and will return to Greece on Tuesday after other scheduled meetings.
There is ongoing wider debate around the place of museums and their collections in a post-colonial world, with Mr Sunak seemingly positioning himself decisively on one side of that argument.
There may also be a divide with Sir Keir Starmer’s Labour party on this issue.
A party spokesman told the BBC: “To pick a fight with a Nato ally for the sake of a headline shows just how weak Rishi Sunak is.
“He should have been talking about the economy, immigration, the Middle East, that’s what the country would expect from a leader but Rishi Sunak is no leader.”
Labour is also distancing itself from reports in a Greek newspaper suggesting it was open to “a legal formula” for the return of the sculptures to Greece.
Instead, the party says its position is that if the British Museum and the Greek government came to a loan agreement, a Labour government would not stand in the way.
A spokesperson for the UK government said there were “no plans” to change the 1963 British Museum Act – which prohibits the removal of objects from the institution’s collection.
But a loan does not require a change in the law and so could happen irrespective of any British prime minister’s position.
A Labour source said the party’s position was long-standing – a Labour government would not change the law to allow the sculptures to be permanently moved – and described Mr Sunak’s behaviour as “pathetic”.
The British Museum’s Chair of Trustees, George Osborne, who was Conservative chancellor from 2010 to 2016, has previously said he was looking to find “some kind of arrangement to allow some of the sculptures to spend some of their time in Greece”.
Speaking to the Culture, Media and Sport Committee last month, Mr Osborne said any deal would have to see “objects from Greece coming here” for the first time.
It is thought any decision is at least months away.
Sign up for our morning newsletter and get BBC News in your inbox.
