In Southampton Mr Letts said he was "immensely disappointed" to have lost his seat after 18 years on the council. The rest are being counted during the day on Friday with results expected during the afternoon. Council group leader Ed Murphy narrowly held onto his seat in Ravensthorpe, but he felt the party had not targeted seats well enough. Conservative Party chairman Brandon Lewis has said he is expecting the results to be "difficult" for his party.
In a night of high drama the Tories, which before the election had half of the seats on the council, increased their number of councillors from 30 to 31 despite defeats for former council leader John Peach and cabinet member Gavin Elsey.
Liberal Democrats have gained a handful of seats in councils such as Liverpool, Hull and Portsmouth and are fighting to take Richmond upon Thames.
Support for Ukip has collapsed with the party winning just two seats almost five hours into counting.
The first results show the UKIP vote going to the Tories with them making gains while Labour have made losses outside of London.
The Lib Dems will be hoping to regain councils they lost to the Conservatives in 2014, as well as picking up seats lost when they were in coalition.
The Conservatives lost three seats on this symbolic council at the heart of London's political district but retained overall control by a wide margin.
Mr Peach, who was also the Mayor of Peterborough, lost his seat in Park to Labour's Aasiyah Joseph.
Meanwhile, social media reports suggested residents in areas piloting the UK Government's controversial ID trials to reduce voter fraud had been barred from voting.
The Electoral Commission said awareness campaigns had been run in the pilot areas, while No.10 said the vast majority were "casting their vote without a problem". Was informed that two people had already turned up without ID this morning so had been unable to vote.