When the first round did not yield a victor with more than half of the votes, the candidate with the least votes and candidates who did not win 5 percent of the total number of votes dropped out.
Hoyle, 62, was the son of a former Labour MP Doug Hoyle.
Welling up with emotion, Sir Lindsay also paid tribute to his late daughter Natalie, who died previous year.
Sir Lindsay also vowed to push on with security reforms to keep MPs, their families, staff and the Commons safe.
The Speaker of the House does not vote on any motion in the House, neither do the deputy speakers.
Mr Bercow stood down from his speaker role at the end of October. He was previously chairman of the Ways and Means committee and one of the three vice-presidents of the Lower House under Bercow.
All the candidates used speeches to a busy Commons chamber to distance themselves from Mr Bercow, who took on a high-profile role in the Brexit process with rulings that broke precedent and allowed MPs to frustrate the government's plans.
In the run-up to Monday's vote, Bryant said he would be "a speaker who is an umpire, not a player", in what appeared to be a reference to Bercow's presumed bias.
Labour veteran Harriet Harman was one of the frontrunners for the role.
Sir Lindsay said he was "truly devastated" at her death.
The "Mother of the House" is well respected within Westminster and would not suffer fools.
She was elected MP for Hackney South and Shoreditch in 2005 and now serves as the Chairman of the Public Accounts Committee.
"Because no matter how fiercely we may disagree, we know that every member comes to this place with the best of motives, determined to solve, to serve the oldest Parliamentary democracy in the world".
The speaker is supposed to be an impartial arbiter of Parliament's rules, but critics accused Bercow of favoring anti-Brexit politicians at the expense of those supporting Britain's departure from the European Union.
Conservative MP Sir Edward Leigh, who was represented Gainsborough in Lincolnshire since 1983, also threw his hat into the ring.
Following the terror attack on Parliament in March 2017, and the killing of PC Keith Palmer, MPs were privately full of praise for Sir Lindsay - at the time still nine months away from being included in the 2018 Honours List for services to political and public life - for the resonant chord he struck.
Third deputy speaker Dame Rosie Winterton also ran.
The proceedings were conducted by father of the House Ken Clarke on his penultimate day in parliament after 49 years as an MP.