Even new Cabinet minister Sajid Javid, who backed Remain in the European Union referendum, came out in the meeting against the "customs partnership" model, under which the United Kingdom would collect tariffs on behalf of the EU.
The group's move came ahead of today's meeting of Theresa May's "Brexit war Cabinet", which is due to discuss the different options to replace membership of the EU customs union.
She insisted "the customs partnership is not meant to keep us in the customs union or the single market by the back door", adding: "It would be an issue of resolving the issue on the Irish border, the need to have no border in the Irish Sea and to give easy, frictionless trade between the United Kingdom and the rest of Europe whilst also enabling our own trade policy with other parts of the world".
Brexiteers want May to agree to a streamlined customs arrangement in which new technology and "trusted trader" schemes minimise the need for checks on goods at the border between the United Kingdom and Ireland.
"There is no question of there being an ultimatum". We also want to ensure that we're commited to delivering on our commitment of no hard border between Northern Ireland and Ireland, and have as frictionless trade as possible with the European Union.
A Number 10 source said there was agreement in the Brexit strategy and negotiations sub-committee that Britain should leave the European customs union in order to be able to have control of its own trade policy.
A spokesman for Mrs May said: "The PM said in the House that there are a number of ways of taking this forward and that's what we're working on".
Some pro-Brexit lawmakers fear the customs partnership is little more than an attempt to stay in the customs union - something they say would not deliver the kind of divorce the government has promised to pursue.
And shortly after the meeting ended, the upper House of Lords inflicted the latest of a series of defeats on May, voting through an amendment to her flagship Brexit bill created to ensure there is no hard border in Ireland.
She also faces increasingly urgent demands from Brussels to come up with a customs plan to avoid a return to a hard border between British-ruled Northern Ireland and the Irish Republic.