However, the home side celebration over the goal did not last long before Robert Lewandowski netted the third goal for the Bavarian and his 10th league goal this season before Collins scored in the 84th minute to leave the scoreline at 3-2.
Gnabry then set up Philippe Coutinho 10 minutes after the restart for their second goal but instead of scoring again it was the hosts who came alive with Kai Proeger's powerful volley in the 68th.
With time nearly up, Lewandowski showed great finishing to put away a Niklas Suele pass to make it 3-1 and became the first Bayern player to score in each of the first six matches of the season, bringing his tally for this campaign to 10.
Even though Proger's goal sparked hopes for the home side, Lewandowski left no doubt about Bayern's will to win as the Poland worldwide set a new Bundesliga record after scoring his 10th goal in his sixth match.
Paderborn's best chance of the first half came from a misplaced return pass from Thiago Alcantara to goalkeeper Manuel Neuer, who nearly got caught flat-footed.
Paderborn are bottom of the log with one point from six games.
"We didn't finish them off", admitted Gnabry.
In Leipzig, Morocco global midfielder Amine Harit inspired Schalke to an impressive away win, winning and drilling home a penalty and setting up a first Bundesliga goal for Welsh teenager Rabbi Matondo.
Leipzig winger Marcel Sabitzer struck the crossbar for the hosts and Schalke goalkeeper Alexander Nubel made a stunning double save to deny Emil Forsberg and Sabitzer.
Milot Rashica grabbed an early lead for Werder in the seventh minute and Mario Gotze levelled matters for Dortmund two minutes later.
In Leipzig, Morocco worldwide midfielder Amine Harit inspired Schalke to an impressive away win, drilling home a penalty and setting up a first Bundesliga goal for Welsh teenager Rabbi Matondo.
"We should have had a third goal and conceding their final goal really hurts".
Bayer Leverkusen are just behind them in fifth thanks to late goals from forwards Kevin Volland and Kai Havertz that added to a Florian Niederlechner's first half own goal in a 3-0 win at Augsburg.