Fiji dominated the first half in Wellington but were too slow and laboured to break the Kiwis down as they failed to add to Apisai Koroisau's penalty.
Some of the players who featured for the Bati in the last two World Cups are Akuila Uate, James Storer and Ashton Sims while Jarryd Hayne played in 2008 and prominently featured for the Kangaroos in 2013 where he scored four tries against the Bati in the semis.
Fiji face reigning champions Australia next.
New Zealand were gifted late field position after Kevin Naiqama lost the ball on his own 40-metre mark but Fiji knew the game was won when Brad Takairangi failed to reach Rapana's kick.
The Kiwis were highly fancied to progress despite last week's shock defeat by Tonga but they produced a flat performance and Fiji took advantage to claim a 4-2 win in a tryless game.
"We slayed a giant tonight", he said.
"I don't think (there was any more we could have done)". It's an unreal feeling, it's unbelievable.
"I had the utmost belief from the moment we went into camp". We got one tonight and it is the biggest test next week.
"I am so proud of these boys". "It's as simple as that.".