Following a period of "temporary detainment", Wellington Police contacted New Zealand's chief governmental wildlife body, the Department of Conservation (DOC), and the penguins have now been taken into their care.
In New Zealand, Wellington police were called in to evict a pair of penguins on Jul 16, after they began building a nest at a sushi stall in the city centre.
"[The] waddling vagrants were removed from their sushi stand refuge earlier today by Constable John Zhu", New Zealand Police said.
Luckily, they were unhurt and have been returned back to a nesting box near the harbour - although the DOC have said they may be back again!
Their cover was blown after a worker heard them making a cooing, humming sound.
It is believed the birds, described as "waddling vagrants" by local police, may have been attracted by the smell of fish in the food truck at Wellington's busiest train station.
"But it's adorable I think - they're probably terrified but it's adorable".
Mace said people should keep their distance from the penguins, which could have a nasty bite.
Little blue penguins typically start looking for nesting spots in July and start laying eggs in August.
The Department of Conservation (DOC) and the Wellington Zoo showed up to help remove the penguins and relocate them back to the beach.
"That's why here, tonight, even though this is a good 200m to 300m away, I wouldn't be surprised if the owner of the sushi bar says "they're back".