Police found "over half a dozen" pipe-bomb like devices on the property, according to officials.
Authorities later said that a suspicious package with a "destructive device" was left at a bar along the Jersey Shore.
Apparently, the man responsible for the homemade explosives left almost half-dozen firework-like devices on his property, which happened to be near the parade route, according to authorities.
He was charged with second-degree possession of a destructive device for an unlawful goal, as well as one count of third-degree possession of a destructive device, according to the outlet.
South Plainfield is in Middlesex County, about 40 miles southeast of New York City.
The governor also was to appear at a parade in Rutherford at 12:30 p.m., but it was cancelled because of rain.
Kaiser's brother told journalist Samantha Liebman that authorities distorted the situation, saying "they're not finding anything" and said authorities were shooting "from the hip".
Gov. Phil Murphy was slated to attend a parade in South Plainfield this morning but it was abruptly canceled, over what police are describing as a security concern. He was taken into police custody, the station reported. It is sponsored by the South Plainfield Public Celebrations Committee.
Multiple agencies were involved in the investigation, which remains ongoing.
Kaiser was transported to the Sea Bright Police Department where he was criminally charged.