Herman had permission to leave the jail Monday but was due to return Tuesday evening, prosecutor Eric Van Der Sypt said.
The carnage in the eastern industrial city of Liege began around 10.30 am (local time), when the attacker armed with a knife repeatedly stabbed the two officers before using their own firearms to kill them, prosecutors said.
He then fired on a parked auto, killing 22-year-old University of Liege student Cyriel Vangriecken, who was in the passenger seat of his mother's vehicle.
Justice Minister Koen Geens said the assailant, who was later killed by police, was on a two-day leave from prison. In addition, Liege police chief Christian Beaupere claimed that the attacker's goal was to "target police".
Belgian police are holding a tribute for the victims of Tuesday's attack.
The country's police and members of the military have worked overtime to guard public buildings since co-ordinated suicide attacks on the Brussels airport and subway system killed 32 people and injured hundreds on March 22, 2016.
He then shot and killed Mr Vangriecken, a student on the verge of qualifying as a primary school teacher, who was sitting in the passenger seat of a parked vehicle.
- Officials said the officers were "executed" in a targeted attack. Police and security officials also assembled in Brussels and Antwerp to mourn the loss of their two colleagues.
Hearses came to transport the bodies of the victims in Liege.
Later in the day, a Belgian federal magistrate said investigators were treating Tuesday's attacks as a "terrorist murder" and were trying to ascertain whether the attacker acted alone. "It is probably too early to make any comments but I wanted here to give all the condolences and solidarity of the people of France towards our Belgian neighbors".
Herman has had several run-ins with police since he was a minor and has been sentenced on assault, drug and insubordination charges, prosecutors said.
The prime minister said Herman did not have his name on a list maintained by an anti-terror assessment group.
A source close to the investigation told the AFP news agency that Herman was suspected of having been "radicalised" in prison. "Otherwise he would have been known as such by all services".
Belgian prime minister Charles Michel has so far described the attack as "a serious incident", while the country's federal crisis centre said there was "absolutely no confirmation yet that the incident is terror-related".
A crisis center official in Belgium says that the situation around the shooting site in Liege is fully safe and has added that there is no reason to increase the terror threat level.