Scott Morrison is staring down a class action against Centrelink's controversial robo-debt welfare recovery scheme.
Lawyers are preparing to argue the Commonwealth must repay debts collected and provide compensation to those affected.
The robo-debt system matches tax office and Centrelink data to claw back overpaid welfare payments.
Recipients of the automated debt letters are presumed guilty and must prove their innocence.
But the government has admitted more than one-quarter of debt notices sent have been wrong.
More than 160,000 of the welfare agency's letters are estimated to have contained errors.
The prime minister argues many complaints against the scheme have been overstated.
"Where the system needs to be improved then we'll always continue to do that," he told the Seven Network on Wednesday.
"But we won't make any apologies for actually making sure we recover overpaid taxpayers' money."
Mr Morrison said welfare recipients should regularly update their income details to avoid falling foul of the scheme.
"I encourage people to do that and then all of this can be avoided," he said.
Opposition frontbencher Bill Shorten said hundreds of thousands of people had been "put through the grinder" in a system often proved wrong.
"It's almost a legalised form of a Nigerian email scam where they say 'you owe us X thousand dollars' unless you can prove that you don't," he said.
"How do individuals take on the government? I mean, most of us are just busy trying to make ends meet."
Australian Associated Press