Anyone who wants to use public office for their own benefit should find a new line of work, Prime Minister Stephen Harper said Tuesday -- for the first time addressing a deepening scandal related to Sen. Mike Duffy's repayment of improperly filed expenses.

In a rare move, Harper invited reporters and cameras to witness him address the Conservative caucus before he flies to South America later in the day for trade meetings. He took no questions from reporters and wasn't in the daily House of Commons question period Tuesday afternoon.

"I don't think any of you will be very surprised to hear I'm not happy, I'm very upset, about the conduct we've witnessed, the conduct of some parliamentarians and the conduct of my own office," Harper said as he opened his speech.

Last week, CTV's Ottawa Bureau Chief Robert Fife reported that Harper's chief of staff, Nigel Wright, had written a personal cheque to Duffy for over $90,000, allowing him to repay the money he owed for improperly filed housing expenses.

Ethics Commissioner Mary Dawson confirmed late Tuesday that her office has launched an investigation into Wright鈥檚 cheque to Duffy under the Conflict of Interest Act.

鈥淭he Commissioner and her Office cannot comment on the examination, as all examinations are conducted in private,鈥 Dawson鈥檚 office said in a statement.

The Canadian Press reported Tuesday afternoon that Liberals in the Senate are trying to trigger special parliamentary hearings in an effort to force Wright and other Conservatives to testify.

Liberal Senate leader James Cowan is expected to argue that the PMO violated the privileges of parliamentarians. If Senate speaker Noel Kinsella agrees, he could send the issue to a special committee.

Wright , while Duffy quit the Conservative caucus on Thursday. Another Conservative-appointed senator facing an expenses audit, Pamela Wallin, from the Conservative caucus on Friday.

On Monday, sources told Fife that Harper鈥檚 former special counsel and legal adviser Benjamin Perrin helped draft a letter of understanding that called for Duffy to publicly declare that he would repay the money. In return, sources said, Wright would give a personal cheque to Duffy to cover the $90,000. Sources also said the agreement stipulated that a Senate investigation into expense claims would go easy on Duffy.

On Tuesday, Perrin said in a statement he was 鈥渘ot consulted on, and did not participate in, Nigel Wright鈥檚 decision to write a personal cheque to reimburse Senator Duffy鈥檚 expenses.鈥

鈥淚 have never communicated with the Prime Minister on this matter,鈥 he added.

The opposition hammered the government about the $90,000 cheque and the agreement during question period Tuesday.

鈥淒uffy says he agreed to 鈥榮tay silent鈥 on the orders of the PMO. In exchange, the Prime Minister鈥檚 Office agreed to cover the cost of the senator鈥檚 fraudulent expenses,鈥 NDP Leader Thomas Mulcair said. 鈥淲hy were taxpayer-funded lawyers used to negotiate this secret backroom deal between the prime minister鈥檚 chief of staff and Senator Duffy? Was taxpayers鈥 money used to bankroll Senate-gate, yes or no?鈥

Foreign Affairs Minister John Baird responded, saying: 鈥淚 reject much of the premise of his question. The government鈥檚 been very clear that the prime minister was not aware of this payment until media reports surfaced last week.鈥

In response to a separate question, Baird also denied that taxpayers鈥 money was used to pay Duffy鈥檚 expenses.

Liberal Leader Justin Trudeau accused the government of having 鈥渓ost its moral compass.鈥

Trudeau asked: 鈥淎pparently when Conservatives break the rules, they get their debts secretly paid off by their friends in high places. It boggles the mind. Nobody over there even thinks anybody did anything wrong except get caught. When will they release this secret document, allow for a full investigation, and while they鈥檙e at it, how about apologizing to Canadians?鈥

Baird denied the existence of a legal deal.

鈥淲ith respect to a legal agreement that the member opposite refers to, our understanding is there is no such agreement. This issue has been referred to two independent authorities, which will look into the matter and we look forward to them reporting back to Parliament and to Canadians.鈥

On Tuesday, Harper touted his own government's efforts to introduce accountability and transparency since they were first elected in 2006, saying "Canada now has one the most transparent and accountable systems of governance in the whole world and this is something Canadians are rightly proud of."

However, he said it is something Canadians can never take for granted.

Without getting into specifics about the recent controversy, Harper said Sen. Marjory LeBreton, leader of the government in the Senate, has been tasked with closing loopholes in the way senators claim expenses. Those not willing to play by the rules should get out, Harper said.

"Anyone who wants to use public office for their own benefit should make other plans or better yet leave this room," Harper said to rousing applause.

He also said that he "didn't get into politics to defend the Senate" and said the Conservatives have pushed to reform the Red Chamber but have been blocked by the opposition parties.

The NDP on Monday as well.

The New Democrats have called on Harper to give Canadians a public explanation of exactly what he knew and when.

Charlie Angus, the NDP ethics critic, said Harper missed an opportunity Tuesday to clear the fog surrounding the repayment of Duffy's expenses, calling it an "epic failure."

"When you write a secret cheque to a politician and there appears to have been perhaps an agreement to go easy on him in the Senate, that breaches all the ethical rules, that breaches the rules of any politician and it also may breach the Criminal Code," Angus told 麻豆影视 Channel.

"And this happened out of the Prime Minister's Office and the prime minister completely dodged that, he refused to take responsibility for it, he refused to talk to the media, he's refused to talk to Canadians."

On Tuesday, Minister James Moore told reporters that any Parliamentarian who doesn鈥檛 respect taxpayers鈥 money 鈥渟hould leave.鈥

Harper departed Tuesday afternoon for Peru, where he'll take part in bilateral trade discussions before heading to Colombia for talks on the Pacific Alliance trade group. Chile, Colombia, Mexico and Peru are permanent members of the trade group, while Canada has observer status.

However, Canada already has free-trade agreements with all four countries and critics have questioned the timing of the out-of-country trip with so much going on in Ottawa.