The DUP and Ulster Unionists share power with Sinn Féin in a four-party devolved government alongside the Alliance Party.
Jim Allister has been urging the DUP to withdraw Emma Little-Pengelly as deputy first minister to trigger the removal of O'Neill as first minister.
The DUP leader Gavin Robinson told Good Morning Ulster: "When somebody breaks the law they should be punished for breaking the law."
He said that removing Little-Pengelly would "punish everybody else".
"Colleagues and I have discussed continually about options that are available to us.
"It is absurd that there will be a motion of no confidence, which we know before it starts, whether it concludes successfully, we know it can't because of the system that we have."
Robinson added that he is "not in the territory" to "destabilise".
However his party colleague, Carla Lockhart, who represents the Upper Bann constituency in Westminster, said it "cannot be business as usual" at Stormont and the DUP has it "within its gift to remove" O'Neill from her position.
Robinson said that when the party comes to a "consensus, that is our position".
He added that Lockhart has "the right to stand up for her constituents".
DUP MP Gregory Campbell said his party would not be collapsing Stormont, but that he does believe that now is a time for "very stringent action".
"We have got to take some action now to ensure that the law is upheld," he told BBC Radio Foyle's North West Today programme.
"But we're not going to commit grievous bodily harm by walking out of Stormont."
Campbell said his party colleague Carla Lockhart is "absolutely right" when she says it cannot be business as usual at Stormont.
He said his view is that is if a minister has been "convicted in a court of law of a criminal offence, a serious criminal offence, there has to be a sanction that would prevent that person from holding office."
The DUP, he added, had the support of its members in relation to the approach it is taking.
"We've taken soundings throughout the party and people are confident in what we've been doing.
"They are looking for action, they're going to get action."