I'm in charge, insists Cameron

Mr Mercer was widely condemned after he said he had encountered "a lot of ethnic minority soldiers who were idle and useless but who used racism as a cover".

He said yesterday he felt "bruised" by the controversy and that his words had been misinterpreted.

Some Tories, including senior party figures, have suggested Mr Mercer was treated harshly and that his sacking was an example of excessive "political correctness".

But Mr Cameron told reporters: "That is completely wrong, and I'm running the Conservative Party."

His swift action, on the grounds that no one could "make excuses" for racism, was seen as a part of his determination to kill off the Tories' old "nasty party" image.

Yesterday Mr Cameron made clear he would not retreat on his mission of changing the party – and he was prepared to act ruthlessly against anyone stepping out of line.

Voters were tired of Labour and ready to look at the Tories again, he told activists at the Scottish Tory conference in Perth. "But everyone should be clear on one thing," he said. "The political weather has changed because the Conservative Party has changed. We're a modern party focused on the future."

Some Tories might want a rest from change, and believe that being ahead in the polls meant the party could "sit back and relax, go back to our old ways", he said. "Well that's not going to happen. The change goes on – faster, wider, deeper."

Mr Mercer, MP for Newark and a former Sherwood Foresters colonel, insisted yesterday his comments had been taken out of context.

"I think the most mendacious angle was taken on what I said. I am a bit bruised. I am very sorry if I have caused offence to the fantastic people who I commanded, many of whom were black," he said.

But, referring to Mr Cameron, he added: "I have had a lifetime of public service and I fully understand that if orders are given from above I have to respect them."

Later, he faced down callers to Daily Express columnist Vanessa Feltz's BBC London show.

After one said he was a "racist'', Mr Mercer replied: "I have not called anybody any names. Please read the papers carefully. What I have been saying is that it was my experience from time to time that these names are used in the Army."

Meanwhile, Mr Cameron faced pressure to discipline Shadow Scottish Secretary David Mundell for scathing comments made about senior Scottish Conservatives.

In a memo to Mr Cameron leaked on the eve of the conference, Mr Mundell, the only Tory MP north of the border, claimed the party had a "lack of thinkers" and criticised its leader and president.