"This man knows how to lead, and he did. The country's better off for it," President Bush said.
The president called Rumsfeld "one of America's most skilled, energetic and dedicated public servants."
Departing after six years in office, Rumsfeld said he felt "a sense of urgency about the real challenges ahead" in a time of terrorism, unstable dictators and threats of nuclear proliferation.
The attacks of Sept. 11, 2001, awakened the world to the existence of a global extremist movement whose adherents believe it is their calling to kill Americans and other free people, Rumsfeld said.
"Ours is also a world of many friends and allies, but sadly, realistically, friends and allies with declining defense investment and declining capabilities and, I would add, as a result, with increasing vulnerabilities," Rumsfeld said. "All of which requires that the United States of America invest more."
Recalling the collapse of the Soviet Union, Rumsfeld said, "That history did not happen by accident. And it most assuredly was not made by people sitting safely on the sidelines."
"Today, it should be clear that not only is weakness provocative, but the perception of weakness on our part can be provocative as well," the secretary said.
Mr. Bush made no mention of the often-harsh criticism of Rumsfeld – that he was arrogant, ignored the advice of critics and made many mistakes in his execution of the Iraq war.
"Every decision Don Rumsfeld made over the past six years, he always put the troops first, and the troops knew it," Mr. Bush said.
"We've been through war together. We have shared some of the most challenging moments in our nation's history," said the president.
But Rumsfeld's conduct of the war in Iraq became a political liability and Mr. Bush decided he needed a new defense chief and asked Rumsfeld to step down, CBS News White House correspondent Mark Knoller reports. Robert Gates will be sworn in as the 22nd secretary of defense on Monday.
Mr. Bush announced Rumsfeld's departure the day after his Republican Party was jolted in the November elections.
"I've never worked harder for a boss and I've never learned more from one, either," said Vice President Dick Cheney, who began his career in politics as an intern for Rumsfeld in 1969.
Cheney praised Rumsfeld as a man with "near perfect recall. He has the way of asking you the one question you are not prepared for. And apparently he does not sleep."
Gen. Peter Pace, chairman of the Joint Chiefs of Staff, said the 2.4 members of the U.S. military joined in saluting "this incredible American, for his leadership and service to our country."
"He's a man of enormous commitment," Pace said. "He pushed us hard. The only person he pushed harder was himself."
A former Navy aviator, the 74-year-old Rumsfeld is the oldest defense secretary in U.S. history and the only person to have held the position twice. He was the youngest defense secretary when he began his first stint as defense chief in 1975.
Rumsfeld is the longest-serving member of the Bush cabinet, Knoller reports, but he leaves office less than two weeks shy of becoming the longest-serving secretary of defense. That distinction is held by Vietnam-era Robert S. McNamara, who left under a cloud of another war gone awry.