Here is how Cass Sunstein discusses the Johnson impeachment:
"Andrew Johnson was impeached in 1868 for just one reason: he fired Edwin Stanton, the secretary of war (now called the secretary of defense), and he tried to replace Stanton with someone he preferred. You might well ask: Isn’t the president allowed to choose the Secretary of Defense? Doesn’t he get to fire members of his own cabinet? Excellent questions. You will remember that the framers created a unitary presidency. That is generally taken to mean that under the Constitution, the president can get rid of members of his own cabinet. Congress has no authority to limit that power. That’s certainly what Johnson believed. And ultimately, the Supreme Court agreed with him. 24 Nonetheless, Congress enacted a law that it called the Tenure of Office Act, which was specifically designed to forbid the president from removing certain executive officials, including the secretary of war, without the Senate’s approval. The law said that those officials “shall hold their offices respectively for and during the term of the President by whom they may have been appointed and for one month thereafter, subject to removal by and with the consent of the Senate.” 25 Believing that the Tenure of Office Act was unconstitutional, Johnson ignored it. So the House impeached him. Of course there was a dramatic political background. Johnson had become president only because of the assassination of Abraham Lincoln.
After the Civil War, the nation was embroiled in a debate about how to reconstruct the defeated South, and how to reunify the nation. Although Johnson was from the South, many in a group within the Republican Party, sometimes described as the Radical Republicans, hoped and believed that he would adopt an aggressive set of programs during Reconstruction, designed above all to protect and assist the newly freed slaves. Johnson badly disappointed them. He proved far more cautious than they expected, and as they saw it, far more solicitous of the defeated South. Emboldened by electoral success, the Radical Republicans enacted the Tenure of Office Act specifically to protect Stanton, who generally shared their views. More than that, the Tenure of Office Act was designed to threaten and to trigger impeachment. It explicitly said that if the president violated it, he would be committing a “high misdemeanor.” Gosh. As far as I am aware, nothing like that has ever happened in American history, either before or since. Johnson paid no attention. In response, the House passed no fewer than eleven articles of impeachment. They’re endless as well as redundant...
Johnson had a good-faith argument that he was acting in accordance with his constitutional authority. 27 For those who sought to impeach Johnson, things were even worse. As I have noted, the Supreme Court eventually ruled that Johnson was exactly right on the Constitution, which forbids Congress from requiring the president to obtain the Senate’s consent before firing members of his cabinet. 28 In the House, the vote against Johnson was overwhelming: 126 to 47.29 Johnson narrowly avoided conviction in the Senate, whose 35 to 19 vote to convict fell just one short of a two-thirds majority. 30 All nine Democrats voted Not Guilty; just ten of the 45 Republicans joined them. Johnson was a terrible president, but his impeachment violated the constitutional plan."
From: Sunstein, Cass R.. Impeachment: A Citizen’s Guide (p. 106). Harvard University Press. Kindle Edition.