A new book reveals his surprise that his supposed opponents defended him at a low point in his post-presidency.