"He didn't want the job at HUD."