"We knew it wouldn't be easy for him"