"Yeah. I went to visit him and was banned from the set."