"I don't want Ashley to have problems because of me."