"I'd be drummed out, Ellen would get shunned."