"And the neighbors would let them stay over but only if I didn't tag along"