"Unfortunately, someone's going to go home"