"The greater risk is allowing any of those groups"