"If it's put to a referendum, the town can decide."