"But following orders just cost 40 people their homes"