"Papa, you always say we're not the owners of Downton but the caretakers."