The suggestion that the world could be completely other than it is always annoys those who are content with the way things are. Stories of imagination tend to upset those without one. Rulers are suspicious of new worlds where their writ does not run. Jailers don’t like escapism.

Terry Pratchett (1948-2015) English author

Foreword to David Pringle, ed., The Ultimate Encyclopedia of Fantasy (1999)
    (Source)

Often just given as "Stories of imagination tend to upset those without one."