How often have I said to you that when you have eliminated the impossible, whatever remains, however improbable, must be the truth?

Arthur Conan Doyle (1859-1930) British writer and physician

Story (1890-02), “The Sign of the Four,” ch. 6 [Holmes], Lippincott’s Monthly Magazine, Vol. 45 (US) / 1 (UK)
    (Source)

The first appearance of the phrase in its most quoted form. However, earlier in the story, chapter 1, Holmes tells Watson:

Eliminate all other factors, and the one which remains must be the truth.
 

Similar expressions occur in The Adventures of Sherlock Holmes ("The Adventure of the Beryl Coronet"), The Memoirs of Sherlock Holmes ("Silver Blaze"), The Return of Sherlock Holmes ("The Adventure of the Priory School"), His Last Bow ("The Adventure of the Bruce-Partington Plans"), and The Case-Book of Sherlock Holmes ("The Adventure of the Blanched Soldier").

The original publication, and Doyle's manuscript (along with many other iterations across media) use "The Sign of the Four" as the title, while others (including the first book publications) use "The Sign of Four." The five-word form is used most commonly in the story, but the four-word form does show up. (More info.)

Published in novel form as The Sign of Four (1890-10).