No matter what you do, some people will criticize you, and, if you are entirely sure that you would not be ashamed to explain your action to someone whom you loved and who loved you, and you are satisfied in your own mind that you are doing right, then you need not worry about criticism, nor need you ever explain what you do.

Eleanor Roosevelt (1884–1962) American politician, diplomat, activist; First Lady of the US (1933–45)

Memoir (1937), This Is My Story, ch. 4 “Home Again”
    (Source)

Not original from Roosevelt, but her recollection of advice given her by her "Auntie Bye" (her paternal aunt, Anna "Bamie" Roosevelt Cowles) when she was young. Also recounted in her memoir (1961), The Autobiography of Eleanor Roosevelt, Part 1, ch. 3 "Home Again," with only trivial changes in punctuation and taking out the "whom."

See also Roosevelt.