The story
Uncertain, but dates back at least to the June 15th, 1957 issue of The New Yorker (Vol. 33 No. 17, page 23). In "The Talk of the Town", which was a collaborative effort by the magazine's staff writers: "INTERROGATIVE WISP FOUND IN GUTTER OF MACDOUGAL STREET: 'I know you are but what am I?'" The phrase is often attributed to Pee-Wee Herman.
Etymology from Wiktionary, CC BY-SA 4.0.