The origins of 'robots.txt'

From, of all places, a Slashdot comment.

"it was written to keep this rather simplistic and brain-dead perl n00b from pillaging Martin's bandwidth."

(Read more at Slashdot.org . . . )