Four Seconds Max

This is an excellent article discussing the speed of web pages: (dead link 2015)

A couple of bullet points from the article:

• The consequences for an online retailer whose site underperforms include diminished goodwill, negative brand perception, and, most important, significant loss in overall sales.
• Online shopper loyalty is contingent upon quick page loading, especially for high-spending shoppers and those with greater tenure.
• JupiterResearch recommends that retailers make every effort to keep page rendering to no longer than four seconds.

Also, Jakob Nielsen has two good articles on the same topic:
http://www.useit.com/papers/responsetime.html
http://www.useit.com/alertbox/9703a.html

Those reference research dating from the 1960s' - 10 seconds is about the max you want to go.

Interesting stuff.

You can also check out the original papers by Goodman and Spence from the 70s