One of the complaints we got while testing the web app version of Gander was that it was too slow (Facebook found this to be the case as well). Now that we've begun building out the native app, Skimbox, we're determined to escape that criticism. However, while we are well aware of what slow is; we're on shakier grounds vis-a-vis what it isn't.
Nielsen's rules of thumb give the following guidelines:
0 - .1 second: feels instantaneous
.2 - 1 second: users will notice the delay
~/+ 10 seconds: users will want to perform other tasks
Nielsen wrote these in 1993, but claims they've held true for the last several decades. For desktop, I agree, but I hold that this changes on mobile.
Of course, there's a lot to be said about designing for tasks and to make sure people are able to do what they need to do. In "The Truth About Download Time," users cited certain websites as "fast" that were in fact slower in comparison to websites perceived as "slow". Here, the fast/slow continuum is really a helpful/frustrating one.
That being said, one does want to be sure not to make users wait unduly. So what is unduly?
Roto and Oulasvitra found while downloading, mobile users' typical gaze time in mobile environments are 4-8 seconds on average. In a lab setting, this increases to an average of 14.3 seconds.
They recommend to notify users if the task is expected to take more than 4 seconds, instead of Nielsen's 10 seconds. I generally agree, with some tweaks.
Any interaction should have accurate feedback within .1 second.
.2 seconds - 4 seconds: have some sort of animation to fill in the time (one example is the Gmail app's color-shifting orb.
Any loading of text or emails should happen within 4 seconds.
Downloading emails or anything that would take longer than 8 seconds should trigger a time estimate/additional user feedback (notification to come back or progress bar or the like).
Above all: Make sure it's not frustrating to use.