There was actually an interesting study a few years back (uhhh, probably like 10?) that compared fMRI scans from several groups of people: habitual caffeine users on caffeine, habitual caffeine users *not* on caffeine, non-habitual caffeine users on caffeine, non-habitual caffeine users *not* on caffeine (jeez typing caffeine that many times in a row sucks).
The study found that non-habitual caffeine users do get a cognitive boost from using caffeine, but the habitual user needed the caffeine to get back to the baseline for the non-habitual user, and was below baseline when not on caffeine.
Not that I let that stop me from my morning and afternoon coffee, but there you have it.