Same deal with coffee beans, which contributes to their higher price. The coffee beans do not ripen at the same time, so they must be hand-picked so the worker can identify which ones are ready. Lots of manual labor.
And with coffee being so widely consumed, savings here would easily be felt by a lot of people, although I doubt Starbucks would like it...