I doubt there are too many people in tech making this mistake, but without numbers there’s a good chance you fall prey to people’s inaccurate perceptions or explanations for their own behavior.

On the other hand, without those first-hand accounts, it’s all too easy to tell a mistaken story about your numbers. In particular, your users’ sentiment towards their time spent in your app is a guess, unless they tell you.

I think timescale is another crucial dimension to evaluating time-in-app, and whether it’s a positive indicator for your business.

For instance, driving up engagement has doubtless been good for Facebook’s financials in the short- to medium-term. Arguably though, that relentless focus has led to the present political climate where they’re fighting off regulation. Whether that’s an existential threat is yet to be seen (one can only hope), but it certainly casts the metric in a different light.

As you say, it comes down to the fact that there is no silver bullet metric; there’s no substitute for thinking about and dissecting the data you collect.