" I strongly suspect that it a lot like Brownian Motion"
1.Mkt isn't like BM. BM permits -ve values & any market index say SPY must be non-negative.
2. We (quants) model market indices like SPY as a log-normal process like so :
dS = rSdt + sigmaSdz
where
S = underlying ( or market index )
dS = change in S, dt = change in time
r = risk free interest rate ( about 0.03 % currently )
dz = normally distributed rv with mean 0, std dev sigma
There is an embedded Brownian component in dz, so also in S. But that doesn't make S itself Brownian.
More here: http://en.wikipedia.org/wiki/Wiener_process
"No observer could possibly predict which direction it will jump next"
Nobody is trying to do that in the very next instant.
OTOH, you can certainly predict where the market will go, say, in a year.
Well known econ result: DP ratio regresses positively with return. ie. Dividend-price ratio predicts stock return with high statistical significance.
Source: Cochrance, French/Fama, et al.
http://dash.harvard.edu/bitstream/handle/1/3122601/campbells...
"there are plenty of people making random guesses...there are also plenty of fools, and they are investing on all sorts of techniques which ignore any of the global considerations, and of course this is how bubbles expand."
Ha ha ha! Very amusing but very inaccurate. I think this is my general problem with geeks on the outside of finance. They happen to think markets are purely stochastic. X has randomness doesn't mean X is unpredictable. Otherwise why have APT, CAPM, covariance matrices, factor models, Heath Jarrow, Black Litterman...you've written off some 100+ years of economics in one breath !
To summarize, there are tons of observers, who predict where the overall market or some portfolio of equities ( or bonds or agencies or munis or treasuries or whatever else ) will go. To get data points for these predictions, you use a historical time window, compute coVar matrices, invert them, build a CAPM....etc. Anyways, its not rocket science. There are tons of CFAs & Finmath engg ( I'm one of them ) who do this sort of work day in and day out. Its our bread & butter. If I said something like "market is a lot like brownian motion & people are making random guesses", I'd be out of a job.