Financial market applications of LLMs (opens in new tab)

> All of this should be done by professionals and nothing is "retail" ready.

Don’t worry, just train the LLM to always append “This is not financial advice.” to their responses. Boom, retail ready.

Hard to waste any time reading about AI because it's likely written by AI. But then I probably shouldn't read anything written past 2022.

(non informed, layman sideline perspective from casual reading on this subject over the years)

Real time (financial) sentiment analysis on financial news sources has been integrated for a long time. Thing about LLM's is, while they could improve on quality, they need to get the latency down before being useful in straight trade. For offline analyst support where time is less of an issue they can ofc be useful, e.g summarizing/structuring lots of fluffed or trawled content.

I'd think the first application would be along the lines of Github Copilot, perhaps locally hosted - quantitative traders will write a lot of (proprietary) code, too

I thin the underlying vector databases should have decent uses in financial markets.

Since they can understand taxonomical-ish relationships, a vector db should be able to codify sufficiently large market mover strategies, assuming those strategies are remotely predictable. Once a rival's strategy is codified, it should be possible to undermine it, like some form of heuristic-based insider trading.

One other area which I think is potentially quite interesting is using LLMs to help in deciphering "Fed-speak". Eg JP Morgan built an LLM to try to predict the impact on interest rate markets of speeches by various central bank policymakers.

As a human, I like anomaly tracking if I understand what you mean by that. LLMs are maybe 99% good and 1% totally wrong (hallucination). Lots of profit betting against the 1% totally wrong. Not hard to see when wrong but do need to act fast.

This makes sense. Can you clarify what you mean by journal management in this context?

The most interesting applications for LLMs in finance are basically all summarization.

Could someone please clarify what "journal management" means?

Can Vision GPT be trained to do technical analysis?

Could something like that proposed in "Training Language Models to Generate Text with Citations via Fine-grained Rewards" [0] work for you?

0. https://arxiv.org/abs/2402.04315

I worked on a similar application and eventually we shelved it. We just could not be confident enough that the numbers in the report produced are correct. There were enough instances of inaccuracies to not use it for important decision making. Which actually meant a lot of double work.

I assume you're ingesting PDFs. If so, how are you handling tables accurately?

This is really interesting.

We build multimodal search engine on day-to-day basis. We recently launched video documents search engine. I made a Show HN [0] post about ingesting Mutual Fund Risk/Return summary data (485BPOS, 497) and searching it with AI search. We are able to pinpoint to exact term on given page. It is fairly easy for us to ingest 10K, 10Q, 8K and other forms.

You can try out demo for finance-application at https://finance-demo.joyspace.ai.

Our search engine can be used to build RAG pipelines that further minimizes hallucinations for your LLM model.

Happy to answer any questions around this and around search engine.

[0]https://news.ycombinator.com/item?id=39980902

Price speculations are hallucinations about future with hope of happening.

I used to work in financial software, and when writing the charting UIs, I'd wire them up to a randomwalk to generate fake time series data. It was a relatively common occurrence for a VP or the company CEO to walk by, look at my screen, and say "What stock is that? Looks interesting."

Unpopular opinion backed up by experience: a randomwalk is the most effective model for generating timeseries that have the "feel" of real stock charts.

As always, when running time series predictions on financial datasets, one need to use daily return (including dividends, corporate actions, etc.) rather than end of day price.

Simply outputting the last value (as more or less shown in these charts) is a pretty good end of day price predictor!

As more of the reports get written by layers of AI it makes me wonder how lossy and noisy this whole pipeline is becoming.

Todays derivatives and their pricing are based on the premise that stock prices can not be predicted and behave like a Brownian motion system. If you take real time data from any stock and calculate in order how many times a stock went up in a row or down in a row you end up almost perfectly with a natural probability distribution. HFT's are involved in market making and arbitrage both of which already involves high speed, the later much more, and earning minuscule profits. There are ghost patterns who can be mined for a certain period of time but they are not solely calculated based on trading time series. They involve complex proprietary calculations, some machine learning and relationships between stocks. There is no pattern in the flow how a particular stock is trading.

Also from a long-term view its very questionable. How should a model be able to predict that in the middle of a high interest environment, a tech bubble burst and a dumping stock market in general, a new platform called Chat-GPT gets launched that basically carries the whole world's stock market to new heights which causes among other things retail investors to liquidate bonds and other high interest environment assets and flood it into the stock market. It is more than completely of the text-book. That can not be predicted. The million dollar spending guy is at the end the same way off as the guy who simply employs a 100 python line trend-following strategy.

The data is reasonably easily acquired, for a price...

> is that gaming financial markets is the only real application of anything scientific

medicine (living longer, curing disease, vaccines, etc), cheaper energy, cheaper transportation, cheaper construction, cheaper food, better communication, new forms of entertainment, just off the top of my head.

What does that even mean? How is the atomic bomb not real?

Many people respond to things in conversations just based on common patterns, without much “thought”, and it’s hard to see the difference.

What would a contribution to financial markets even look like?

The only meaningful contribution to financial markets that I can see can come from asking the question 'what are we even doing with our lives?', followed by elimination of 99% jobs in finance and many other industries.

No, not at all, given the context of stock trading. Stocks do not trade the same way today as they did in 2014. Similarly there is no point in using trading data from the 1850s given that that kind of market with those kinds of traders will never ever exist again. You can only pick a few recent months or weeks to capture current trading sentiment/technique and even then everything could get blown away after the next rate hike or international incident.

Generally I don't think there is any alpha in training transformers to predict the next price point just given historical price data, because the price is determined by humans (and algorithms trained on data generated by humans) that react to news. If you can predict the news, you can probably predict stock prices, but if you could predict the future you'd have AGI and not some dingy time series calculator.

For rough, high-level comparisons, it might be seen as "not far off," but for detailed, technical assessments, the difference is considerable.[1]

[1] https://chat.openai.com/share/a19a3b57-398c-49e7-a140-f58784...

Good enough. The comparison is silly though: time series data is not anything like tokenized text data.

All you'd get are projections with percentage error margins; you can choose the riskier plays, but it is literally priced in.

You'd also get clapped by the HFT bots.

The real magic is pairing real human intuition and the LLM's innate ability to discover hidden intuitions and articulate them to find an "asymmetry"-where you believe you have found a gradient/play that is under/over valued and play the opposing side - or selling/further leveraging that information.