undefined | Better HN

0 pointsdcl11mo ago0 comments

Search engines are still required for me. LLM's still get lots of very important things wrong.

Last night, I asked Claude 3.7 Sonnet to obtain historical gold prices in AUD and the ASX200 TR index values and plot the ratio of them, it got all of the tickers wrong - I had to google (it then got a bunch of other stuff wrong in the code).

Also yesterday, I was preparing a brief summary of forecasting metrics/measures for a stakeholder and it incorrectly described the properties of SMAPE (easily validated by checking Wikipedia).

I constantly have issues with my direct reports writing code using LLM's. They constantly hallucinate things for some of the SDK's we use.

0 comments

throwaway42243211mo ago

Asking for a list of companies in a specific sector also gives you made up tickers, or at best a list it found on some blog.

Was a bit more useful at questions like "Rank these stocks by exposure to the Chinese market", as you can prioritise your own research but in the end you just have to go through the individual company filings yourself.

j / k navigate · click thread line to collapse