undefined | Better HN

0 pointssmallstepforman5h ago0 comments

Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”.

Stunned to see that Gemini threw its digital arms in the air and gave up.

0 comments

16 comments · 11 top-level

base6985h ago· 3 in thread

That's better than the loop grok got stuck in trying to use git and push the work it did leading to a $15 api credit deduction.

whh5h ago

Getting AI/ML to acknowledge "I don't know" is such a challenge.

taneq2h ago

This is why the world model approach is so important. It allows you to feed back the prediction accuracy of the model to itself at training time, enabling it to predict (to some degree) its own uncertainty. If you jump through a couple of hoops you can also do this at run time to give it “spidey sense” that something’s not right with current inference.

jgalt2124h ago

Not true regarding ML, most ML methods support RMSE even if they are non parametric methods.

1 more reply

hashta4h ago· 1 in thread

That's interesting because my experience has been almost the opposite. A few months ago I tested Gemini on converting screenshots of tables from PDF files into CSV. I tried it on several different tables and it got every one right. It consistently outperformed ChatGPT.

jatora3h ago

anyone who has used both knows this is inaccurate or dishonestly stated (ie. you were using gpt nano or some nonsense)

anitil2h ago· 1 in thread

My go-to for this is to screenshot and use the built-in text extraction in the screenshot tool (I'm on a mac), then pass on that text data to whatever processing. It's a pretty good tool so long as the PDF is in OK shape (I've had errors in scanned images).

nradov2h ago

It's so horrible that in 2026 people are still publishing important data and specifications in a format like PDF that's difficult for LLMs to consume. We need to drag them kicking and screaming to HTML or Markdown. Heck, even Microsoft Word DOCX is superior for reliable parsing and content extraction.

anigbrowl2h ago

It's extremely hit or miss. I've had it one-shot a pretty decent analytic prototype from a brief description, but also had it get trapped in hour-long back and forth regression hell over incredibly simple things like adding a static favicon (ie it would add it, then keep taking it away with every subsequent iteration, breaking something else every time it was asked to put the favicon back etc.).

BobbyTables23h ago

Tabula + Excel could probably do it quicker.

staticman24h ago

You didn't say whether you were using the App but the App's performance seems to be severely throttled compared to API.

fsmv4h ago

You should just have it OCR a screenshot of the PDF that would probably work better

mjcohen2h ago

Years ago, I used Acrobat to extract tables from a PDF. Had to do it manually, but it pasted nicely into Excel.

jjice5h ago

I haven't heard any accounts of it doing that since Gemini 2.5, but it was pretty easy to get it to do it with a programming task back then after a few failed attempts. Very interesting to hear it'll still do it.

staindk4h ago

We've been quite impressed with GCP Document AI. Not sure if it has a free tier but perhaps that's where Google is putting all the good OCR.

suuuuuuuu4h ago

I envy you that it admitted that rather than simply making up data and lying about it.

j / k navigate · click thread line to collapse