undefined | Better HN

0 points_the_inflator18d ago0 comments

Exactly, and all on an embedded system with quite restrictive settings and no overclocked Intel lastest generation combined with NVIDIA's 10k graphic cards.

0 points_the_inflator18d ago0 comments

Exactly, and all on an embedded system with quite restrictive settings and no overclocked Intel lastest generation combined with NVIDIA's 10k graphic cards.

0 comments

5 comments · 1 top-level

charcircuit18d ago· 4 in thread

Embedded systems can make network calls to powerful, GPU equipped servers.

ceejayoz18d ago

Sure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications.

charcircuit17d ago

You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too.

2 more replies

Chu4eeno18d ago

They really shouldn't, though.

charcircuit17d ago

It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things.

1 more reply

j / k navigate · click thread line to collapse

0 comments

5 comments · 1 top-level

charcircuit18d ago· 4 in thread

Embedded systems can make network calls to powerful, GPU equipped servers.

ceejayoz18d ago

Sure. Claude does that. "Cogitated for 1m 50s" doesn't work for real-time applications.

charcircuit17d ago

You can submit many queries in parallel to increase throughout. Smaller models and faster hardware can reduce the time per query too.

2 more replies

Chu4eeno18d ago

They really shouldn't, though.

charcircuit17d ago

It can offer a ton of user value. There is a whole industry built upon this idea, Internet of Things.

1 more reply

j / k navigate · click thread line to collapse