Show HN: Run LLaMa2 on the Browser with Ggml.js (opens in new tab)

(rahuldshetty.github.io)

23 pointsanonymousd3vil2y ago8 comments

You can now build serverless AI inference web application with ggml.js's LM backends.

8 comments

6 comments · 2 top-level

quickthrower22y ago· 3 in thread

What is up with that, seems too fast and not that accurate. Is it the smallest model?

pawelduda2y ago

This is the model used: https://huggingface.co/karpathy/tinyllamas/resolve/main/stor...

Makes sense why I ask it to answer 2+2 and it tells me stories about Bob, Alice and so on.

quickthrower22y ago

Lol I was going to say it had a tinyGPT feel about it

anonymousd3vilOP2y ago

Lol Okay, I should have been clear in the title. It is the TinyStories version of the LLAMA2. Its not good quick maths for sure.

1 more reply

gardenfelder2y ago· 1 in thread

Somewhat amusing:

Me:

What are the key bullet points?

A substantial percentage of patients with ulcerative colitis (UC) have irritable bowel syndrome- (IBS-) like symptoms despite adequate treatment and endoscopic remission. In this study, we evaluated the clinical efficacy of probiotic therapy for residual IBS-like symptoms in patients with UC in endoscopic remission.

LLaMa2 (edit typo)

What are the key bullet points? A substantial percentage of patients with ulcerative colitis (UC) have irritable bowel syndrome- (IBS-) like symptoms despite adequate treatment and endoscopic remission. In this study, we evaluated the clinical efficacy of probiotic therapy for residual IBS-like symptoms in patients with UC in endoscopic remission. eat. When Halloween arrived,

sigmoid102y ago

It's not the "actual" LLaMA2, although you have too look at the code to see what it is: It's a miniature mock up trained on the tinystories dataset. The model is orders of magnitude smaller than even the smallest official LLaMA2 release and it should honestly be an immediate giveaway if anyone thought that the real stuff would actually run inside browsers on today's hardware. Quantization or not, we're just not there yet.

j / k navigate · click thread line to collapse