BLOOMChat, a 176B parameter, Multi-lingual, fine tuned chat (opens in new tab)

LLMs being bad at math is a known issue, but what they are good at is writing programs. For example:

>>> Write a program that calculates if 120 is greater than 0.7.

>Sure, here's a program in Python that calculates if 120 is greater than 0.7:

    if 120 > 0.7:
        print("Yes, 120 is greater than 0.7")
    else:
        print("No, 120 is not greater than 0.7")

For straight input/output like what this model is trained on, questions like this don't work well. However if LLMs are equipped with tools (like a code interpreter), they get a lot smarter.

Try to not use it on math. Only GPT-4 has reasonable performance there. GPT 3.5 is also pretty awful. It's apparently extremely hard for any LLM to actually understand math. Maybe because they're language models, not math models, so math is a pretty far fetched "emergent property".

Bloom models are hopelessly under-trained. This one is worse than a 13B Vicuna.

I've got similar garbage out of ChatGPT (though tbf, pre-GPT4), I don't think LLMs can understand math

Why do people rush to test these on math problems, the things computers are already really good at?

you just need to teach it to use a calculator https://arxiv.org/pdf/2302.04761.pdf