MSFT's WizardLM2 models have been taken down (opens in new tab)

I can't imagine how toxicity in a LLM can be harmful. Can't we write stories with toxic characters, for example?

It feels like the greatest minds of our era are creating an amazing piece of technology. And we are hindering it in the name of corporate cover ass and bullshit jobs.

So this is a good thing for people like me who don't care if some piece of software is "toxic" with me or not.

It was trained on content from the internet. I'd be massively surprised if it somehow wasn't toxic. Humanity (or a small portion of it) is full of assholes. As much as that sucks, shouldn't the embeddings reflect the reality of the training content? If you want fluffy bunnies and flowers and happy people holding hands, shouldn't you just train it on that content?

Toxicity testing? That a self-imposed requirement by Microsoft?

No, the 8x22 (~140B) is more powerful than the 70B, it had higher eval score according to the blog. But 70B was based on LLama whereas the 8x22 and 7B were mixtral/mistral-based.