Primary sources of AI training data

2 pointskishoredbn3y ago1 comments

What are some of the primary sources of public data that are utilized in the training of large AI models?

1 comments

1 comments · 1 top-level

The Facebook Llama paper[1] calls out a number of the ones they used.

Briefly, they include:

- CommonCrawl

- Wikipedia

- Github

- StackExchange

- arXiv

and "Books" which appears to be made up from Project Gutenberg books among others.

j / k navigate · click thread line to collapse