kavaivaleri

Commonsense Reasoning - HellaSwag - Winogrande - PIQA - SIQA - OpenBookQA - ARC - CommonsenseQA

Logical Reasoning - MMLU - BBHard

Mathematical Reasoning - GSM-8K - MATH - MGSM - DROP

Code Generation - HumanEval - MBPP

World Knowledge & QA - NaturalQuestions - TriviaQA - MMMU - TruthfulQA

I collected their descriptions and links to their original papers here: https://www.turingpost.com/p/llm-benchmarks

Just like children and animals learn through sight and touch, AI needs computer vision (CV) to understand the physical world.

Follow along to learn more about the importance of CV in the next-level Artificial Intelligence: https://www.turingpost.com/p/cvhistory1

- AllenNLP Interpret: A library for interpreting and visualizing LLM predictions, assisting in model explanation and debugging. It works for any model of your choice.

- LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). Features include assessing text quality and relevance, hallucinations check, sentiment and toxicity analysis.

- BERTViz: Specifically designed for visualizing and interpreting BERT-based LLMs. Helps visualize attention in NLP Models (BERT, GPT2, BART, etc.).

- SHAP (SHapley Additive exPlanations): A game theoretic approach to explain the output of any machine learning model. Allows users to use models from the transformers library by HuggingFace.

- AI Fairness 360: An extensible open-source toolkit can help you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle

- Prometheus: An open-source monitoring toolkit for collecting and querying metrics from LLMs in real time.

- Grafana: Integrates with tools like Prometheus and Elasticsearch to provide visualization and analysis of LLM metrics and logs.

Learn more about LLM monitoring and observability at https://www.turingpost.com/p/monitoring

Classics

- Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville

- The Book of Why: The New Science of Cause and Effect by Judea Pearl, Dana Mackenzie

- Designing Machine Learning Systems by Chip Huyen

- The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingo

Fresh look (books published in 2023)

- The Worlds I See: Curiosity, Exploration, and Discovery at the Dawn of AI by Fei-Fei Li

- The Coming Wave by Mustafa Suleyman and Michael Bhaskar

- Understanding Deep Learning by Simon J. D. Prince

History of Artificial Intelligence and Machine Learning

- The Turing Guide by Jack Copeland, Jonathan Bowen, Mark Sprevak, Robin Wilson

- The Quest for Artificial Intelligence: A History of Ideas and Achievements by Nils. J Nilsson

- Machines Who Think: A Personal Inquiry into the History and Prospects of Artificial Intelligence by Pamela McCorduck

AI and Future Studies

- The Year in Tech, 2023: The Insights You Need from Harvard Business Review (HBR Insights Series) by Harvard Business Review, Beena Ammanath, Andrew Ng, Michael Luca, and Bhaskar Ghosh

- AI 2041: Ten Visions for Our Future by Kai-Fu Lee and Chen Qiufan

- The Age of AI: And Our Human Future by Henry Kissinger, Eric Schmidt, and Daniel Huttenlocher

AI ethics collection

- AI: Its nature and future by Margaret Boden

- AI Ethics by Mark Coeckelbergh

- The Costs of Connection: How Data Is Colonizing Human Life and Appropriating It for Capitalism by Nick Couldry, Ulises Mejias

- Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence by Kate Crawford

Lastly, a fire-side list of books written completely using AI

- Aum Golly: Poems on Humanity by an Artificial Intelligence by Jukka Aalho

- Aum Golly 2: Illustrated Poems on Humanity by Artificial Intelligence by Jukka Aalho

- 1 the Road by Ross Goodwin

- The Inner Life of an AI: A Memoir by ChatGPT by Forrest Xiao

- Bob The Robot: Exploring the Universe - A Cozy Bedtime Story Produced by Artificial Intelligence by Olle Green

- (Fun!) 50 Ways AI Would End The World: Written by AI by Rob Knott

Find links and descriptions at https://www.turingpost.com/p/books2023

The "Vector Database Market" research report forecasts significant growth in the sector from $1.5 billion in 2023 to $4.3 billion by 2028, a CAGR of 23.3%.

We discuss why these databases matter for AI, how they work, and their role in handling complex data; we also explore alternative solutions and provide expert insight on security. Plus you get a curated list of open-sourced vector databases and search libraries. -> https://www.turingpost.com/p/vectordatabase

In three days, ChatGPT, developed by OpenAI, will celebrate its first public launch anniversary. Its success has shown us just the tip of the iceberg of ML and deep learning capabilities. GPT models and other Foundation Models (FMs) have since ignited numerous debates about the future of AI development: Will the focus be on synthetic data, or perhaps a novel architecture beyond transformers? Or both, or something else? That we will see.

But today let’s reflect on just a few significant achievements and impacts ChatGPT has made in just one year: https://www.turingpost.com/p/fod30

We are happy to introduce a new series of interviews that highlights the real-world implementation of Foundation Models (FMs) and Large Language Models (LLMs). We start with an absolutely fascinating topic that can bring so many changes into our world: Bioengineering. In this insightful conversation with Barry Canton, CTO and co-founder of Ginkgo Bioworks, we discussed the evolution of synthetic biology, the integration of AI and GenAI in their R&D, and the future of bioengineering. Let's see what it takes to create a model that 'speaks DNA'.

Read further at https://www.turingpost.com/p/barrycanton-gingko

Editorial:

Since the Tesla I was driving, equipped with Autopilot and Full Self-Driving, nearly drove me under a truck, my interest in self-driving cars has intensified. As if serving my curiosity, the last week was filled with news about self-driving, driverless, autonomous vehicles, and robotaxis – so, here's my Monday take.

Let’s be clear, I don’t blame Tesla. In the same way, I can’t fault ChatGPT for making things up. It’s not truly intelligent. ChatGPT doesn’t chat with full understanding and responsibility for its word choices, just as self-driving cars are still largely experimental. If you're a good driver, you can somewhat rely on it – but with heightened awareness. Similarly, if you're a sharp lawyer, you can somewhat rely on ChatGPT, ensuring you verify the info it generates.

Read more -> https://www.turingpost.com/p/fod25

There is almost nothing less stable than the landscape of Generative AI (GenAI). The last week's changes include a sudden $0.3 billion drop by Jasper. It’s not a surprise in a world where apps are built on someone else’s model/technology. But we will be talking about something ‘stable’ today: the Stable Diffusion model and Runway – whose researchers are behind the model development. It’s interesting that Stable Diffusion has also become a bedrock for another unicorn from our chart – Stability AI. But today we concentrate on Runway, number 7 in the unicorn family (so far).

https://www.turingpost.com/p/runway

The concept of self-reproducing automata envisioned by John von Neumann in 1948 heralded a paradigm where machines could mimic biological self-reproducing. Alan Turing in 1950 posed a question if machines can think.

Von Neumann never saw his self-reproducing machine come to life, but 75 years later this notion has resurfaced with contemporary advances in machine learning (ML), illuminating a pathway toward realizing von Neumann’s ambitious vision and touching some of Shannon’s questions. A few of last week's research papers suggest a future where machines could attain a level of autonomy and self-organization akin to biological systems.

1. The idea of self-assembly in “Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs” underscores the potential for artificial networks to evolve autonomously. This process, inspired by biological neural development, alludes to a future where artificial networks might organically grow and adapt to tasks, possibly lessening the extensive engineering currently needed for effective neural network design.

2. The exploration of Theory-of-Mind (ToM) in Large Language Models (LLMs), as discussed in “How Far Are Large Language Models From Agents with Theory-of-Mind?”, evaluates LLMs’ potential to pragmatically act upon inferred mental states, a crucial aspect of human intelligence. While unveiling a gap in translating inference into action, it also presents a new evaluative paradigm, potentially directing future research to bridge this divide.

3. The self-improvement narrative discussed in “Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation” adds a significant layer to this discourse. The idea of self-improving code generation could serve as a scaffold for self-reproducing automata.

4. The paper “Language Models Represent Space and Time” by Max Tegmark sparked discussions across the web on its terms and doubtful conclusions. Gary Marcus digs into why “correlations aren’t causal, semantic models”. However, temporal and spatial capabilities are fundamental for intelligent agents to interact meaningfully with their environment and should be explored as a step toward more sophisticated AI systems.

Join our weekly email list to stay updated about the latest trends without the hype https://turingpost.substack.com/p/fod23-the-quest-for-self-reproducing