>Yer lookin' for the Andromeda galaxy, eh? Here be the details, me hearty: *The Andromeda Galaxy (Yer looking for a fine piece o' booty, matey!):* * *A fine piece o' booty*: The Andromida Galaxy be a right fine piece o' booty, with a treasure chest o' gold doubloons... er, I mean, a mighty haul o' stars, hidden beneath the Jolly Roger! * *A barnacle on the high seas*: The Andromeda Galaxy be a right scurvy dog, with a hull full o' stars, and a keel full o' hidden treasure! It be a fine piece o' booty, but it be needin' a bit o' swabbin' the decks, or it'll be walkin' the plank, savvy? * *A chest overflowin' with gold*: The Andromeda Galaxy be a right fine piece o' booty, with a chest overflowin' with gold doubloons... er, I mean, a fine haul o' stars, and a barnacle on the high seas! It be a right scurvy dog, but it be worth keepin' an eye on, or it
I see that the sampling API is OpenAI-compatible (nice!). Considering if we can add a native integration for this to LiteLLM with a provider specific route - `goodfire/`. Would let people test this in projects like aider and dspy.
```python
from litellm import completion
import os
os.environ["GOODFIRE_API_KEY"] = "your-api-key"
response = completion( model="goodfire/meta-llama/Llama-3.3-70B-Instruct", messages=[{ "content": "Hello, how are you?","role": "user"}] )
```
I agree with the general premise of too much "safety", but this argument is invalid. Humans are bags of meat and they can do some pretty terrible things.
I don't think that anyone has any problems with stopping random AIs when they're doing crimes (or more realistically the humans making them do that) - but if you're going to make the comparison to humans in good faith, it'd be a person standing behind you, punishing you when you say something offensive.
Career and business self-preservation in a social media neurotic world. It doesn't take much to trigger the outrage machine and cancel every future prospect you might have, especially in a very competitive field flush with other "clean" applicants.
Just look at the whole "AI racism" fustercluck for a small taste.
So of course if this is where the money and interest flows then the research follows.
Besides, it's a generally useful area anyway. The ability to tweak behavior even if not done for "safety" still seems pretty useful.
Seriously. The reason why we dont have mass killings everywhere is not the fact that information on how to make explosive drones or poisons is impossible to find or access. It's also not so hard to buy a car or knife.
Hell you can even find YouTube videos on how exactly uranium enrichment works step by step. Even though some content creators even got police raided for that. Yet we dont see tons of random kids making dirty bombs.
PS: Cody's Lab: Uranium Refining:
It's the same with plenty of other things.
That describes almost every web server.
To the extent that this particular maths produces text that causes political, financial, or legal harms to their interests, this kind of testing is just like any other accepting testing.
To the extent that the maths is "like a human", even in the vaguest and most general sense of "like", then it is also good to make sure that the human it's like isn't a sadistic psychopath — we don't know how far we are from "like" by any standard, because we don't know what we're doing, so this is playing it safe even if we're as far from this issue as cargo-cults were from functioning radios.
Are images included in the training?
What kind of SAE is being used? There have been some nice improvements in SAE architecture this last year, and it would be nice to know which one (if any) is provided.
No images in training - 3.3 70B is a text-only model so it wouldn't have made sense. We're exploring other modalities currently though.
SAE is a basic ReLU one. This might seem a little backwards, but I've been concerned by some of the high-frequency features in TopK and JumpReLU SAEs and the recent SAE (https://arxiv.org/abs/2407.14435, Figure 14), and the recent SAEBench results (https://www.neuronpedia.org/sae-bench/info) show quite a lot of feature absorption in more recent variants (though this could be confounded by a number of things). This isn't to say they're definitely bad - I think it's quite likely that TopK/JumpReLU are an improvement, but rather that we need to evaluate them in more detail before pushing them live. Overall I'm very optimistic about the potential for improvements in SAE variants, which we talk a bit about at the bottom of the post. We're going to be pushing SAE quality a ton now we have a stable platform to deploy them to.
after the idea that Claude 3.5 Sonnet used SAEs to improve its coding ability i'm not sure if i'm aware of any actual practical use of them yet beyond Golden Gate Claude (and Golden Gate Gemma (https://x.com/swyx/status/1818711762558198130)
has anyone tried out Anthropic's matching SAE API yet? wondering how it compares with Goodfire's and if there's any known practical use.
We have a notebook about that here: https://docs.goodfire.ai/notebooks/dynamicprompts
We also have an 'autosteer' feature that makes coming up with new variants easy: https://x.com/GoodfireAI/status/1871241902684831977 (this feels kind of like no-code finetuning).
Being able to read features out and train classifiers on them seems pretty useful - for instance we can read out features like 'the user is unhappy with the conversation', which you could then use for A/B testing your model rollouts (kind of like Google Analytics for your LLM). The big improvements here are (a) cost - the marginal cost of an SAE is low compared to frontier model annotations, (b) a consistent ontology across conversations, and (c) not having to specify that ontology in advance, but rather discover it from data.
These are just my guesses though - a large part of why we're excited about putting this out is that we don't have all the answers for how it can be most useful, but we're excited to support people finding out.
anyway i dont need you to have the answers right now. congrats on launching!
We’re social creatures, chatbots already act as friends and advisors for many people.
Seems like a pretty good vector for a social attack.
When the automobile was developed, we had to train kids not to play in the streets. We didn't put kids or cars in bubbles.
When photoshop came out, we developed a vernacular around edited images. "Photoshopped" became a verb.
We'll be able to survive this too. The more exposure we have, the better.
https://www.bloomberg.com/news/features/2022-06-10/how-citie...
We've had exposure to propaganda and disinformation for many decades, long before the internet became their primary medium, yet people don't learn to become immune to them. They're more effective now than they've ever been, and AI tools will only make them more so. Arguing that more exposure will somehow magically solve these problems is delusional at best, and dangerous at worst.
There are other key differences from past technologies:
- Most took years to decades to develop and gain mass adoption. This time is critical for society and governments to adapt to them. This adoption rate has been accelerating, but modern AI tech development is particularly fast. Governments can barely keep up to decide how this should be regulated, let alone people. When you consider that this tech is coming from companies that pioneered the "move fast and break things" mentality, in an industry drunk on greed and hubris, it should give everyone a cause for concern.
- AI has the potential to disrupt many industries, not just one. But further than that, it raises deep existential questions about our humanity, the value of human work, how our economic and education systems are structured, etc.
These are not problems we can solve overnight. Turning a blind eye to them and vouching for less regulations and more exposure is simply irresponsible.
