undefined | Better HN

0 pointsceejayoz14d ago0 comments

Yes, but as with spam/phishing/abuse prevention, too much information about what does and doesn't trigger things can be very useful to attackers. An explicit error is something you can feed into another AI to find jailbreaks.

I think it's a fundamentally impossible thing to fix, though. There's no 100% correct answer.

0 comments

1 comments · 1 top-level

mapontosevenths14d ago

I understand completely, and respect the tough spot they're in. They have a choice between human safety and cybersecurity/business needs here. I don't envy that position.

That said, this thing is in real production use with war fighters, doctors, and financial experts. Just YOLO'ing to a dumber model midway through a multi-step process and pretending everything is fine is not a real or defensible option. Someone is going to die, and its going to be the fault of whoever decided to make this the default rather than opt-in.

Personally, I couldn't live with myself.

j / k navigate · click thread line to collapse