It would be nice if people understood the consequences of it, and I do find in my conversations with developers that people generally do not understand that by forwarding the agent, anyone with sufficient access on the remote can use the agent. (E.g., another user w/ sudo.)
Your agent _should_ always be where you are (ie not inside a container, a bastion host, or whatever else) because otherwise that means you aren't actually in possession of the key material and there's plenty of opportunity for much _worse_ surprises than with SSH agents if somebody else has the key material.
Because it's where you are, and you're probably not on a 1970s video terminal link but a laptop or something, the agent could just ask you to OK each request out of band, e.g. popping up a "Really log into machine X?" request. Once such a mechanism existed it could be refined (should it let you say "Yes always to requests from machine X" ? How about "Yes always for the next five minutes" ?) and if necessary SSH auth could even be tweaked to better support any real world behaviours that are popular (e.g. I don't recall off the top of my head if the agent can tell from what it's signing either where you're signing in, or where that sign-in was used, but the binding mechanism in SSH auth could certainly enforce either of those if they're determined to be important and don't exist today)
1) clearly display what "local" machine is making the connection, under what user
2) clearly display what remote machine is the local connecting, to what user
3) allow me to select a specific key for that connection pair, and only present one key to the remote
4) if the key is unlocked (or, gasp not passphrase-protected), then allow me to accept/deny the agent request
5) give me some mechanism to permanently disable the agent for my user if I decide I don't want to risk some software "accidentally" forwarding an agent (pebcak, bug, malice, whatever)
