"What happens if someone tries to extract training data?" CC6.7 covers data loss and data transfer restrictions. I've typically included controls related to monitoring data transfer, including flagging and highlighting potential breaches. Documented procedures on what happens if data loss or unauthorized data transfer occurs. These can be reviewed, but may be hard for the auditor to test unless they were executed and there's evidence that they were executed as written.
"Can this agent be manipulated into accessing data it shouldn't? How do you test for adversarial attacks?" I'm struggling to understand the difference between these questions. It seems like part of the answer likely overlaps with controls to address CC6.7 and data loss or data transfer restrictions. CC8.1 discusses testing the product or service.
"How do you prevent prompt injection?" This may be a bit specific for a SOC 2 Type 2 report, since it really gets into requirements, architecture, and design decisions rather than controls over the requirements, architecture, and design. That is, you can essentially not require preventing prompt injection and follow all of your controls related to, for example, CC8.1. CC8.1 talks about managing, authorizing, executing, and documenting changes. You can do all of these things well without that requirement in place.
"What guardrails are in place, and have they been validated?" This is the entire SOC 2 Type 2 report. It lists all evaluated criteria, describes the organization's controls, and provides an audit of those controls. It's up to the organization being audited, however, to think about what controls are necessary for their context. The controls that should be in scope of the audit will differ for an AI product or service than for something else. The recipient of the SOC 2 report can review the controls and ask questions.
Part of the burden is on the organization getting the SOC 2 audit report to think about what controls they need. But there's also a burden on the organization reviewing the audit report not just to see that there are no exceptions, but to review the controls described to make sure the controls are in place for the given product or service. And this detailed information about the controls is what makes something like the SOC 2 audit report a whole lot more useful than something like an ISO 27001 certificate, which says that whatever policies and procedures are in place meet the requirements of the standard and doesn't offer details on how those requirements are met.
