1
I rarely have a problem with all of the requirements laid out and just the implementation is missing.
Are there any LLM coding benchmarks that have a human in the loop? That would be more helpful for me. Maybe with a large subset enough of humans you can take the average without the human performance being the main differentiator.
But there are some interesting challenges that I feel can't be solved unless inference providers allow us to include the concept additional entities in a conversation.
As far as I know the three most basic ones shared alongside all providers are:
- System
- Assistant
- User
That's fine and it allows for simple conversational-based approaches (ChatGPT, Claude, Gemini, etc). However in our use case we allow our customers (not the final user who is talking with the AI) to configure the AI in different ways (personality, RAG, etc), which poses a problem.
If we inject those customer settings in the System prompt then that's a risk because there might be conflicting prompts with our internal rules. So the easiest option is to "clean" the customer prompts before injecting them, but that feels hacky and just adds one more level of indirection. Cleaning the prompt and injecting it with common patterns like XML tags seems to help a bit but still feels extremely risky for some reason.
Injecting it in the assistant or user also seems flaky and prone to prompt injection.
Creating a fake tool call and result like "getPersonalityConfiguration" seems to work the best, from our testing it is treated as something between the System and Assistant roles. And our top system prompt rules are still respected while allowing the customer some freedom to configure the AI.
The problem comes when you need to add more parties to what essentially is a 2 entity conversation. Sometimes we want external agents to chime in a conversation (via subagents or other methods) and there is no good way to do that AFAIK. It gets the occasional confusion and starts mixing up who is who.
One of our typical scenarios that we need to model:
System: Your rules are: You will never use foul language...
Store owner: You are John the customer agent for store Foo...
User: Do you have snowboards in stock?
Assistant->User: Let me check with the team. I'll get back to you soon.
System->Team: User is asking if we have snowboards in stock. Do we?
Team: We do have snowboards in stock.
Team->User: Yes we do have snowboards in stock!
User: Perfect, if I buy them will the courier send it to my country? [country name].
Assistant->User: Let me check, I need to see if our courier can ship a snowboard to your country.
Assistant->Third party logistics: I have a user from [country] interested in buying a snowboard. The dimensions are X by Y and the weight is Z. We would send it from our logistics center located at [address].
Third party logistics -> Assistant: Yes we can do it, it will be 29.99 for the shipping.
Assistant->User: Yes they can ship it to [country] but it does incur in 29.99 extra charge...
I obviated tool calls and responses, but that's basically the gist of it. Spawning sub-agents that have the context of the main conversation works but at some point it is limiting (we need to copy all personality traits and relevant information via summarization or injecting the conversation in a manner that the sub-agent won't get confused). It feels like an anti-pattern and trying to fight the intended use case of LLM's, which seems to be focused in conversation between two entities with the occasional external information going in through System or tool calling.
It would be amazing if we could add custom roles to model messages, still with special cases like agent or assistant.
Has anyone worked with similar problems? How did you solve it? Is this solved in the model lab or at the inference provider level (post-training)?