This is exactly (one of) our use cases at Eraser - taking code or natural language and producing diagram-as-code DSL.

As with other situations that want a custom DSL, our syntax has its own quirks and details, but is similar enough to e.g. Mermaid that we are able to produce valid syntax pretty easily.

What we've found harder is controlling for edge cases about how to build proper diagrams.

For more context: https://www.eraser.io/decision-node/on-building-with-ai

Agree that your use-case is different. The papers above are dealing mostly with adding a domain-specific textual corpus, still answering questions in prose.

"Teaching" the LLM an entirely new language (like a DSL) might actually need fine-tuning, but you can probably build a pretty decent first-cut of your system with n-shot prompts, then fine-tune to get the accuracy higher.