I'm using GPT and its functions, which are basically JSON schemas for its output. It makes the formatting a lot more stable, but even then it's still doing tokenized completion and the function definition is just another aspect of the prompt.

I've done some productive collaborating with someone who only works at the prompt level, but you can't really hand off in my experience. You can do some things with prompts, but pretty soon you are going to want to change the pipeline or rearrange how data is output. Sometimes you just won't get a good response that is formatted the way you want, and have to accept a different function output and write a bit of code to turn it into the representation you want.

Also the function definition (i.e., output schema) looks separate from the prompt, but you absolutely shouldn't treat it like that; every description in that schema matters, as do parameter names and order. You can't do prompt engineering without being able to change those things, but now you will find yourself mucking in the code again. (Though I make the descriptions overridable without changing code.)

Anyway, all that just to say that I agree that code and prompt can't be well separated, nor can pipeline and prompt.