Programming with Code and Prompts
There are several ways to design AI with software development. It's often discussed as an aid to development, for generating code. We also sometimes talk about agents that are themselves systems with code and prompts, but they often contain more logic in the prompts than in the agent's code itself. No, I want to talk here about the integration of AI into the code of a larger system where the code represents a large part of the program's logic. This is the kind of system I've been building in recent months, so I decided to write about it to try and clarify my practice.
Is a Prompt Code?
When prompt execution happens within a program's code, there are several ways to look at things. LLMs execute a kind of code and return text as output (or another type of media: image, video, audio...).
We can view the prompt we write for an LLM as lines of code written with a programming language. Even if, in this case, there's no strict syntax or compilation. Nevertheless, it seems the analogy goes further. The whole prompt + LLM that infers the result is like a function call. The prompt is the input, the LLM is the function that executes the code and returns an output. A black box function. So of course, to interact with other parts of the code, it's good to request a structured output format in the prompt, such as JSON, YAML, XML, or even code in a programming language.
There are many reasons to call on AI, but it rarely comes down to asking for simple unstructured output. If your application or idea is already structured, you want to adopt this convention for the output expected from the LLM.
In a prompt, we describe both the function's instructions (more or less precisely), the expected output format, sometimes with a few examples, and of course this function takes parameters, so we pass existing system data into the prompt.