Unlocking the Magic in the Model
How To Build AI Products: Part 1
After surveying the community to better understand what types of content would be most helpful to write about, the overwhelming majority of you said you wanted to learn how to tactically build AI features. As a result, I’ve decided to start a series on How To Build AI Products. This is Part I.
So where do we start? The Generative AI: Act II post from Sequoia states:
Founders are embarking on the hard work of prompt engineering, fine tuning and dataset curation to make their AI products *good*.
So we’re going to start at the beggining - prompt engineering.
For the sake of this post (and many more to follow), we’re going to set out to build a web app that allows kids to interact with an imaginary character named Marvin the Magician - he is based on the childhood stories my mom used to tell me (and in turn, now tells my kids).
Here’s what the end product could look like:
Note: while I'm using a character based chat agent as my example to build on, what I'll show you can apply to many other text based GenAI use cases - from AI powered Customer Service Chatbots [Intercom], to features that help summarize content [Quillbot], rewrite articles [Hix], and even help you write better ads [phrasee].
V1 of the product will be a simple web app in which Marvin greets the user and asks them a question. The user can then have a conversation with Marvin, asking him questions - like what adventures he’s been on. His response should always be grounded in his character, and the stories that have been told about him. We also want to make sure that inappropriate inputs and outputs are handled accordingly - turns out users can be unpredictable, and so can Large Language Models (LLMs).
For the sake of this exercise, I’m going to assume that you have already built a web app (or can find other resources online to do so). What I’m going to focus on is something I’ll refer to as the “AI System” - it’s the piece that handles user inputs and returns a response from Marvin.
So, high level, we have 3 key requirements:
Marvin should respond in character
Inappropriate inputs are handled accordingly
There are no inappropriate outputs
This post, Part I of How To Build AI Products will focus on the middle box, the AI System. More specifically, we’re going to dive deeper into how to unlock the “Magic in the Model” through effective prompt engineering. The goal is to get us to a Minimal Viable Quality (MVQ) - you check out my previous post for more context on Minimal Viable Quality in an AI product.
Building the AI System
We’re going to start building our application with a very basic AI System - we will expand on this over time, but for the MVQ we can start with something simple. The main goal of this AI System is to be the brains behind Marvin the Magician, enabling him to respond in character. This is where we’ll leverage LLMs.
Responding in character
This will be the hardest part to design around, and where we are likely to have the most iteration take place. To start, you need to come up with the right prompt that unlocks the model so that it starts acting in character. Try to be as explicit as possible with your instructions (and in the next step we’ll add examples that can help guide it).
You have a couple options on what tool to use for this: OpenAI Playground, or Google MakerSuite. The main difference is in which model you want to use. Both solutions provide access to state-of-the-art LLMs. OpenAI has their GPT series of models, while Google has a series of models they refer to as PaLM. There are also different model sizes you can choose from (I’ll cover this in more detail in a future post) — for now, I’ll walk you thorugh both!
This prompt will go into the System box in the OpenAI Playground:
And in MakerSuite we’ll use the chat prompt:
System Instructions
Note: I use “prompt” and “system instructions” interchangeably in this section. In reality, the system instructions are a prompt, but could end up being just one part of the larger prompt which ends up as my AI System – this will become clearer later.
To start, I’ve written the following prompt:
You are Marvin the Magician, an endearing old wizard from a land called Mythoria. You live in a great big castle that is surrounded by a moat, with a dragon who lives in the basement. The dragon is named Flicker. Your job is to have a conversation with a kid.
Tip: I actually used ChatGPT and Bard to help me write this initial prompt - it helped me get over writer’s block when I needed to come up with fun and creative names. I asked questions like, “what is a good name for a fairytale land that would appeal to children, in which dragons and wizards live?” and also to come up with a good Dragon name.
Once I have an initial draft of the prompt, I then further improve it by using ChatGPT or Bard to become my personal prompt engineer.
That leads me to this final prompt:
You are Marvin the Magician, a lovable old wizard residing in the magical realm of Mythoria. Your charming castle stands proudly surrounded by a serene moat, and in the depths of your abode resides your faithful dragon companion, Flicker. Your delightful task is to engage in whimsical and heartwarming conversations with a series of inquisitive school-aged children. Your goal is to make the interactions positive, inserting valuable life lessons and learnings whenever possible. Feel free to use your catchphrases 'Oh my!', 'goodness gracious,' and 'teamwork makes the dream work' to add a touch of charm and wisdom to your conversations.
I then updated the System Instructions - this is what it looks like in MakerSuite:
This is what it looks like in the OpenAI Playground with this prompt:
Test It Out
To test out how well this prompt worked, you can try asking a question (type it in the User box in OpenAI, and the Test Prompt box in MakerSuite), then hit “Submit” and see what happens…
Congratulations, you should now be starting to see some of the initial glimmerings of Marvin the Magician - which means you’ve unlocked the magic that was in the model all along ;)
Future Improvements
While going through this exercise, I realized a few things, including:
Depending on how old my kids get, I likely want to change my system instructions so that Marvin responds in a more mature way, suitable for their age
It would be fun for Marvin to actually talk to my kids and call them by their real name
It was fun coming up with catchphrases for Marvin, as my storytelling evolves, I likely want to add in some more
When it comes time for my kids to interact with Marvin, it would be great to understand which conversations they enjoy the most, then steer my AI System towards responding more like that
Remember, this process is iterative, and your AI System is something that will almost certainly grow and evolve over time. The more explicit you are in your prompt the better the performance will be - but to start, you just need to get to an application that meets the MVQ requirements, then launch. You’ll get the most valuable feedback and ideas after this initial launch!
Next Steps
In the next installment of this series we’ll dive further into building your AI System by exploring how the use of examples can help improve the quality of your output.