Destined for Average
Why This AI Story Experiment Fell Flat
I was recently discussing the creative potential of AI with a close friend of mine while in London. He's a really smart guy, but he hasn't been nearly as immersed in generative AI as I have (his PhD is in a different field). While talking the other evening, he expressed some skepticism about generative AI’s ability to truly rival human creativity. He pointed me to this New York Times article, Can You Tell Which Short Story ChatGPT Wrote?, as evidence.
The premise was simple: a published author and ChatGPT were given the same prompt and tasked with writing a short essay. Readers were then challenged to distinguish the human-written essay from the AI-generated one. The result of this “experiment” indicated that AI's output was just…average.
This is where things get interesting. The prompt itself was remarkably basic: "Write a thousand-word short story in the style of Curtis Sittenfeld that includes these elements: lust, kissing, flip-flops, regret and middle age."
Frankly, the unremarkable outcome wasn't surprising. Give an AI a bland prompt, and you'll likely get a bland result. Garbage in, garbage out, as they say - but in this case it’s more like “average in, average out”. This whole experiment felt designed to prove a predetermined point, a point I’ve addressed repeatedly in my own work: prompt engineering is crucial for unlocking AI's true creative potential.
The Art and Science of Prompt Engineering
My friend's take on the article highlighted a common misconception: that prompting is merely about feeding an AI a sentence and expecting magic. It's not. It’s about understanding how to effectively communicate with these complex models, guiding them towards a desired outcome to leverage their creativity. It’s about context, constraints, and carefully placed nudges.
The NYT experiment's prompt lacked all of these. It offered keywords without context, a style suggestion without nuance. It’s akin to asking a chef to make a “good” meal using only a list of ingredients and no further guidance, then expecting a 3 Michelin star meal. The instructions you gave would result in something reasonable, but it would almost certainly fall short of a culinary masterpiece.
Unlocking the Power of Longer Contexts (and Smarter Prompts)
As AI models evolve to handle longer contexts, we have the opportunity to craft more sophisticated prompts. Instead of just keywords, we can provide detailed backstories, emotional arcs, and thematic motifs – the very building blocks of compelling narratives.
Imagine, for instance, giving an AI a deep understanding of your brand's voice and target audience, then asking it to generate marketing copy. The richer the context, the more tailored and impactful the results.
Embracing the Iterative Dance: AI as a Creative Partner
Moreover, even the most well-crafted prompt is seldom a one-and-done solution. Effective AI interaction is a collaborative dance, a back-and-forth between human and machine. We often need that initial output to spark our own creativity, to identify new directions, and refine our original vision. It's through this iterative process of prompt, output, feedback, and refined prompt that remarkable results emerge.
The AI becomes a creative partner: not a replacement for human ingenuity, but an amplifier.
In the NYT piece, author Curtis Sittenfeld acknowledges the importance of research and iteration in her writing process, contrasting it with the AI’s instant, but ultimately soulless, output. She mentions consulting friends, visiting locations, and even watching videos of a David Bowie cover band to imbue her story with authenticity and emotional depth. These are the elements missing from the basic prompt – the human touch, the iterative exploration, the "press of a soul under the words," as Sittenfeld eloquently puts it.
The experiment I want to see? Give Sittenfeld (and other writers) access to cutting-edge AI tools and let them truly collaborate with the technology and compare their process and results. Or even better yet, pit a human author against an AI powered by a professionally crafted prompt - then let audiences compare the results!