This past weekend, I embarked on a creative experiment. Fueled by curiosity and a desire to push the boundaries of what's possible with generative AI, I set out to create a short film using only AI tools. The result is The Cardboard Navigator, a whimsical, slightly surreal tale about finding magic in the mundane.
My Generative Toolkit
Creating The Cardboard Navigator involved a symphony of AI models and services, each playing a distinct role in bringing the story to life:
Google Gemini
This powerful multimodal model served as my creative brainstorming partner. Through a series of back-and-forth prompts and refinements, Gemini helped me shape the core narrative, develop the scene structure, and even craft the descriptions for the soundtrack.
To get started, you can head to AI Studio. Here are some examples of the prompts I used:
Plot & Scenes: “You are a world renowned movie producer, director, and writer… I want to come up with a short video. Can you help me brainstorm ideas for a something that will be about 45 seconds in length, and shot in 5 second shots.” → once I picked an idea I liked, I then refined it further through conversing with Gemini. Some scenes I had it rewrite, while others I proposed changes and then had Gemini help me work through how that impacted the rest of the film.
Title: “what's a good title for this movie?” → Gemini always gives me a list, and in the case of this movie, I actually had my friends and family help me vote on which title they liked best!
Soundtrack: “describe the specific soundtrack that would be perfect for this. Use 200 characters or less” → I love using Suno for music generation but I still have a hard time coming up with a prompt that describes exactly what I want - this is where Gemini shines. Not only does it help me identify the type of music that would be good based on the context (in this case, the movie plot and scenes), it also helps craft the perfect prompt that I then use in Suno.
Runway
This platform was my go-to tool for generating each scene. While the prompting process required quite a bit of iteration, Runway ultimately delivered the visually striking, dreamlike sequences that form the backbone of the film.
To get started, you can head to Runway. Here are some examples of the prompts I used:
Opening shot: “Backlit shot of a janitor mopping an office building hallway.”
Gemstone Dessert: “wide angle shot: a desert where the sand is made of shimmering gemstones. The camera slowly zooms out to reveal giant sand dunes that sparkle with gemstones.“
Holding Box: “50 mm lens: old man in his 60s, weathered face, wearing coveralls and holding a cardboard box in front of him. A faint glimmer in his eyes, but his face slowly goes from a smile to being worried and distressed. He is standing in an old office building basement.”
Suno
This AI music generator provided the orchestral score that underscores the film's most emotionally resonant moments. I was able to use different songs for different sections of the movie to best match the mood I wanted to evoke. Not only does Suno let me decide if I want to generate an instrumental song or one with lyrics, it also names the songs for me, and comes up with an image for each one - I find these little details absolutely delightful!
To get started, you can head to Suno. Here are the prompts I used:
Intro: “Ambient melancholic minimalistic”
Dream Sequence: “triumphant orchestral pop rhythmic”
Transition: “indie rock electronica edgy”
ElevenLabs
This platform brought both voice and sound effects to life. I used ElevenLabs to generate the narrator's voice, as well as subtle sound cues like gasps and twinkling noises.
To get started, you can head to ElevenLabs.
Google GenType
This AI-powered design tool allowed me to create the custom cardboard-and-duct-tape lettering that appears in the ending credits. After I generated the alphabet, I downloaded each letter, added it to a Google Slide, and animated them in. I then used QuickTime to screen record my screen in presentation mode so that I could capture the letters animating in.
To get started making your own custom alphabet, you can head to Google Labs - GenType - I used the following prompt for my alphabet: “cardboard box and duct tape.”
iMovie
While not technically an AI Tool, this is my trusty video editing software that proved invaluable for stitching together all the individual elements generated by these various AI tools.
Lessons Learned: From Frustration to Inspiration
Creating The Cardboard Navigator was a journey of discovery, filled with both triumphs and frustrations. Here are a few key takeaways:
Small Feature Requests Can Lead to Big Workarounds: While GenType was great for creating a custom alphabet, its limited functionality meant I had to download individual letters, import them into Google Slides, and manually arrange them to create the desired layout for the ending credits. It's a reminder that even small feature gaps can create significant friction in the creative process. While this makes sense for GenType's experimental nature, in a full-fledged product, such limitations could lead to unnecessary user frustration. (I did, of course, pass this feedback along to the GenType team!).
Video Generation - The Wild West of AI: Despite studying prompt structures and online tutorials, generating consistent, high-quality video with Runway proved to be the most challenging aspect of the project. It required countless iterations, abandoned plotlines, and even a few workarounds to overcome limitations in scene consistency, facial rendering, and overall control.
The Disconnected Nature of AI Models: One of the biggest opportunities for improvement I encountered was the lack of seamless integration between the various AI models. Each platform operated independently, which meant I had to manually bridge the gaps, ensuring the music, sound effects, and visuals aligned coherently. It highlighted the need for more interconnected AI ecosystems where models can communicate and collaborate to streamline the creative process and produce even better outcomes.
The High Cost of Creative Exploration: AI-powered tools offer incredible creative potential, but that potential comes at a cost. Between the numerous iterations required to generate satisfactory video outputs and the credits consumed for each generation, I ended up spending around $30 on Runway alone. While this is a relatively small price to pay for a creative experiment, it underscores the importance of thoughtful planning and efficient prompting when working with AI tools, especially if budget constraints are a concern.
Embracing Constraints, Pushing Boundaries
The limitations of current generative AI models can be frustrating, but they can also serve as a powerful source of creative inspiration. As I explored in my previous post on LinkedIn about "building a house around a tree," sometimes the most innovative solutions emerge from working within constraints, rather than trying to force our will onto the world
The experience of creating The Cardboard Navigator was a reminder that generative AI is still in its early stages. The tools are evolving rapidly, and the boundaries of what's possible are constantly expanding. There's a thrilling sense of discovery inherent in this process, as we learn to navigate the fuzzy rules, embrace the limitations, and push the boundaries of this exciting new creative frontier.
A Few Closing Thoughts:
The Power of Prompting: Mastering the art of prompting remains crucial in this current age of generative AI. It's a skill that requires both precision and creativity, as we learn to communicate our intentions to models in a way that elicits the desired outputs.
Practice Makes Perfect (or at Least Better): Experimenting with different AI tools and exploring their strengths and weaknesses is essential for unlocking their full creative potential. It's a process that requires practice, patience, and a willingness to learn from both successes and failures.
The Future of Generative Video: The future of generative video is brimming with possibilities. As models become more sophisticated, interconnected, and cost-effective, we can expect to see a surge in AI-powered video creation, opening up new avenues for storytelling, artistic expression, and even commercial applications.
Five years ago, the thought of making a short film in my spare time would have seemed absurd. And yet, here I am. Turns out, the future is full of surprises!