Did I Win AI BINGO in 2024? Grading My Predictions from Last Year
Ringing in 2025 with a Recap of Last Year's AI Forecast - Hits, Misses, and Lessons Learned
Happy New Year everyone! To end last year, I shared my 2025 AI BINGO predictions with you all. To kick off the new year, I wanted to take a look back at my 2024 AI BINGO card predictions and see how they fared. As we embark on a new year, full of exciting possibilities and inevitable surprises, it's always fun to reflect on the past and learn from both our accurate forecasts and our missed guesses. So, without further ado, let's dive into the results!
I used a standard grading scale of A+ through F and then grouped the predictions into three categories:
Green (B+ or above, indicating confident hits)
Yellow (B- or B, suggesting promising progress but not complete fulfillment)
Red (C+ and below, for predictions that missed the mark)
Here’s where things landed:
The Verdict:
No perfect BINGO this year, but 2024 showed real progress in the predicted directions. No home runs in the green zone (B+ or above), but if you consider the "Yellow" predictions (grades of B- or B) as wins, that is two successful BINGOs!
Here’s a more detailed breakdown of how we landed at these scores…
#1 Small Models Become a BIG Deal
Grade: A-
Stamped: Green
Status: Small but mighty, these models have proven their worth!
Analysis: There's strong evidence that small models have gained significant traction in 2024 [source 1, source 2, source 3]. The trend toward smaller, more efficient AI models for specific tasks, especially on edge devices, has really taken off. They are being seen as more practical for real-world applications with limited resources, not to mention that they tend to be more cost-effective and fine-tuned for specific tasks and domains, indicating a shift from only focusing on large models.
#2 On-Device AI Models in All High-End Hardware
Grade: B
Stamped: Yellow
Status: Integrated, but not ubiquitous. On-device AI is making its way in.
Analysis: While many high-end devices now incorporate AI chips, it's not yet in "all" of them. There has been a strong push towards on-device AI to improve speed and privacy with some examples being new smartphones and computers. However, it's an ongoing trend and not fully universal.
#3 AI Auto-Writers in All Internet Textboxes
Grade: C+
Stamped: No
Status: AI help is there, but still not everywhere.
Analysis: AI auto-writers are becoming more common in online platforms, including text boxes, especially those related to email, and some online documents. However, it's far from a standard in all internet textboxes. There are a lot of options for AI writing tools, but they aren't yet built into every text input field.
#4 Chief AI Officer Becomes a Common Role in Large Companies
Grade: B
Stamped: Yellow
Status: The C-suite is getting AI-savvy, but it's not a full house yet.
Analysis: The role of Chief AI Officer is definitely becoming more common, especially in larger companies [Gartner reports that More Than 50% of Organizations Have an AI Leader, and that was as of June 2024]. There's increasing recognition of the need for dedicated leadership in AI strategy, but it's not a completely universal position yet and it's still evolving.
#5 This Will Become the Year of “Agents”
Grade: B-
Stamped: Yellow
Status: Agents have entered the chat, and they’re here to stay.
Analysis: AI agents have become a major area of focus, especially towards the end of 2024. The development of autonomous agents capable of completing tasks independently has seen a lot of progress with many companies researching and building them. Between Claude’s Computer Use, and Google DeepMind’s Project Mariner (which I helped launch in December!) It seems like this prediction was spot on to describe the end of this year, but I think the real game changers in this space will come in 2025 (OpenAI is already rumored to enter this space early in 2025) - so I’m going to only give this one a partial pass.
#6 Elon Declares AI Sentient
Grade: F
Stamped: No
Status: Still waiting for the tweet heard 'round the world on this one. Silence on the sentience front (officially, at least).
Analysis: While Elon Musk has been known to openly voice his opinion in this space - predicting that AI will surpass human intelligence by 2026, and urging lawmakers to regulate AI before we have robots “walking down the street and killing people” (a comment that dates back to 2017)…There is no evidence of Elon Musk declaring AI sentient in 2024.
#7 Monetary / Environmental Cost of AI Gains Attention
Grade: A
Stamped: Green
Status: The bill is coming due, and people are definitely starting to ask about the energy consumption and the financial investment.
Analysis: The monetary and environmental costs of AI have definitely gained significant attention in 2024. There are many discussions and concerns about the energy consumption and resource intensiveness of large AI models, which makes this prediction accurate.
#8 Focus Moves from “Prediction” to “Reasoning”
Grade: A
Stamped: Green
Status: Reasoning is creeping in, taking center stage to end 2024.
Analysis: The end of 2024 saw some big launches in this space. In September, OpenAI announced o1-preview, then followed it up in December with the announcement of o3-preview. December also saw some big announcements from Google with the launch of Deep Researcher in the Gemini App, as well as a new Gemini 2.0 Flash Thinking Model.
#9 We Will See an Influx of AI-First Hardware Devices
Grade: B-
Stamped: Yellow
Status: AI-powered devices are pushing onto the shelves, but the floodgates aren't fully open.
Analysis: New gadgets with AI in their DNA started popping up in the beginning of the year (Poem/1, Ray-Ban Meta Glasses, Halo Headband, Brilliant frame AI glasses and the XREAL Air 2 Ultra AR Glasses), indicating a definite trend, but then we saw a lot of set-backs with overpromises and lacklustre results (major flops like the Humane AI Pin and Rabbit R1). Interestingly, TED AI this year had a big focus on robots (a theme popping up more and more) - hosting the first ever Robot Festival. And then there’s Tesla Optimus which continues to see improvements, but Elon states won’t start being deployed at scale until 2026.
The hype here is definitely growing, but I may have perhaps jumped the gun a little on this one. Nevertheless, I think this area will continue trending up in 2025!
#10 AI Blamed for Causing Election Upheaval
Grade: C
Stamped: No
Status: Despite the AI-fueled fire, things played out ok
Analysis: There was certainly an abundance of speculation and concern about AI being used to influence elections this year (from deepfakes to misinformation) - yet despite these worries (or maybe because of them), it seems like AI managed to avoid causing a massive upheaval. I’ve never been so happy to have been wrong on a prediction before.
#11 10+ Unicorn Companies Building SOTA Open Source
Grade: A+
Stamped: Green
Status: The open source unicorn stampede is underway
Analysis:
The open-source AI movement is gaining serious traction - here’s a list of some of them (which takes us over 10):
Google building Gemma
Meta building Llama
NVIDIA building Nematron
Cohere building Command R+
Alibaba building Qwen2
01.AI building Yi
Mistral building Mixtral
Stability building Stable Diffusion
Databricks building DBRX
Anyscale provides a managed Ray platform, but their core technology is open source.
Weights & Biases has built a core product around open standards and is heavily used by the open-source AI community
Scale AI have open-sourced datasets and tools and are known for making meaningful contributions for open-source AI
#12 A New Eval Standard Replaces Academic Benchmarks
Grade: B+
Stamped: Green
Status: The old guard is still holding court, though Chatbot Arena has entered the scene.
Analysis: While “Academic” benchmarks are still used, Chatbot Arena (formerly LMSys) has definitely taken the AI world by storm - becoming the de facto way new model launches are evaluated and talked about publicly. It’s also worth mentioning that new benchmarks and eval sets continue to emerge - take the recent launches of Claude’s Computer Use and Google DeepMind’s Project Mariner, both of which reference the WebVoyager dataset; or the newly released Facts Leaderboard. Another growing trend in this space is using auto-rater models to evaluate, instead of human raters. When it comes to eval standards, the space is still evolving, but I’m giving this a pass at a B+ as the traditional Academic Benchmarks are no longer the one and only gold standard.
#13 A Currently Hyped Up AI Company Goes Bankrupt
Grade: B-
Stamped: Yellow
Status: While there's been some turbulence in the market, the major players are still afloat, though there have been some choppy waters.
Analysis: While there have been some AI companies facing difficulties (we were tracking Stability and Inflection in our midyear check in), there has not been an actual bankruptcy of a significantly hyped-up AI company. Inflection appears to have pivoted its initial business plan, while Stability is continuing to launch. That said, AI is expensive, and even the most popular companies, like OpenAI, continue to make headlines for being on the brink of bankruptcy at times (we call this a success disaster). Another notable headline this year was Google’s acquihire of the Character.AI founders - although even that company continues to keep running.
#14 Synthetic Data Gains in Popularity
Grade: A-
Stamped: Green
Status: Synthetic data is making its entrance as a must-have in the AI world!
Analysis: Synthetic data has gained significant popularity in AI, especially in areas where real data is scarce or hard to obtain. Other benefits include privacy protection, data augmentation, cost reduction, and bias mitigation. This prediction is correct, and synthetic data is becoming more and more essential in AI and trends predict it to keep growing.
#15 A Full-Length Movie Will Be Almost Entirely AI Generated
Grade: C-
Stamped: No
Status: AI has its eyes on the silver screen, but it's not quite ready for its close-up.
Analysis: While there are AI tools that can be used in the film production process, there has been no entirely AI generated full-length movie released in 2024. There have been AI movie trailers and shorter works, but not a full length movie, making this prediction not yet accurate.
#16 Demand Will Keep Growing for Prompt Engineers
Grade: C+
Stamped: No
Status: The whisperers of the AI world are still diamonds hiding in the mines.
Analysis: The demand for prompt engineers hasn’t taken off as much as I would have expected. For those in the know, the value in this skill set has become quite apparent - but this insight hasn’t seemed to have made its way into mainstream knowledge just yet. There is an ongoing need for people who can effectively communicate with AI models through prompting, but given how new this entire field is, that realization hasn’t yet made its way into job postings.
#17 Focus Moves from Text & Image to Video & Audio
Grade: B+
Stamped: Green
Status: Video and audio are stealing the spotlight, but text and image are still in the mix.
Analysis: There is definitely a strong shift towards video and audio generation and processing, however, the compute required for these has made them slower to roll out to mainstream audiences than text and image (which continue to have considerable focus). That said, the announcement of Sora in February, followed by its broader release in December; along with Veo 2 in December; and Suno in March, definitely bring focus to video and audio generation.
#18 An AI-Written Piece Will Win a Pulitzer Prize
Grade: F
Stamped: No
Status: The Pulitzer committee is keeping the human touch for now.
Analysis: No AI-written piece has won a Pulitzer Prize in 2024. If anything, the benefits of having a human-in-the-loop for creative works is the theme that shined the brightest - Ben Affleck’s post continues to resonate.
#19 Ilya Leaves OpenAI
Grade: A+
Stamped: Green
Status: Ilya has indeed left the building!
Analysis: Ilya Sutskever did indeed leave OpenAI in 2024, making this prediction accurate.
#20 a16z Raises a $5B+ Fund to Invest in Generative AI
Grade: A
Stamped: Green
Status: a16z has put its money where its mouth is, and it's betting big on AI.
Analysis: Andreessen Horowitz (a16z) has raised a $7.2 billion fund to invest in a variety of things, and with one clear priority - AI. This makes the prediction accurate.
#21 An AI Lawsuit Makes its Way to the Supreme Court
Grade: F
Stamped: No
Status: The legal battles are brewing, but the highest court hasn't taken center stage on this topic yet.
Analysis: There has not been any AI lawsuit that has made its way to the Supreme Court in 2024, so this one is a miss.
#22 AIs Calling Other AIs to Complete Tasks
Grade: B
Stamped: Yellow
Status: The robots are networking! AI agents collaborating is becoming a more tangible reality.
Analysis: AI agents using other AI agents to complete tasks has been an active area of development in 2024. From complex AI systems that can also use tools, to Mariner using Gemini to help complete a task. I’m giving this a B+ because it still feels like early days in this area, but this is a trend that I predict will continue to make advancements.
#23 An Alternative to the Transformer Architecture Gains Traction
Grade: C
Stamped: No
Status: While there's always research and exploration, the transformer still reigns supreme for the time being. Challengers are emerging, but haven't dethroned the king.
Analysis: While there are some new architectures being researched, transformers are still the dominant architecture for AI models. There hasn't been a large-scale alternative to the transformer that has gained significant traction, making this prediction only partially accurate.
#24 Nvidia Becomes a Full-Scale Cloud Provider
Grade: B
Stamped: Yellow
Status: They're not just selling the shovels anymore, they're building the gold mines too! Nvidia's cloud presence is definitely expanding.
Analysis: Nvidia has indeed expanded its cloud presence through their DGX offering. It's not a full-scale cloud provider to the extent of an Amazon or Google, but is definitely making strides in that direction, with services targeted at AI developers.
#25 A Major Insurer Will Offer an AI-Specific Hallucination Policy
Grade: F
Stamped: No
Status: Insurance companies aren't ready to cover the AI dream world just yet.
Analysis: There has been no evidence of any major insurer offering an AI-specific hallucination policy in 2024, making this another miss.
Footnote: To ensure a thorough and accurate assessment, I enlisted the help of several AI tools for grading my predictions. This included Gemini's Deep Researcher functionality, the Gemini Flash Thinking model in AI Studio, the Gemini Flash Experimental model with Search Grounding in AI Studio, and AI Overviews in search. While AI provided valuable assistance, the grading process still required several hours of careful review and analysis.