Insights

AI in May 2024: The Multimodal Race Goes Real-Time

May 2024 in AI: GPT-4o, Google's Gemini era, Copilot+ PCs, the EU AI Act, and OpenAI's safety shakeup — what each means for builders.

May 2024 was the month AI stopped feeling like a chat window and started feeling like an interface. Within nine days, OpenAI and Google both demoed assistants that see, hear, and respond in real time; Microsoft tried to bake the same idea into the operating system; and the EU finalized the rulebook that will govern all of it. It was also the month OpenAI’s most prominent safety researchers walked out. For operators, the signal isn’t “AI got better” — it’s that the surface area where AI touches your product just expanded, and so did the obligations that come with it.

GPT-4o made multimodal the default

On May 13, OpenAI launched GPT-4o (“o” for omni), a single model that handles text, audio, image, and video input and responds with near-conversational latency. The headline wasn’t raw intelligence — it was roughly GPT-4-level — but speed, voice, and price. OpenAI made it free to all ChatGPT users and cut API costs sharply versus GPT-4 Turbo.

Why it matters for builders: the economics changed more than the IQ did. A faster, cheaper, multimodal model makes a class of features viable that were previously too slow or too expensive to ship — live voice support, image-driven workflows, real-time assistants. If you shelved an idea last year on cost, it’s worth re-pricing now.

Google declared the “Gemini era”

A day later at Google I/O (May 14), Google reframed its entire product line around Gemini, said the word 121 times, and shipped substance alongside the slogan: Gemini 1.5 Flash in public preview with a 1-million-token context window, plus Project Astra, a real-time multimodal agent demo aimed squarely at OpenAI’s launch the day before.

Why it matters for builders: 1.5 Flash is the quietly important release. A cheap, fast model with a massive context window changes how you architect retrieval — you can stuff more context in directly and lean less on brittle RAG pipelines for some workloads. Don’t over-engineer chunking before you’ve tested whether a long-context model just solves it.

Microsoft tried to make Windows an AI OS

On May 20, Microsoft introduced Copilot+ PCs at Build — a new hardware category with on-device NPUs and AI features like Recall, which makes everything you’ve seen on your machine searchable. Recall drew immediate, fierce privacy backlash and was walked back before broad release.

Why it matters for operators: this is the on-device AI bet made concrete, and it’s a preview of the trade-off you’ll keep facing — capability versus the trust cost of capturing everything. Recall is a useful cautionary tale: shipping a powerful feature without a defensible privacy story is a liability, not a moat.

The EU AI Act became law

On May 21, the Council of the EU gave final approval to the AI Act, the world’s first comprehensive AI law. It takes a risk-based approach — the higher the potential for harm, the heavier the obligations — and entered into force later in 2024 with staged compliance deadlines.

Why it matters for builders: if you sell into Europe or process EU users’ data, this is now a roadmap input, not a future hypothetical. The practical move is to know which risk tier your use cases fall into before you scale them, not after. Most teams won’t be in the high-risk bucket — but knowing that with confidence is the point.

OpenAI’s safety exodus

The same week, OpenAI dissolved its Superalignment team after co-leads Ilya Sutskever and Jan Leike resigned (Sutskever announced his departure May 14). Leike’s parting note — that “safety culture and processes have taken a backseat to shiny products” — landed hard right after the GPT-4o launch.

Why it matters for operators: vendor governance is now a real procurement variable. When you build on a frontier model, you’re inheriting that vendor’s stability, incentives, and risk posture. Diversify your model dependencies where it’s cheap to do so, and don’t assume the lab building your foundation is a settled institution.

The throughline for May: AI got faster, cheaper, more embedded, and more regulated — all at once. The teams that win this stretch aren’t the ones chasing every launch. They’re the ones with clear judgment about which of these shifts actually changes their roadmap. If you’d like a second opinion on where AI genuinely fits yours, let’s talk.