For years, streaming MTG meant the streamer had to constantly name or show every card so viewers could follow along.

That changed with interactive Twitch overlays that let viewers see card details themselves.

Here's how one project, Cardboard Live, kicked off MTG streaming's tech leap, and how it matured.

How Cardboard Live Worked

Cardboard Live began as a hackathon project in early 2019 with a pretty ambitious goal: make physical Magic streams just as interactive for viewers as digital games.

The team (led by MTG enthusiasts Sławomir Pruchnik, Wilson Hunter, and James Hsu) built a Twitch extension powered by computer vision that could see the cards on a playmat and let viewers click on them in the stream video.

On the backend, it used an image recognition service written in Python (with OpenCV and FFmpeg as the foundation) that constantly analyzed video frames for card-sized rectangles and then tried to match those to known card images.

As such, it was like magic about Magic.

In internal tests, it reportedly recognized about 96% of the cards correctly out of roughly 4,000 instances, which showed how feasible this approach was even for fast gameplay.

From Hackathon Demos to Real World Usage

Back in 2016, a Reddit user going by Adaptive_Automaton created a Chrome extension called Ocular Automaton that let viewers click on cards in a Magic stream to identify them.

By 2019, Cardboard Live took the concept to a whole new level and into the spotlight of official events.

After polishing the extension, the Cardboard Live team partnered with Wizards of the Coast to deploy it at high-profile tournaments. It premiered for Paper Magic at MagicFest Los Angeles 2019, and the feedback from the community was overwhelmingly positive.

Just a few weeks later, Cardboard Live was featured on the biggest stage yet, the $1 million Mythic Invitational at PAX East 2019.

Come 2020, the COVID-19 pandemic unexpectedly accelerated the need for good card recognition tech, but in a different context. With in-person play shut down, the Magic community flocked to webcam-based play from home.

A small team of developers led by Jonathan Rowny built a web application called SpellTable to make playing paper Magic over video calls easier.

Under the hood, SpellTable used its own image-recognition database to recognize cards from the live video, essentially the same idea as Cardboard Live, but built into a dedicated remote-play platform.

The response was huge, and Wizards of the Coast acquired SpellTable in 2020 to integrate it into their official offerings.

Meanwhile, on the Twitch side of things, another major evolution was happening.

In mid-2021, the stats and deck-tracking site Untapped.gg released their own Twitch extension, and it quickly became the overlay of choice for Arena players.

The Untapped.gg extension took inspiration from Cardboard Live's interactivity but applied it to the digital realm in a very polished way.

Since MTG Arena is a digital game, Untapped's approach didn't need any camera vision. Instead, it connected to the game data via the Untapped.gg companion app on the streamer's PC, to pull all the relevant info and display it to viewers.

Untapped.gg basically threw in all the bells and whistles that made streams highly interactive and informative without the streamer needing to do anything extra. Unsurprisingly, it became the most popular Twitch extension for MTG Arena streamers very quickly, especially as Cardboard Live's own Arena extension started falling out of maintenance around that time.

Tech Choices That Shaped MTG Streaming

Cardboard Live's solution was computer vision, perfect for paper Magic and even digital streams where direct game data wasn't available.

On the other hand, Untapped.gg's idea was deep integration with game data, essentially pulling the answers straight from MTG Arena's logs, which bypasses the need for any visual recognition.

Each approach has its pros and cons.

Cardboard Live's tech can work for any card game in a video (not just Arena, but a physical table or even other games entirely), and it doesn't require the streamer to run additional software aside from the Twitch extension.

However, image recognition will never be 100% perfect, i.e., lighting, card art changes, or new card frames can throw it off.

Meanwhile, Untapped's data-driven approach is extremely accurate, but it's limited to MTG Arena.

It also requires streamers to run the Untapped companion app and viewers to be watching on an interface that supports extensions.

So, what did Untapped.gg's extension improve that helped it overshadow Cardboard Live?

The big thing was user experience.

It's simply a smoother ride when you tap directly into game data, no occasional misreads or missing card images, as sometimes happened with Cardboard Live's camera-based approach.

Untapped's extension also packed in those extra features like zone exploration and draft pick overlays, which went beyond Cardboard Live's initial scope.

Another factor is that Untapped, as a platform, was already widely used for stats tracking, whereas Cardboard Live was a more niche startup project.

AI Image Generation Becomes a Part of the Broadcast Toolbox?

Generative-art platforms will probably also complement recognition overlays in many MTG streams. Tools such as MidJourney and MagicDiffusion model let creators whip up proxy illustrations in a matter of minutes, be it full-art lands, bespoke tokens, or placeholder images for unreleased cards. Custom character creation is also important; platforms like Candy AI showcase what can be done in that regard (mostly in relation to girlfriend AI topics), and crossovers that combine the art from the first two allow players to get some really unique-looking play pieces.

Streamers could then drop these assets into OBS so that the live card tracking software can identify a brand-new preview, where viewers will then see a polished graphic instead of a blank rectangle.

The same images often serve as play-test proxies offline; players print them, slip them into sleeves, and let SpellTable declare the art as a stand-in until the real card is available. In this way, AI image generation plugs the gaps in official databases and keeps both the tabletop and the broadcast overlay visually consistent.

Future in Focus

We're likely to see further refinement in accuracy for camera-based recognition.

Perhaps we'll see the tech use AI and machine learning to improve identification of cards even with glare or at weird angles.

There's also the exciting possibility of Augmented Reality, such as physical MTG tournaments where spectators with an AR headset or a mobile app can point at the table and get live card labels and stats floating above the battlefield.

For online platforms, developers are already extending these overlay concepts to other games (Untapped.gg has expanded into Marvel Snap and other digital card games with similar Twitch extensions).

The success of Magic's overlays has proved that viewers love being in control of the information they see.

Therefore, it's not hard to envision future streams where viewers can customize their experience even more—maybe choose which player's hand to focus on, or get live probability readouts and advanced metrics as part of the overlay.