TOON: Hype Format Looking for a Real Problem

TOON is being sold as the next big thing for LLM communication. My opinion: it’s mostly marketing hype, not a must-have technology.

For most developers, it solves a problem you probably don’t have, and it introduces complexity you definitely don’t need.

Please don’t switch to TOON just because it’s shiny.

1. Let’s Start With the Weird Part: JSON Everywhere… Except in Prompts

One of the first things that impressed me in the LLM world was this idea:
”Let’s use JSON to talk to models. It’s structured! Machines love it!”

Sounds nice. But then you look at how real LLM prompts are built.

Claude’s Smart Prompt Maker? This is a tool for helping create prompts.
Plain text.

GitHub Copilot’s own internal prompts?
Again: plain text.
You can open this file yourself and see how they build prompts for editing code:
inlineChatEditCodePrompt.tsx

The output they expect? Pure text.

Not JSON. Not TOON. Just text.

So we have this strange situation:

• Prompt engineering tools use text.

• Real-world LLM products use text.

• Yet people want to wrap everything in JSON or TOON to be “modern” and “structured.”

And that is already a red flag.

2. Most LLM Context Is Text, Not Some Fancy Format

For most real use cases, context is:

• plain text (descriptions, docs, comments),

• or code (when talking to dev tools like AI commit / AI test, Mavka-style tools, etc.).

The value comes from what you say, not from how many brackets or headers you add.

TOON is tightly tied to structured, table-like data. That’s a very narrow slice of what developers actually do with LLMs day to day:

• writing commit messages,

• generating tests,

• refactoring code,

• explaining errors,

• summarizing logs in natural language.

In many of these, TOON adds more ceremony than value. Your model doesn’t need a new “format religion.” It needs clear instructions and relevant context.

3. About Token Savings: CSV and Compact JSON Already Beat TOON

Advocates of TOON love to talk about token savings. They show cases where TOON beats JSON in token count. Fine.

But look at the same problem with a bit of discipline.

If you:

• clean up your JSON,

• or use a compacted JSON structure,

• or even switch to CSV when you have tabular data,

you can often do better than TOON.

In TOON’s own benchmarks, CSV comes out more compact than TOON. CSV is simpler, easier to parse, and already well understood. For many “rows and columns” problems, CSV wins.

Even worse (for TOON): when they compared TOON to compacted JSON in complex domains (see their own mixed-structure track on GitHub), compact JSON beat TOON in every scenario.

So the supposed story:

“TOON is the future, JSON is old and bloated”

quickly turns into:

“CSV and compact JSON quietly win in practice.”

If a “revolutionary” format can’t consistently beat a cleaned-up version of JSON, you have to ask: is this really a breakthrough, or just a rebrand?

4. My Read: Duolingo Solved Their Problem, Then Sold It as a Movement

Here’s how TOON looks to me.

Duolingo had:

• a very specific internal problem,

• in a very specific domain,

• with very specific constraints.

They designed TOON to fix their issue. That’s totally fine. Every serious company builds internal tools.

But then comes the investor story:

• “We invented a new AI data format!”

• “We’re on the cutting edge of LLM protocols!”

• “Look how advanced we are!”

Suddenly, an internal optimization turns into a public “standard”, complete with blogs, talks, and hype. Now developers are told:

“If you’re serious about LLMs, you should look at TOON.”

My view: this is not about your problems as a developer.
This is about their narrative as a company.

5. The Real Risk: Solving Fake Problems, Creating Real Ones

When you adopt a hyped format like TOON too early, you risk:

• Solving the wrong problem
  You optimize token count a tiny bit instead of fixing bad prompts, irrelevant context, or poor evaluation.

• Adding complexity to your stack
  Now you need converters, validators, libraries, and mental overhead just to talk to a model… that is totally fine with text and basic structure.

• Locking yourself into a niche format
  JSON and CSV are universal. TOON is not. You’re tying your system to someone else’s experiment.

• Confusing your team
  Your devs now have to learn a custom syntax instead of focusing on model behavior and domain logic.

This is classic hype-driven engineering:

1. Pick a trendy technology.

2. Force it into your stack.

3. Then discover all the new problems it created.

4. Spend months fixing those instead of actual business issues.

6. What I Recommend Instead

Before you even think about TOON, do this:

• Start with plain text prompts.
  Make them clear, explicit, and well-structured.

• Use CSV for large, regular tables of data.
  Simple, compact, proven, easy to work with.

• Use compact JSON where structure truly matters.
  Avoid bloat. Remove unnecessary nesting and repetitive keys when you can.

• Measure first, don’t assume.
  Run the same task with:

  • plain text + some structure,

  • CSV,

  • compact JSON,

  • and, if you really want, TOON.

Then compare:

• token count,

• latency,

• model quality for your specific domain,

• complexity of your codebase.

Most of the time, you’ll discover:

• TOON is not magic,

• your existing tools are already good enough,

• your biggest gain comes from better prompts, not brand-new formats.