The Art (& Science) of finding the most important AI Use Cases - Part 3
Google’s "Find special character" in Google Docs (Part 3)
In an earlier post, we talked about how teams can go about spotting the right AI use cases. Then, in part two, we walked through our first case study. Today, we're jumping into the second one — a great example that really shows the art of finding the right AI opportunities and reminds us why Google continues to lead the way in innovation.
Problem Statement:
Users of Google Docs and Slides sometimes need to insert special characters — things like α, β, Γ, ↔, ▶, ◑, ☁, ☎, € and many more. But with nearly 2,000 special characters to choose from, finding the one they need can quickly become difficult, tedious, and frustrating.
Design a solution to help the end users in such situations - i.e. the end users can very quickly get to/find ‘the’ special character(s) they need. The end user needs ‘that’ one special character only and often cannot do with some other ‘similar’ special character in its place.
Put yourself in the user’s shoes and think: How would you design a simple, effective solution? Once you have your ideas, share them in the comments below.
Attempt 1: Widget Approach
The first idea that comes to mind is to treat special characters the same way we treat lines, shapes, and other elements — by showing a widget that displays all the special characters.
However, there's a big catch: unlike lines and shapes, the number of special characters is huge. This leaves us with two options:
Either create an enormous widget that tries to show everything at once (which isn't practical), or
Force users to scroll endlessly inside a small widget.
Neither option is ideal. Trying to spot the right character inside a cramped, scroll-heavy widget leads to a frustrating, clunky user experience — exactly what we want to avoid.
Attempt 2: Widget for the Select few
One might then think of a smaller widget — one that shows just 5, 10, or 15 special characters, featuring the most frequently or recently used ones. It would definitely make access quicker for common characters.
But here's the problem: what if the user needs a character that's not in that limited list? which often the case! They’d still have to go through the painful scroll process — and the frustration we were trying to avoid comes right back.
Attempt 3: Search Bar
If you need something specific, your instinct is to search for it — and for that, you need a search bar! To make special characters more accessible, one obvious suggestion is: add a search bar where users can simply type the name of the character they're looking for, and we show the exact or similar ones.
Sounds simple, right? But here's the catch: traditional search doesn't work well here because most users don't know the official names of the special characters they need.
Not convinced? Try recalling the names of just 20 special characters off the top of your head! It's surprisingly hard — recalling the names of ~2,000 special characters? impossible!
But there might be a way out — we often search even without knowing the exact name.
Think about it: if you're trying to find Robert De Niro but can't recall his name, what do you do? You describe him — "highly acclaimed American male actor from the movie 'Taxi Driver'; a method actor, a legendary performer, often plays 'loner' roles."
And guess what? Search engines are smart enough to figure out who you're talking about. The same idea can work here: instead of forcing users to know the exact name of a special character, we can let them describe it in plain language — and have the system intelligently figure out what they mean.
What if the user simply describes the special character they need in words? Turns out, that's even harder! Try this: write a textual description for α (alpha) without using its name. It's tricky, right? You might say something vague like "a fish-like symbol" or "a rounded 'a' with a tail," but it’s hardly precise — and different users would describe it very very differently.
This makes traditional "text-based search" unfit. It’s not just about adding a search bar — the challenge is much deeper.
Light at the end of the Tunnel
"The biggest breakthroughs come not from new tools, but from new ways of seeing the problem."
Here’s the interesting part: users can easily visually describe the special character they need. What do we mean by "visual description"? Well, rather than trying to find the right words, users can simply draw the character they’re looking for.
It turns out that most people can very easily sketch a symbol rather than describe it with text. And this simple but powerful observation opens up a whole new possibility!
This approach would make it much easier for users to find the exact special character they need, without needing to recall its name or describe it in words. All they have to do is sketch it!
To leverage this idea, we need an AI algorithm that can perform "visual" matching based on the user's drawings. To achieve this, Google designed a new & special algorithm to make this work — sketch-RNN [3].
Sketch-RNN is a neural network that understands stroke-based drawings of common objects, and it works really well for recognizing the shapes we draw. Google actually used this technology to build a solution where users can draw the special character they need, and the AI identifies it in real-time.
Getting to this solution wasn’t easy. To train the above network, they needed massive data. For this David Ha and Douglas Eck from Google created QuickDraw [6], a dataset of vector drawings obtained from Quick Draw [5], an online game where the players are asked to draw objects belonging to a particular object class in less than 20 seconds. QuickDraw dataset consists of a collection of 50 million drawings across 345 classes of common objects. Each class of QuickDraw is a dataset of 70K training samples, in addition to 2500 samples in validation & test sets.
The team [6] very innovatively created a secondary system (a gaming system) to collect the precise data they needed. More on this aspect in posts on data collection.
References
https://googlesystem.blogspot.com/2014/08/special-characters-search-engine-in.html
https://experiments.withgoogle.com/sketch-rnn-demo
https://github.com/magenta/magenta/tree/main/magenta/models/sketch_rnn
David Ha and Douglas Eck, "A Neural Representation of Sketch Drawings", 2017. https://arxiv.org/abs/1704.03477
If you found this useful, please cite this write-up as:
Gupta, Anuj. (Apr 2025). The Art (& Science) of finding the most important AI Use Cases - Part 3. anujgupta.co https://pragmaticai1.substack.com/p/the-art-and-science-of-finding-the-257
or
@article{gupta2025ai-use-cases-1,
title = {The Art (& Science) of finding the most important AI Use Cases -
Part 3},
author = {Gupta, Anuj},
journal = {anujgupta.co},
year = {2025},
month = {AprIL},
url = {https://pragmaticai1.substack.com/p/the-art-and-science-of-
finding-the-257}
}