Saturday Errands: Rigid & Squishy Problem Solvers

It's Saturday morning. You have 12 errands to run. Woof! Making sure to stop at the coffee shop first, you order each stop in your head in an efficient route.

How did you do that?

Would AI make the same route?

Would an algorithm/code make another route?

Why you might care to read this

  • if you want a better sense of how AI solves problems
  • if you want to know when AI would solve a problem better with tools
  • if you wanna know the difference between AI and algorithms/code
  • if you wanna know the trade offs to each approach
  • if you wanna start seeing which parts of your own workflow could be improved with AI or AI + tools

Playground If you wanna go experience all this yourself, before reading the post, check out the Saturday Errands playground! Then come back to have it all explained.

Numbers get big

We are looking at driving routes cause it is a common concern and is mathematically "very tricky" (I'll define very tricky in a bit.)1

To get a sense of the numbers:

So wait, how are we casually solving this all the time?

There is a common sense approach to this and we only care about it being good enough...

Human: Common Sense

When we make a route in our heads we often do so using a simple rule: "given where I am right now, what is the next closest stop?" So we start at home. Choose the coffee shop as the first stop and then ask: "whats the next closest errand?" At the next stop, we ask the same thing.

It turns out that common sense approach is the same as a common route finding algorithm...

Algorithm: "Nearest Neighbor"

We can write out that common sense approach as a set of instructions:

  1. Start at home.
  2. Travel to the coffee shop.
  3. Look at all the destinations we haven't visited yet.
  4. Go to the closest one.
  5. Repeat until it has visited all our destinations.
  6. Return home.

These instructions are an algorithm known as "Nearest Neighbor".

Not all lists of instructions can be algorithms.

A recipe is a list of instructions that is not an algorithm. When a recipe says "a splash of wine", this instruction contains some ambiguity requiring judgement.

The instructions must be precise without gaps or ambiguity. They must be rigorous. If someone needs to step in to interpret, use surrounding context, or make judgement call, then its not rigorous enough to be an algorithm.

rigorous: complete and exact enough to follow without judgement

There is a incentive for being this exact. You can take your instructions and have them run by computers!

An algorithm is a set of instructions rigorous enough that it can be written as code and then run by a computer.3 (And when we say computer we don't mean AI, that'll come in later.)

algorithm: list of rigorous instructions that can be written as code and run by a computer.

Good Enough

For the chronic maximizers among us, your inefficiency detecting intuition is probably unsettled right now.

But is that the best route? You'll say.

It's probably not. But you wanna be careful with this very tricky problem.

Now that we know what an algorithm is we can define a very tricky problem. We call a problem very tricky when there is no algorithm that gives us the best solution. To absolutely know that you've found the best route you have to try all of them.4

very tricky: no algorithm exists to help us find the best solution. we must try all solutions to find the best

To get a sense of how long it would take to check every route:

But wait, we have the "Nearest Neighbor" algorithm?

Yea, that is an algorithm. But the algorithm doesn't give us the absolute best route. It gives us a good enough route. When an algorithm only provides a good enough solution thats called a heuristic algorithm.5

heuristic: a shortcut through a complex problem that gives good enough solutions

Ok, you've made 4 definitions so far and we haven't even talked about AI??

Up to this point, this might have been a blog post we could have written before the 2020s. But now we have a new problem solver we interact with nearly everyday. That new problem solver is this AI. How does it approach tricky problems?

AI...

Lets get a bit more clear about what we mean by AI.

"Artificial Intelligence" has meant a lot of things over the last 70 years since the term was coined by John McCarthy in 1955.

When we say "AI" right now we mean Large Language Models aka LLMs.

LLMs

LLMs is a very new thing that we can use to solve problems that, like an algorithm, also runs on computers.6 But it is not an algorithm.

An LLM can do amazing new things. The first amazing thing we maybe all saw was it's ability to write out sentences language! This is something algorithms had kept coming up short on. You might say that the rules of language are not rigorous enough to be captured as an algorithm.

If language can't be written down as a complete set of rules, then how is an LLM "talking" back to us?

Can I use the bathroom?

Come along on this digression with me, I promise it motivate where we are going with LLMs. Look how hard it is to pin down language with rules:

If a student says "can I use the bathroom?" We know exactly that this is a request even though the sentence is actually posed as a question of capability. ("I don't know, can you?" says the annoying teacher.) The meaning is interpreted from the situation.

If your friend says "Nice job!" It is gonna hit different if you just made the game winning shot or dropped and broke a glass. The meaning shifts dramatically with context.

If you are told "she's at the bank." If you're about to go canoeing, then she's at the riverbank. If you are running errands, then she's at the financial bank. Its a judgement call.

When interpretation, context, and judgement are required to make sense of something we are far from the realm of rigid problem solving. We are doing something new. Something squishy.

LLMs: The Squishy Thinkers

LLMs are not made from a massive collection of rigorous instructions/rules. They come to know things from exposure to a huge number of examples (not unlike how we first learned language). This exposure process is called training.

The LLMs main super power is ability to interpret language, make judgements with its context, and write down many complex ideas and instructions.

So if we gave an LLM the locations of all our Saturday morning errands it would try to work through that problem in numbers and language on "scratch paper". It would cluster stops, guess at a path, catch its mistakes and keep trying. It doesn't have a calculator. It doesn't have a computer. Not the LLM alone.

Here's an example of an LLM trying to solve this problem from the playground:

We've now seen two incredible problem solvers. The algorithm which is rigid but fast algorithm. The LLM which is squishy but slow.

What if we could combine the best of both?

LLMs + Algorithms

You remember how an LLM is constructed from a massive amount of training on various examples?

If something is not in the training set, the model doesn't know about it. If its highly repeated in the training set, it'll be more completely encoded into the LLM. The LLM can repeat back common poems, speeches, etc from its encoded memory alone. (If you are wondering if the LLM you are using knows about X, just ask it. Thats how I start a lot of conversations anyway to get us started in the right direction.)

One huge source of examples are all the various code that LLM has been trained on! Because of that it has "learned" how to write code. And because of its squishy ability to make judgement on current context, it can adapt its code writing, like its language writing.

Another large source of examples of training are education materials including lots of math problems, algorithms, and approaches to solving different types of problems. Thus when it sees this "find the shortest route" problem it uses its squishy ability to interpret this to be a "traveling salesmen" problem which is well documented in math/computer science.

But right now it can only write code. Not run it.

What we need to provide to the LLM is a way to execute the code it has written. Thats something we can do by giving it a tool. We tell it the exact words and symbols to write back when it wants the tool to be called. Then on our side, we call the tool. And then we pass back the outputs of the code execution to the LLM. The same way you use a calculator.

tool: a set of instructions we tell the LLM to use if it wants an action to happen. the execution of the action never happens "in" the LLM. it happens elsewhere.

So in this case, we can provide the LLM a tool that when given TypeScript code, we will go run that code and then give the LLM back the results of that code.

Here's how that would go:

  1. LLM thinks about the problem
  2. LLM writes out Nearest Neighbor algorithm in code to solve the problem
  3. LLM requests to have the code tool called with that code
  4. LLM reviews code output and either corrects any code issues or reports the answer

This is how the LLM can be a squishy thinker and still re-use the best of the rigid, algorithmic tools we have spent decades developing.

Playground If you wanna go experience all this yourself, before reading the post, check out the Saturday Errands playground! Then come back to have it all explained.

  1. This particular problem has a well worn history in mathematics and is referred to as the traveling salesmen problem ↩
  2. The total number of routes here is hard to even explain, but lets try: If you took the total atoms in the universe, assume each atom contains a copy of our universe and summed up all the atoms of all the copied universes, we'd have a number 161 digits long. Which is not even close to the 327 digit number that captures the total possible routes of 180 stops. ↩
  3. For more on the definition of algorithm.  ↩
  4. Bumping into a tricky problem isn't so exotic as it appears. There is no algorithm to check for the best way to packing a bag. There is no algorithm for cutting wood to find the most efficient use material. To know that you've found the best case in each would require trying all cases. To go way deeper on this, you might dive into reading about NP-hardness ↩
  5. Let me know if you wanna see what we mean by good enough! We can take a look at quality vs effort graph where we compare how many times you might run an algorithm to get within some % close to the best route. For a fun read that touches on how to live a good life, see satisficing ↩
  6. Some building blocks of language models have existed for decades, like the neural network. While a very important type of neural network, the transformer, was created by researchers in 2017. Attention is All You Need ↩