AI-generated code feels scattered. They seem to have no point although we can assume that they're built on a goal.
In a good-old properly architected code, you could see significant points. These parts are allowed to be complex, but intelligible and meaningful.
2400 years ago a Greek wrestler-philosopher made an observation that in this world things have essence. He calls it eidos/forms.
Let's say an innocent child asks you: What makes a chair? Wood with four legs. What about metal chair? Okay, solid objects with four legs. There is a big wood over there and people put their plates and eat on it, is it a chair? No, it's a dinner table, you sit on a chair. Is a horse a chair? And so on, and so on.
There's a point, an idea, about a chair (by the way, the word idea comes from eidos). You know it, but you just don't have the complete definition written. You make proposals (4 legs, wood, sits on it, etc) to try to understand and describe the idea. This process of getting to the idea is what's called logos.
Logos is what logic comes from, but logos isn't about logical operations which is making a conclusion from existing truths. Logos is much more.
Remember that time you came home and your mom asked why you smelled like smoke. Truth is, you smoked, but the idea was to get out of the situation scot-free. So you did not use logical operation and tell the truth. You make up excuses, reasons that presuppose the idea that you did not smoke. You huffed on your mother; a moment ago you prepped heavily on peppermint mouthwash. You walked, because you convinced your mother of a truth you made up.
Science is built on this making up things: hypotheses. How does it feel to chase a light beam? What if the continents were once a single puzzle piece? What if microscopic organisms cause infections?
Algorithm, actually
When we hear "algorithm", usually we think about reels recommended to you in Instagram, TikTok, and all these social media. But computer folks (should) know different.
An algorithm is actually a mathematically rigorous (sounds so serious) steps to solve a particular problem and get a guaranteed correct result. It can guarantee result because the problem is clear, the goal is clear, the boundary is clear.
"How to sort a list?", "How to search a database?", "How to get to your favorite bar?", those have clear goals, clear constraints, clear boundaries.
Algorithm has a cousin that doesn't guarantee a correct result. It is called heuristic. Heuristic, since the 1950s, has been researched to solve ill-defined problem.
"What to eat for dinner?" The goal, the boundary, the constraints, those are the questions themselves. What's your goal of eating? Do you have food restriction? How far can you go for a restaurant and what's your bare minimum restaurant looks like?
"Where to go on the weekend?", "How to reject an invitation from a sensitive friend?"
Inference by "metaphor/analogy"
I argue that LLM-based AI won't ever be able to do abductive reasoning. Lots of literature out there. AI also can't make new ideas because a model is static and the smartest (i.e. most abstract) reasoning it can do is the one that exists in its training data or in the prompt.
That means, AI is not creative. Can't create real eidos. It can't do actual reasoning. No real logos. Ultimately it can't actually decide what's good for you.
But an LLM is naturally a very powerful metaphor/analogy engine. It finds the closest possible answer that's usually the actual answer. If you ask it nonsense question, it will regurgitate answers that make some sense to you because that's how language "works".
An AI coding tool can build a website for a shop because it's been trained on countless codebases, websites, and shops. It infers, i.e. read between the lines, your actual concrete goal and then uses that to figure out the path around the constraints to get that goal.
But what about ill-defined problems?
The Factory that got late parts
Fact 1. A famous problem in software development is that "naming things" is hard. Software code is full of one word that can mean many different things all the time. Ambiguity messes with AI's metaphorical nature. Fact 2. Serious proprietary software has a niche problem from complex business requirements and usually it is ill-defined (because if it's not then we'll have no job).
Combine these two and you'll get a mess. AI will do its best to bridge it, but what will come out is a structure that's a result from the AI hallucinating your goal, constraints, and path to getting there.
Don't think of an extremely complicated problem. Let's say you're building a "just-in-time" ice cream factory. "Just-in-time" means the factory runs when the parts come, but considering it's manufacturing ice cream, when the part comes late the assembly halts and the ice cream melts and production is ruined. You're logistics. You ask AI to simply "track how quick parts arrive" and it comes with a program that tracks the average time. So the tracking system shows that on average the parts come in 10 hours. Manager asks, "can you guarantee the parts will arrive within 12 hours?" "Yes," you confidently answer. So manager sets the assembly to work with the 12-hour assumption.
But here's the plot twist: The traffic where the parts delivery goes through turns out to be unpredictable. Your part can take 5 hours to arrive, but it can also take 15 hours. And when it takes 15 hours, your production is ruined.
Only then you realize that deviation here is as important as average.
So the problem is not the problem! Of course that is an contradiction. But if you notice, all the AI guess of how to "track how quick parts arrive" is correct, it's just that we didn't put enough information (what's enough anyway?).
So, the problem is how you formulate the problem. It is reasoning about what actually matters and is solvable, reasoning towards the essence of the problem rather than the symptoms. It is work to be done at the axiomatic level, or in software development's term, "not implementation detail".
"The problem is your attitude about the problem" -Captain Jack Sparrow
I'll lay here the three things that are basic in software and computation, but rarely talked about these days:
- In computation, we continuously realize, in both sense, become aware and making it real because, as most would point out, "we seldom get it right the first time".
- Half of the purpose of computation is simulation, building a small model of the real thing so it's cheap to play around. For example, we simulate in a computer how we get a spacecraft to Mars before actually launching it. Simulation itself is a form of abstract thinking, but with symbols and mathematical precision.
- Making sure the model small and computation not done repeatedly are important to keep it small. Optimization. One of the most famous optimization paradigm is dynamic programming, which is about breaking computations down to mathematically meaningful sub-structure or sub-systems.
You see these everywhere.
Database engines have distinct parts: connection management, SQL processing, transactions and concurrency management, data storage management, and each of these parts is a meaningful point that's kept distinct. And the thread that connects them all is also a meaningful point that's kept distinct.
Game engines have physics, graphics, animation, audio, AI, world and script editors, asset management. Each is a meaningful point that's kept distinct.
Web UI frameworks have compilers, bundlers, runtime, reconciliation system, rendering, etc. You get the point!
If you notice, the sub-system that works earlier is often a "simulation" of the later sub-systems. Compiler works first makes sure many later runtimes don't break. SQL processing works first to make sure what the user asks is valid for what the rest of the database engine will do. Etcetera.
And these simulations must be certain. You can't have a compiler that says "your code is kind of correct, but I can't guarantee" or an SQL parser that says "your SQL is kind of correct, but I can't guarantee". Therefore, it can't be heuristic, it must be algorithm. And for that, these meaningful points must be well-defined.
And computations and computers, in our culture, have been useful because we have been maintaining these meaningful points that are kept distinct and well-defined. It is a form of optimization, it allows for simulation of simulation; and, more importantly it allows realization to be affordable.
The last bit, realization, is very important because it is basically the evolution of software, software actually being fixed and made better. To evolve something, to change something, you need to be able to understand it and change it. And to understand and change it, you need to be able to first, point at the thing that needs a change and understand it. So how would you point at something if it doesn't have a point and understand something if it doesn't mean anything?
A detour: software rewrites. Sometimes software needs to evolve, because of business requirements or regulations, but it doesn't have meaningful points in it so to change it, you must understand how the entire thing works. Changing the entire thing is expensive, you've heard this ad nauseam. So you rewrite in hope that a better code will be better understood next time.
Hence, meaningful point, essence, substance, idea, eidos is what allows logos, reasoning, optimization, evolution.
I swear I will get back to my point
My first point is, with AI or no AI, software that needs to evolve needs the establishing of these "meaningful points that's well-defined". It could be structure, it could be type, it could be constraints, it could be proposal to how to evolve the point itself, and it doesn't need to be a spatial point. Specifically with AI, it has to be written clearly so it's not ambiguous. This proposition points to the same thing as what Martin Fowler means by (software) architecture.
My second point is, AI infers things you don't define and therefore may hide important details from you. Worse, AI can be "confused" by your lacking or improper definition. You must actively develop the problem the AI is trying to solve and the sub-structures/sub-systems to maintain its well-definedness.
My third point is, we can apply simulation culture to AI-assisted coding itself. Since the code is a "derivation" of the problem definition, defining the problem well first is cheaper than correcting the code after it's made. This reveals that specification matters (it has mattered all this time, we only remembered it recently).
My fourth point, code becomes meaningless when the specification is ill-defined. In the same sense, a product becomes meaningless when the problem is ill-defined. But also, constraints, certainty, isn't always low-level "implementation" detail the same way as goals and meaning isn't always a high-level consideration. This reveals that essence/idea/forms/eidos, reasoning/logos/redefining/proposing, and optimization (curiously) are universal, exist in all abstraction layers.
Reference
This piece is heavily inspired by cognitive science literature https://meaningcrisis.co/