Recently, my team and I had success building a software system of massive complexity relative to the one and a quarter month ramp we had to get it running. It performed much better than its predecessor, more reliable, more maintainable, and less stressful. I won’t get too deep into what we actually made, only that it is an ETL with cooperative work-stealing, leader election, and a control layer.
AI is involved. We picked Rust. Both have much to do with the success. We’ll talk about them, but neither is the main topic.
Between 1946 and 1953, there was a famous group called the Macy Conference full of extraordinary minds from many disciplines. There was Warren McCulloch and Walter Pitts, who first built the mathematical models of neurons that eventually evolved to the GenAI we know today. There was John Von Neumann whose name is now synonymous with the shape of the brain of modern computers, the CPU and memory. There was Claude Shannon, the father of information theory.
And then there was Norbert Wiener who came up with cybernetics, the main topic of the conference. Cybernetics will be our entry point.
Cybernetics
Fun facts: Cyber, cyborg, cybersecurity, cyberpunk—these words came from cybernetics.
Cybernetics is a science of communication as it relates to machines, and living things. It's interesting that you don't hear this word these days. Disciplines have become so specialized and how cybernetics relates to other subjects is mostly by application of the theory, not the theory itself.
Cybernetics' core tenet is the circular causal, where something loops so that its input is fed back to its output. The classic image of cybernetics is an arrow where the end meets the start. The typical metaphor is a sailor steering a ship in which the sailor checks where he’s going, where he’s now and based on that, he decides where to go. And he does it constantly. That’s the key.
To put it in a practical term, classically when you program, you think of point A, point B, and how to get from A to B. Input. Output. Algorithms. Cybernetics thinking, very simply put, is akin to wrapping that process with a loop and reframing the process as differentials.
The subsystems of the system that we built were a bunch of cybernetic loops, long-living and non-crashing automation, and automations of automations. That kind of thinking turned out to be free lunch because by just that simple act of turning things into loops you’ll get free qualities, like your product being more robust, easier to maintain and manage, and focused on constant adjustment towards the goal, whatever it is, business goal, robustness, self organization, and so on.
Some new intuitions start to develop, like realizing these loops can be stacked so that one loop manages the other—or how to keep all these heavy calculations sustainable—or things as simple as how to frame the output from the previous loop to be the next loop’s input.
Rust and cybernetics
Feeding back output into input might sound obvious. “Of course that’s the way to do it!”. But check, if you’re a programmer, or ask if you have a programmer friend, how much of the system’s output is unreadable by itself. There’s this thing called error logs where a software throws a bunch of text into a file or the terminal to tell the human operator that it’s not well.
Most error logging and error handling code is just “screw it, if that ever happens, I’ll just log it, and probably crash it”.
First of all, we picked Rust. For those who don’t know, it is a relatively new programming language with rich features, that yells at you more than other languages when you make mistakes in it, and is notorious for being difficult because of a novel paradigm; I am of the opinion that this “difficulty” is only because it’s not widely taught yet.
So, our major reasons for picking Rust were 1.) how easy it is to write parsers in it, 2.) its safety, 3.) its enum, and 4.) cargo. Parser is pretty obvious because we’re writing a data transformer. Type safety is pretty obvious because we just want to lay back and feel safe while the production server is running. Enum and cargo are the interesting parts here.
Enum, for those who don’t know, is Rust’s way to converge many types into one. It’s like saying, “Ok we need a general name for boots, sneakers, sandals. Let’s name it footwear.”—footwear here being the enum. It is a much more elegant form of polymorphism in the sense that it doesn't force you to converge the behaviors if you want to only converge the structure.
That structural convergence point as a language construct is a very important language feature for looping back the output into the system as input because you can describe many scenarios, successes, failures, anomalies, into a single object that for us programmers feels almost tangible, you can almost hold it in your palm. That closes the output-to-input loop.
Meanwhile, Cargo is the tool that closes the loop of agentic AI.
Cargo and AI
Cargo is Rust’s companion tool for building and testing, but better than most other tools for other languages.
In other languages you need setups. Sometimes if you get these setups wrong, the tests fail. Tests are meant for keeping fallible things from failing. If the tests themselves are fallible, then you’ll eventually need to test your tests to some degree.
During that period, I got the “luxury” of using AI with three different languages: Rust, TypeScript/Node, and (god help me) gitlab-ci YAML file.
I don’t have the number, but I'll make the claim anyway because the phenomenon is so obvious. When you work with an AI agent sometimes you need this back-and-forth interplay with the agent, usually because it didn’t understand you correctly or because you forgot something important.
I will use TypeScript/Node as a baseline of this interplay. In TypeScript/Node, I often have to intervene and make manual fixes because the AI doesn’t have the grasp of the situation. Mostly this is caused by side effects, like mocks side-effects, test tool misconfiguration, wrong compiler configuration.
In Rust, the interplay is near zero. Most of the time it was because my specification was ambiguous or lacking and the agent made assumptions instead of trying to clarify the "plot holes". Mostly when the specification was decent I could trust it to run, leave it for the gym and lunch, and then when I got back I got the full working feature.
Now, with gitlab-ci YAML, I was basically the testing tool. It writes the code, I run the script manually, it fails, I put the error message back to the agent, and so on. Most of the time both me and the agent are stumped by the error message.
My observation is that Rust works really well with AI because both cargo's test and Rust's type system give a ridiculously unambiguous guarantee to whoever uses it. There's a meme for that: “if it compiles it will likely work.” The AI agent uses that guarantee to keep its own quality.
From the cybernetics point of view, AI agents (like humans) work in the build-test loop. What cargo provides is a great test quality that connects the test step back to the build step. Essentially, it closes the cybernetic loop of software development.
I'm calling this property "degree of given agency". Basically, if you give an AI agent the instruction to guard its own quality, complemented by the resources to do so, then it can develop the same self-organization property as any other thing modelled as cybernetic loops.
Another important point to make is that density, accuracy, and expressibility are among the important properties of a programming language, assuming equal mastery. It means, if you master a certain language, then the limit of that language is determined by how you can express complex meanings in a non-ambiguous manner with as few symbols as possible. And people might not realize this but Rust’s syntax is very dense because you can describe a thing's spatial configuration, temporal relation, Liskov’s substitution principle, etc, in one or two declarations. AFAIK, you can’t do that in Go, TypeScript, Java, Kotlin, etc.
Rust may be hard to learn, but AI doesn’t really mind. In fact, the information density means the AI spends less context window to “understand” the same thing in another language. While in TypeScript/Node things can go wrong because you misconfigured something, in Rust there’s not much to set up because all of those configurations are baked into the language itself with sane defaults.
Before we enter the main topic, a caveat: I’ve been telling this story of cybernetics thinking as if it’s novel, but I have used it more than five years ago and this idea itself has permeated many in the industry, although not explicit nowadays.
Another caveat: I’m comparing Rust and TypeScript because they reside in an obviously different layer of abstraction, yet, they share a lot of similarity as pointed out in many forums. People transition from TypeScript to Rust with ease.
Bound melds and bound restitching
Let’s look at a hypothetical pre-looped algorithm that has three operations between input and output, [0,1,2]. If we loop them together and put them on a timeline, we’ll get [...,0,1,2,0,1,2,0,1,2,...]. This is now a sequence and we can frame this as a boundless continuum where 0 is before 1 but also after 2.
I’m going to use the above example to generalize the framing of this act of closing the loop into two aspects: the circular stitching and the dissolution of boundaries (or in short, bound-melds). These two properties are used a lot in existing applications that need to scale. Video streaming, for example, dissolves the file boundary and treats the data as a continuous flow of frames. BitTorrent protocol dissolves the outer boundary of files as well as chunking the bytes of each file and then regards the entire package as one continuum. This “treating a system as pipes” aspect is pretty much ubiquitous in modern computing.
Fun fact: Rust's enum is sort of a bound-meld.
So, this act of dissolving and restitching of logical boundaries seems to give free stuff or at least cheap bonuses. This seems as if by thinking about the circular causal in concept, we are led to think about the foundational aspects that are necessary to manifest the circular causal. In layman’s terms, “think long life, get long life.”
Let’s generalize more. What about dissolving and restitching, not on operations, but on concepts—I mean categories, boundaries, and distinctions that we’ve made over time because it’s just convenient.
Take the microservice vs monolith shenanigans circa 2018. The choice was presented as an either-or thing while the two actually don't even conflict in the same battlefield. Microservice was about service-oriented architecture, ownership, deployment orthogonality. Monolith was about having a formal, programmatic, or, at least, explicit dependency and data flow.
The criticism pointed at microservice was how every communication involves network, how transactions are hard to manage because they’re distributed. Well, let’s be imaginative. How about deploying a microservice not as a kubernetes instance but a WASM-based stored procedure? A modular team can just deliver a binary that's attached to a bigger long-running binary at runtime, which eliminates the network dependency while keeping responsibility separated. How about using a different model of distributed computation since if you replicate monolith apps, you’ll have more or less the same problem as microservice?
The criticism that a single module failure can bring down the entire monolith application is really an error boundary problem, not a consequence of centralized code. Microservice doesn’t guarantee that the same problem will not bring it down at the business operation level.
Take the OOP vs FP. The lazy answer to OOP vs FP is often “use them pragmatically”, but if we break down OOP vs FP it’s ultimately about the smaller aspects of the language like state mutation management, module separation, composition over inheritance, hierarchy vs non-hierarchy. Meanwhile, OOP vs DOD is about cache optimization, code coherence at system level, and orthogonality of operation/structure.
The point is that these dilemmas are often false. The things being fought are the big labels, like OOP, microservice, FP, DOD, that serve as semantic shortcuts rather than the actual essence under them. The solution is often to dissolve these arbitrary bounds that might no longer be useful, to look directly at the essence behind these bounds, and to restitch them where it makes sense.
Important: dissolution does not mean dividing the problem into small pieces and working with them in isolation because with that approach the whole will lose cohesion.
Fringe things
There’s one topic I like to poke around with this dissolving and restitching: AI and consciousness. Between 2022 and 2024 this topic was a bit warm. In 2022, there was an incident involving a Google engineer claiming that LaMDA was sentient. Anthropic posted a blog post “exploring model welfare” which shows that it still considers AI consciousness a possibility.
An easy-ish way to dismiss AI’s consciousness is by asking whether the weather is conscious because it shows some degree of interactivity and behavior, seemingly retaliating when humans make extreme effects on the environment. Another point to ask is that if we train transformers with only numbers instead of text, is it still conscious? You can't communicate with it, yet it has the same architecture as GPT.
Let's ask the bound-melding questions: What if consciousness is not a matter of yes or no, but a continuous scale? What if sentience is broken down into many aspects like memory, intelligence, self-affect, agency, which are aspects that make up a degree of sentience? What if LLMs are just proxies and reflectors of sentience the way a language is, just much richer and more coherent such that our minds perceive them as somewhat alive?
How can LLMs be conscious when weather or fluid dynamics is not? But why would silicon-based lifeforms not exist when carbon-based lifeforms do?
There's a shower thought.