So I finally used the US holiday weekend to look into it.

Here is what I learned:

Nix is a set of tools for helping you configure your software environment exactly the same way every time. That is what it is.

However…

The ideas behind Nix are advanced, so much so that it started life as a guy named Eelco Dolstra’s Ph.D. dissertation1. It has a learning curve that goes beyond “steep” and does a full loop-the-loop, like one of those fairground rides that unfolds off the back of a truck. The biggest reason Nix is still a niche technology, rather than the One True Way that we manage our software packages, is that most professional software engineers who take one look at Nix say “Dear God, no. And also, what is it? Please make it stop.” For them, the cure (reshaping their brains to speak Nix) is worse than the disease.

But those who persevere—interesting software shops like Replit and Shopify—choose Nix for a good reason. Because there is a disease. And it lies in wait for us all.

THE DISEASE

The disease is software itself. Specifically, software dependencies.

All the little packages that your code imports, plus all the little packages that THEY import, right down to things like the C compiler included with your operating system (OS)…

The metadata used to compile your system libraries…

The minor version of your operating system…

All those things your code needs to run are dependencies and collectively they are known as your software environment.

(Sometimes Nix people prefer to say “the closure of your dependencies” which is a FUNCTIONAL PROGRAMMING way of saying “the whole universe of everything that you depend on” but FUNCTIONAL PROGRAMMING is one of those things a lot of us don’t feel comfortable thinking about and so we are not going to think about it.)

Any time you deploy software — to paraphrase our buddy Eelco, that’s when you move a program from computer A to computer B and it still works when it gets there—you are betting that the software environment on computer B will interact with your code in the same way that it did on computer A.

Historically, this has been a sucker’s bet. Linux uses what is called “dynamic linking” which means that the system libraries you depend on can change out from under you when your OS updates. The default package managers for your Linux distros, like yum and apt-get, like to install a single version of your dependencies GLOBALLY which means that if different apps require different versions then haha good luck.

If you have ever tried to update Python on your Macbook, you know how easy it is to end up with a nasty dirty developer environment full of broken packages of unknown origin.

What we all want is REPRODUCIBILITY which means I want to specify EXACTLY what dependencies my software environment needs and I want them to deploy EXACTLY THE SAME WAY every time on every machine. That is the problem Nix is trying to solve.

And at the time Eelco was writing his dissertation in the early 2000s there was not really a great way to ensure this.

THE NIX CURE

The cure was functional programming!

This is why so many people get hung up on Nix, because functional programming hurts our brains. But the big concept is not hard to understand:

Nix wants to help you build your software “without side effects” - that is, without changing other things in your system that you didn’t specify. This is sometimes referred to as a PURE FUNCTIONAL approach. Again, all it means is that what you put in (the description of what your environment should look like) is exactly what you get out.

Nix uses several pieces to make this happen, including among others:

• A little mini-programming language that helps you say what packages you want

• A package manager (a thing that downloads the packages you want and saves them in a unique location)

• A special build of Linux called “NixOS” that you can put your packages in if you want to have an end-to-end Nixified software environment

A tricky part is that people might refer to all of these things, or some of them, or just one at a time, as “Nix”. And that is why when you ask ten different Nix gurus to define “Nix” you will get ten different answers, like blind men describing an elephant.

But don’t be dazzled. Nix is a set of tools for helping you configure your software environment exactly the same way every time. That is what it is.

THE INSUFFICIENT CURE

Docker. Docker is the insufficient cure to the software dependency disease.

If you want to prickle the hackles on a Nix devotee, say “Why don’t you just use Docker?”

The Nix devotee will say that Docker containers do not give you truly reproducible builds in the way that Nix does. There are quite a few things that can change out from under you, even with a “pinned” Docker image. Plus, stacking Docker images on top of each other quickly leads to duplicated packages, whereas Nix hashes references to everything in the environment to avoid duplication.

The Nix devotee will also say that Dockerfiles are IMPERATIVE and Nix is DECLARATIVE. A Dockerfile says “run this command and then that command, in that order, to build the environment.” A Nix configuration says “Here is what the environment is supposed to look like, go make it so.”

That said, you can actually use Nix to generate Docker images and lots of people do.

I asked ChatGPT to give me an analogy comparing Nix and Docker, and it said if your software is a sandwich, Nix is like a recipe and Docker is like a lunchbox.

That is sufficiently insane that I’m going to roll with it.

THE TRAGEDY OF NIX

The tragedy of Nix is that the documentation is famously poor, the mini-language is hard to learn (it’s functional, natch), and over time other solutions such as Docker (for shipping reproducible environments) and Alpine Linux (for minimal OS builds) sprang up that were “good enough” for a lot of use cases.

That said, these days people are developing their software in cloud environments that need special automation, and the overall increase in sophistication of software infrastructure teams suggests that maybe Nix has brighter days ahead. There’s really nothing else out there quite like it.

If you want to get into the HOW of Nix, I recommend these Zero to Nix tutorials.

Just for fun

I drew this cartoon to help promote the fireside chat I’m doing with Rubrik next week. See you there!

Today’s sponsor / victim stepping into the arena is Archera, a Freeman & Forrest client. Kudos to them for being good sports about this - I only do these write-ups with permission, but Archera has no editorial control over what I say.

As far as I can tell, there are two types of cloud cost management products.

The first kind, like CloudHealth and its many “FinOps” alternatives, primarily focuses on planning and reporting. Their value prop is that they give you better tools to figure out where your IT budget is going, and how to allocate it, than the AWS Billing console does.

The other kind promises to sell you cloud at a discount, and those products tend to have a red flashing warning label on them that goes SCAM SCAM SCAM SCAM. Either they’re playing weird arbitrage games with resource reservations, or they’re asking you to give them way too much control over your billing accounts, or they’re charging you a massive 15-40% fee for “savings” you could probably get on your own with a few hours of effort.

There’s a reason AWS is cracking down on some of these shady practices. Products that promise to charge a percentage of what they “save” you by booking EC2 instances on your behalf have a fundamental incentive mismatch: the more cloud you buy, the more money they make. This is the opposite of what you want your cloud cost management partner to be rooting for.

So when I landed on Archera’s homepage, my sketchiness antennae were way, way up.

At first glance, Archera looks like it could be another “we book your AWS Savings Plans and charge you a fee for it” vendor. The buzzwords are all there on their homepage: flexibility! Discounts! Usage! Commitments! Incentives!

It took me a minute to realize that Archera is not a cloud reseller at all. Actually, it doesn’t fit into either of the cloud cost management categories I defined up top. They do have a basic FinOps platform that can book native AWS savings plans in your own account for you, but it’s free to use, no fees at all.

Archera is up to something a lot weirder and more surprising. It turns out that Archera is an insurance product.

This is where cloud architects get bored and tune out, because insurance is boring. But it’s where finance people should get really interested.

As I understand it, Archera’s main thing is not selling you cloud discounts, but selling you peace of mind. The founder/CEO came from AWS and has seen first-hand that people are understandably squeamish about ponying up tens or hundreds of millions of dollars for a three-year cloud commitment when they have no idea how much cloud they will actually need by then.

Archera’s solution to this problem is simple, much simpler than the buzzwords on the homepage might make you think.

• Step 1: you pay Archera a premium, like an insurance premium, to book shorter-term cloud savings plans and reserved instances directly in your AWS or Azure account1. Way shorter than 3 years, as low as 30 days in some cases. The shorter the commit term, the higher the premium.

• Step 2: you use your committed resources.

• Step 3: if for some reason you DON’T consume all your committed resources, Archera automatically pays you back to cover the difference. With real money.

Step 3 is the wild part. There are a few other cost management products out there that throw around words like “cloud commitment insurance”, but what they mean is something like “here is store credit you can use to buy even more of our knockoff savings plans, since you didn’t use all of them last time.” Archera, by contrast, puts actual dollars back in your bank account. I genuinely don’t think I have seen any other FinOps tool that does this.

Again, the reason they can pull this off is that they are not really a developer tooling product; they are a financial product, like car or boat insurance. Archera relies heavily on “reinsurance”, which is a jargon term meaning insurance for insurance companies. Somewhere, an actuary has figured out how likely it is that Archera will have to rebate all its customers overnight because us-east-1 got attacked by direwolves, and has spread that risk among other insurance companies so you will get your money back either way.

I like what Archera is doing because I think they have fixed the incentive problem we talked about that plagues so many reseller-type companies. You are not paying Archera to “save you money”, you are paying them to take the risk out of not knowing how to forecast your cloud spend. This is a useful thing.

I do not particularly like their marketing website, though. It’s long on high-level jargon (“Why Archera?”) and short on explanation (“How does Archera actually work?”). I would foreground the extremely clear and well-written docs that are currently buried in the site footer, probably with an architecture diagram that shows how Archera interacts with your AWS account.

My advice to Archera is to stop using the same buzzwords that the cloud resellers do. You don’t want to be confused with those people because SCAM SCAM SCAM SCAM.

Talk about yourself like what you are: travelers’ insurance for the cloud.

My wife and I took a 10th anniversary trip to Scotland recently. I bought trip insurance ahead of time. Not because I especially wanted to spend hundreds of additional dollars, but because I wanted to zero out the risk of losing the larger amount of money we spent on airline and hotel bookings if, oh I don’t know, Crowdstrike decided to push out another minor Windows update.

The trip went great (thanks for asking!), and we enjoyed it even more knowing that our bookings were insured.

A product like that should exist for CFOs who are staring at their cloud forecasts and imagining what would happen if … well, if Crowdstrike decides to push out another minor Windows update. It seems to me that Archera has built that product. They should publish more content showing exactly how it works.

If you would like your product explained in a future edition of Tech the Heck?, let me know.

Links and events

1. I’ll be back at AWS re:Invent this fall for the first time in several years. Let me know if you’d like to grab coffee!

2. Emily and I recently appeared on The Cloudcast to explain why we started Freeman & Forrest and what we have been up to lately.

3. It is mid-October, which means you probably have about 30 days left to accomplish whatever goals you set yourself this year before the holidays kick in. Seems like a good time to redeem your 30 free days of learning from Cloud Resume Challenge sponsor Pluralsight.

Look, most of the new technologies we deal with as software engineers are conceptually not that complex. We’re just not very good at explaining them, either to non-techies or to each other.

Sure, we all like to think we’re above using marketing buzzwords: “leverage transformative innovation”, “supercharge ROI by driving accelerated adoption”, etc. But man, do we ever have a problem with technical jargon.

When I land on some developer-tooling startup’s product page, I find myself glazing over at words like “Programmatically provision customer-representative environments, for comprehensive cross-distro testing that prevents customer issues”. (That’s real copy, name withheld to protect the guilty.) That description paints NO picture in my mind’s eye. I can read it five times and it just drains out of my brain like water through a sieve.

The obtuse way we talk about technology has a couple of bad effects: it makes us feel dumb, and it helps vendors foist expensive crap onto us before we realize what we’ve gotten ourselves into.

“Tech the Heck?” is a new series I’m trying that explains WHY a software product exists and WHAT problem it is trying to solve, in as plain English as I can come up with. Think of it not as the missing manual, but as the missing confessional. Also, there will be cartoons.

Huge round of applause to consulting client CloudTruth, who were incredibly good sports to let me share today’s notes publicly. Seriously, this is VERY cool of them. I wouldn’t be doing this if they hadn’t given the okay, and I also wouldn’t be doing it unless I thought their product was legitimately useful - and deserves to be better explained.

Tech the heck? - CloudTruth

Selling professional developer tools is really hard. People think this is because developers don’t like to pay for things. And they don’t, but that doesn’t really matter, because in a business context the developers aren’t usually the ones buying the things.

The real problem is a mismatch in values. Developer tools are usually built by developers who care a LOT about doing software engineering The Right Way. They care about good abstractions and clean code and hygienic ops. They care about writing fast, maintainable apps. They might even have left their old day job because they were so frustrated by the slow, out-of-date practices and DIY tooling. They believe they can build a better path. And they assume that the better path ought to be worth money, because what’s more valuable than happy, productive developers?

This is why so many technical founders market their tools with phrases like “easy to use / better abstractions / less toil”: because those are the things they themselves value. If you want to get developers to adopt an open-source product, definitely talk about the great abstractions and ease of use.

But if you want engineering directors to pay for a tool, you’re probably gonna have to do better than that. It is 2024. Nobody has budget and everybody is laying people off. Nice-to-haves are not need-to-haves. When you, a developer, ask your boss to buy a product that will help you design more aesthetically-pleasing build pipelines, the boss is hearing “Can we spend money on a more fun way to do non-revenue-generating tasks?”

This is where CloudTruth comes in. They asked me to take a look at their homepage and see if I could think of any ways to explain their product better. When I first landed on it, the main hero copy looked something like this. (They’re A/B testing some new things now, so you might see something different when you click):

OK, based on just that information, what problem would you guess that CloudTruth is trying to solve?

They’re apparently selling something that’s supposed to help with “config”. When I hear “config”, I think “YAML templates.” Or maybe it’s some kind of infrastructure-as-software product that helps you manage config with code? Maybe it’s a configuration management database (CMDB) for tracking IT assets! I don’t know, and context clues aren’t helping. It definitely FEELS like I should know what they’re referring to, though, so I catch myself nodding along.

The next thing on the page is a demo video. We could do a whole essay about making a great demo video, but just note that the very first thing you see in that video is a slide called “The need for ConfigOps in modern SDLCs.”

Hang on. I already don’t know what they mean by “config”. Now we’re talking about ConfigOps. What is ConfigOps? It’s a term CloudTruth is trying on for size to encompass what they do, but it creates no picture in my mind. I’m pretty sure I have both config and ops in my SDLC already. Maybe I don’t need this product, whatever it is.

Here’s where I pause again to clarify that I am not dunking on CloudTruth. I think they have built something interesting and it probably even speaks well of them that so far they have put 99% of their energy into building it, rather than tweaking copy on their landing page. But what the heck is it?

Simply put: CloudTruth is a single pane of glass for configuration data. Configuration data is secrets, environment variables, and other parameters—the stuff you inject into your app at deploy time to make it work.

Yes, there are tools that manage certain types of config data today. Hashicorp has Vault. AWS has Secrets Manager and SSM Parameter Store (or whatever they’re called these days; I’m not looking it up). The Linux Foundation has a vaguely terrifying open-source thing called External Secrets Operator that sucks all your config data into Kubernetes. Some of us are unfortunate enough to have used Secret Server.

These tools tend to be best at one thing (eg, secrets handling), or are provider-native; most require a good bit of manual spackle, and keeping track of different versions of parameters as they change over time tends to be an unfriendly adventure.

CloudTruth’s founding insight, as with so many developer-tool founders before them, is that this situation sucks! There should be an easy way to version and push out config data across lots of different teams and cloud environments. There should be a great set of abstractions for wrangling the data, and that should reduce ops toil as you … aaaand you see where this is going.

Easy, great abstractions, reduced toil. The rogues’ gallery of Nice-to-Haves. If those really were CloudTruth’s primary selling points (and when I looked at their marketing site I kept seeing “easy and fast” pop out like a scary clown), I would not be writing this post because I would not believe in CloudTruth’s potential and I would not be defending them in this weird backhanded who-needs-enemies-with-friends-like-these kind of way.

But there is another way to look at CloudTruth, and that view is: oh crap, this is not a devtools startup, this is a data startup. And then it suddenly becomes very interesting.

Look, let’s be brutally honest: platform / DevOps / SRE / whatever teams generally don’t excel at managing data. This is why the DBAs are always refusing to let them get their slimy hands on the production databases.

The most mission-critical data that platform teams usually get to touch is probably configuration data: the Kubernetes ConfigMaps, the VPC layouts. Because it’s “just config”, we’re often pretty casual about handling it, and that is a massive misperception of risk. It just so happens that mistakes in handling config data are the culprit of something like 80% of all production outages. The more apps and teams and cloud environments you juggle, the more likely you will push a dev parameter to prod, or ship an environment variable that really should have been a secret just because your secret manager is way over there. (I’ve seen what you all do with your OpenAI API keys.)

Building a prettier dashboard for your config data is a Nice-to-Have. Building a thing that lets you responsibly manage, audit, secure, and version the kind of data that is statistically more likely to take your apps offline than anything else—THAT seems like it could be a Need-To-Have.

If I’m trying to convince the head of my platform engineering team to use CloudTruth, I’m not talking about how easy it is to use. I’m talking about the fact that it might be the only product on the market that has a coherent plan for taking care of the most mission-critical data our team is allowed to touch. Config data is load-bearing, and it might be smart to share the load.

The CloudTruth homepage has a bunch of possibly-hypothetical “things our customers used to say” near the top of the page. I don’t find hypothetical testimonials compelling and I suggested they get rid of them entirely. What I do find compelling is the list of REAL “things our customers say now” with logos waayyy down near the bottom of the page, which boil down to “uh, holy crap, we really were not taking config data seriously enough before CloudTruth came along.” I would move those testimonials way up.

The last thing I want to call out is this “how CloudTruth works” section near the middle of the homepage:

The copy is … a lot of words. It uses bolded jargon terms like configuration artifacts and platform components that I am not totally sure I define the way CloudTruth does. I feel like I would grok what they are telling me much quicker if I could see it in a diagram. Fortunately the right side of the screen is taken up with … oh. Oh no.

I call this style of graphic “Developer Impressionism”. It’s a dreadful genre that pops up in so many product announcement videos and investor slide decks. It looks sort of like an architecture diagram, but it conveys no information. Nothing is labeled. It is essentially abstract art, meant to invoke the feeling of an architecture diagram.

Your target audience is scrolling this page with two minutes and half a brain cell to spare before diving back into a quarterly roadmap meeting. If you are going to take up screen real estate with a diagram, make it a real architecture diagram. I would like to see a napkin sketch that that shows how CloudTruth actually fits in my deployment process:

Okay, I know that’s crude. But it contains exactly the same information as the original wall of text, with the crucial distinction that I can follow at a glance what CloudTruth’s role in my CI/CD process is supposed to be.

My overall recommendation to CloudTruth was to focus on their killer value prop: they are stewards of mission-critical data. That means getting really precise with language.

CloudTruth is testing some new landing page ideas. When I last checked that main hero copy, it looked like this:

Gone is the oblique reference to “config.” They are now calling themselves “The Configuration Data Platform.” That already creates a better picture in my mind, but in case it doesn’t, we have context clues above and below: variables, secrets, and parameters. The word “easy” does still appear in the sub-head, but crucially, the thing they are making easy is an important thing: accurate config data.

Maybe you think a configuration data platform is a Need-to-Have, maybe you don’t. But at least now, you’ll be having a real conversation about it, not just saying: “Tech the Heck?”

Thanks again to CloudTruth for giving me permission to share these reflections based on my work with them. If you would like me to walk through your product page with you, I’ve opened up a few 60-minute meeting slots.