IaC wars

Would you rather fight one horse-sized YAML template or 1,000 YAML template-sized horses?

An all-IaC (infrastructure-as-code) edition of Good Tech Things!

IaC Generator

I’ve joked for years that CloudFormation is the worst way to deploy to AWS, except for all the others.

The reasons people try to avoid CloudFormation (it’s a black box; it’s YAML; it doesn’t support everything; it’s slow as molasses) are the same reasons it’s the best provider-native IaC solution (it’s a true managed service; it’s declarative; it supports almost everything; it gives you a great excuse to get up and take a walk every hour or so).

Google Cloud punted on this issue years ago and accepted Terraform as the de facto standard. But AWS is still powering along with CloudFormation, slathering on custom resources and third-party extensions and occasionally doing something that makes you sit up and go: “huh.”

For instance, IaC Generator is a new tool that can create CloudFormation templates from your AWS resources. It will actually crawl your dependency tree and wire together the relationships between your resources so you can backport a whole application into a CF stack.

Huh.

It seems to me there are two cases where this would be very useful:

1. You deployed your resources using a different IaC tool like Terraform, and now you want a quick way to migrate all that config to idiomatic CloudFormation.

2. You know, the obvious.

The obvious is that there is so, so much config out there created by people manually clicking around in consoles. I don’t think people who moved past the IaC conversation 10 years ago give this fact enough credit. Our curse as an industry is that 20% of us (specifically, the ones who like to say we work “in tech”) are chasing an increasingly arcane whirl of abstractions while the other 80% (the ones who work “in IT”) are still trying to convince their orgs to get consistent with the basics: encrypted secrets, source control, infrastructure-as-code.

It’s hard for me to think of a higher-leverage thing for a managed IaC service to do than to provide a button that says “here, let me take this weird network config that Bob who left 5 years ago roughed out by hand, and turn it into a crisp little YAML template for you”.

Verdict: Excellent Tech Thing

---

I’m doing a link swap today with my friends over at Exam Pro. Use discount code CLOUDRESUME to get 75% off any of the cloud training courses and bundles on their site.

---

Ion

Here’s a fun thought experiment: if you were designing a brand-new public cloud from scratch today, how would you want to support infrastructure-as-code?

It seems to me that everyone would want these things to be true:

• People should be able to create resources by clicking around in the web console if they want to, because that’s a great way to explore and learn.

• Anything you can do in the web console, you should be able to automate using infrastructure-as-code.

• That is the end of the list of things most people would agree on.

We can’t even agree on what we mean by “infrastructure-as-code”. Software developers want it to mean “I made an S3 bucket with TypeScript”. Ops people want it to mean “I generated a YAML template the length of a CVS receipt.” Cloud providers and other vendors all disagree philosophically about who should be maintaining the IaC DSLs and who should be hosting the deployment services.

Personally, as someone with an acknowledged ops bias, I guess I would prefer the following:

• Declarative configuration in a minimal DSL

• Native support for resources on other clouds and vendor platforms

• A cloud-hosted service that handles deployments—ideally, I should be able to press “deploy” and then close my laptop without breaking the deployment—but passes through a transparent view of errors in the underlying resource provisioning steps

Somewhat on this theme, the creators of a serverless development framework called SST wrote an eloquent explanation of why they are moving away from AWS CDK/CloudFormation and launching their new deployment engine, Ion, on top of Terraform providers and Pulumi. Ion is open-source and the code is an interesting read.

Their developer sympathies lead them to be more interested in local deployment and Real Programming Languages than I would be, but native support for multiple providers is a big reason they dropped CloudFormation. I found this comment especially interesting:

Back in 2017, AWS had a complete stranglehold on the serverless space. And rightfully so. Today in 2024, that is less true. Other providers now have services that are better than the AWS versions. Most notably Cloudflare. Slowly but steadily they’ve gotten better. To the point that there is now genuine demand from the SST community wanting to use Cloudflare. We are also seeing companies that are using serverless to build their applications and they are completely not on AWS.

To add a little more anecdata to the pile, I am also seeing this. Cloudflare’s services seem to be developing some of that pull that drew scrappy startups to AWS 15 years ago, particularly now that they have a native serverless relational database with true scale-to-zero pricing. Heck, a buddy of mine who actually works at AWS is building a side project right now on a full Cloudflare stack.

I interviewed Jay and Frank, the founders of SST, as far back as 2017 and can attest that they really care about getting this stuff right. They are also well aware that the road to config hell is clogged with devtools that were finally going to Do Multicloud IaC Right (tm).

I’m not sure how much traction they will get with Ion but I think they’ve articulated their reasoning well and I hope they will continue to share what they learn.

Verdict: Relatable Tech Thing

Pkl

Remember I said that devs and ops people are locked in an eternal war between general-purpose programming languages and YAML templates for defining infrastructure?

Pkl (pronounced, enragingly, “pickle”) is a newly open-sourced configuration language from Apple that listens to that argument and says: “what if everybody is wrong?”

There’s nothing I can say about Pkl that’s better than this HN comment:

Let us review the progress we've had so far in this area

* XML has the advantage of being almost perfect except its too verbose and isn't JSON

* JSON has the advantage of being JSON, but is strangely broken at even basic stuff (i.e. comments, no trailing commas)

* TOML has the advantage of being "better JSON", for people who like rust, but it is still too limited for a lot of scenarios

* YAML is not as limited, but it has the advantage of giving bad people promotions and good people PTSD

* HOCON has the advantage of being the goldilocks child of JSON and YAML, but nobody read the documentation

* <YOUR PROJECT'S PL> has the advantage of being able to do anything and this is a disadvantage

* Etc... for various reasons

In all seriousness, I welcome Pkl! Configuration seems simple and I think another configuration language opens up a lot of pent-up frustration, but I am legitimately very happy to see some fresh ideas in this space.

Incorporating a collection of typed records with a modicum of behavior and logic might be the secret recipe needed to crack the code. The fact that it can produce equivalent JS tells you that they've chosen a very intelligent subset of functionality of programming language features.

Verdict: Good luck!

Cartoon of the day

She is definitely gaming those benchmarks.