Closed as unhelpful: an elegy for Stack Overflow

It's the end of an era.

I posted my very-first-ever StackOverflow question on the second day of my first job after college. I really wanted to link the post here, but it was so long ago that I can’t find the question, or even the throwaway account I was using at the time. I think it was something about talking to SQL Server with PowerShell.

I do remember that I felt nervous, overwhelmed, out of my depth in my new job. I was too embarrassed to ask a coworker for help. Posting on Stack Overflow felt like a last-ditch effort, a Hail Mary pass. The next day I logged in and checked my post, hoping against hope that somebody had taken pity on my plight.

The post had been closed as off-topic.

I didn’t post again for years.

---

Even a decade ago, Stack Overflow had already developed a reputation as kind of a prickly, intimidating place. Users weren’t shy to let you know if they thought you had posted something dumb. The top contributors (like the legendary Jon Skeet) were so fast at gobbling up answers to the good questions that it was hard for new contributors to amass reputation points. The system of relying on user upvotes to rank multiple answers, rather than crowdsourcing a single canonical response to each question à la Wikipedia, meant that out-of-date or just plain wrong answers often appeared at the top of the page.

None of it mattered because Stack Overflow was a quantum leap in surfacing help for programmers. The upvoting system was janky, but it worked better than any previous tech forum. (Remember “Tom’s Hardware”? The 27th page of Google search results sure does.) The userbase was polyglot and omniplatform: you could find answers on just about any programming topic. And it was certainly faster than reading man pages. Or, crucially, it felt faster.

Stack Overflow gave stumped programmers the dopamine hit of looking up the solution to their problems, rather than learning how to solve them.

In this way, too, Stack Overflow was ahead of its time.

---

You know something weird? When Legendary Jon Skeet passed one million reputation points in 2018, Stack Overflow celebrated by creating a Markov chain bot that spit out Skeet-shaped answers to arbitrary questions. At the time everyone seemed to think this was harmless fun.

In retrospect, it looks like foreshadowing.

---

LLMs have not, so far, proved to be as generally useful as a lot of true believers hoped. They’re bad at surfacing insights not found in their training data, they’re asymptotically mediocre at “creative” tasks, and they’re untrustworthy in any field requiring accurate information.

But they have turned out to be reasonably accurate at producing answers to common programming questions. And GitHub Copilot and ChatGPT are much faster and easier to use than navigating the labyrinth of a question-and-answer site.

Most importantly, the thing Stack Overflow was worst at — providing a welcoming place to learn — is the thing LLMs are the best in the world at. Whereas posting on Stack Overflow can be like releasing a fragile bird into a cage full of furious snakes, ChatGPT does not care if your question is a duplicate. It does not care if you did not provide enough context. It is not hung up on parliamentary procedure. ChatGPT is infinitely patient and (more or less) constantly available.

My younger self, two days into a job I barely understood, would have greeted ChatGPT like the messiah.

---

The people who work at Stack Overflow are not stupid. They understood immediately that LLMs were an existential threat to their business. The problem is what to do about it. They’ve tried launching their own AI tools, but it’s not clear that they’re better than any of the other 500,000 coding-help models out there.

Instead, they’ve just announced a wide-ranging partnership with Open AI. These partnerships are sometimes public relations niceties more than they are substantive collaborations, but it does sound as though Stack Overflow is going to license its data for Open AI to train future models on.

It’s hard not to see this as anything other than an attempt by Stack Overflow’s newish corporate overlords to cash in on their intellectual property before it becomes entirely worthless.

What remains of the Stack Overflow community is disgusted by the idea that their contributions will be absorbed into anonymous AI models, perhaps devaluing their hard-won reputation points and community credibility. Some people are trying to delete their own questions out of spite, only to discover that they don’t actually own their own content. And I don’t know why anyone would willingly submit more unpaid answers (or questions!) to Stack Exchange forums under these conditions. After all, huge pools of people in Africa and South America are being paid to write code examples for model-training companies right this minute. Why should I do it for free?

---

When LLMs first became available, a whole bunch of people tried to use them to generate Stack Overflow answers and rack up cheap karma; the overall quality of answers, though, was low enough that Stack Overflow has banned AI-generated content entirely from the platform.

This raises the most troubling problem going forward, not just for SO but for all of us. The relationship between user-generated training data and AI-generated results so far appears to be one-way. LLMs do not add to humanity’s body of knowledge; they only synthesize and regurgitate.

Particularly in the programming world, where many questions are niche, context-specific edge cases, the only way to get high-quality answers is through a lot of net-new human experimentation. Stack Overflow, to its credit, built a community that incentivized this type of knowledge sharing.

It is not obvious that LLMs will do the same.

---

A coda: I discovered today a whole trove of questions on Stack Overflow related to the Cloud Resume Challenge. Beginning programmers, just like I was all those years ago, are crying out for help—stumped by DNS SERVFAILs, by Python path woes, by responsive CSS. And in almost every case, they are getting help. Real advice from real people, who are not just showing them how to solve their problems, but leaving a public record for the next new programmer to find. I saw little if any judgmental behavior—the community has matured over the years. I saw encouragement, respect, and growth.

Despite all its warts, perhaps even because of them, Stack Overflow has carried the torch for an extraordinarily human way of exploring technology. Open AI would do well to consider how they could put more of that into the world, rather than less.

---

Good Sponsored Thing

I’m super excited to announce that Pluralsight, a leading provider of tech courses, labs, and cert prep, has signed on as the first official sponsor of the Cloud Resume Challenge.

What does this mean for the CRC? I've always said that if the Cloud Resume Challenge community were to work with sponsors, it would have to be in exchange for something that would clearly benefit the community.

I believe Pluralsight has met that bar. They are offering 30 days free on their Premium plan for all Cloud Resume Challengers. This is 3x longer than their standard personal free trial, and should give you enough time to study for an intro-level cert without having to worry about finances.

The link to the free 30-day offer can be found on the CRC challenge pages, or you can redeem it directly here.

I hope this is helpful to many of you - feel free to share the redemption link with friends, and if you want to thank Pluralsight, I'm sure they'd be thrilled if you tagged them in a social post.

---

Cartoon of the day

The US Department of Homeland Security’s new AI Safety and Security Board is stocked with a remarkable assortment of the clueless and the self-interested.