[1]Blog [2]About [3]Moonbound This is a post from [4]Robin Sloan’s lab blog & notebook. You can [5]visit the blog’s homepage, or [6]learn more about me. [7]Is it okay? February 11, 2025 Macbeth Consulting the Witches, 1825, Eugène Delacroix [8] Macbeth Consulting the Witches, 1825, Eugène Delacroix How do you make a language model? Goes like this: erect a trellis of code, then allow the real program to grow, its development guided by a grueling training process, fueled by reams of text, mostly scraped from the internet. Now. I want to take a moment to think together about a question with no remaining practical importance, but persistent moral urgency: Is that okay? The question doesn’t have any practical importance because the AI companies —  and not only the companies, but the enthusiasts, all over the world — are going to keep doing what they’re doing, no matter what. The question does still have moral urgency because, at its heart, it’s a ques tion about the things people all share together: the hows and the whys of humanity’s common inheritance. There’s hardly anything bigger. And, even if the companies and the enthusiasts rampage ahead, there are still plenty of us who have to make personal decisions about this stuff every day. You gotta take care of your own soul, and I’m writing this because I want to clarify mine. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ A few ground rules. First, if you (you engineer, you AI acolyte!) think the answer is obviously “yes, it’s okay”, or if you (you journalist, you media executive!) think the answer is obviously “no, it’s not okay”, then I will suggest that you are not thinking with sufficient sensitivity and imagination about something truly new on Earth. Nothing here is obvious. Second, I’d like to proceed by depriving each side of its best weapon. On the side of “yes, it’s okay”, I will insist that the analogy to human learning is not admissible. “Don’t people read things, and learn from them, and produce new work?” Yes, but speed and scale always influence our judgments about safety and permissibility, and the speed and scale of machine learning is off the charts. No human, no matter how well-read, could ever field requests from a million other people, all at once, forever. On the side of “no, it’s not okay”, I will set aside any arguments grounded in copyright law. Not because they are irrelevant, but because … well, I think modern copyright is flawed, so a victory on those grounds would be thin, a bit sad. Instead, I’ll defer to deeper precedents: the intuitions and aspirations that gave rise to copyright in the first place. To promote the Progress of Sci ence and useful Arts, remember? I hope partisans of both sides will agree this is a fair swap. Put down your weapons, and let’s think together. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ I want to go carefully, step by step — yet I want to do so with brevity. Lan guage models produce so … many … WORDS, and they seem to coax just as many out of their critics. Logorrhea begets logorrhea. We can do better. I’ll begin with my sense of what language models are doing. Here it is: lan guage models collate and precipitate all the diverse reasons for writing, across a huge swath of human activity and aspiration. Start to enumerate those reasons: to inform, to persuade, to sell this stupid alarm clock, to dump the CUSTOMERS table into a CSV file … and you realize it’s a vast field of desire and action, impossible to hold in your head. The language models have many heads. In this formulation, language models are not merely trained on human writing. They are the writing: all those reasons, granted the ability to speak for themselves. I imagine the PyTorch code as a mech suit, with squishy language strapped in tight …  To make this work — you already know this, but I want to underscore it — only a truly rich trove of writing suffices. Train a language model on all of Shakespeare’s works and you won’t get anything useful, just a brittle Shakespeare imitator. In fact, the only trove known to produce noteworthy capabilities is: the entire internet, or close enough. The whole extant commons of human writing. From here on out, for brevity, we’ll call it Everything. This is what makes these language models new: there has never, in human history, been a way to operationalize Everything. There’s never been any thing close. Just as, above, I set copyright aside, I want also to set aside fair use and the public domain. Again, not because they are irrelevant, but because those intuitions and frameworks all assume we are talking about using some part of the commons — not all of it. I mean: ALL of it! If language models worked like cartoon villains, slurping up Everything and tainting it with techno-ooze, our judgment would be easy. But of course, digiti zation is trickier than that: the airy touch of the copy complicates the sce nario. The language model reads Everything, and leaves Everything untouched — yet sud denly this new thing exists, with strange and formidable powers. Is that okay? ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ As we begin to feel our way across truly new terrain, we can inquire: how much of the value of these models comes from Everything? If the fraction was just one percent, or even ten, then we wouldn’t have much more to say. But the fraction is, for sure, larger than that. What goes into a language model? Data and compute. For the foundation models like Claude, data means: Everything. Compute combines two pursuits: 1. software: the trellises and applications that support the development and deployment of these models, and 2. hardware: the vast sultry data centers, stocked with chips, that give them room to run There’s a lot of value in those pursuits; I don’t take either for granted, or the labor they require. The experience you get using a model like Claude depends on an ingenious scaffolding. [9]Truly! At the same time: I believe anyone who works on these models has to concede that the trellises and the chips, without data, are empty vessels. Inert. Reasonable people can disagree about how the value breaks down. While I believe the relative value of Everything in this mix is something close to 90%, I’m willing to concede a 50/50 split. And here is the important thing: there is no substitute. You’ve probably heard about the race to generate novel training data, and all the interesting effects such data can have. It is sometimes lost in those dis cussions that these sophisticated new curricula can only be provided to a lan guage model already trained on Everything. That training is what allows it to make sense of the new material. Also, it is often the case — not always, but often — that the novel training data is generated by … a language model … which has itself been trained on … you guessed it. It’s Everything, all the way down. Would it be possible to commission a fresh body of work, Everything’s equal in scale and diversity, without any of the encumbrances of the commons? If you could do it, and you trained a clean-room model on that writing alone, I con cede that my question would be moot. (There would be other questions! Just not this one.) Certainly, with as much money as the AI companies have now, you’d expect they might try. We know they are already paying to produce new content, lots of it, across all sorts of business and technical domains. But this still wouldn’t match the depth and richness of Everything. I have a hypothesis, which naturally might be wrong: that it is precisely the naivete of Everything, the fact that its writing was actually produced for all those dif ferent reasons, that makes it so valuable. Composing a fake corporate email, knowing it will be used to train a language model, you’re not doing nothing, but you’re not doing the same thing as the real email-writer. Your document doesn’t have the same … what? The same grain. The same umami. Maybe one of these companies will spend ten billion dollars to commission a whole new internet’s worth of text and prove me wrong. However, I think there are information-theoretic reasons to believe the results of such a project would disappoint them. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ So! Understanding that these models are reliant on Everything, and derive a large fraction of their value from it, one judgment becomes clear: If their primary application is to produce writing and other media that crowds out human composition, human production: no, it’s not okay. For me, this is intuitively, almost viscerally, obvious. Here is the ultimate act of pulling the ladder up behind you, a giant “fuck you” to every human who ever wanted to accomplish anything, who matched desire to action, in writing, part of Everything. Here is a technology founded in the commons, working to undermine it. Immanuel Kant would like a word. Fine. But what if that isn’t the primary application? What if language models, by collating and precipitating all the diverse reasons for writing, become flex ible general-purpose reasoners, and most of their “output” is never actually read by anyone, instead running silent like the electricity in your walls? It’s possible that language models could go on broadening and deepening in this way, and eventually become valuable [10]aids to science and technology, [11]to medicine and more. This is tricky — it’s so, so tricky — because the claim is both (1) true, and (2) convenient. One wishes it wasn’t so convenient. Can’t these companies simply promise, with every passing year, that AI super science is just around the corner … and meanwhile, wreck every creative industry, flood the internet with garbage, grow rich on the value of Everything? Let us cook—while culture fades into a sort of oatmeal sludge. They can do that! They probably will. And the claim might still be true. If super science is a possibility — if, say, Claude 13 can help deliver cures to a host of diseases — then, you know what? Yes, it is okay, all of it. I’m not sure what kind of person could insist that the maintenance of a media status quo trumps the eradication of, say, most cancers. Couldn’t be me. Fine, wreck the arts as we know them. We’ll invent new ones. (I know that seems awfully consequentialist. Would I sacrifice anything, or everything, for super science? No. But art and media can find new forms. That’s what they do.) Obviously, this scenario is especially appealing if the super science, like Everything at its foundation, flows out into the commons. It should. So — is super science really on the menu? We don’t have any way of knowing; not yet. Things will be clearer in a few years, I think. There will either be real undeniable glimmers, reported by scientists putting language models to work, or there will still only be visions. For my part, I think the chance of super science is below fifty percent, owing mostly to the friction of the real physical world, which the language models have, so far, avoided. But, I also think the chance is above ten percent, so, I remain curious. It’s not unreasonable to find this wager suspicious, but if you do, I might ask: is there any possible-but-unproven technology that you think is worth pur suing even at the cost of itchy uncertainty in the present? If the answer is “yes, just not this one”: fair enough. If the answer is “no”: aha! I see you’ve answered the question at the top of this page for yourself already. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Where does this leave us? I suppose it’s not surprising, in the end: If an AI application delivers some profound public good, or even if it might, it’s probably okay that its value is rooted in this unprecedented operational ization of the commons. If an AI application simply replicates Everything, it’s probably not okay. I’ll sketch out my current opinions more specifically: I think the image generation models, trained on the Everything of pictures, are: probably not okay. They don’t do anything except make more images. They pee in the pool. I think the foundation models like Claude are: probably okay. If it seemed, a couple of years ago, that they were going to be used mainly to barf out text, that impression has faded. It’s clear their applications are diverse, and often have more to do with processes than end products. The case of translation is compelling. If language models are, indeed, the Babel fish, they might justify the operationalization of the commons even without super science. I think the case of code is especially clear, and, for me, basically settled. That’s both (1) because of where code sits in the creative process, as an inter mediate product, the thing that makes the thing, and (2) because the commons of open-source code has carried the expectation of rich and surprising reuse for decades. I think this application has, in fact, already passed the threshold of “profound public good”: opening up programming to whole new groups of people. But, again, it’s important to say: the code only works because of Everything. Take that data away, train a model using GitHub alone, and you’ll get a far less useful tool. Maybe (it turns out) I’m less interested in litigating my foundational question and more interested in simply insisting on the overwhelming, irreplaceable con tribution of this great central treasure: all of us, writing, for every conceiv able reason; desire and action, impossible to hold in your head. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Did we make progress here? I think so. It’s possible my question, at the outset, seemed broad. In fact, it’s fairly narrow, about this core mechanism, the operationalization of the commons: whether I can live with it, or not. One extreme: if these machines churn through all media, and then, in their deployment, blow away any prospect for a healthy market for human-made media, I’d say, no, that’s not what we want from technology, or from our future. Another extreme: if these machines churn through all media, and then, in their deployment, discover several superconductors and cure all cancers, I’d say, okay … we’re good. What if they do both? Well, it would be a bummer for media, but on balance I’d take it. There will always be ways for artists to get out ahead again. More on that in another post. I also think there are some potential policy remedies that would even out the allocation of value here — although, these days, imagining interesting policy is a sort of fantastical entertainment. Even so, I’ll post about those later, too. In this discussion, I set copyright and fair use aside. I should say, however, that I’m not at all interested in clearing the air for AI companies, legally. They’ve chosen to plunge ahead into new terrain — so let them enjoy the fog of war, Civ-style. Let them cook! [12]To the blog home page I'm [13]Robin Sloan, a fiction writer. The main thing to do here is sign up for my newsletter: [14][ ] [15][Subscribe] This website doesn’t collect any information about you or your reading. It aspires to the speed and privacy of the printed page. Don’t miss [16]the colophon. Hony soyt qui mal pence References: [1] https://www.robinsloan.com/lab/ [2] https://www.robinsloan.com/about/ [3] https://www.robinsloan.com/moonbound/ [4] https://www.robinsloan.com/ [5] https://www.robinsloan.com/lab/ [6] https://www.robinsloan.com/about/ [7] https://www.robinsloan.com/lab/is-it-okay/ [8] https://www.clevelandart.org/art/1962.109?utm_source=Robin_Sloan_sent_me [9] https://www.youtube.com/watch?v=ugvHCXCOmm4#t=9780 [10] https://research.google/blog/accelerating-scientific-breakthroughs-with-an-ai-co-scientist/?utm_source=Robin_Sloan_sent_me [11] https://darioamodei.com/machines-of-loving-grace?utm_source=Robin_Sloan_sent_me [12] https://www.robinsloan.com/lab/ [13] https://www.robinsloan.com/about?utm_source=Robin_Sloan_sent_me [16] https://www.robinsloan.com/colophon/