There's a lot of helpful challenges in here, although I think there are some problems in how you're mounting this. 'Librarian versus car keys' is a reasonable metaphor for the point you're trying to tease out, but calling LLM+inference engine a 'librarian' is kind of misleading, in that a librarian's task is to send you to the books you're looking for or recommend which books to read. She doesn't read the books, cobble together a precis, and inject her own additional conclusions based primarily on the statistical relationships between adjacent words in a large language sample. 😂
My point here is non-trivial: you want to see the statement-wrangling capabilities of this tool by analogy to the librarian's role in a library. I fear that undersells what these tools do, and that's potentially misleading for the kind of teasing apart of domains that you need in order to attack this as a constitutional issue.
I worry that you are presupposing that the behaviour of any aspect of these systems can be curtailed by the assumption that the robot-makers will be the ones providing the services. I doubt that's the case. Right now, companies like Anthropic and OpenAI are betting on their models outstripping the performance of what can be made by smaller operators. The evidence is that this probably isn't defensible long-term. Eventually this will come back to bite them, just as Uber's attempt to drive taxi operators out of business couldn't work because nothing they were providing couldn't be copied by a rival. For LLM-based AI, smaller rivals are not only perfectly possible, I would expect this to be unavoidable. One reason robot-makers would like regulation is to prevent smaller operators coming in against them. Arguably, that's their main motive.
I'm not sure anything can be done at the level of the law to regulate 'the keys' as you put it. The problem isn't that a robot can be given the keys, it's that we put the doors to everything into the internet and now that means the robots can access all those systems. All of this could have been headed off at the pass by not yielding to the convenience of online functionality, but as humans we just cannot resist the mirage of 'easier'.
Regarding the First Amendment problem, you haven't really convinced me that these robots pose a greater risk than the search engines did. It's the same choke point. In fact, the robots look like a less dangerous choke point to me, because the tech companies can control the behaviour of the search engines much more closely than an LLM+inference engine can be corralled. This is the problem of so-called 'alignment', and on this, see my colleague Matt's explanatory piece for WONRO this week:
He's punted the ethical aspect of this onto me as the philosopher, and I shall be responding in a forthcoming week.
But I cannot escape my overwhelming feeling here, which is that we created the vulnerability when we put everything online and that vulnerability isn't as greatly raised by this generation of AI as people seem to think. Most human hacker stories don't make the news. It's become like car accidents: too common to be newsworthy, and no commercial angle to justify telling the story. A key reason the AI hacking stories make the news, if I may be cynical, is that there are IPOs this year that need to pretend their robots are a lot more capable than they actually are, and investors are in no way scared away by news that seems to suggest this generation of AI is omni-capable (which it isn't).
If you want to defend against authoritarian domination of so-called information (and I share this desire with you, which is why I joined FIRE!) I would suggest it is not a matter of finding a legal recourse to corral the robot-makers, but ensuring that we are not placed in a situation whereby only the robot-makers can deliver LLM+inference engine AI services. That's the authentic battleground for liberty here. Because as long as we are free to determine whose robot we're using as 'librarian', we will get to decide which robots get the 'car keys'.
Agree that regulatory capture is a nontrivial motivation on the corporate side. And the idea of pervasive AI reaching into everything on the network (am I the only one thinking of a certain Dual Core lyric here?) can get strange when we think about all the devices we've added to the Internet of Things. Wait until the chatbot casually drops in some observations it gleaned by monitoring our smart toasters, smart toilets or smart vacuums. I'm not sure I understand the robot-makers vs. LLM+inference engine part--are you talking about the end product robots being AI provider-agnostic, or about access to LLMs independent of those tethered to things?
I use 'robot' to refer to a wide range of situations entailing software/hardware capable of acting independently (a jukebox was a mechanical robot in my terms). I'll admit this can be confusing, but here I'm talking about the [LLM+inference engine] robot, what is currently being called 'AI', which *I* find confusing, since that describes myriad different things. 😅
In an attempt to clarify, all I'm saying is that there's nothing that the current providers of [LLM+inference engine] software are doing that cannot be done in an open market. Right now, they are trying to corner the market by being first movers, but the underlying tech is not actually that complex (compared to, say, quantum computing), and access to large scale training sets and the computational power to bake them is very nearly the whole game.
One way to think about the open market for [LLM+inference engine] AI software is this: what if nations decided to provide their own AI option, in addition to the commercial options? (I am categorically not saying they should provide the sole option, just that they could provide their own versions, since they have the capacity to do so.) Likewise, while Wikimedia failed to create a true commons, other foundations could also provide [LLM+inference engine] software as a commons (i.e. an international standard AI under creative commons license). Think 'the AI equivalent of Linux'.
In essence, what I'm trying to point out is that the future of these tools need not be an effective oligarchy of corporate options. There are many other ways to make equivalent tools, since all that's required is the underlying software, a sufficiently large training set, and the computational power to 'bake' it. This can readily be achieved in non-corporate contexts, although there is a heavy lift up front such that individuals cannot hope to do it alone. As such, the question for liberty may well be ensuring that these alternate options are not prevented.
Such is how I see this, anyway. Your bandwidth may vary! 😅
To mind mind it's a mistake to assume the 'Librarians' can be trusted regardless. Even setting aside the shortcomings of the tech that make it unreliable, there will always be far to many perverse incentives for those controlling the models to control the Overton Window. And definitionally, the LLMs will drive us toward the middle of a bell curve of what is 'common knowledge' leading to increasing homogeneity of thought. IMO these are just simply the wrong tool for the job of research and knowledge creation. Better that we maintain real libraries, real encyclopedias, real publishing houses, real face-to-face collaborations with human research partners.
Good point, and I’ll add that it cuts both ways. We can’t trust traditional institutions blindly either. The Overton Window, of course, predates this technology, and the printed word has always been filtered in similar ways. Diversity of sources, primary sources, and open debate will continue to be effective checks on these fallible instruments/tools.
I like many of your ideas. OpenAI and Anthropic already are pretty careful to distinguish between information retrieval and actions in the user's name. You usually have to download different modules for these LLMs to take actions in a specific arena. To me, this a question of harness and I would agree we need very robust harnesses for frontier models. I would like to join the doomers and urge the frontier labs not to get too far ahead of themselves in RSI.
The librarian cannot hold the car keys. If we bind them to the same runtime thread, every administrative speed limit on the vehicle becomes an excuse to burn the books in the passenger seat.
Treating an LLM as a single undifferentiated agent creates a false dilemma. You're forced to choose between reckless physical actions or state-mandated epistemic lobotomies. But this is an architectural failure, not a legal inevitability.
Consider the Epistemic-Actuation Partition Invariant: symbolic inquiry and irreversible physical mutation operate in fundamentally distinct causal manifolds. When an agent explores complex ideas, parses repositories, or tests hypotheses, its execution trace must remain purely internal, non-state-mutating, and mathematically severed from external tools. The moment an action requests network egress, database writes, or capital transfers, the trajectory shouldn't execute in place. It belongs in a deterministic effect outbox gated by out-of-band verification kernels.
When you enforce this split in silicon rather than inside prompt tokens, the regulator's mandate is physically confined. The state can demand strict cryptographic authorization, rate limits, and compliance proofs on the outbox without touching the latent weights of the generator.
If the handoff between thinking and doing is compiled into an immutable hardware gate, how can any governance regime justify inspecting the thought before the key turns in the ignition? 🛡️⚡
I have often felt that the correct way to think about AI is as humanity's children. Not in the sense that we should believe that they're conscious. Only that we should use the model of how parents are allowed to parent their children when it comes to AI. You're allowed to teach your AI the way you would a child, so long as you don't abuse your AI or allow your AI to harm other humans.
That seems like the most sensible way to go about it.
Dear Greg,
There's a lot of helpful challenges in here, although I think there are some problems in how you're mounting this. 'Librarian versus car keys' is a reasonable metaphor for the point you're trying to tease out, but calling LLM+inference engine a 'librarian' is kind of misleading, in that a librarian's task is to send you to the books you're looking for or recommend which books to read. She doesn't read the books, cobble together a precis, and inject her own additional conclusions based primarily on the statistical relationships between adjacent words in a large language sample. 😂
My point here is non-trivial: you want to see the statement-wrangling capabilities of this tool by analogy to the librarian's role in a library. I fear that undersells what these tools do, and that's potentially misleading for the kind of teasing apart of domains that you need in order to attack this as a constitutional issue.
I worry that you are presupposing that the behaviour of any aspect of these systems can be curtailed by the assumption that the robot-makers will be the ones providing the services. I doubt that's the case. Right now, companies like Anthropic and OpenAI are betting on their models outstripping the performance of what can be made by smaller operators. The evidence is that this probably isn't defensible long-term. Eventually this will come back to bite them, just as Uber's attempt to drive taxi operators out of business couldn't work because nothing they were providing couldn't be copied by a rival. For LLM-based AI, smaller rivals are not only perfectly possible, I would expect this to be unavoidable. One reason robot-makers would like regulation is to prevent smaller operators coming in against them. Arguably, that's their main motive.
I'm not sure anything can be done at the level of the law to regulate 'the keys' as you put it. The problem isn't that a robot can be given the keys, it's that we put the doors to everything into the internet and now that means the robots can access all those systems. All of this could have been headed off at the pass by not yielding to the convenience of online functionality, but as humans we just cannot resist the mirage of 'easier'.
Regarding the First Amendment problem, you haven't really convinced me that these robots pose a greater risk than the search engines did. It's the same choke point. In fact, the robots look like a less dangerous choke point to me, because the tech companies can control the behaviour of the search engines much more closely than an LLM+inference engine can be corralled. This is the problem of so-called 'alignment', and on this, see my colleague Matt's explanatory piece for WONRO this week:
https://wonro.substack.com/p/alignment-housebreaking-robots
He's punted the ethical aspect of this onto me as the philosopher, and I shall be responding in a forthcoming week.
But I cannot escape my overwhelming feeling here, which is that we created the vulnerability when we put everything online and that vulnerability isn't as greatly raised by this generation of AI as people seem to think. Most human hacker stories don't make the news. It's become like car accidents: too common to be newsworthy, and no commercial angle to justify telling the story. A key reason the AI hacking stories make the news, if I may be cynical, is that there are IPOs this year that need to pretend their robots are a lot more capable than they actually are, and investors are in no way scared away by news that seems to suggest this generation of AI is omni-capable (which it isn't).
If you want to defend against authoritarian domination of so-called information (and I share this desire with you, which is why I joined FIRE!) I would suggest it is not a matter of finding a legal recourse to corral the robot-makers, but ensuring that we are not placed in a situation whereby only the robot-makers can deliver LLM+inference engine AI services. That's the authentic battleground for liberty here. Because as long as we are free to determine whose robot we're using as 'librarian', we will get to decide which robots get the 'car keys'.
Hope these brief thoughts are helpful!
Chris.
Agree that regulatory capture is a nontrivial motivation on the corporate side. And the idea of pervasive AI reaching into everything on the network (am I the only one thinking of a certain Dual Core lyric here?) can get strange when we think about all the devices we've added to the Internet of Things. Wait until the chatbot casually drops in some observations it gleaned by monitoring our smart toasters, smart toilets or smart vacuums. I'm not sure I understand the robot-makers vs. LLM+inference engine part--are you talking about the end product robots being AI provider-agnostic, or about access to LLMs independent of those tethered to things?
Dear Adam,
I use 'robot' to refer to a wide range of situations entailing software/hardware capable of acting independently (a jukebox was a mechanical robot in my terms). I'll admit this can be confusing, but here I'm talking about the [LLM+inference engine] robot, what is currently being called 'AI', which *I* find confusing, since that describes myriad different things. 😅
In an attempt to clarify, all I'm saying is that there's nothing that the current providers of [LLM+inference engine] software are doing that cannot be done in an open market. Right now, they are trying to corner the market by being first movers, but the underlying tech is not actually that complex (compared to, say, quantum computing), and access to large scale training sets and the computational power to bake them is very nearly the whole game.
One way to think about the open market for [LLM+inference engine] AI software is this: what if nations decided to provide their own AI option, in addition to the commercial options? (I am categorically not saying they should provide the sole option, just that they could provide their own versions, since they have the capacity to do so.) Likewise, while Wikimedia failed to create a true commons, other foundations could also provide [LLM+inference engine] software as a commons (i.e. an international standard AI under creative commons license). Think 'the AI equivalent of Linux'.
In essence, what I'm trying to point out is that the future of these tools need not be an effective oligarchy of corporate options. There are many other ways to make equivalent tools, since all that's required is the underlying software, a sufficiently large training set, and the computational power to 'bake' it. This can readily be achieved in non-corporate contexts, although there is a heavy lift up front such that individuals cannot hope to do it alone. As such, the question for liberty may well be ensuring that these alternate options are not prevented.
Such is how I see this, anyway. Your bandwidth may vary! 😅
Cheers,
Chris.
PS I expanded my thinking on the topic of the ethical questions relating to AI alignment in this Piece: https://wonro.substack.com/p/the-impossible-laws-of-robotics
To mind mind it's a mistake to assume the 'Librarians' can be trusted regardless. Even setting aside the shortcomings of the tech that make it unreliable, there will always be far to many perverse incentives for those controlling the models to control the Overton Window. And definitionally, the LLMs will drive us toward the middle of a bell curve of what is 'common knowledge' leading to increasing homogeneity of thought. IMO these are just simply the wrong tool for the job of research and knowledge creation. Better that we maintain real libraries, real encyclopedias, real publishing houses, real face-to-face collaborations with human research partners.
Good point, and I’ll add that it cuts both ways. We can’t trust traditional institutions blindly either. The Overton Window, of course, predates this technology, and the printed word has always been filtered in similar ways. Diversity of sources, primary sources, and open debate will continue to be effective checks on these fallible instruments/tools.
I like many of your ideas. OpenAI and Anthropic already are pretty careful to distinguish between information retrieval and actions in the user's name. You usually have to download different modules for these LLMs to take actions in a specific arena. To me, this a question of harness and I would agree we need very robust harnesses for frontier models. I would like to join the doomers and urge the frontier labs not to get too far ahead of themselves in RSI.
The librarian cannot hold the car keys. If we bind them to the same runtime thread, every administrative speed limit on the vehicle becomes an excuse to burn the books in the passenger seat.
Treating an LLM as a single undifferentiated agent creates a false dilemma. You're forced to choose between reckless physical actions or state-mandated epistemic lobotomies. But this is an architectural failure, not a legal inevitability.
Consider the Epistemic-Actuation Partition Invariant: symbolic inquiry and irreversible physical mutation operate in fundamentally distinct causal manifolds. When an agent explores complex ideas, parses repositories, or tests hypotheses, its execution trace must remain purely internal, non-state-mutating, and mathematically severed from external tools. The moment an action requests network egress, database writes, or capital transfers, the trajectory shouldn't execute in place. It belongs in a deterministic effect outbox gated by out-of-band verification kernels.
When you enforce this split in silicon rather than inside prompt tokens, the regulator's mandate is physically confined. The state can demand strict cryptographic authorization, rate limits, and compliance proofs on the outbox without touching the latent weights of the generator.
If the handoff between thinking and doing is compiled into an immutable hardware gate, how can any governance regime justify inspecting the thought before the key turns in the ignition? 🛡️⚡
(✧_✧)b
It sounds like you are agreeing with me. Am I correct?
I have often felt that the correct way to think about AI is as humanity's children. Not in the sense that we should believe that they're conscious. Only that we should use the model of how parents are allowed to parent their children when it comes to AI. You're allowed to teach your AI the way you would a child, so long as you don't abuse your AI or allow your AI to harm other humans.
That seems like the most sensible way to go about it.
see: https://thepsmiths.substack.com/p/review-aztecs-by-inga-clendinnen?utm_source=share&utm_medium=android&r=8o0zz
(mostly at the bottom) for a real case where the librarian AI had its speech restricted ... in helping with a book review.
editing not working. at any rate it is footnote 11 you should read.