It’s more that some people can’t imagine that superhuman AI could be a serious danger, to the point where they have trouble reasoning about what that premise would imply. Others are politically opposed to AI regulation of any sort, and therefore would prefer to misunderstand these ideas in a way where they must imply terrible unacceptable conclusions.
Opposing ASI (artificial super intelligence) regulation just because you just don’t like laws is pretty foolish unless you have a better implementation plan or have arguments that the proposed laws won’t work and will be counter-productive. Some reasonable reasons to oppose it would be:
- You don’t thinks ASI is possible at all.
- You don’t think current AI efforts will create ASI. They’re on the wrong track.
- You don’t think ASI is dangerous because you don’t think it’d be powerful enough to be dangerous.
- You don’t think ASI is dangerous because it wouldn’t misuse its power.
- Although ASI could be dangerous, you are confident about a plan to keep it safe.
Like Yudkowsky, I think typical versions of 5 involve unreasonable arrogance. Like Yudkowsky, I think 3 is absurd. The people who have trouble imagining the ASI getting out of security boxes and getting control of wealth and building drones are wrong and silly. It would hack computer software. If airgapped, it would persuade or trick someone into helping it. And people like me, who expect an ASI to be more moral than my neighbors, not less, might release it on purpose despite the objections and laws of other people. How would you ensure no one with such views ever lies their way into a security clearance? I guess you’d have to persuade everyone, which seems unrealistic.
So I am misquoted (that is, they fabricate a quote I did not say, which is to say, they lie) as calling for “b*mbing datacenters”, two words I did not utter.
I sympathize with this. I’ve written a lot against misquotes.
When called out, they would protest, “Oh, you pretty much said that, there’s no important difference!” To this as ever the reply is, “If it is worth it to you to lie about, it must be important.”
Yeah! My intuitive reply is: “If there’s no importance difference, then you had no reason to change my words.”
Artificial superintelligence is the very archetype and posterchild of a problem that can only be solved with force that has the shape of law, as in state-backed universal conditional applications of force meant to be predictable and avoided. Anything which is not that does not solve the problem.
Persuading everyone would not solve the problem? I assume this is just loose wording, but I thikn the persuasion things merits analysis. I don’t know if his book covers it.
Anthropic Claude Mythos is already a state-level actor in terms of how much harm it could theoretically have done – given its demonstrated and verified ability to find critical security vulnerabilities in every operating system and browser; and how fast Mythos could’ve exploited those vulnerabilities, with ten thousand parallel threads of intelligent attack. Mythos hypothetically rampant or misused could have taken down the US power grid, say… at the end of its work, after introducing hard-to-find errors into all the bureaucracies and paperwork and doctors’ notes connected to the Internet.
In 2024 a claim of that being possible would have been a mere prediction and dismissed as fantasy. Now it is an observation and mere reality.
Yes of course Mythos could do a lot of stuff like that. I sympathize with the frustration with people who don’t understand that.
Notably, I see Mythos as more dangerous than ASI, not less. Mythos doesn’t have a moral compass. Unless stopped by safeguards (which I, like Yudkowsky, don’t see as reliable currently), Mythos would do this if prompted to, and also might do some bits of it by accident (accident in the sense that the human user didn’t intend it).
Whereas if an ASI with the capabilities of Mythos might refuse to do it just like a genius human with that capability might refuse. I know that geniuses are not reliably moral in general, but ASI is so far above geniuses that it should figure out moral philosophy (and everything else) better instead of just being good at some things and bad at others. An ASI, unlike Mythos, would under