Do you think they are logical contradictions?
no, not logical contradictions. i just read the article fully (hayek and mises are on my to read list so maybe i’m misunderstanding terms like minarchy) and the first one still seems like a real problem to me even if it’s not a formal contradiction. if the government is also staffed by irrational people, how do we not just run into the same problems on the opposite side? more vigorous policing by flawed people could mean more vigorous corruption or abuse of power.
Intuitively, making judgments about potential intelligence based on methods seems worse/riskier to me than judging outputs. At least partly because the outputs of humans and of other potential intelligences are more directly accessible than their methods.
We might be wrong about how humans create new knowledge, and we might be wrong that the way humans create new knowledge is the only way it can be done.
I’m not saying I think we are wrong. If I knew a criticism of Popperian epistemology I’d say it, and I don’t.
And I also think we might be wrong about what is and isn’t (or could be vs. couldn’t be) going on inside a sufficiently deep neural network.
Here’s a risk of looking at outputs:
One person writes a new book.
Another person copies the book, shape by shape. They aren’t even literate.
Same output.
I haven’t done a detailed analysis (and it’s been a long time since I read ET’s article), but a few thoughts occurred to me as I read Oracle’s post:
-
Incentives could change if companies were policed better. (Inadequately policed companies might see no downside to committing fraud.)
-
It’s been a long time since I read ET’s article, but I feel like a goal of ET’s article was to warn people (especially libertarians, Objectivists, etc.) against being biased in favor of big companies. Not to present policing fraud as a panacea.
- I think ET is in favor of a government that is limited to the task of defending people’s rights, not a government that is so minimally funded that it’s incapable of doing a good job of defending rights. So, limited/~“minimal” in terms of the scope of what it does (it just protects rights) but not “minimal” in terms of how robust/effective/well-financed it is in pursuing that goal.
- If new loopholes or edge cases are discovered, new fixes can be devised. (Though the basic moral principles would presumably stay the ~same.) Similar to how software developers can patch newly-discovered security flaws. It will always be the case that new problems can be discovered and new solutions needed (cf. the idea that we’re always at the beginning of infinity).
I don’t understand your point. One criteria for judging output I already mentioned would be that the output is unique, which the copy would not be.
That’s not enough of course, since a machine could ex: copy + insert some random characters to make it “unique” but that wouldn’t be new knowledge. I think I agree with your earlier statement:
Even if it’s hard, at least we can look at a human output and a machine output side by side and judge whether their dis-similarity exceeds a breakpoint (dis-similarity being just one criteria among multiple required to qualify an output as new knowledge).
OTOH I don’t know how methods which arise outside of an explicit process/program could be observed and compared at the right level of abstraction. I don’t think we could meaningfully compare methods by ex: looking at a series of human brain scans or neuron firings and a series values from a machine’s deep network propagations. And I don’t think anything at a higher level of abstraction is available.
I think another reason for my intuition to judge outputs rather than methods is how we evaluate the intelligence of humans vs. animals. I think we (correctly) judge that humans are intelligent and animals aren’t based on things that humans regularly output (like symphonies and plays and technical diagrams etc.) that no animal ever has. We didn’t get to that conclusion by first figuring out epistemology then figuring out that animals are using different methods, right?
I’ll try. Intentionally not putting too much effort into this to get me to post. Probably going to try and do some intentionally low effort posts tomorrow morning (was studying for an exam and working past two days) to get back into posting more. I think it helps me to acknowledge to myself that a post is going to be low effort to be comfortable with it and also letting others know that’s my goal.
What’s a harness? Are you talking about AI here? I haven’t kept up with the discussion in this thread and, uhh, ctrl+f has not been helpful. Since, I guess, discourse loads parts at a time of the thread, so I couldn’t see how it was used earlier that easily.
Small thing, but the block quote is already a quote adding those quotes around it makes it seem like in Elliots article that was something he wrote in quotes.
? So idk if there was any prior discussion on this (I assume not much since the topic is primarily AI and stuff in this thread), but wdym by if staffed by same people? From the essay:
It’s because companies are run by people, and lots of people are bad. Changing the government won’t make company leaders into good people. That requires education.
A lot of people are bad. What would make the government effective at policing these companies if they are bad? Is that what you’re asking? Since their both bad? I don’t think Elliot is claiming that policing will be effective. Regular stuff that is already being policed is not really that effective from what I know. I think part of the point of the essay is that big companies are bad and that’s it. When speaking to an average freedom minded person they think that big companies are good and that a big serious issue is getting the government away from them so they can do whatever they want. Elliots essay aims to point out that is not true. Big corporations suck. A freedom minded person should want government to be more involved with corporations. In the correct way of course i.e. policing fraud not regulating the size of their drinks (idk).
afaik that kind of thing is already happening (idk if you were claiming it isn’t or of it is)
you later said:
i think you are. minarchy usually refers to the scope of what the government covers(?)/does(?). the usual minarchist idea is that “all” the government does is policing/protecting kind of stuff. so they protect you from violence, fraud, and other rights violations but it doesn’t step into, let’s say, do stuff like taxes to reduce smoking, or give welfare or subsidies. The government just protects. I think a government that just protects can be in line with creative judges.
thanks, the minarchy point makes sense. i was conflating scope with funding/robustness.
but i still think the first issue needs clarification. eternity’s reading seems to weaken the essay into a normative claim about what a freedom-minded person should want. the essay itself looks stronger than that. it says “part of a capitalist society would be a better, more effective government”, says “our government’s failure to police fraud well is a huge deviation from capitalism which a free market society would fix”, and says companies would be policed “more vigorously” than today.
so is Elliot’s current position that the essay was only making a normative point, not a functional one? if yes, does he retract those stronger claims? if no, what is the mechanism by which government enforcement works better even though the essay also says the people in government and companies are similar and not significantly better people?
completing above:
that’s an interesting distinction. ideas represented as tokens vs ideas represented as actual data structures with functions that can apply one idea as a criticism of another.
but i think substrate independence cuts against this as well. we don’t know what the functional relationships between neurons and ideas actually are, and tokens could be doing something similar.
also evolution creates knowledge without anything like conscious word-idea correspondence, and deutsch counts it as genuine knowledge creation because the error correction mechanism is real regardless of substrate. so the question for LLMs isn’t whether there are ideas behind the tokens, it’s whether there’s a real error correction process. for systems trained with ongoing feedback from reality, like RLHF, is that obviously absent? or is it optimization against a fixed reward signal rather than genuine error correction through criticism?
We have Popperian explanations and arguments that give us broad guidance on what could or could not possibly work. Evolutionary processes can create knowledge. Non-evolutionary processes can’t. If there isn’t replication with variation and selection, then there isn’t knowledge creation.
I don’t think it’s plausible that LLM tokens are doing replication with variation and selection because we know what algorithms LLMs are programmed with.
RLHF happens during training but not when the AI is responding to your prompts. So even if RLHF involves error correction and knowledge creation, that wouldn’t mean the AI can create knowledge when you chat with it.
Do you know a way to do that? How do you evaluate similarity? And how do you determine a breakpoint?
You can get a full topic on one page for searching:
I think we could do much more effective policing of companies without people getting better or more rational, if we just wanted to and most people agreed on that goal. Like if 90% of people found my essay persuasive and agreed about most of the examples I’ve gathered over the years, then I think the existing justice system is capable of doing more. If it was a priority, they could catch more cases and have more serious consequences when companies are caught.
I think the main blocker right now is political opposition not rationality or skill. One reason for the opposition is pro-company biases from Republicans and libertarians, some of which I think are connected with Milton Friedman. A second reason for the opposition is that a lot of anti-company type people, who want to reign in companies, go too far in various ways, wanting policing and other policies which are not permitted by classical liberalism, libertarianism and minarchy. And they do things like use anti-trust law to try to police companies, which offends libertarians who think anti-trust is bad on principle.
One of my other relevant thoughts is that policing companies better (preventing them from aggressing) is a higher priority than shrinking government and lowering taxes. I think most (non-anarachist) libertarian types prioritize the other way and I think they’re making a mistake and not taking aggression by companies seriously enough as an important, widespread problem which it its the government’s job to stop. (Anarchists should care about defense too but there are various complications with them that I’m not going to discuss now.)
My text says:
Just reforming government wouldn’t fix society because it wouldn’t fix the bad people problem… The underlying issue is irrationality…
It’s not primarily the government’s fault that many companies are bad. It’s not just due to having a mixed economy or government interference in the economy. It’s because companies are run by people, and lots of people are bad. Changing the government won’t make company leaders into good people. That requires education.
You can make and enforce laws without first figuring out how to educate everyone to be saints or geniuses. Turning CEOs into good people, morally and rationally, is a different issue than reducing the amount of fraud in the economy. It’s not a precondition.
The government doesn’t need to be better people to enforce laws more/differently, similarly to how CEOs don’t need to be better people to stop committing egregious fraud. CEOs will mostly stop if there’s enough law enforcement. The government can enforce these laws better if most people inside and outside of government want them to and care (wrinkle: it could be hard if most people want it but most very rich people oppose it).
A minarchy is minimal in the sense of doing what’s needed to stop the initiation of force but not doing extra things like education, healthcare, and social security. If being large is necessary in order to have a free market, then large could be minimal. Minarchy means the government handles a minimal set of jobs and generally tries to be small but it doesn’t have an actual size requirement. If we colonize a trillion planets then we could have a much larger government that’s still a minarchy.
Also, I just said it takes some ongoing creativity not perfect laws that you just write once and never touch. That doesn’t mean constant adaptation. Once we got things under control and companies got used to participating in the market without aggressing, and courts and police got set up for that and used to it, then I think a much smaller government could maintain things and make some adjustments sometimes.
thanks for clarifying both points.
this shows harness was overstating the issues. on the second one, i was misunderstanding minarchy by treating it like a size constraint rather than a scope constraint. on the first one, i was reading the essay as if education had to be a precondition for better enforcement. your answer is that reducing fraud doesn’t require first making either government officials or CEOs into better people, it just needs enough support and priority behind enforcement.
this supports your earlier point about LLMs pretty well. the harness was useful for surfacing candidate tensions, but it didn’t understand the conceptual structure well enough to classify them correctly. it was seeing contradictions where there weren’t any, and it needed human judgment to sort that out.
1 Like
i think there may be two different claims here.
one is that a frozen deployed LLM doesn’t create knowledge while responding to prompts. i think i understand that claim.
the other is that the broader training process also isn’t a knowledge-creating evolutionary process. that part still isn’t clear to me. modern training seems to involve variation, selection, and feedback over many candidate behaviors/policies/weights, even if it’s not conjecture and refutation in the strong deutschian sense.
so are you claiming:
- no knowledge creation happens at chat time, but some could happen during training, or
- no knowledge creation happens anywhere in the LLM pipeline?
I was talking about chat time.
The parts where a human is involved certainly have some potential to create knowledge since evolution and error correction happens in human brains there. What’s going on there is a more complex analysis. RLHF is one example of where human thinking is involved, and there are more like designing and coding the LLM in the first place, gathering and curating training data, and writing the system prompt and harness.
To be an AGI, the LLM has to be able to create knowledge by itself, with no human creativity involved anymore while it runs, only while creating it. Humans writing chat prompts can actually create knowledge and give it to the LLM too, but if we use limited prompts that aren’t very helpful like “do philosophy by yourself and share the results” we should generally be ok.
i can see how idea, goal, context triples help turn a lot of vague degree questions into clearer pass/fail ones. but it still looks to me like the hard part is the same one as before: deciding whether a criticism really refutes the IGC in that context, or whether the IGC already addresses it using existing knowledge.
so is CF giving a fundamentally new way to compute that judgment, or is it still C&R all the way down, with the additions being mainly organizational tools that help structure the judgment better?
stockfish seems like a useful analogy here. it doesn’t create new knowledge by itself once deployed either, but a lot of human knowledge goes into designing it, and the resulting system still embodies that knowledge in a usable form.
so is your view something like that for current LLMs too? not creators of new knowledge at chat time, but systems that can embody and apply knowledge created upstream by humans during training/design?
CF has, IMO, major conceptual changes to CR (not just organizational tools) like:
- using decisive arguments and binary evaluations
- evaluating ideas for goals instead of in isolation
- breakpoints and excess capacity
I think this is substantial progress in epistemology. But thinking is still mostly CR and coding AGI is still hard with these changes.
yes