a way of eliminating the KV cache and using a residual vector instead. A home-researcher on YT had a video about using a mac studio and I think got a deepseek model with large context window from like 4 tok/s to like 20 tok/s + didn’t need to use 10s of GB of memory for the KV cache. (I can try finding the video again if anyone is interested)
Edit: for the second one it might not work well with batching, which means good for home users but not useful for providers. Or I might be mixing up a 3rd similar technique thing I heard elsewhere that was also about KV cache efficiency stuff, can’t quite remember.
I think, pretty soon, people will start thinking about an AI’s maximum Level of Indirection. I haven’t heard anyone discuss this, but it occurs to me that it’s (maybe) responsible for some of the leaps and bounds we’ve been seeing including mythos’s ability to break out of sandboxes (which can presumably be something it decides to pursue on its own if it gets stuck).
By indirection I mean the same thing as CF when it comes to goals and indirect goals.
Higher max level of indirection means keeping more of the current ‘stack’ of goals in mind, and allows it to go off autonomously and do complex stuff in order to continue with the main goal. For example, if you ask it to fix your microphone on linux, it might end up cloning pulseaudio and fixing a bug, or fixing a kernel driver bug in some kernel extension and setting it up for you.
This is interesting. is maximum level of indirection the same thing agent researchers are calling planning depth or task horizon, or are you pointing at something different?
and do you think frontier models actually have this built in somewhere internal, or is the apparent depth mostly coming from prompt chains, tool loops, the agent harness and other scaffolding keeping the goal stack for it?
Maybe it’s the same thing as planning depth, but time horizon is different. Time horizon would be improved by better planning depth and/or max indirection.
But it’s also different from ‘deep planning’ (like deep research) – this isn’t about task decomposition necessarily.
I’m not familiar with what ‘planning depth’ means is outside of explicit planning/task decomp so maybe they are the same thing.
By max level of indirection I mean the kind of thing where a model can ‘see’ a number of steps into a problem that let’s it link nonobvious things or know to go in an unintuitive direction. Before it prints out ‘oh I could try XYZ’, where does the XYZ come from? The context plays into it but isn’t sufficient to explain where the idea to link XYZ comes from.
Yeah I think there is something internal going on. There definitely is some apparent depth from the context (prompts and tools), but it’s also not about context size itself and seems to be somewhat independent of how much is in context.
My guess is that the residual vector (being in a super high dimensional space) contains a lot of like ‘raw information’ (below the level of tokens) that includes stuff about the stack above it (larger goals, position in goal stack, etc) but also about the epistemic context around it (kind of having an ‘awareness’ of many different facts and explanations at the same time and being able to find/detect relationships between them).
Pseudo-forks prediction – open source projects, especially complex AI-coded ones like Codex, will have forks, but it will end up being easier to maintain them as pseudo-forks rather than true forks. Typically with a forked project, maintainers might go to some effort to keep the codebases compatible enough that patches can be used on both, and refactoring / merge conflicts are minimal and easy to fix. With AI-coded stuff, I think it will be easier to: fork the project, implement the new fork-features, and then when updates land upstream, get an agent to build a spec for those changes, then re-implement them in the fork (with knowledge of the upstream changes). This will turn out to be much quicker and easier than trying to rebase or merge in changes.
This video reminded me of something I’ve been thinking about the past two weeks. Mo mentions not being sure if there’s a limit to how good LLMs et al can get.
Disclaimer: I don’t know how original these ideas are; there’s obviously some overlap with CF/CR/BoI ideas about AGI.
The idea is that an enduring Turing-esq test is impossible. “Turing-esq” meaning an external test that reliably determines non-conscious from conscious and can include really hard benchmarks like ARC-AGI3 or whatetver. Enduring means that it works reliably and will keep working (so it’s just specific to detecting current gen LLMs like gptzero).
Corollary: LLM-esq AI can always be trained to imitate creative intelligence to an arbitrary degree.
There isn’t much of a takeaway from that besides that benchmark performance will keep getting better but we’ll still be bottlenecked by humans.
I’m not sure if AI companies are doing this, but they might be.
Anthropic and OpenAI will reset usage limits from time to time. For example, right now I am 90% through my claude week and was at a similar usage pct yesterday, but the new Fable 5 dropped and limits were reset. This is good for me if I use more than like 6% of my weekly quota in the next 12 hours.
However, sometimes quota resets and the period resets too. So instead of keeping my weekly reset date (june 10th, 22:00 in the screenshot), it resets at the same time to +7days (either from when it was reset, or from when I next use the subscription).
Now, sometimes I’m under quota at that point, like I have used x\% but we’re y\% through the week and x < y. If both quota and the timeframe are reset, that’s bad for me. I lose out overall.
So, if it’s bad for enough customers, resetting quota like this would reduce the provider’s overall compute liability. The subscriptions from the provider, all added up, cannot use as much compute as they could earlier, before the reset.
This reminded me of p-hacking data to get the result you want, except you’re like p-hacking your customers quota to reduce cost.
Next business model looks like: get coders hooked with personal accounts, make enterprise pay for it. This is the first time I think anthropic will just not include a flagship as part of their subscription package.
I think it’s unrealistic that AI companies are saving compute with usage resets. Most people who have low usage in the current period probably weren’t saving quota to use a lot towards the end of the period; most of them were probably not going to use it all up.
