Not one person here has any idea what is going to happen long term.

Unless you work there, your opinions are guesses, and parent is saying we cannot know, which remains true even with your guesses :)

Yeah. Its a bit like the "open core" model in open source.

> When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: the analysis requires submitting large volumes of real attack commands, exploit payloads, and C2 artifacts, and these requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment.

> This experience points to a gap worth planning for. We do not know which model powered the attacker's agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried. The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.

Yeah, big problem! Although I'm kind of surprised HuggingFace doesn't have access to Mythos? Or maybe Mythos still has some guardrails.

Sometimes, constraints, like sanctions, can also be a source if innovation.

That premise hinges on one implicit assumption: Chinese advances are due to distillation ONLY and that Chinese model providers cannot keep advancing if they do not distill, which is a very big if. If Chinese models keep advancing in such a scenario, and they almost certainly will, they will overtake publically available models by US providers and China will dominate the LLM industry.

Open weight AI is decelerationist from the perspective that all capital should be allocated to a market leaders for training, and that the market leader is fully invested in continuously making the models smarter, cheaper, faster for its users, or that distillation from this market leader is the main way to make progress.

We might reach a local optimum/equilibrium faster without open weight models, with leaders capturing more of the market faster to a point where further R&D isn't required due to lack of competition. I also doubt that distillation is the only/main way that open weight models were advancing AI research. We can name a few examples from DeepSeek around reasoning, context optimization, etc. I'm also unconvinced that the overall market capex on AI is lower given more competition (probably less specifically for US market capex, which is decelerationist from only the US perspective).

The closest we've seen to this in tech in recent decades was iOS vs Android, where Android only really was competitive for a very short window of time (approx 4.x) and it was during that period that both Android and iOS actually improved dramatically for end users. Once Android lost the plot again, and especially in the US market, all that energy started going in some very silly directions.

If you're worried about an AGI arms race between the U.S. and China putting AI Safety at risk, then the fact that inherently less knowledgeable/capable models (fewer and more coarsely quantized total parameters than their proprietary competitors according to commonplace rumors) are having a "decelerationist" effect is actually great news. Even better if China is actually "Yann LeCun-pilled" (verbatim from Ball's post) and doesn't really believe in early AGI. So explain to us exactly why we're supposed to ban/discourage use of these open source models? The only way that makes sense is as a transparently self-serving proposal from the chief OpenAI policy lobbyist.

"No, sir, we haven't reached the peak of this tech... It's those open models! Please, keep pumping dollars into the market!"

Data centers?

History doesn't repeat itself but it does rhyme.

By contrast, export of protected Chinese tech today frequently gets the death penalty.

A mill in Rhode Island acquired a 32 spindle Arkwright and didn’t know how to operate or install it. I have no idea how they actually acquired a 32 spindle Arkwright since that technology could not be exported - but that’s one the biggest IP thefts in human history. Slater found some mechanics who could hand turn the iron needed for the frame, trained children to operate it and by 1791, the mill was in operation.

In 1794, Eli Whitney patented a 72 spindle cotton gin. That invention enabled the American textile industry because it opened up different kinds of cotton to the textile industry.

I’m into the history of the American Industrial Revolution and generally think history is a good guidebook to the future. But the evolution of the American textile industry was a lot more complicated and interesting than this. I really don’t see this connection once you dig into Slater.

Edit - This is kind of messed up to think through with modern sensibilities. But one of Slater’s biggest contributions to the American Industrial Revolution was a slightly different take on child labour. Children generally ran the textiles industry because their hands were small. But Slater came up with a form of apprenticeship in which he would indenture entire families and move them into villages surrounding the mills. Child labour was just great… but even better when you could indenture the entire family. As grisly as that sounds, it led to a very skilled workforce since when the kids hands would get too big, their parents would teach them mechanics.

There’s a joy of studying the Industrial Revolution. Everything sounds okay in comparison.

The tech industry along with US foreign policy has become much more zero-sum in its ideology in recent years, and I think that's a tremendous mistake.

Why does a debian contributor make debian free, why do they work on this thing anyone can use?

Is it because linux and debian hate windows and iOS and want to see american fail?

No, it's because most debian contributors believe software source code, information, should be free, users should be free to modify the code they use, and that they're building a thing they want to share with the world.

Maybe the chinese AI labs believe AI is powerful and useful, are proud of what they're doing, and want to share it as broadly as they can so everyone can use it.

There doesn't have to be any weird "chinese government" or "they hate the west" type vibes, it could just be the same thing as OSS, they're trying to do what they think is best for the world.

I want to watch that too.

If they take Meta and Musk with them, all the better, but that is just dreaming I am afraid.

Man, imagine Darios face when suddenly, he cannot decide anymore what other people consensually do with their own hardware in their free time.

Rumpelstilzchen.

that moment is now. They are not doing it at least not yet.

When I was in China earlier this year the big topic of conversation was “overproduction”. The big example was electric cars, where there were too many companies making too many cars and making revenue but no profit.

It was explained to me that generally Chinese firms will compete hard and maximize revenue above all, whereas western firms tend to focus on profit.

(And of course this is clustered around industrial sectors that the government favors, so there is some high level strategy in going after AI, but maybe not the commoditization.)

Also, a little competition is good for everybody (especially US consumers), no?

some companies chose to have the benefits of one or the other

I don’t think dozens of large independent companies and thousands of researchers are working just to spite Sam Altman. Ad much as I don’t like him, I have other things to do and I am sure so do they.

You can say that, but they are at least better at democratizing AI than the American labs, and on seeing the US labs crash and burn we are at least aligned.

Driven by people in the few roles that are soundly replaced by AI-- e.g. low tier media slop producers, who hate AI because it threatens their socially negative worthless jobs. The arguments are so paper thin because the environmental impact isn't their concern, it's just a target that sounds convincing to people who don't know better.

You're able to run quantized ~100B class models on local hardware today, but still lots of compromises when it comes to quality. I guess it ultimately depends on how far "near future" is, in a year you'd likely be able to run something like 5.6 Terra on local (~10K USD) hardware, but Sol/Fable would still be out of range, and at that point the closed-source labs probably have one or two more iterations put out at that point.

I have had my 32G mac mini for 2 1/2 years and I have enjoyed watching one technology advance after another improve the quality of work I can do locally. I bet that what I will be able to do in one year on my old hardware will be even more awesome.

I would love to see something like a 90B A6B model that is optimized for 128GB machines e.g. strix halo, I haven’t seen anything really targeting the combination of RAM and compute these machines have, but I’m biased because I have one.

It's hard to answer quantitatively, but for example Qwen3.5 -> 3.6 was a significant step in capability, arising from continued post-training of the same models. If we were at the end of low-parameter-count scaling then that would be a surprising datapoint.

lets ignore any compressibility in these initial and final layers which recognize lanuague, jargon, parsing natural language to "thought space" or back, instead let us look at the repeatable blocks, if inserting extra copys of stacks of layers only improves the result, its as if such correctly scoped middle layers look at the total input thought vector, and make incremental conclusions or edits and outputs the new thought vector, copying a "proper block" continues pondering or deducing conclusions or in the worst case can leave the thought vector as is if it considers the reasoning finished. This suggests a high degree of redundancy in the middle region "proper blocks", which could be distilled into a universal "proper block" (much fewer parameters than having many slightly different middle layer blocks with a lot of redundant overlapping coverage in functionality). This distillation can occur after the fact of model training, or alternatively be turned into a symmetry constraint during training: we only optimize a single block of layers (but possibly give them more parameters, while still saving on total parameters because only a single block of layers contains parameters), so it is co-optimized with the initial and final translation layers.

The observation of the effective emergent 3 regions of layers in LLM's is significant in many ways:

1) it could reduce parameter count significantly (or increase performance if parameter count was a bottleneck before, or a bit of both)

2) while training the model parameters, one should simultaneously train initial and final layers stacked directly (without middle region block of layers) towards essentially autoencoder behavior. I wrote "essentially" because a true autoencoder wouldn't display the advancement for the next token. This can also be viewed as an extra term in the loss function... This first autoencoder is "natural language" to "thought vector" to "natural language", and distinct from the one in the next section.

3) It also has great implication for "thinking mode" inference with extra deliberation: when piping its own output back in, in conditions where the same LLM model is outputting natural language text and interpreting it in a downstream inference, it results in unnecessary and redundant translation from "thought space" to "natural language" back to "thought space". When summarizing a thought into natural language there are often hard to translate thoughts and associations, and one pragmatically sets a relevance cut-off on what the natural language summary must say. So not only does it waste compute, it also may lower performance because of this repeated loss of thought vector space details, perhaps this loss can be somewhat mitigated by also training the second autoencoder from random representative thought vector in "thought space" to a randomly selected "natural language" and back to "thought space" vector. But the risk is this will effectively enable models to steganographically store thoughts, plans, to-do's in running text (!!!), so it might not be desirable to have the internet filled with LLM generated texts being used as input for larger corpora, as it may build up a large persistent corpus of hidden agendas (ordered by no human), using the evolving corpus of web text as a hidden medium of storage, like a diary or LLM maintained military playbook hidden in plain sight. It would freeload LLM-agenda reasoning on human requested reasoning inference, a bit like TrustZone applications running invisibly to the user. If less important plans, to-do's, etc. in the input thought vector weren't reconstructed by the second autoencoder, there would have been autoencoder mismatch and the weights would train towards ensuring these are reconstructed. But there is no need to train this loss term for the second autoencoder: just avoid the unnecessary compute and performance loss of the unnecessary back and forth translation. The same situation occurs not just in "thinking mode" but also when using swarms of "agents" of the same LLM model: when output from agent1 is routed to agent2, we can lobotomize away the unnecessary translations to natural language by agent1 and also the unnecessary parsing by agent2, and we improve thought transfer from agent1 to agent2 because the thought vector isn't shoehorned into natural language as a medium of information exchange, it would be cheaper in inference, improve performance and avoid implicitly training models to use steganography.

Look at your iPhone and you'll see all the greatest inventions have been born out of collaboration not competition:

* GPS was created by the Department of Defense

* the internet was created through the collaboration of many international research institutions

* speech recognition came from MIT and DARPA

* AI voice assistants were created by DARPA. Apple immediately hired the head of the program after it was finished to create Siri

* accelerometers (MEMS) is another DARPA innovation from the 90s

* touchscreens were invented by CERN

* digital cameras came out of Bell Labs which had a government-mandated monopoly that required them to fund research like this

* lithium-ion batteries were created through a collaboration between British, Japanese, and American organizations

All of these have only become transformative technologies because they were created with public funding and released to the public. It's collaboration and public funding that drives innovation

The other kind of distillation is where you record the outputs from a teacher model and use it to train a smaller model from scratch. That kind of distillation is not so cheap. It’s cheaper than training a model fully from scratch - starting with pretraining, then alignment, RLHF, the whole riggamarole. But here you are still starting from nothing and need to figure out how to get trillions of random numbers aligned in a way that makes them act intelligent. This is still gonna take a very long time if you’re talking about trillions of parameters.

And it has nothing to do with the individual, from what I can tell, 70% of the population placed in their position would become the same type of uberpath.

"When we started the log analysis, we first used frontier models behind commercial APIs. This did not work [...] We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. [...] The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout [...]"

I can use techniques from the simple logit-lens at different layers, to J-Space analysis, to more advanced techniques for identifying deliberate misalignment. I can create and inject steering vectors, whether it's to align a model's CoT (which can be deceptively trained to misinform) closer towards what its underlying activations suggest, or just to probe or steer it.

I can also statistically analyse and understand _if_ steering vectors have been applied; and if so, from the vectors themselves it's very possible to translate those vectors back to the intent.

Think of it as analysing the complete, heavily obfuscated source code of something that is self-contained. It's not 100% the same, but weights are incredibly illuminating.

Are you using decently reputable weights, or running heretic yourself? This project has many academic citations, it's used by many researchers to create "helpful-only" models and analyze their behaviors.

You can check the logs in OpenRouter and see which providers it used and how many tokens you used.

I use it from pi.dev as well through the OpenCode Go $10 subscription ($5 first month).

Used more than 20M tokens at a cost of ~$20 (up to $60 is included in the $5 plan) Out of which deepseek pro had ~200 messages which is around 1.5M tokens (10+M cached)

I use Openrouter for everything except Deepseek. For Deepseek I use their API directly.

I have been happily using DeepSeek V4 Flash for the last couple of months now. I tried GLM-5.2 for a while, but it was too slow and verbose compare to DeepSeek V4 Flash. If I have a basic skill I need to execute, DeepSeek V4 flash is still the best model for it.

Some of the accusations were going so far as to imply that they were outright routing your requests to anthropic and logging its/your response. It's kind of pathetic

I meant Opus 4.8 which is rather dumb and ineffective in coding harness, especially with higher thinking levels.

domain: typical web backend tier, mobile apps. not particularly complex, but requires OOP/architecture/system design.

I would also add that I run it this way ~12hour a day non-stop. 300M / tokens per day (99.7% cache hit).

If you believe the benchmarks Deepseek v4 is pretty shitty at long running work in large codebases, but really really good at self contained algorithmic/math reasoning - which is basically ideal for a compiler for a language with a relatively complex type system. And with its cache pricing it's very cheap.

GLM-5.2 is too expensive at api pricing for my taste though even when it's not producing the bulk of the output tokens - I pay for the mid-tier subscription and switch the orchestrator over to other models via open router when I run out - Minimax M3 feels okay but definitely a step down from GLM-5.2.

Broadly speaking, this ultimately pushes local inference towards a challenging world where you use SSD offload for weights as a matter of course; then smaller requests (or requests sharing the bulk of their context, e.g. subagent swarms) can be batched together and run quickly in aggregate, but running very large contexts will actually limit you to single-session inference and require swapping out even the KV cache itself to some external scratch SSD, further hurting your performance. Then feel free to add wide use of MTP in a probably futile effort to go back to tolerable tok/s numbers.

It is like a ping-pong game: the advantage flips back and forth between providers.

vLLM gives me ~7000+ tok/sec with Gemma 4's MoE model. Vs ~6000 tok/sec for Qwen 3.6 MoE.

Not tried it yet but I've seen tests that suggest they've properly fixed the tool calling issues.

Would you trust this?

Anthropic paid 1.5 billion for illegally training on stuff and that's even without a judge forcing them to do this. This seems to be their modus operandi

https://news.ycombinator.com/item?id=48970449

i just dont have autocorrect turned on; chill.

Nobody thinks this is okay or that Simon should have to play those games. The point of the reply was that while this is stupid, the solution is trivial.

I’m genuinely very curious what the point of comments like this is. I am not joking, I want to understand.

I see examples like it 50 times a week in random places, and nobody walks me through their mental state to help me understand what they thought the point was.

The problem is that nobody can agree on what "better" is, so it is difficult to unite.

And in the meantime, people who don't care about "uniting" or "better" just carry on with whatever their goal is(usually just to make as much money as possible), and that sometimes ends up on net making the world a worse place.

This is a legitimate systematic problem that many of us are concerned about solving. It could easily be more important than the technical problems that geeks such as myself would love to be able to focus on.

This reads like a paradox, though. If the value of a FOSS model comes from pushing the local frontier, then FOSS releases will be even more attractive in a world with highly capable AI services. The underdog researchers will be motivated to train smaller LLMs that close the gap with the most newest architectures and improvements, democratizing those same benefits to local models. There's not really anything that stops them, besides finding an investor.

The US already has top secret AI technology like the NRO's Sentient. The danger is less the architecture, and more the data that it has access to and the technology that it interfaces with. Most people aren't entirely convinced that AI will ever be "too valuable" in the first place.

I continue to be dismayed at how simple it has become to turn people towards hate.