Basically like mandating the electric chair have sufficient lumbar support

The Epicurean paradox.

And yes, you can train out biases. Rather easily, actually. You don’t even have to adjust weights. You post-train through rules files.

The assertion is not about whether ‘you’ can do it. It is that the model cannot do this itself.

A human recognizing a bias in the model and correcting it is not evidence robot intelligence, it is further evidence for human intelligence.

Training is absolute, it determines the set of relationships the model can represent between input and output. Context, rules files, system prompts, etc. operate only on the conditional probabilities within this set and are impermanent. This is ‘priming’ in human psychology, and why a person in a herpetology class will talk about Pythonic features differently than in a coding class.

As an example of how this is not real learning and is inadequate to solve real problems, here is a paper where the authors challenged multiple models to solve unfamiliar games where they need to discover the rules through play and feedback. Even the ‘best’ model performed pitifully compared to humans.

There’s some semantics here.

Large neural networks have been shown to build parallel models of the world that they learn through their training data. A chess model trained only on chess move notation does have an interior board and state that it tracks, despite never being told about a board, the number of pieces, how they move, etc.

LLMs do something similar with language, mapping tokens to ideas in ways that can produce meaningful and grammatically correct output without needing to be formally taught any of this.

Our brains also do things like this - we observe X and then try to intuit Y, the truth of the world and reason for X.

Where this breaks down is humans can then take our guess for Y and propose Z, and try to find it out in the world. If we can’t, then Y is probably wrong, and our new understanding of the world is Y* that is consistent both with X and the absence of Z.

I cannot stress enough how far we are from machines that can do this. They can learn to approximate Y through training, but they have no means to extend that to propose an unobserved Z, no means to update their internal state given the results of that observation (without fully retraining the model on new data - which makes it a new model), and no way to question X given Z - they can talk at great length about data biases but can literally never excise bad data from their training weights. There have been experiments with training models on time-boxed datasets (e.g. stuff up to AD 1880) and asking it about modern concepts - predictably hilarious results.

All the current ‘AI’ stuff is smoke and mirrors around this very hard problem. The machines can talk, but they can’t think (Z) and can’t learn (Y*).

‘oh no the terminators are coming’

‘oh shit, how many?’

‘Three, skynet couldn’t afford any more RAM’

Doing something yourself always teaches you better than watching it, or just being told the answer.

A neat result from pedagogical studies is that being given the worked solutions to new problems to study from is much more effective for learning the material than simply practicing.

Students who were assigned practice problems performed worse on later tests than those given the same problems and all of the worked-out solutions for them. The explanation from the research was - when learning a new topic, immediate practice isn’t beneficial because the student doesn’t yet know what to practice. If they are lost when trying the problem, they can easily end up practicing the wrong skills by not knowing where to start. Those with the solutions have some of the cognitive burden removed, and are better able to internalize the task and generalize it to later problems.

I completely agree with you on LLMs and I think they’ll be a disaster for education. But I’ve always found it interesting that being given the answer can be useful for learning in certain circumstances.

50B is annualized revenue, accrued revenue will be almost certainly be quite a bit lower this year.

Tech companies love reporting ARR because it is always bigger than GAAP revenue (if the business is growing at all).

When peasants were able to copy ‘intellectual property’ for personal benefit and enjoyment, it was a massive crime. When the oligarchs can steal - everything - for the slight chance of paying less in wages, it is an innovation.

I think it is pretty much certain the model companies escape any penalty for their mass theft of IP. There will be some settlements and token penalties that sound big on paper, but are nothing compared to the market valuations.