What works for me is to delegate the time consuming things I don’t like to do so I can focus more on the coding.

I’m not saying that rich people have never had an advantage. I’m saying that the only thing that gives lower income people a little bit of extra advantage is completely lost.

It doesn’t matter if you’re the best programmer in the world. You’re no match to what the rich brat can output.

Future dystopia: investing in your own skills is no longer valuable. That kid with rich parents will always be ahead of you.

Sometimes you’re not even allowed to even suggest that AI might be a thing that exists

It’s healthy to change mind in light of new evidence. AI has improved since GPT 3 days - in particular in coding tasks.

“This is a BIG merge window,” he wrote in his weekly state of the kernel update, as it contains almost 20,000 commits compared to the usual ~15,000 that developers submit for a new cut of the kernel.

Great to see so much momentum!

Anti-intellectuals think science is like in the movies: write gibberish on a chalkboard and time travel is suddenly invented. “Who needs experiments? My equation is correct!”

Wake me up when their AI can propose and conduct novel scientific experiments.

Maths is an “easy” target for AI because it’s self contained. It’s easier to brute force training data in mathematics when there’s no external forces.

Can’t have that cheat code in science, unless they let their models study the behavior of real ant colonies as part of their reinforcement learning process.

Future dystopia: AI vendors may refuse to serve certain individuals, which will make them unemployable because no company wants to hire people who are locked out of AI access.

I think it’s unlikely mathematicians happened to have 300+ almost finished solutions to long standing problems, and then that pesky AI snatched them all just in the last moment.

So far!

I doubt models are “topping out “. It feels like no prediction about the supposed “fundamental limits” of AI models have held.

Two years ago it seemed like LLMs were fundamentally incapable of doing maths. It’s just a next word predictor! Of course it can’t do maths!

Now it’s solving high profile maths problems left and right faster than any mathematician alive. Still a next word predictor.

There’s a difference between pre-training and reinforcement learning.

Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.

Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.

As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.

Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.

There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it.

That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.

They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.

Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).

People are different. People value different things. Some people are more susceptible to chatbot addiction, just like how some people are more susceptible to gambling addiction. It’s simple as that.

They’re going to use AI to install the AI.

Recursive Self Installation

Oh no! A person doesn’t have English as their first language. The HORROR!

It’s a losing battle.

Projects that seem to take a clear anti-AI stance today can change their stance in the future. Linus went from being kinda AI-cautious to AI-pilled in the span of a year. I’ve seen the same with today AI-pilled coworkers who just one year ago were AI-skeptic.

An operating system also consists of many different parts made by different people, each having their own stance with regards to AI use. It’s impossible to keep a track how all of these projects change their stance over time.