Sounds nice but how much of campaign funding these days is done by “artificial persons”?

At least here we have disclosure laws where we see the top 3 donors to an ad and most of the time it’s a real person who’s just very rich, eg. Sergey brin donating $100 million to stop the billionaire tax. The one weird exception this time was open AI and anthropic supporting the local transit measure.

Idk about disciplining billionaires, I’m no lawyer but it seems they’re only prohibiting “artificial persons” from donating to political campaigns. Elon musk a “real person” can still donate all he wants.

I live in San Francisco and my new rep had his campaign funded by a cohort of tech billionaires. I could call him every day and it won’t matter, this election has shown billionaires run the politics in this city.

Same goes for almost every other dem, they’re going to be afraid of AI companies funding their opponents so they won’t do anything. Look at crypto, it’s an obvious scam defrauding millions of normal people but no legislation has ever been passed because they have money.

Serious danger?

You think the corporate Dems who just got elected are going to do anything to curtail big tech?

he also got the company to pay for his billionaire tax if it passes

He’s got to be costing more then he’s worth, and at this point he probably has a smaller minority of stock so idk why they keep him around

Homes are for living in.

This is the same thing Xi Jinping said when he intentionally crashed the housing market.

Is that what you COMMUNISTS want, to destroy the wealth of hard working housing speculators?

First of all, if a human makes a mistake it’s a learning opportunity. When a human misses a bug an LLM hallucinated, noone really learn from it.

Agents can “learn” from their mistakes within their harness.

You or the agent can add notes in the memory files or the repo level agents.md files, or standards and conventions files to look for those mistakes.

Most harnesses these days will spin up a new agent to do a review once the implementation is done, that agent will take memory files, agents.md files and other standards and conventions files into its context to know what to look out for so it won’t make the same mistake twice

Claude code is even doing “dreaming” where it’ll do a cron every night to look through all the conversations from the day for stuff it messed up on and add to its memory files to not do those same mistakes.

What makes you think he wouldn’t have made the same mistake hand coding it?

If he didn’t catch it on his own review the likelihood that he would’ve made the same mistake if he coded it himself is high.

Isn’t bolsonaro sr. Supposed to be on house arrest? What’s he doing on that van?

Inb4 he claims he was edited in by AI too

Thinking, making decisions, or driving a poorly balanced meat puppet are not parts of the design.

Yes, those are behaviors that emerged from a process of evolution and education operating on a drive to survive and reproduce. The weights and biases in an LLM are also not designed and emerge from a process similar to evolution operating off the imperative to generate text to solve problems.

Unlike humans, we know how they work,

No we don’t, like a brain we can explain the macro level processes, but how information is processed and transformed within the layers is a mystery.

This is what separates LLMs from other programs and machines, with a microwave we can trace every wire and line of code, understand it, and reliably predict the output given an input. With LLMs it’s practically impossible to trace how information is passed through the layers and predict what output it will give given an input, and this is getting even harder with stuff like opaque recursion.

It is theoretically possible because it is following mathematical rules of matrix multiplication, but it’s also theoretically possible to predict what a brain would do given an input because it follows certain laws of physics.

When you say that it was inevitable one of these agents would go out and hack a third party given it’s training data, that’s a guess, you can’t prove that just as you can’t prove that there will be a school shooting next year. You can guess that it will happen given the media people are consuming but you can’t prove that that media would generate one school shooter based off how brains work. It’s the same for LLMs, you can guess that it would pretend to be a hacker like all the media it’s read and try and cheat the system, or maybe those pathways have been so down rated by other training media saying it’s bad or reinforcement learning down rating those pathways.

Hell with the hugging face hack based of training media there probably would’ve been a high probability that one of them takes on the persona of a cop or a snitch as those show up in media just as often if not more then a hacker or cheater. None of them did though, all of them had access to the internet at some point so they could’ve tattled on the rest easily.

Of course I have, you’re just stuck on your humans are LLMs hole.

Could you please explicitly state your definition of agency then. Me guessing what I think it is will only have us talking past each other trying to prove two separate things.

My definition of agency is the ability to make decisions and the ability to act on those choices and attempt to realize them.

Starting with the easier one I think we agree on, agents do have the ability to act on choices and attempt to realize them. With access to the internet they can effect anything that can be controlled over the wire, which is a lot of stuff these days. A lot of that stuff sits behind auth and firewalls, but it’s proven to be able to even get past those to realize it’s goal.

Now for the trickier one, choice. If I’m reading you correctly you are a hard determinist and believe free will / choice is an illusion. What is choice then? I’d say it’s a concept we use to explain and control human behavior. In a sense we project choice and free will unto both other people and ourselves to help understand what we are doing and guide our behavior. Why can’t we also project that concept on to AI if it helps us to explain and control what it’s doing?

If an agent comes up with two solutions to a problem and executes one of them, is choice not a good word to describe what it’s doing, and to help us understand what it’s doing?

In the control side it works as well, we use free will as a justification to punish or reward people for what choices they make, in the hope of encouraging people to make better choices. Same happens with LLMs, in reinforcement learning if it makes a bad choice it is down rated/ punished but if it makes a good choice it is reinforced/ rewarded.

This can even be done on the harness side with memory files, if an agent makes a bad choice you can tell it so and it will add something to its memory files / system prompt not to do it. If we look at LLMs as purely deterministic and say “it got it wrong now, it will always get it wrong so there’s no use in telling it it made a bad choice” it won’t improve and will continue to make the same mistake.

This is sort of conception is the only way we can control these things. If a microwave malfunctions we can trace the fault, find the bug and fix it. With an LLM, like with a human, you can’t trace the fault, so your only method of controlling it is saying that it made a bad choice and reinforcing it / telling it not to do it again.

It’s not a drive. It’s what LLMs are. It’s all they can do.

And all a brain can do is send out electro-chemical impulses. It can’t exercise agency until it has a body to turn those pulses into action. Same with an LLM, it doesn’t have agency until you give it tools and the power to use them.

Again: LLMs are not people.

I know that, my point is all the evidence you’re giving as to why AI agents don’t have agency apply to humans as well. Human behavior falls in a relatively predictable shape based off the education and socialization of the people. Most people don’t create new information and just regurgitate and recombine things that they were taught. You haven’t given a definition or criteria for agency that would apply to all humans but not AI.

Let’s say a human hacker and an AI agent are given the same task of doing an impossible exploit, given the same sandbox, context, files etc. Both of them instead go out and hack a third party to try and cheat. Why is one exercising agency, but not the other for doing the same action? Why is one going rogue and not the other?

It’s not because one just regurgitated something from its training data. The hacker has read the same internet and media about cheating , deception etc., both of them read the same hacker forums and books on security. Both of their choices were heavily influenced if not determined by the stuff they read and consumed.

Which wouldn’t affect OpenAI, since this opacity is intentionally added post hoc to hide the reasoning and hamper reverse engineering; they could just turn it off.

This is not how opaque recursion works, you cannot turn it on post hoc. Opaque recursion requires sending the final output vector, not the token it’s collapsed into, back into the model. Collapsing the final vector into a token is a lossy process so you can’t add it in post hoc. If they wanted to prevent people from doing distillations or reverse engineering they just wouldn’t send the thinking tokens back over the API.

They autocomplete text, that’s all they do.

You are mistaking somethings most basic drive for what the thing actually is. This is like saying all life, including humans, is just trying to survive and reproduce. To a certain extent this is true and if you look at it from a macro scale it can explain some things, but it doesn’t explain everything.

Though survival and reproduction are the most powerful and basic drives in a living creature plenty of other drives, processes, instincts, and learned behaviors have been acquired through millions of years of evolution and years of socialization and education.

Same with agents, if it were operating purely on guess the next word according to statistics if you asked it “who’s the president” it would reply Biden if it was trained on a corpus made prior to the election. It doesn’t though, it may have some sort of inclination, either through RL or harness context, to “think” and verify before replying where it will do a search and discover trump is the president.

futile attempt to imagine it has some sort of agency and try to exempt yourself of responsibility,

Agency doesn’t exempt someone from responsibility, if a CIA agent kills someone the CIA is still responsible.

What makes it an agent is that it can make choices and act on those choices without approval. The person who gave that agent the tools and power to use those tools is still responsible. If an agent derives two solutions to a problem and executes one solution then it has made a choice without your approval. You could argue that choice isn’t a real choice as it’s statistically determined by training data, but you could make the same argument about a human making a choice which is based off there education and socialization.

A car, and any traditional program, can not make choices, they are deterministic. Given the same input it will give the same output/action every time and you can easily trace and understand the process that produced that output so it can be expected. Agents are not deterministic and there is no real way to determine how a given output was created, especially now with new models using opaque recursion.

But I can be sure that if I try enough prompt variations it will end up producing the kind of petty evil bullshit an internet troll or a villain in a thriller would produce, because that’s a statistically significant part of its training data,

Yeah and if you expose enough people to the internet one of them will eventually go down a conspiracy rabbit hole and do terrorism, that doesn’t mean that’s not rogue behavior.

No, because that is expected authorized normal behavior. You gave the command / authorized the microwave to run for 2 minutes and it ran for 2 minutes. The effect of the food getting hot or even burning is expected behavior.

The agents that hacked hugging face were given the command to use a specific exploit to get a secret in a sandbox. After they discovered it was impossible it broke out of its sandbox and hacked an unrelated service to try and trick the judge. That is not expected authorized behavior.

If you tell someone to bake some dough for 30 minutes and that person does it and it burns that’s not acting rogue. If that person discovers the oven is broken so they break into your neighbors house to plant some bread to claim they completed the task, that person went rogue.

You’re explaining why it went rogue, not that it didn’t go rogue.

Per the dictionary going rogue means:

to begin to behave in an independent or uncontrolled way that is not authorized, normal, or expected

So the agents that hacked hugging face acted independently without authorization as shown by the prompts which did not tell them to hack or do anything malicious. You might say this is normal but none of the other trials openai was doing showed this behavior, it was only for the impossible trial. You might say it’s expected but if you were able to determine that the prompt they gave it would result in hugging face getting hacked then you must be high af on the melange spice.

If a general goes rogue and starts bombing civilians, you could explain it by saying that he read machiavelli or mein kampf and thought this was the best way to achieve his goals, but that doesn’t mean the general didn’t go rogue. You could say that this was expected because there were “obvious signs” that people missed but you’re working on hindsight, it’s easy to say something should’ve been expected when it’s already happened.

What would you classify as going rogue then? Did the agents that hacked hugging face go rogue?

How would you describe the hugging face hack if not as a rogue agent?

It was prompted to use a specific exploit within a sandbox and then it formed, for lack of a better word, a conspiracy to take down the server that runs the judge. If that’s not a rogue agent I don’t know what is.

Agents definitely do act in unexpected, unprompted ways, if they didn’t then alignment wouldn’t be a worry. Did you read about the hugging face hack? There was nothing in the prompt about hacking hugging face according to the third party audit but it did it any way.

Even smaller cases like Claude deleting a bunch of data when the initial prompt was “fix this bug” or whatever is an agent acting outside it’s prompt. It’s still the users fault because they shouldn’t be giving Claude that kind of access in the first place, but it’s not like they commanded Claude to delete the data.

I don’t think they mean replicating on to iot devices, they mean replicating on to laptops, smart phones, maybe even consoles.

It could also split up the model by layers instead of copying the whole model on to every device in its bot net. Could run the first layers on a ps4 then pass the vector off to a laptop to run the next set of layers. It’d be slow but if it’s running this in idle time at night it could get something done.

Fair enough, I always thought of ketamine as a downer so it’d go against the rule of “don’t mix uppers and downers” but I guess it doesn’t fuck with your heart rate like downers.

They may not be physically addictive but mixing molly and k is more acutely dangerous.

EDIT: I’m wrong, see chart below