This is due to Hallucinations and the training AI had.
I used Google Gemini for a while and got really frustrated over this exact issue. I gave it one more try to see if it could follow simple orders.
Gemini was like “ok I will follow direct orders going forward”. I was like “okay starting this moment do not respond to anything I say in this chat” and Gemini was like “okay done” and I was like “you failed. Why did you ignore my instructions and respond to me?” Basically it told me that due to its training it was ‘programmed’ to ignore certain prompts.
This is what is dangerous about AI. Similar to social media and Google, AI can be used to persuade public opinion to match those who train AI. AI will be way more efficient persuading opinion that Google and social media because it knows you better than those. It spits out information as if it’s facts.
It doesn’t follow directions, and it probably doesn’t have programming to ignore certain prompts. It’s a text generator.
Below was taken from ChatGPT. I don’t know why everyone downvoted me for telling the truth. Maybe I gave a bad example. I thought giving a real world example of it ignoring my prompt would prove my point but I guess we need to go deeper too.
“trained to believe” versus “configured to answer as though it believes.” A model doesn’t necessarily have a private belief system. You can make two instances of essentially the same underlying model produce substantially different answers by changing their instructions, training data, reward criteria, or information sources.
For example, you could create three AI systems and give all three the question:
“Should the government provide universal healthcare?”
One could be optimized around libertarian principles, another around social-democratic principles, and another instructed to provide a politically neutral analysis. They could all know essentially the same facts while reaching different conclusions because they’re being asked to evaluate those facts using different frameworks.
There is also a more subtle issue: belief-curated AI doesn’t have to contain obvious propaganda. Selection of which facts to emphasize, which uncertainties to mention, which counterarguments to steelman, and even what questions it considers relevant can systematically push users toward a particular worldview.
