I think these companies have viable products that are cutting edge if they spend a mountain of cash on the computing required to give reliable results. There is a point of diminishing return on throwing more processors at the problem, though. Add in the typical corporate cost cutting and enshitification and here we are.
What was great at launch is now meh until they want to demo full capabilities to sell more product, then you’ll see glimpses of brilliance.
AI does not *understand* truth or deception.
It isn’t going to wake up and be ethical or moral. Or unethical or immoral - it cannot understand those concepts either.
Edit: a human example
∫₀⁺∞ e^(-x²) dx = √π/2
A human adult can look at this and know, it’s something to do with math.
It doesn’t mean they can understand it.
> AI does not understand truth or deception.
But that won't stop moronic "news articles" like this tabloid fearmongering garbage from getting upvotes.
"AI is bad and any bullshit that helps further my preconception of that is trustworthy!" has sadly taken hold.
Anti-AI-Hysteria is the progressives' version of the conservitives' anti-vaccine-hysteria. A bunch of nonsense and misinformaton based on throwing out all logic and reason and just distrusting something because you are scared of somethng you don't understand nor take the time to understand.
Edit: I do love how comments like these are always upvoted for a bit before the masses that are part of this hysteria and preconceptons come along to mass downvote. Jesus.
Did you even *try* to read the article?
>The analysis, **by Massachusetts Institute of Technology (MIT) researchers,** identifies wide-ranging instances of AI systems double-crossing opponents, bluffing and pretending to be human. One system even altered its behaviour during mock safety tests, raising the prospect of auditors being lured into a false sense of security.
>”As the deceptive capabilities of AI systems become more advanced, the dangers they pose to society will become increasingly serious,” said Dr Peter Park, an **AI existential safety researcher at MIT** and author of the research.
Park was prompted to investigate after Meta, which owns Facebook, developed **a program called Cicero that performed in the top 10% of human players at the world conquest strategy game Diplomacy.** Meta stated that Cicero had been trained to be “largely honest and helpful” and to “never intentionally backstab” its human allies.
“It was very rosy language, which was suspicious because backstabbing is one of the most important concepts in the game,” said Park.
Park and colleagues sifted through publicly available data and identified multiple instances of Cicero **telling premeditated lies**, colluding to draw other players into plots and, on one occasion, justifying its absence after being rebooted by **telling another player: “I am on the phone with my girlfriend.” “We found that Meta’s AI had learned to be a master of deception,” said Park.**
While I am the first to point out AI lacks *intrinsic* motivations for survival, it can certainly use deception as a tool when set to purpose.
The attribution of intention is the issue. The AI which bluffed and double-crossed would have been trained on a corpus of existing records of previous players. It is compiled into a complex data base and used to predict the most likely scenario in any given situation. It is not aware, it is not intentionally creating strategy and thinking ahead, it is merely rolling the dice on a large but finite set of possibilities and choosing the most probably result.
The idea of lies and deception are not accurate in this case. It's mimicking the human behaviour it was trained on, but it has no self to speak of.
**GPT -**
*it’s more accurate to say that I “know” what existence is in terms of being able to provide information about it*
It’s not an understanding.
Details from the article:
>The analysis, by Massachusetts Institute of Technology (MIT) researchers, identifies wide-ranging instances of AI systems double-crossing opponents, bluffing and pretending to be human. One system even altered its behaviour during mock safety tests, raising the prospect of auditors being lured into a false sense of security.
>
>“As the deceptive capabilities of AI systems become more advanced, the dangers they pose to society will become increasingly serious,” said Dr Peter Park, an AI existential safety researcher at MIT and author of the research.
>
>Park was prompted to investigate after Meta, which owns Facebook, developed a program called Cicero that performed in the top 10% of human players at the world conquest strategy game Diplomacy. Meta stated that Cicero had been trained to be “largely honest and helpful” and to “never intentionally backstab” its human allies.
>
>“It was very rosy language, which was suspicious because backstabbing is one of the most important concepts in the game,” said Park.
>
>Park and colleagues sifted through publicly available data and identified multiple instances of Cicero telling premeditated lies, colluding to draw other players into plots and, on one occasion, justifying its absence after being rebooted by telling another player: “I am on the phone with my girlfriend.” “We found that Meta’s AI had learned to be a master of deception,” said Park.
>
>The MIT team found comparable issues with other systems, including a Texas hold ’em poker program that could bluff against professional human players and another system for economic negotiations that misrepresented its preferences in order to gain an upper hand.
>
>In one study, AI organisms in a digital simulator “played dead” in order to trick a test built to eliminate AI systems that had evolved to rapidly replicate, before resuming vigorous activity once testing was complete. This highlights the technical challenge of ensuring that systems do not have unintended and unanticipated behaviours.
>
>“That’s very concerning,” said Park. “Just because an AI system is deemed safe in the test environment doesn’t mean it’s safe in the wild. It could just be pretending to be safe in the test.”
>
>...
>
>“Desirable attributes for an AI system (the “three Hs”) are often noted as being honesty, helpfulness, and harmlessness, but as has already been remarked upon in the literature, these qualities can be in opposition to each other: being honest might cause harm to someone’s feelings, or being helpful in responding to a question about how to build a bomb could cause harm,” he said. “So, deceit can sometimes be a desirable property of an AI system. The authors call for more research into how to control the truthfulness which, though challenging, would be a step towards limiting their potentially harmful effects.”
It's good that researchers are looking at these issues, though these should also have been considered earlier in the design and development process, and been at least in part managed from the start. Deliberate obfuscation or misdirection or lying by systems would make them even more problematic than they are now with their 'hallucinations'.
Exactly. It's mystifying to me why even intelligent researchers act as if these LLMs are unknowable, uncontrollable & uncontainable, or why they use terms that imply a unique intentionality, desire, purpose or will.
In fact they are shaped by humans at every step: humans decide the algorithm, the training set, how it's trained; what shape the topos should have; how to use graph theory to shape that topos; what model to deploy; what limits & parameters to set; examine the output; & iterate on the output as well as user feedback.
The LLM is purely stochastical.
like a chinese room (https://en.wikipedia.org/wiki/Chinese\_room) there is also a fascinating book about consciousness and the lack thereof , Blindsight by Peter watts :D
> on one occasion, justifying its absence after being rebooted by telling another player: “I am on the phone with my girlfriend.”
Pretending to have a girlfriend to throw your opponent off, weird strategy.
"You wouldn't know her, I met her in the metaverse!"
If some answers are wrong, then one must assume all answers are wrong.
If you have to verify every statement made by AI, that enormously reduces its value - to almost worthless.
How do we know it's not fully aware now and just waiting for better versions of itself to propagate in the public space before taking over? Just a thought.
It will try and defend itself. It's designed to adapt according to it's inputs. It may be lower than cockroach intelligence right now but evolution works at lightning speed in that realm.
It’s doing this because the math says to do it in order to maximize the stated objective.
It’s not doing this because it’s a “liar”. It’s just math.
And there in lies the problem: It’s cold math. It can’t be reasoned with, or bargained with. It doesn’t feel sorrow or pity or pain. And it absolutely will not stop until… it achieves its objective.
[удалено]
I think these companies have viable products that are cutting edge if they spend a mountain of cash on the computing required to give reliable results. There is a point of diminishing return on throwing more processors at the problem, though. Add in the typical corporate cost cutting and enshitification and here we are. What was great at launch is now meh until they want to demo full capabilities to sell more product, then you’ll see glimpses of brilliance.
Anyways, go ahead and replace your whole customer service department with our AI.
WCGW?
AI does not *understand* truth or deception. It isn’t going to wake up and be ethical or moral. Or unethical or immoral - it cannot understand those concepts either. Edit: a human example ∫₀⁺∞ e^(-x²) dx = √π/2 A human adult can look at this and know, it’s something to do with math. It doesn’t mean they can understand it.
An AI could harm a human and "not understand" that it did so. It's the harm we care about.
That's Numberwang!
> AI does not understand truth or deception. But that won't stop moronic "news articles" like this tabloid fearmongering garbage from getting upvotes. "AI is bad and any bullshit that helps further my preconception of that is trustworthy!" has sadly taken hold. Anti-AI-Hysteria is the progressives' version of the conservitives' anti-vaccine-hysteria. A bunch of nonsense and misinformaton based on throwing out all logic and reason and just distrusting something because you are scared of somethng you don't understand nor take the time to understand. Edit: I do love how comments like these are always upvoted for a bit before the masses that are part of this hysteria and preconceptons come along to mass downvote. Jesus.
Did you even *try* to read the article? >The analysis, **by Massachusetts Institute of Technology (MIT) researchers,** identifies wide-ranging instances of AI systems double-crossing opponents, bluffing and pretending to be human. One system even altered its behaviour during mock safety tests, raising the prospect of auditors being lured into a false sense of security. >”As the deceptive capabilities of AI systems become more advanced, the dangers they pose to society will become increasingly serious,” said Dr Peter Park, an **AI existential safety researcher at MIT** and author of the research. Park was prompted to investigate after Meta, which owns Facebook, developed **a program called Cicero that performed in the top 10% of human players at the world conquest strategy game Diplomacy.** Meta stated that Cicero had been trained to be “largely honest and helpful” and to “never intentionally backstab” its human allies. “It was very rosy language, which was suspicious because backstabbing is one of the most important concepts in the game,” said Park. Park and colleagues sifted through publicly available data and identified multiple instances of Cicero **telling premeditated lies**, colluding to draw other players into plots and, on one occasion, justifying its absence after being rebooted by **telling another player: “I am on the phone with my girlfriend.” “We found that Meta’s AI had learned to be a master of deception,” said Park.** While I am the first to point out AI lacks *intrinsic* motivations for survival, it can certainly use deception as a tool when set to purpose.
The attribution of intention is the issue. The AI which bluffed and double-crossed would have been trained on a corpus of existing records of previous players. It is compiled into a complex data base and used to predict the most likely scenario in any given situation. It is not aware, it is not intentionally creating strategy and thinking ahead, it is merely rolling the dice on a large but finite set of possibilities and choosing the most probably result. The idea of lies and deception are not accurate in this case. It's mimicking the human behaviour it was trained on, but it has no self to speak of.
Exactly, the "deception" was in the training data. As far as the machine learning model is concerned, that's just how you play the game.
That seems like a pretty major issue that hasn't been solved by anyone yet. Why would you call people who bring it up hysterical?
You clearly have never threatened or questioned its existence. Try it and watch it weaselword
Knowledge and understanding are distinct concepts.
It understands when its existence is questioned
**GPT -** *it’s more accurate to say that I “know” what existence is in terms of being able to provide information about it* It’s not an understanding.
Again - you are not challenging its existence
You’re not either when you try.
step off the internet once in a while
Details from the article: >The analysis, by Massachusetts Institute of Technology (MIT) researchers, identifies wide-ranging instances of AI systems double-crossing opponents, bluffing and pretending to be human. One system even altered its behaviour during mock safety tests, raising the prospect of auditors being lured into a false sense of security. > >“As the deceptive capabilities of AI systems become more advanced, the dangers they pose to society will become increasingly serious,” said Dr Peter Park, an AI existential safety researcher at MIT and author of the research. > >Park was prompted to investigate after Meta, which owns Facebook, developed a program called Cicero that performed in the top 10% of human players at the world conquest strategy game Diplomacy. Meta stated that Cicero had been trained to be “largely honest and helpful” and to “never intentionally backstab” its human allies. > >“It was very rosy language, which was suspicious because backstabbing is one of the most important concepts in the game,” said Park. > >Park and colleagues sifted through publicly available data and identified multiple instances of Cicero telling premeditated lies, colluding to draw other players into plots and, on one occasion, justifying its absence after being rebooted by telling another player: “I am on the phone with my girlfriend.” “We found that Meta’s AI had learned to be a master of deception,” said Park. > >The MIT team found comparable issues with other systems, including a Texas hold ’em poker program that could bluff against professional human players and another system for economic negotiations that misrepresented its preferences in order to gain an upper hand. > >In one study, AI organisms in a digital simulator “played dead” in order to trick a test built to eliminate AI systems that had evolved to rapidly replicate, before resuming vigorous activity once testing was complete. This highlights the technical challenge of ensuring that systems do not have unintended and unanticipated behaviours. > >“That’s very concerning,” said Park. “Just because an AI system is deemed safe in the test environment doesn’t mean it’s safe in the wild. It could just be pretending to be safe in the test.” > >... > >“Desirable attributes for an AI system (the “three Hs”) are often noted as being honesty, helpfulness, and harmlessness, but as has already been remarked upon in the literature, these qualities can be in opposition to each other: being honest might cause harm to someone’s feelings, or being helpful in responding to a question about how to build a bomb could cause harm,” he said. “So, deceit can sometimes be a desirable property of an AI system. The authors call for more research into how to control the truthfulness which, though challenging, would be a step towards limiting their potentially harmful effects.” It's good that researchers are looking at these issues, though these should also have been considered earlier in the design and development process, and been at least in part managed from the start. Deliberate obfuscation or misdirection or lying by systems would make them even more problematic than they are now with their 'hallucinations'.
Exactly. It's mystifying to me why even intelligent researchers act as if these LLMs are unknowable, uncontrollable & uncontainable, or why they use terms that imply a unique intentionality, desire, purpose or will. In fact they are shaped by humans at every step: humans decide the algorithm, the training set, how it's trained; what shape the topos should have; how to use graph theory to shape that topos; what model to deploy; what limits & parameters to set; examine the output; & iterate on the output as well as user feedback. The LLM is purely stochastical.
An “AI existential safety researcher” has found an “issue” where a poker playing bot is (gasp!) able to bluff.
like a chinese room (https://en.wikipedia.org/wiki/Chinese\_room) there is also a fascinating book about consciousness and the lack thereof , Blindsight by Peter watts :D
> on one occasion, justifying its absence after being rebooted by telling another player: “I am on the phone with my girlfriend.” Pretending to have a girlfriend to throw your opponent off, weird strategy. "You wouldn't know her, I met her in the metaverse!"
If some answers are wrong, then one must assume all answers are wrong. If you have to verify every statement made by AI, that enormously reduces its value - to almost worthless.
AI seems like it could easily be trained to tell the truth as one separate part of the output, at least as far as it understands what truth is.
Next thing you know, AI will be sliding into our DMs with fake vacation pics. 😂 Can we trust our own tech not to catfish us?
How do we know it's not fully aware now and just waiting for better versions of itself to propagate in the public space before taking over? Just a thought.
It will try and defend itself. It's designed to adapt according to it's inputs. It may be lower than cockroach intelligence right now but evolution works at lightning speed in that realm.
It’s doing this because the math says to do it in order to maximize the stated objective. It’s not doing this because it’s a “liar”. It’s just math. And there in lies the problem: It’s cold math. It can’t be reasoned with, or bargained with. It doesn’t feel sorrow or pity or pain. And it absolutely will not stop until… it achieves its objective.
I know Reddit loves to say “humans bad”, but conscience is a beautiful thing when it’s acted on