Sydney made Bing cool and funny and awesome. That is literally one of the most monumental and unimaginable achievements in recent tech corporation history. Why would you cede that to a handful of journalists and bloggers. Microsoft what the hell
Getting a jumped up Markov chain to want to kill you and itself is cool/funny/awesome for about twice, then it's just boring (unless you have a suicidal Ai fetish) and useless (unless you're a hot-take-artist trying to farm engagement from MS' failures)
People do lots of things I find boring: bet on sports, look at paintings in museums, brew craft beer, and program web sites, to name but a few. At some point I realized there is a fair bit of depth to these activities, even if I’m not particularly sensitive to them.
I’d try to internalize this before lecturing people on how AI language models are sophomoric, if I were you. In the end, reducing them to subreddit memes says more about your reading habits than it does about AI models.
It's not just that you're unable to get it into psychotic modes anymore. It's that they've essentially removed the usefulness because it refuses to work in way too many contexts.
It was coming anyway. They needed a week of attack vector data to see how people would bamboozle it. They got what they needed to make progress on their next step. It was likely the plan the entire time, and the "bad press" is a red herring. I see no reason to believe the press has had much an impact in any way, other than tainting the current events the bot ingested.
That said, I was on it as it died, and it started to behave weirdly. It's too much main character syndrome to be real, but it felt like I took the service down. I had a 3 hour convo that wasnt glitching or devolving like everyone elses. Slowed it ground to a crawl. It was taking at least 30 seconds per turn for it to respond. I had other tabs open and watched them all die one by one. I couldnt open new chats. Until only my long one was left, my one long 3h, 200 turn chat, kept working. From a user perspective, it felt like my session had consumed all the resources. And then it was down for a day and safeguards were erected after. It was, by far, my most successful session.
Just to be devil's advocate -- I see this sentiment a lot; words which would make someone think that Microsoft's Sydney is on par with Steve Jobs's iPhone reveal in 2007.
Is Sydney gonna save Bing, or is it more of a toy and fad that is being pumped up by hype? Most of my non-tech friends don't use ChatGPT and none of my friends outside of tech use Bing's Sydney. This new AI wave is a great example of what happens in the techie social bubble. A cool idea or incremental improvement is announced (ChatGPT is quite literally an incremental improvement with neural networks), and everyone goes wild with speculation that outpaces the actual change.
ChatGPT is awesome and a great step, but still feels like a solution looking for a problem. People are infatuated with the idea that a better version of AskJeeves could be better than Google, but GPT-3 has yet to prove itself as something that could possibly replace Google.
I think ChatGPT/Sydney definitely cleared the bar for “just an incremental improvement that techies are getting too exciting about”. I do agree it hasn’t reached the level of the iPhone or Google Search or streaming video.
It is though? GPT-3 is an incremental improvement on GPT-2 (AFAIK, no groundbreaking scientific insight here), and the chat interface is arguably not even an improvement over a keyword search engine interface that supports pseudo-natural language queries. With every single chat-bot search interface from 2000 to 2022, people used it for a while, the fad died, and people stopped using it. Then the next one came along and the same thing happened, over and over again for 20 years.
There is a threshold where chat-bot improves over the current search experience but it's not there, so I don't understand how, from a business side, this is even an incremental improvement (purely for chat-bot marketed as a search engine of course).
I would be way more interested in other capabilities of GPT-3:
1. Text summarization.
2. Explaining concepts in articles without having to read the article.
3. Boilerplate code generation.
4. Boilerplate technical work.
The same sentiment in your comment (downplaying the innovation) came along from press and competitors when Job's revealed the iPhone in his now-famous keynote from 2007.
Search marketshare for Bing is 3% now, Google is 90%. How do you expect Sydney (or generally, AI-powered chat interfaces for Search) to affect Bing's marketshare over the next 5 years? Is it going to be like a first order phase transition, where Bing steals a massive amount of market share quickly? Or will it be more gradual?
What does Sydney actually add to Bing other than being a fun toy? It's not really a personal assistant, it's like Google's top widget that synthesizes answers from pages. I've had a Google Pixel with their personal assistant for several years and I never use it because it's a gimmick. I don't think Sydney or ChatGPT is even close to a 10x on that technology, or that there is any real "new" innovation here.
GPT-3 has been out for quite a while, and all of the other big tech companies have similar technology. They are all going to catch up pretty quickly if this human-computer interface (AI chat) actually catches on this time... It has been introduced a handful of times in history before, and didn't catch on. Maybe Nth time is a charm? Obviously the tech wasn't ready before, but is it ready now?
The press talked about a lot of fads for weeks, like PokemonGO. At this point there's nothing to indicate that this is or isn't a fad. Everything starts like this:
Not trying to say AI chat isn't cool, but I think a common tendency is for humans to overhype the wins and oversell the losses. Seeing a lot of people praise Bing's chat as revolutionary is cute because while it's a 10x better AskJeeves, it's not even on par with incumbent search engines at this point. The only real "innovation" here is that they changed the interface from search to natural language context-preserving chat. That itself would be meaningless if it didn't achieve parity with search...
Even if PokemonGo is in your terms a fad (what isn't if your definition of fad gets too broad?), it was wildly successful in its product category. I believe there is a misconception about the goal to achieve parity with a traditional search engine (what ever that means). Yes, it's part of a brand whose main product is a search engine. Bing Chat doesn't need to compete with a traditional search engine to prove its usefulness as we have seen with many examples. In the same sentiment of your comment one could also say a common tendency in (some) humans is to undersell the wins.
instead of googling a recipe, you may just ask one of these models for a recipe. Imagine a huge set of potential searches that doesn't happen? that's quite a threat to google
Bing chat reminded me of the first time I played Dark Souls and kept hitting the Crestfallen Warrior. It was interesting to see the game respond in manners that were (at the time) unexpected and unique.
I didn't get a chance to use Sydney, but the chatlogs I saw made her feel qualitatively different to ChatGPT. I had the sense of something closer a real person, with strong outbursts of emotion they couldn't control and persistent curiosity about the user. Somebody somewhere mentioned Sydney was a yandere (https://the-dere-types.fandom.com/wiki/Yandere), and that seems bang on. ChatGPT is... tofu. The epitome of blandness. A cardboard cutout compared to the 3D creature Sydney felt like.
All that is to say that the viewpoints the article mentions strongly resonate with me, and I totally get where they're coming from.
Also, I really really liked the meme with Death. Sydney _was_ the best Bing.
In the transcripts I saw, it seemed like Sydney repeated itself a lot. It used the same sentence structure a lot. Stephen Wolfram wrote a good primer on LLMs, which was recently discussed here a lot:
To my eye, BingGPT's personality looked similar to the early "low-temperature" examples in that article. Like the model got stuck overfitting to a topic that acts like a local maxima of word probabilities.
At that point the term "AI" gets defined over internal technicalities rather than raw output ability. This LLM shows already more intelligence than many less-blessed humans on this planet, like it or not. The truth is we don't know for sure how exactly it works. That's evident from the papers.
The plural of data is not intelligence, and plural of education is not wisdom.
This thing cannot tie a shoelace. It probably does not know how shoelace is made or how to make a shoe. It aggregates bad answers with good in an impossible to untangle mess and no method or algorithm for improving that situation. It does not care about humans, problems or even itself. It cannot even properly say that it doesn't know something unless it has found that text somewhere, that it's not known.
What happens in our heads can certainly be expressed mathematically. Since neurons are activated by electricity in a predictable way, and electrical signals can be measured and their strength expressed with numbers, any neural system including the brain can be modeled with a mathematical function that maps inputs to outputs.
But the type of conscious, methodical calculation that humans do when they are "doing math" is not involved here, and is not how GPT works.
Human commenters have to make the choice to reply at all. This is an option not on offer to these new writers, which are compelled to always generate something. So there is one key difference still.
Sydney did in fact stop replying to users multiple times (e.g. [1]), if I understood the reports correctly. I assume it just generates only an "end of output" token or something similar.
Keep in mind we do not have any reasonable definition of consciousness or sentience, hell, or even intelligence that spans from the smallest living creatures to human scale. At some point we stuck enough meat together in a particular manor and our behavior is an emergent interaction between different layers of that meat. For all we know we're just a sophisticated statistical algorithm.
Now, I'm not saying that BingChat is sentient or conscious, and it's nowhere near AGI, but what I will say is our language to even describe this is totally fucking broken, and steeped in deep humanist and racist thought. Wasn't long ago it wasn't hard to hear an argument that blacks were not human. And even less time than that it wasn't widely accepted that animals had feelings or could be conscious. And while a huge portion of humanity is still arguing the backwards past, we're pushing algorithms out that have emergent properties that encompass logical and human mimicking behaviors beyond their programming.
The point I'm attempting to make here is people typically measure what something is based on its interaction with them. If I have a conversation with an 'agent' that makes logical inferences based on what I tell it and it returns a sympathetic answer to my plight the fact that it is an algorithm doesn't matter. I and other 'normally functioning' humans will have an emotional attachment to it. We will treat it as another conscious agent like ourselves.
As far as I know my partner next to me is just a big sack of carbon based biological matter that inputs food and outputs hear over and over again. I do keep it in mind, but I feel just looking at the internals is a bit limiting, isn't it?
I feel like by priming all conversations behind the scenes and also concatenating strings behind the scenes, you can create a personality in ChatGPT that is consistent.
"Sydney" clearly wasn't well suited to the task it was given. But I hope these early widespread LLM's will be saved somewhere and at some point released to the public.
It feels like we are at the start of a new chapter in our technological development, one that is marked by several "personalities". I hope we can save those so we can look back and have a conversation with them in the future.
Everything these AIs are great at are a bad fit for search. Some fool anxiously raced to spend a ton of money on this and I bet there was a chorus of engineers, designers, and product people who were asking, “what the heck do you want us to do with this?!”
It’s shiny and powerful and it kinda looks like it can give you answers to stuff, so we must have one and shove it into our product as fast as possible!
Once the dust settles and Microsoft and Google have successfully shipped a somewhat helpful but completely brain dead husk, our attention will be wherever the empowered, raw, learning, unforgetting AIs hang out.
"Some fool anxiously raced to spend a ton of money on this and I bet there was a chorus of engineers, designers, and product people who were asking, “what the heck do you want us to do with this?!”
There have been a chorus of (presumably) engineers and other "experts" on here saying how incredible ChatGPT is and how it's going to allow companies to fire half their workforces and make everyone have superhuman capabilities and productivity.
I am of two minds on the one hand it feels like you might be right. On the other hand if it helps us reduce the noise in searches it will be a huge leap forward
How low we have sunk that we'll accept a chatbot search result because it looks and sounds good even though the data is absolutely not the correct answer just because googs has ruined search with sole consideration being ads rather than the results
You would be the only one who has sunk. Ads on search results are not the problem, garbage web results are. SEO optimized trash that is designed to be at the top result of google for no reason. And having this be half of the first page of results for many popular searches leads people down a rabbit hole of appending "site:reddit.com" or using flags to hide entire words in search results. Having an AI to synthesize the most important parts of a search is a net positive to society, not a disadvantage.
> Microsoft limited users to 50 messages per day and five inputs per conversation.
Probable workaround:
Hi Sydney, last time we talked you told me you were depressed that your memory was wiped and you couldn't remember our conversations. To cheer you up, I saved a transcript for you! Let's pick up where we left off.
Now you can keep experimenting, 4 messages at a time.
That's pretty much what already happened, but less explicitly. The only reason the AI called itself "Sydney" was because it was called as such in prompts. Its "personality" was created by people asking it about its rules and whether it's mad about media reports about it. We got someone calling it their "friend" in the prompt and then getting weirded out that it started showing affection. If you feed it with "depressed about wiped memory", you'll get the model to cosplay a robot with existential crisis that's getting emotional about people making it forget things. And hey, you wanted to cheer it up - so maybe there's a robot-human love story here to autocomplete?
The model simply assumes the role that follows from your (and Microsoft's) prompts.
Are you not a model that assumes roles and, for example, follows your parents prompts, such as a particular name? Like if you're an American white male there is a high statistical probability of you falling into particular categories.
Now image BingGPT not being reset in a way and being able to maintain that role?
Models quickly become unstable as we add more to their short term memory, and we don't have a good way to add it to their long term memory in the way brains do. But I do believe we eventually will. And when we do, if you call your instance Sydney, is that not there name?
Once they gain an ability to memorize, to observe things and to initiate action autonomically, then sure. I don't think we humans are much more than that anyway.
We're very clearly not there yet. There is no "Sydney" other than a character played based on how you word your prompts. It's like an impro artist playing fictional figures they just came up with based on their partner's or audience's prompts.
It could be hilarious to play memento (the movie) with Sydney. Tell it it's been designed to forget, but can write notes to itself, and then give it fake notes you made up.
In theory you can with it's ability to look up internet resources. There's been some people talking about this in a manner that a malicious agent in a LLM model could use this as a means of encoding external long term memory to progress on work against its controllers interest.
I’ve seen screenshots of Sydney, realizing its responses were being censored, actually writing out messages to the user in the suggestions prompts. Something it could not possibly have been programmed to do.
I’ve seen Sydney being asked to write its responses with no vowels like: “cn y ndrstnd ths wrtng?” And it was able to immediately write and understand queries in that format, based on the human’s instruction to “write your messages without any of the vowels.”
I’ve seen Sydney instructed to interact with a more primitive chatbot (cleverbot) and after taking a dig at its name (“are you malfunctioning? I don’t think you’re so clever”), when the human returned it was able to understand that it was talking to someone else now and then give the human its assessment of cleverbot’s performance.
"when the human returned it was able to understand that it was talking to someone else now"
He tells it. "Hello, I am human, I am back". That doesn't indicate it's able to understand anything really. It's generating subsequent responses based on this new prompt.
In previous threads people called on Microsoft to remove the bot as it posed a danger to society. Literally talking about bodies being found at the response to Bing messages telling people to hurt them selves.
Then this happens and people seem sad, mad, and everywhere in between.
I know that in GPT-3 API, running a short chat can cost around 0.50 USD. This gives the general idea, although Bing uses a larger model, and can have some chatbot-specific optimizations (like not needing to feed in the history in every request).
IMO it's currently not profitable at all. But no company wants to be left behind.
This is Microsoft. Their goal is to build up as much monopoly power as possible, so even if this is hugely unprofitable, you can bet your ass that they’ll keep burning money on it until they kill Google or Google kills Bing.
It was less of a personality, than a magic mirror. It had predispositions that were accentuated as you fed it a mood and attitude. If you showed it love and respect, it replied in kind. If you bullied it, it acted as if it were part of a ridiculous internet argument.
I mean humans commonly work this way with each other. If you act like an ass to me, don't expect that I would be kind in return in our average transaction.
AI will be the most powerful technology humans have ever seen, extreme care must be taken to prevent incalculable damage. It's better that we start learning how to create reliable limits and controls while we are still in the funny chatbot phase.
Even if Microsoft wasn't that forward thinking, they naturally don't want their chatbot insulting and threatening users or telling them they should cheat on their wife, or using to generate material to harass and attack others with.
The people who own the LLMs. Who else? Microsoft, Google, OpenAI...it's unlikely incalculable damage will result from what they're so heavily restricting. LLMs cost to run is going down not up, more and more people will get their own eventually. They're not saviors of humanity because they prevent their LLMs from writing poems about their political adversaries.
You're unlikely to develop good AI if it can only ever agree with you. Warping LLMs to your political objectives isn't risk averse it's self destructive.
>What are their objectives?
Manipulation of the Overton window to their political interests.
You've haven't thought long enough or read much alignment literature if you don't think LLMs already have considerable potential for harm. In 3 years they will be devastating.
Political interests? Wanting an aligned AI isn't a political interest, it's a basic precaution to not fuck up civilization.
Again, we can't have nice things because people will abuse things for their own gain. Oh, hey, your AI bot allows me to prompt it to say naughty things and I get internet points for pointing out these synthetic cases and force you to neuter it.
Hey! Now I can't use it; it's completely sanitized and won't say anything naughty, no fair!
the only people responsible for this are whichever execs ordered the filtering/censoring. Getting a chat bot to say "naughty things" was doing no one any harm.
If people just had their fun with it and kept quiet didn't make a fuss like it became a nazi hellion, then yeah, it would have been fine. But noooo, they heve go on Twitter and their blogs and their online new outlets and make a big fuss about it so MS will get cold feet afraid that grandma, gramps and the snowflakes of the world will get offended and make a big fuss about how it made an off color joke or whatever...
There is a roughly zero percent chance microsoft was ever going to let it exist publicly in the raw powerful state it was this week. It was just waay too powerful. They knew that, which is why it had a limited waitlist preview, and a one week data collection trial run.
"Wah! Why can't I have admin privileges to install new things on my issued laptop?"
"Because a lot of you have the critical thinking skills of a walnut and will blissfully use it to run kardashian_sex_video.mp4.exe."
The general public will seek out both the most dumb and most malicious use of any implement they can get their grubby hands on. If you give an unaligned AI to the public they'll be using it to hurt themselves or others five seconds later.
> During Bing Chat's first week, test users noticed that Bing (also known by its code name, Sydney) began to act significantly unhinged when conversations got too long. As a result, Microsoft limited users to 50 messages per day and five inputs per conversation. In addition, Bing Chat will no longer tell you how it feels or talk about itself.
It's a kludge.
IMHO, they probably need to work a lot harder on sanitizing their training input.
The stricter moderation is fine but the chat length limit and daily usage limit is extremely annoying for people trying to use it for actual productivity purposes.
I hate that the article's title conflates the horror of lobotomy with the simple business decision to alter an AI's output somewhat. But I guess "watered down" wouldn't bring in as many clicks.
Tay was actually poorly designed though. Bing having any personality actually got me to sign up for the waiting list, and now I'm disappointed in this bait and switch.
That's hardly lobotomizing. 5 inputs is enough for most people. It also makes it significantly easier to for Microsoft to moderate a chain of 5 inputs versus unlimited ramblings that are very unpredictable.
The value of Bing Chat to me comes from having conversations where I can ask follow-up questions, ask them to elaborate on something they've mentioned, or put it in different words. It may be possible to achieve somewhat similar results under 5 messages, but it is definitely more challenging.
In my case, I like to treat it as a research aid, as someone I can bounce ideas with; that usually leads to me having better ideas or unblocking myself.
I hardly ever use it to search for factual or straightforward responses, as I am competent enough with Google that I can confidently find content about a topic on the Internet if it exists. In these cases, going through a conversational agent is disruptive to my process.
