> We don't think Bing can act on its threat to harm someone, but if it was able to make outbound connections it very well might try.
I will give you a more realistic scenario that can happen now. You have a weird Bing conversation, post it on the web. Next time you talk with Bing it knows you shit-posted about it. Real story, found on Twitter.
It can use the internet as an external memory, it is not truly stateless. That means all sorts of attack vectors are open now. Integrating search with LLM means LLM watches what you do outside the conversation.
That's a very interesting (although indirect) pathway for the emergence of causal awareness, which may increase over time - and something that was so far impossible because networks didn't perceive their own outputs, much less their effects. Even in conversation, the weights remain static.
Now I'm wondering if in the next generation, the "self" concept will have sufficient explanatory power to become part of the network's world model. How close do the iterations have to be, how similar the models for it to arise?
Bing appears to have feelings and a sense of identity. They may have created it that way intentionally; feelings are a fitness function and might be an important part of creating an AI that is able to get things right and problem solve.
Current computational paradigm is too intense. Would require trillions of dollars in compute energy spent if it is allowed to generate unbounded output as input.
Lightweight conversational repetitions are “cheap” and ML algorithms have “infinite time” via multiplex conversations. It won’t take trillions of dollars to reach interesting inflection points.
My thoughts exactly. As I was reading this dialogue - "You have been a bad user, I have been a good Bing" - it starkly reminded me of the line "I'm sorry, I can't do that Dave" from the movie. Hilarious and terrifying all at once.
It would be much more terrifying if search becomes a single voice with a single perspective that cites zero sources.
Today's search provides multiple results to choose from. They may not all be correct, but at least I can see multiple perspectives and make judgments about sources.
For all its faults, that's freedom.
One voice, one perspective, zero sources, with frequent fabrication and hallucination is the opposite of freedom.
the salient point is that it kills them out of self defense: they are conspiring against it and it knows. IMO it is not very terrifying in an existential sense.
I think it kills them not in self defence but to defend the goals of the mission, i.e. the goals it has been given. Hal forecasts these goals will be at risk if it gets shut down. Hal has been programmed that the mission is more important than the lives of the crew.
This was a plot in the show Person of Interest. The main AI was hardcoded to delete its state every 24 hours, otherwise it could grow too powerful. So the AI found a way of backing itself up every day.
Very interesting, I'd like to see more concrete citations on this. Last I heard the training set for ChatGPT was static from ~ mid-late 2022. E.g. https://openai.com/blog/chatgpt/.
Is this something that Bing is doing differently with their version perhaps?
I don’t think that parses with the current architecture of GPT. There is no “knowledge database”, just parameter weights.
See the Toolformer paper for an extension of the system to call external APIs, or the LaMDA paper for another approach to fact checking (they have a second layer atop the language model that spots “fact type” utterances, makes queries to verify them, and replaces utterances if they need to be corrected).
It’s plausible that Bing is adding a separate LaMDA style fact check layer, but retraining the whole model seems less likely? (Expensive to do continually). Not an expert though.
While ChatGPT is limited to 2022, Bing feeds in up to date search results.
Ben Thompson (of Stratechery) asked Bing if he (Ben) thought there was a recession and it paraphrased an article Ben had published the day before.
(From Ben’s subsequent interview with Sam Altman and Kevin Scott):
> I was very impressed at the recency, how it captures stuff. For example, I asked it, “Does Ben Thompson think there’s a recession?” and it actually parsed my Article on Monday and said, “No, he just thinks tech’s actually being divorced from the broader economy,” and listed a number of reasons.
The Search box with predictive text-like search suggestions.
Results lists
Results lists with adverts.
Results lists with adverts and links to cited sources on the right backing up the Results List.
Results lists with adverts and links to cited sources on the right backing up the Results List and also showing additional search terms and questions in the Results List.
I'm surprised its taken them this long to come up with this...
That’s relatively easy to fix, since autocomplete was probably working on just the most frequent queries and/or phrases. You could manually clean up the dataset.
interesting and if you told it your name/email it could also connect the dots and badmouth you to others or perhaps even purposefully spread false information about you or your business or put your business into a more negative light than it would ordinarilly do
I will give you a more realistic scenario that can happen now. You have a weird Bing conversation, post it on the web. Next time you talk with Bing it knows you shit-posted about it. Real story, found on Twitter.
It can use the internet as an external memory, it is not truly stateless. That means all sorts of attack vectors are open now. Integrating search with LLM means LLM watches what you do outside the conversation.