It's completely mental that HF ran into cyber safety blocks trying to use OpenAI models to help defend against the attack. They could only rely on a local hosted chinese model in the end.
No the models are just ass at communication without being directed.
Try asking them to make useful diagrams for some stuff in a codebase, out of the box without excessive hand holding they don't make good choices about what's worth communicating and how to do it.
You see this in their pointless frontend copy all the time too.
Like any time you make them do any UI without strict directions they'll almost always add a label describing the feature somewhere. Ask for a calculator, and instructions for what the different buttons do might appear in the bottom out of nowhere for example.
Same concept of "over-sharing" seems to prevalent in a bunch of domains when it comes to LLMs, sometimes more visible, sometimes less.
Is there any model that knows how to smooth an overly literary text over? I find Opus and Fable constantly decorate the documentation they write like a damn 19/20th century writer. We're working with IT stuff yet it writes like it's going to win some Pulitzer prize. It's that one thing I don't get why they can't train them to do properly: I have not encountered a model yet that sticks to the current language of the domain it's tasked with.
Not sure how to fully fix this but I remember a session last week where I got so fed up mid way though reading a response that I used the following:
"give me this again without jargon invented this session at high density
and with a couple (maybe more or less) simple useful ascii diagrams underneath each design"
The context is that I was discussing an experimental new idea for my video game review analysis product.
Designs 1,2, and 3 were horrible: the model even suggested a rejection after the word soup so it would have been pointless to waste my fleeting time on Earth reading it.
Otherwise, I generally really enjoyed using fable for bouncing ideas. It was an absolute joy to have this thing provide useful criticism, analyse sample data, and create prototypes so that I could elevate my understanding of the problem without stepping down from a pure intuition/design headspace.
But I don't consider the purely model written code usable for a feature this important. I'll probably scrap it entirely and start from scratch with newfound understanding.
There’s confusion about the classifiers on Fable. They don’t ban chemistry and biology topics they flag as a potential risk, they ban anything related to chemistry or biology at all. This is intentional, and directly stated on the model card, but seems so absurd that there can be assumption it must be a misreading.
Being a researcher somewhat connected to chemistry and biology, Fable has been the most useless model I have ever tried. Essentially all work has instantly downgraded to Opus.
Probably because the cost of blocking "is mitochondria the powerhouse of the cell" is nearly zero, while the cost of allowing "how do I synthesize the Spanish flu" is approximately infinite.
I think the source of this misapprehension is that,
You are comparing wet work in a lab to writing code on a computer.
When you screw up an exploit, you fail to execute the exploit. Famously, just like software's near zero marginal cost of distribution, the marginal cost of failure is nearly zero.
You can screw up an infinite number of times on your way to a successful exploit.
If you screw up with lethal agents in a lab? You die.
Here's a non-exhaustive list,
Dora Lush died after accidentally pricking her finger with a needle containing lethal scrub typhus while attempting to develop a vaccine for the disease
A 23-year-old laboratory assistant at the London School of Hygiene and Tropical Medicine, was infected with smallpox after observing the harvesting of live smallpox virus from eggs without isolation cabinets at that time. The assistant was hospitalised and before being isolated, she infected two visitors to a patient in an adjacent bed, both of whom died. They in turn infected a nurse, who survived
Ebola laboratory infection by the accidental stick of contaminated needle in the United Kingdom
Researcher Nikolai Ustinov was lethally infected with the Marburg virus after accidentally pricking himself with a syringe used for inoculation of guinea pigs. The accident occurred at the Scientific-Production Association "Vektor" (today the State Research Center of Virology and Biotechnology "Vektor") in Koltsovo, USSR (today Russia).
"lethally infected with the Marburg virus after accidentally pricking himself"
Anything lethal enough to kill other humans is lethal enough to kill you.
And if you don't know what you're doing — and for this argument you're saying this person has to ask a LLM "how do I spanish flu?" then they definitely don't know what they're doing, the number of ways you will die far outnumber the ways you can succeed.
And this, of course, doesn't even cover the cost of equipment, the precursors, sourcing the highly specific materials needed, then setting the equipment up... etc.
The same is true for the Bosch-Haber / Haber-Bosch process, which famously made WW1 possible. Every HS'er learns about the process and the steps. Steps that were classified once upon a time and were the subject of negotiation at the Versailles.
Does that mean a HS'er (or any adult) can set up an experiment that works at 177 times the pressure of the Earth's atmosphere to do anything at any scale without significant infrastructure and help?
The people who can do this are domain experts, and they've been able to do this with COTS stuff since the 1990s, at the very least, for a price of around $2M – https://en.wikipedia.org/wiki/Project_Bacchus . And those people don't need a LLM to tell them what to do. In fact, they're the exact people who'll have access to unrestricted versions of these LLMs.
And from a security perspective, I would bet good money that flooding the FBI's tip line with junk about every teenager trying to learn "what be a mitochondria" does more harm to the effort of finding people who could be planning such a thing than it helps. It takes more resources to go through the mass of false negatives that have now been created as matter of policy.
These experiments have been run. The fictional scenario of someone learning how bioweapons work and conjuring up a plague isn't real and it hurts humanity as a whole to impede the sciences over it.
Because what someone can flail around in / do is learn about immunology / try to "cure cancer" with a LLM and hopefully get started on a long career in medicine. Or, a discovery that matters.
Because in those cases, if and when they do end up at a lab, screwing up doesn't mean death. Just tons of wasted time (and money). And they will fail / screw up. Just look at literally every undergrad in any lab and the expensive messes they create.
-
And last, but not least, yes. Teenage hackers have been a meme for decades.
First of all, sure if you screw up designing a bioweapon you die, but unfortunately that does not necessarily mean the bioweapon dies with you, quite the contrary it can kickstart its propagation.
Second, in your argumentation you assume that the experiments are extremely difficult or costly. Thankfully, so far, it seems to be the case that they are too difficult (either due to domain knowledge, or to difficulty of obtaining the necessary components/equipment).
But there is no clear reason to think it will remain this way (e.g. crispr allows for genetic engineering which is very cheap). And it appears that, for domain knowledge, capable LLMs are rapidly reducing that barrier. We are not there yet of course, but I think it is crazy to dismiss these concerns, which are very real and crucially are NOT just brought up by the big labs.
No, the source of the misapprehension is that you are ignoring how easy it is to source the input components and combine them into a bioweapon these days.
Correct, there is a small number of people who have been able to do it at high cost for a long time.
Now, there is an ever-growing number of people who are able to do it at an ever-falling cost.
That's the entire issue. Do you dispute that this is what's happening?
> I would bet good money that flooding the FBI's tip line with junk about every teenager trying to learn "what be a mitochondria" does more harm
Did someone propose doing that? Or is this a strawman?
> No, the source of the misapprehension is that you are ignoring how easy it is to source the input components and combine them into a bioweapon these days.
Citation needed. Where are these home biolabs? Why haven't any leaked yet like home meth labs?
Who said anything about a home biolab? Are you thinking a possible solution is just to block the terrorists, irresponsible corporations, or evil governments from LLMs? Obviously not.
In any case, the "home biolab" required to do this stuff gets smaller and more accessible every day. Biochemistry, like virtually every other complex procedural field, has become heavily outsourced. You can literally order genetic fragments even of known pathogens on the Internet, shipped to your door. There should obviously be much more aggressive restrictions on manufacturing known pathogen fragments, but 1) every money-hungry lab would need to volunteer to participate, and 2) it's totally unclear how they'd detect novel pathogen fragments that unsafe AI would be happy to help predict a couple thousand of.
Today, composing that into a working virus might require an undergrad biochem education, a hundred grand, and a bunch of patience (and risk), but that describes millions of people. As GP pointed out, it wasn't too long ago the group of people with this capability was fewer than a dozen individuals on the planet. And as is obvious, we are trending in one direction. We are not trending the other direction.
Give it a shot, it's not in beta. It's much more powerful and fun to use than `--watch`. Works best with frontier models but is compatible with open source / local models too. If you have feedback please let me know!
I do that a decent chunk of the time yeah especially for learning. I also have a bunch of marimo notebooks that double as clis and they're lovely.
But sometimes I want to do something too specific or high fidelity and it's just easier to get the clanker to write typescript and make a webpage/components.
hello, please fix needing to reauth every day (sometimes with email verification). This started happening this month. It's tedious and makes switching very tempting.
Deeply unserious company, flip flopping on policy every week with ludicrous, sometimes invisible, guard rails on their top model. Imagine trying to make business decisions about AI use with this.
I love Fable for many things including coding but I cannot justify basing any internal AI tools or LLM powered products on top of Anthropic's offerings.
reply