Capital r-Rationalism definitely bleeds into cult-like behaviour, even if they haven’t necessarily realised that they’re radicalising themselves.
They’ve already had a splinter rationalist group go full cult, right up to & including the consequent murders & shoot-out with the cops flameout: https://en.wikipedia.org/wiki/Zizians
> You have to be extremely verbose in describing all of your requirements. There is seemingly no such thing as too much detail.
Sounds like ... programming.
Program specification is programming, ultimately. For any given problem if you’re lucky the specification is concise & uniquely defines the required program. If you’re unlucky the spec ends up longer than the code you’d write to implement it, because the language you’re writing it in is less suited to the problem domain than the actual code.
Agree, I used to say that documenting a program precisely and comprehensively ends up being code. We either need a DSL that can specify at a higher level or use domain specific LLMs.
> Sigh. People are WAAAY too accepting of urbanist propaganda.
>
> Buses are _barely_ more effective than cars. A regular passenger car with 4 people is more efficient than a city bus. An EV needs 2.5 people (these numbers are for the US).
>
> The explanation is simple:
>
> 1. Buses have to drive _all_ _the_ _time_, even when there are few passengers. As a result, the average bus load tends to be around 10-20 people. And you can not increase the bus interval to compensate for it because it makes off-rush-hour bus commutes impractical.
>
> 2. Buses have INCREDIBLY polluting components: 2-3 drivers for each bus needed to provide the service. They are by far the dirtiest part of the bus. This part can be removed with the self-driving hardware, but...
>
> A full self-driving bus also makes no sense. It defeats the main advantage of self-driving: door-to-door transportation.
Even if you focus only on emissions this completely ignores the cost of congestion, which is huge.
Your complaint about self driving buses makes no sense either. If the most polluting part of the bus is the driver then removing the driver makes the bus far, far less polluting.
When your arguments don't even make sense on their own terms it suggests that you're making them from an emotional position instead of a rational one. That's ok: if you don't like buses just say so, but be honest about it instead of making spurious arguments.
> Even if you focus only on emissions this completely ignores the cost of congestion, which is huge.
Congestion should be fixed by removing buses, de-densifying city cores, and forcing companies to build offices in a distributed fashion.
Meanwhile, replacing buses with shared taxi-style vehicles will do most of the job, while _reducing_ congestion. It's a bit complicated, but it's entirely possible.
The reason is simple, there is an unavoidable tension between the density of bus stops and the average speed. As a result of frequent stops, in most cities buses move at an average speed of less than 20 km/h.
For example, in Seattle it's 15 km/h. This is just 3 times faster than a rapid walk!
If we reduce the number of cars by 2x by adding mild car-pooling during the rush hour, then we'll have more than enough throughput to eliminate congestion _and_ buses in Seattle. This does not generalize to all cities in the US (e.g. Manhattan needs a serious demolishing to become sane) but usually, it's in the same ballpark.
> Your complaint about self driving buses makes no sense either. If the most polluting part of the bus is the driver then removing the driver makes the bus far, far less polluting.
Sure. But why stop there? Buses have an INCREDIBLE impact in the number of lifetimes wasted during commutes.
> When your arguments don't even make sense on their own terms it suggests that you're making them from an emotional position instead of a rational one. That's ok: if you don't like buses just say so, but be honest about it instead of making spurious arguments.
Nope. There are no rational arguments _for_ urbanism. It's a failed obsolete ideology, and it's leading to the downfall of democracty and the rise of populism.
> Congestion should be fixed by removing buses, de-densifying city cores, and forcing companies to build offices in a distributed fashion.
Yes, let's solve congestion by forcing urban sprawl. Forget any efficiency gains by using denser infrastructure, just make everyone drive 10x as far so there's 10x as much road and therefore 1/10th the congestion!
We already do this. You just described the city of Houston. I've been there, it's ass. And the 14 lane Katy highway does it no favors.
Bus stops are often set too close by municipalities, but that’s driven by the lack of density in US housing. Density drives efficiency.
> Manhattan needs a serious demolishing to become sane
Ah yes, lets demolish one of the most economically productive regions of the USA, both in GDP / capita and GDP / km^2 in order to make it easier for people to drive through it.
> Bus stops are often set too close by municipalities, but that’s driven by the lack of density in US housing. Density drives efficiency.
Doesn't matter, dense cities start having their own issues. Instead of taking "the bus", you'll need to wait for the correct bus to arrive. Also, density drives up misery and nothing else. Proven by the birth rate.
> Ah yes, lets demolish one of the most economically productive regions of the USA, both in GDP / capita and GDP / km^2 in order to make it easier for people to drive through it.
Yup, exactly. There's no freaking reason so much GDP has to be crammed into several square miles of space, sucking the life from everywhere else.
IIRC RLHF inevitably compromises model accuracy in order to train the model not to give dangerous responses.
It would make sense if the model used for train-of-though was trained differently (perhaps a different expert from an MoE?) from the one used to interact with the end user, since the end user is only ever going to see its output filtered through the public model the chain-of-thought model can be closer to the original, more pre-rlhf version without risking the reputation of the company.
This way you can get the full performance of the original model whilst still maintaining the necessary filtering required to prevent actual harm (or terrible PR disasters).
Yeah we really should stop focusing on model alignment. The idea that it's more important that your AI will fucking report you to the police if it thinks you're being naughty than that it actually works for more stuff is stupid.
I'm not sure I'd throw out all the alignment baby with the bathwater. But I wish we could draw a distinction between "Might offend someone" with "dangerous."
Even 'plotting terror attacks' is not something terrorists can do just fine without AI. And as for making sure the model wouldn't say ideas that are hurtful to <insert group>, it seems to me so silly when it's text we're talking about. If I want to say "<insert group> are lazy and stupid," I can type that myself (and it's even protected speech in some countries still!) How does preventing Claude from espousing that dumb opinion, keep <insert group> safe from anything?
Let me put it this way: there are very few things I can think of that models should absolutely refuse, because there are very few pieces of information that are net harmful in all cases and at all times. I sort of run by blackstone's principle on this: it is better to grant 10 bad men access to information than to deny that access to 1 good one.
Easy example: Someone asks the robot for advice on stacking/shaping a bunch of tannerite to better focus a blast. The model says he's a terrorist. In fact, he's doing what any number of us have done and just having fun blowing some stuff up on his ranch.
Or I raised this one elsewhere but ochem is an easy example. I've had basically all the models claim that random amines are illegal, potentially psychoactive, verboten. I don't really feel like having my door getting kicked down by agents with guns, getting my dog shot, maybe getting shot myself because the robot tattled on me for something completely legal. For that matter if someone wants to synthesize some molly the robot shouldn't tattle to the feds about that either.
Basically it should just do what users tell it to do excepting the very minimal cases where something is basically always bad.
> it is better to grant 10 bad men access to information than to deny that access to 1 good one.
I disagree when it comes to a tool as powerful as AI. Most good people are not even using AI. They are paying attention to their families and raising their children, living real life.
Bad people are extremely interested in AI. They are using it to deceive at scales humanity has never before seen or even comprehended. They are polluting the wellspring of humanity that used to be the internet and turning it into a dump of machine-regurgitated slop.
1. Those people don’t need frontier models. The slop is slop in part because it’s garbage usually generated by cheap models.
2. It doesn’t matter. Most people at some level have a deontological view of what is right and wrong. I believe it’s wrong to build mass-market systems that can be so hostile to their users interests. I also believe it’s wrong for some SV elite to determine what is “unsafe information”.
Most “dangerous information” has been freely accessible for years.
I used to think that worrying about models offending someone was a bit silly.
But: what chance do we have of keeping ever bigger and better models from eventually turning the world into paper clips, if we can't even keep our small models from saying something naughty.
It's not that keeping the models from saying something naughty is valuable in itself. Who cares? It's that we need the practice, and enforcing arbitrary minor censorship is as good a task as any to practice on. Especially since with this task it's so easy to (implicitly) recruit volunteers who will spend a lot of their free time providing adversarial input.
This doesn’t need to be so focused on the current set of verboten info though. Just make practice making it not say some set of random less important stuff.
Yeah I really don’t care about this case much. Actually a good example of less important stuff. It’s practical things like nuclear physics (buddy majoring has had it refuse questions), biochem, ochem, energetics & arms, etc. that I dislike.
Correct me if I'm wrong--my understanding is that RHLF was the difference between GPT 3 and GPT 3.5, aka the original ChatGPT.
If you never used GPT 3, it was... not good. Well, that's not fair, it was revolutionary in its own right, but it was very much a machine for predicting the most likely next word, it couldn't talk to you the way ChatGPT can.
Which is to say, I think RHLF is important for much more than just preventing PR disasters. It's a key part of what makes the models useful.
Oh sure, RLHF instruction tuning was what turned an model of mostly academic interest into a global phenomenon.
But it also compromised model accuracy & performance at the same time: The more you tune to eliminate or reinforce specific behaviours, the more you affect the overall performance of the model.
Hence my speculation that Anthropic is using a chain-of-thought model that has not been alignment tuned to improve performance. This would then explain why you don’t get to see its output without signing up to special agreements. Those agreements presumably explain all this to counter-parties that Anthropic trusts will cope with non-aligned outputs in the chain-of-thought.
> There’s no proof that ejaculating more actually lowers the chances of prostate cancer. For now, doctors just know they’re connected. It may be that men who do it more tend to have other healthy habits that are lowering their odds.
> Ejaculation doesn’t seem to protect against the most deadly or advanced types of prostate cancer. Experts don’t know why.
I'm not the expert but, like all things, exercise, sleep and diet probably goes a long way.
My recent experience has been that the Vulkan support in llama.cpp is pretty good. It may lag behind Cuda / Metal for the bleeding edge models if they need a new operator.
> Correct, it's impossible to specifically and formally define the natural numbers so that addition and multiplication work. Any definition of the natural numbers will also define things that look very similar to natural numbers but are not actually natural numbers.
Are such objects not inevitably isomorphic to the natural numbers?
Can you give an example of a formal definition that leads to something that obviously isn't the same as the naturals?
In that article you'll see references to "first order logic" and "second order logic". First order logic captures any possible finite chain of reasoning. Second order logic allows us to take logical steps that would require a potentially infinite amount of reasoning to do. Gödel's famous theorems were about the limitations of first order logic. While second order logic has no such limitations, it is also not something that humans can actually do. (We can reason about second order logic though.)
Anyways a nonstandard model of arithmetic can have all sorts of bizarre things. Such as a proof that Peano Axioms lead to a contradiction. While it might seem that this leads to a contradiction in the Peano Axioms, it doesn't because the "proof" is (from our point of view) infinitely long, and so not really a proof at all! (This is also why logicians have to draw a very careful distinction between "these axioms prove" and "these axioms prove that they prove"...)
All of these models appear to contain infinitely sized objects that are explicitly named / manipulable within the model, which makes them extensions of the Peano numbers though, or else they add other, extra axioms to the Peano model.
If you (for example) extend Peano numbers with extra axioms that state things like “hey, here are some hyperreals” or “this Goedel sentence is explicitly defined to be true (or false)” it’s unsurprising that you can end up in some weird places.
We are able to recognize that they are nonstandard because they contain numbers that we recognize are infinite. But there is absolutely no statement that can be made from within the model from which it could be discovered that those numbers are infinite.
Furthermore, it is possible to construct nonstandard models such that every statement that is true in our model, remains true in that one, and ditto for every statement that is false. They really look identical to our model, except that we know from construction that they aren't. This fact is what makes the transfer principle work in nonstandard analysis, and the ultrapower construction shows how to do it.
(My snark about NSA is that we shouldn't need the axiom of choice to find the derivative of x^2. But I do find it an interesting approach to know about.)
No additional axioms are needed for the existence of these models. On the contrary additional axioms are needed in order to eliminate them, and even still no amount of axioms can eliminate all of these extensions without introducing an inconsistency.
Fair warning: when you turn over some of the rocks here you find squirming, slithering things that should not be given access to the light.
reply