paper looks nice! i think what they found was that they can recover the input sequence by trying all tokens from the vocab and finding a unique state. they do a forward pass to check each possible token at a given depth. i think this is since the model will encode the sequence in the mid flight token so this encoding is revealed to be unique by their paper. so one prompt of 'the cat sat on the mat' and 'the dog sat on the mat' can be recovered as distinct states via each token being encoded (unclear mechanism but it would be shocking if this wasn't the case) in the token (mid flight residual).
anyone who spends 10 years writing performance reviews and rebuilding a service they already rebuilt last year, all to collect RSUs, was never going to land us on mars
Well don't do it and instead of using an off the shelf library that is known to work while the rest of the development team isn't reinventing the wheel.
Doing it for fun and education is fine of course.
I am an AI interpretability researcher and have a new proposal for a way to measure the per token contribution of each head and neuron in LLMs.
I found that the normalisation that happens in every LLM is avoided by modern attribution methods despite it having a large impact on the model's computation.
Here is the full preprint paper and the code I used. https://github.com/patrickod32/landed_writes
Happy to some insight from any interested people and would like to know if other people here have been working on anything similar. This seems like a real gap in the research to me.
the average human certainly can not run that fast. treadmills in gyms usually cap at 20km/h and how many people on the street do you think could handle it? (very few of course)
this should be the real headline. I am certain and beyond any doubt that I speak for all of us hackernews readers when I say that I have no frame of reference on the value of cryptocurrencies without comparing first with the Malaysian Ringgit
In case you intentionally left out the /s at the end of your comment...
You'd be surprised how advanced the banking systems are in some 3rd world countries. In many cases the banks skipped over a lot of the legacy cruft that 1st world countries are struggling to get rid of now.
Also, in countries where the banking system or currency is falling apart, they use USD cash. Zimbabwe is doing this, Argentina is doing this, I'm sure more places also do this.
Also also, often where you have crumbling banking or currency, you also have crumbling internet and/or electricity infrastructure. Nobody is sitting on an upturned milk crate in their dilapidated corrugated iron shack in the middle of a slum fucking trading bitcoin on Coinbase so that they can buy bread.
Bitcoin is 99% used as a speculative investment instrument and as a means to participate in illicit markets (drugs etc). There is nothing altruistic or democratising about it.
the certainty of this post is its undoing. Someone once relayed this story .. while speaking between two intelligent but not expert adults, about the "size of the Internet" .. one said .. the Internet is so big, that there are whole social groups online, for example a popular multi-player game in Korea, that you have never heard of.. yet there are hundreds of thousands of active participants in this thing that you have never heard of.. that is how big the Internet is..
So wrt. this Bitcoin.. one armchair Admiral is going to pronounce that "electronic cryptographic money exchange Only Does This (list of things that are obnoxious).
It is parochial.. the definition of uninformed.. people who are directly involved don't know all the things that are being done with Bitcoin.. dismiss simplistic portrayals
I'd bet the biggest use case is scamming/extorting people. It has been the fastest growing use case, it must move more money than all other use cases combined.
I mean a lot of people speculate that BTC is going up due to Trump's policies and promises, but an equally logical reason is that people expect the fall of USD. Currencies tend to fall after heated elections anywhere.
USD/EUR isn't necessarily a good reference either because they could be headed the same direction - an aggressive POTUS could be viewed as bad for NATO. Even gold, because that gold is going somewhere physical.
But there's not much other evidence that the USD is weakening after the election.
That's simply untrue. There's been constant swings in valuation of around 20x recorded through the past 3000+ years (and decent evidence this also happened before that).
Pretty much all of recorded history is local govts trying to fix the price of gold to remove this instability, and in pretty much every case those govts had to abandoned the fix as actual market values caused local mints to implode.
There's ample research available on google scholar covering all of these points.
This is one of those things that people say as though it's a good thing, but since precious metals are a negative carry asset (ie it costs money to store and insure gold or silver) this means that when you take compounding into effect they have been responsible for a devastating destruction in wealth over that time. Even more so if you consider the opportunity cost of holding metals when you could be holding a productive asset.
