Yes, it has to in fact. If you have zero context to attend to in a transformer, and you try to predict the first token, you effectively are multiplying a zero-vector by the attention head, making all tokens equally likely in the final softmax (unless the lm_head has a bias, but at least in GPT it does not).
So the <|beginning of text|> token, with no context before it, learns to predict the first-token-in-a-document distribution. That's not quite the same as predicting nothing at all.
For finding interesting Ethereum addresses on the GPU (uses SHA3 instead of MD5) there's a tool called profanity.
Amusingly the tool was archived by the author years ago, and just last year a stupid bug in the random initialization code was found (32 bytes of state initialized with ~31 bits of entropy), leading to ~millions of $ in stolen ETH by exploiters.
I haven't looked at the project but would assume that several people generated their wallets by using this tool. The attackers were thus able to generate keys for existing wallets by simply bruteforcing the measly 31 bits that were used in the tool's random initialization.
> If anyone has an idea why the TIA’s designers used LFSRs for this stuff, I’d love to hear about it.
I'm guessing because a counter would have ripple carry (i.e. five extra gate delays when rolling over from 111111 to 000000) or need extra gates for carry lookahead and an LFSR is constant-delay.
The latency notwithstanding, you can implement a counter with an LFSR that hits every unique value in a 2^n-1 sequence with less transistors than a standard counter.
I have to admit I stole the idea from this 256-byte demo (after it wiped the floor with the one I made at a demo party): http://www.pouet.net/prod.php?which=5327
Yeah, that one wasn't too hard but it's nice to have it to check your work. I didn't go into it in the post, but the real power of SymPy is its cse() function, which I used to do all the common subexpression elimination for the integer math version at the bottom.
I think the built-in code generator is a bit strange, though: for example, it always prints integers as integers, and not as floats depending on the context, so code often doesn't typecheck.
There are two internal registers (IND and OPR) that aren't visible to the programmer. See the block diagram at the bottom of the post. The flags are in the ALU, not in the register file.
after updating to Catalina I was suddenly unable to edit my own ~/.ssh/config. I still can't figure out why not. I could rename it and make a copy, though, and the old one shows this:
part 3 is to find the secp256k1 private key for satoshi's bitcoins