Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Powers of 3 don't pack well into binary memory...

A 1 bit multiplier in silicon is a single logic gate, but a ternary decoder to decode a packed tri-state 'weight' is bigger.

I therefore suspect that this method will be extended to make all weights simple 1 or 0 (ie. Binary). Perhaps that will be done by having half the weights have 1 or 0 values, while the other half are -1 or 0.



5 trits fit into 1 byte pretty well, since 3^5 = 243 is just under 2^8 = 256.

That should be called an 8/5 = 1.6 bit model though, while the paper names it 1.58 bit, closer to log_2(3) ~ 1.5849625


But the decoder for that will be 25+ gates, which is huge compared to the handful of gates to use the resulting weights.


Would be nice to have hardware instructions that work on 5 tris natively.


You can build dedicated silicon with ternary gates: https://medium.com/@rxseger/exploring-ternary-logic-tnand-an...

Not sure if it's more efficient than just binary digital circuits in highly integrated chip, though.


It's optimal if your program is naturally ternary, which this one is. Using three signals, rather than ternary gates, is less effective, because you need much more precision to detect two different voltage levels rather than just up and down.


I think it's the right chain of thought. You could either have 0/1 and then have additional nodes with negative activation functions, or -1/1

-1/1 is appealing to me (0 = -1) because bit hackery could be used instead of the multiplication function, presumably on integral or fixed-point representations. The goal would be to eliminate any "if/then" like "if 0 do this if 1 do that" to avoid the need for branch prediction - there are bit-hackery ways to bypass this. That would lend itself well to all existing processors, ASICs, FPGAs, GPUs, etc.


can't you have 2 bits ? first bit for the sign second bit for the 1 0 you can represent -1 +1 +0 -0




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: