Hi mark, Wanted to know if the float4 training is possible with torchao as we tr... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		darkninja on Oct 1, 2024 \| parent \| context \| favorite \| on: PyTorch Native Architecture Optimization: Torchao Hi mark, Wanted to know if the float4 training is possible with torchao as we trying to fit a large model on a single GPU for training.

formalsystem on Oct 1, 2024 [–]

we have experimental support for float4 training with the mx formats https://github.com/pytorch/ao/tree/main/torchao/prototype/mx...

But that's waiting for Blackwell to be released so we get the hardware support. SO recommendation for now would be to use either fp8 training or int8 training

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact