- “your classifier is secretly an energy-based model and yiu should treat it like one” paper
- self-supervision
- distance metric learning
Places where you can read implementations:
- lucidrains’ github
- timm computer vision models library
- fastai
- labml (annotated quite nicely)
Biggest foreseeable headaches:
- not suuper easy to do test-driven development
- data normalization (floating point error, not using eg batchnorm)
- sensitivity of model performance to (hyper)params (layer sizes, learning rates, optimizer, etc)
- impatience
- lack of data
I’d also recommend watching Mark Saroufim live code in PyTorch, on YouTube. My 2 cents, you can only get really fast as well as good at this with a lot of experience. A lot of rules-of-thumb have to come together just right for the whole system to work.
Early days, but trying to solve the lack of data problem with a project called "Oxen".
At it's core it's a data version control system. Built from the ground up to handle the size and scale of some of these ML datasets (unlike git-lfs or DVC which I have found to be relatively slow and hard to work with). Also building out a web hub similar to GitHub to collaborate on the data with a nice UI.
Would love any feedback on the project as it grows! Here's the github repo:
- devdocs.io for pytorch
- conda for packaging w CUDA
- einops
- tensorboard
- huggingface datasets
Interesting models/structures:
- resnet
- unet
- transformers
- convnext
- vision transformers
- ddpm
- novel optimizers
- generative flow nets
- “your classifier is secretly an energy-based model and yiu should treat it like one” paper
- self-supervision
- distance metric learning
Places where you can read implementations:
- lucidrains’ github
- timm computer vision models library
- fastai
- labml (annotated quite nicely)
Biggest foreseeable headaches:
- not suuper easy to do test-driven development
- data normalization (floating point error, not using eg batchnorm)
- sensitivity of model performance to (hyper)params (layer sizes, learning rates, optimizer, etc)
- impatience
- lack of data
I’d also recommend watching Mark Saroufim live code in PyTorch, on YouTube. My 2 cents, you can only get really fast as well as good at this with a lot of experience. A lot of rules-of-thumb have to come together just right for the whole system to work.