For learning (and for development on most projects, until it actually comes time to train the real model) the K80 or whatever the lower tier is on colab is fine as a gpu.
The problem with colab IMO is that if it's your main platform, you'll be pushed to use notebooks for everything which is not really a good practice. Whatever you use, I'd suggest focusing on building a real train.py script (I'm assuming you'll be using python) that takes command line arguments for the hyperparameters. Don't get sloppy and just have things run as a bunch of cells.
If you are learning, my unsolicited advice is don't use built in datasets, make sure you can write datasets / dataloaders yourself so you understand what is going on and can adapt to your own work. All the stock examples using built in mnist or whatever gloss over the most important parts of setting up the data
I want to add to your advice: not all of your code has to live in Colab. If you create a public repo on GitHub, then on Colab you can simply do a pip install using the git URI for your repo. Your GitHub repo will need to be setup as a proper Python library, but there are many simple examples you can find on the web.
I find this technique to be particularly useful since the same GitHub based libraries that I use on Colab I sometimes also use from Common Lisp locally on my laptop using py4cl.
The problem with colab IMO is that if it's your main platform, you'll be pushed to use notebooks for everything which is not really a good practice. Whatever you use, I'd suggest focusing on building a real train.py script (I'm assuming you'll be using python) that takes command line arguments for the hyperparameters. Don't get sloppy and just have things run as a bunch of cells.
If you are learning, my unsolicited advice is don't use built in datasets, make sure you can write datasets / dataloaders yourself so you understand what is going on and can adapt to your own work. All the stock examples using built in mnist or whatever gloss over the most important parts of setting up the data