How to learn PyTorch for Free! - A Step-by-step Guide

How to learn PyTorch for Free! - A Step-by-step Guide

It can be daunting for a lot of us when starting out with learning PyTorch. If you are wondering how to learn PyTorch as a beginner without getting overwhelmed in the process, then this article will help you get started. I’ve compiled a list of the best free resources I’ve found to date that would help you with the learning process.

The steps for learning PyTorch are as follows:

Alright, I know what you’re thinking: “That’s a lot of content to cover. I thought you said that you won’t make it overwhelming. What are you doing, man?”. Well, fear not, because I don’t expect you to do everything I mentioned :-). Let me explain each step in more detail and the process should make a lot more sense to you.

Step 1: Follow PyTorch’s tutorial

Deep Learning with PyTorch: A 60 Minute Blitz tutorial homepage A 60 Minute Blitz tutorial homepage

[Link]

I think the reasoning behind this one is fairly straightforward. It’s an excellently made tutorial and it helps to get a feel of how PyTorch works. Also, each page of the tutorial can be opened in Google Colab, so you can try out the operations as you read the tutorial (which I highly recommend).

Since the tutorial isn’t too long, it would be good to actually follow it to the end. The key is, don’t expect yourself to complete it in one sitting. I know it says 60 minutes, but take breaks between chapters. If possible, play around with what you learned in each chapter before going to the next. That’ll help you really understand the basics, which is something that a lot of people forget to mention.

The following is an outline of the “A 60 Minute Blitz” tutorial:

  • Tensors

    • Tensor Initialization

    • Tensor Attributes

    • Tensor Operations

    • Converting between Numpy arrays and tensors

  • torch.autograd

    • What is it?

    • How to use it

    • How it works

    • Explanation of the computational graph

    • How to exclude from the computational graph (and why)

  • Neural Networks

    • A small introduction and context relating to torch.autograd

    • Defining a network

    • Loss functions and how to define them

    • Backpropagation

    • Updating the weights

  • Training a Classifier

    • Introduction to datasets

    • Steps to train an image classifier

    • Training on GPU

Now, by the end of this tutorial, you may still have doubtful areas. That’s perfectly fine and that’s where the other resources come in.

Step 2: Watch the Example YouTube video

[Link]

The video walks through an example, line-by-line, and gives you a better idea of the main structure behind most PyTorch programs. So, this video should help you clarify some of the doubts you may have had after the end of the previous step.

For convenience, I’ve embedded the video below:

https://youtu.be/Jy4wM2X21u0

Once you’re done watching the video, try to replicate the steps (train an MNIST classifier) without looking at the video. Unless you have a photographic memory or something, you most likely will need to take peeks at their code. That’s alright. Practice until the assistance you need is minimal.

Step 3: Read the mentioned blog article

[Link]

Alright, so this one is quite a long blog article. So, I don’t expect you to read it end-to-end. But, it’s one of the best resources out there for learning PyTorch and I can’t recommend it enough.

A quick summary of the article

The article begins by explaining some of the foundational topics of neural networks and machine learning, in general. It’ll begin by explaining some concepts using Numpy, and then it’ll move to PyTorch. Now, just because it’s foundational, doesn’t mean it’ll be easy. You may not understand it right away, which is totally fine. So, take your time with it.

The remaining sections of the article cover pretty much all the topics that you may have seen in the previous two steps. These are explained using an example. So, if you think you have the time to read the entire thing, then that would be the best thing. On the other hand, if not, I’ll explain my recommended approach in the next section

My advice for using the article

  • Begin by reading the first three sections

    • A Simple Regression Problem

    • Gradient Descent

    • Linear Regression in Numpy

  • Read through the table of contents

  • Identify the topics that you don’t feel comfortable with

  • Navigate to the respective sections and read them

This approach should help you with getting the basics down.

Step 4: Practice with the GitHub repo

[Link]

At this point, you should be able to understand and work with most scripts that use PyTorch. However, something that I’ve noticed after practicing for some time is that your progress tends to stagnate a bit, and yet, you know that there’s more to learn. This GitHub repo helps with getting over that roadblock.

The repo has tutorial scripts that cover the concepts under three levels of expertise:

  1. Beginner

  2. Intermediate

  3. Advanced

I’d recommend trying out each of these scripts and attempting to recreate them. However, don’t learn the scripts by heart. Instead, focus on the stages that the script goes through to address the specific task. Begin with the Beginner tutorials and progress up, level by level when you feel comfortable. That is, you can recreate the scripts from scratch without looking at the tutorial too often.

Also, don’t forget to check out the Tensorboard tutorial under the 4th section (Utilities). This can be very helpful to keep track of different metrics that you use during training. Additionally, you can start it in parallel with the 1st section.

What Next?

Even after going over all this, there’s still a lot you can learn. The next step is to read actual research papers and try implementing them. Often, research papers provide their own code repository. First, try implementing the proposed algorithms yourself, and then refer to the code repository for help. Once all that’s done, you should feel very comfortable working with PyTorch. Good luck and have fun learning! :-)