PyTorch tensor objects for storing and updating model parameters
We covered tensor objects in Chapter 12, Parallelizing Neural Network Training with PyTorch. In PyTorch, a special tensor object for which gradients need to be computed allows us to store and update the parameters of our models during training. Such a tensor can be created by just assigning requires_grad
to True
on user-specified initial values. Note that as of now (mid-2021), only tensors of floating point and complex dtype
can require gradients. In the following code, we will generate tensor objects of type float32
:
>>> a = torch.tensor(3.14, requires_grad=True)
>>> print(a)
tensor(3.1400, requires_grad=True)
>>> b = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
>>> print(b)
tensor([1., 2., 3.], requires_grad=True)
Notice that requires_grad
is set to False
by default. This value can be efficiently set to True
by running requires_grad_()
.
method_()
is an in...