Linear Regression

Linear regression is a way to find the linear relationship between the dependent and independent variable by minimizing the distance.

Linear regression is a supervised machine learning approach. This approach is used for classification of order discrete category. In this section, we will understand how to build a model by which a user can predict the relationship between the dependent and independent variable.

In simple terms, we can say the relationship between both the variable, i.e., independent or dependent, is known as linear. Suppose Y is the dependent and X is an independent variable, then the linear regression relationship of these two variables is

Y=AX+b

A is the slope.
b is y-intercept.

Initial State

Final State

There are three basic concepts which are essential to understand to create or learn basic linear model.

1. Model class

It is very typical to code everything and writes all the function when required, and it's not our motive.

It is always better to writing numeric optimization libraries rather than writing all the code and functions, but business value can also be increased if we built it on top of prewritten libraries to get things done. For this purpose, we use the implementation of the nn package of PyTorch. For this, we have first to create a single layer.

Linear layer use

Each linear module computes the output from the input, and for weight and bias, it holds its internal Tensor.

There are several other standard modules. We will use a model class format, which has two main methods, which are as follows:

Init: Used for defining a linear module.
Forward: With the help of forwarding method predictions are made on the basis of that we will train our linear regression model

2. Optimizer

The optimizer is one of the important concepts in PyTorch. It is used to optimize our weight to fit our model into the dataset. There are several optimization algorithms such as gradient descent and backpropagation which optimize our weight value and fit our model best.

Various optimization algorithms are implemented by torch.optim package. To use torch.optim, you have to construct an optimizer object which will update the parameter based on the computer gradient and will hold the current state. The object is created as follows:

Optimizer=optim.SGD(model.parameters(), lr=0.01, momentum=0.9)
Optimizer=optim.Adam([var1, var2], lr=0.0001))

A step() method is implemented by all optimizer, which updates the parameters. There are two ways to use it

1) Optimizer.step()

This is very simple method and supported by many optimizers. After computing the gradients using backward () method, we can call the optimizer.step() function.

Example:

for input, target in dataset:
	optimizer.zero_grad()
	output=model(input)
	loss=loss_fn(output, target)
	loss.backward() 	
	optimizer.step()

2) Optimizer.step(closure)

There are some optimization algorithms such as LBFGS, and Conjugate Gradient needs to re-evaluate the function many times, so we have to pass it in a closure which allows them to recompute your model.

Example:

for input, target in dataset:
def closure():
	optimizer.zero_grad()
	output = model(input)
	loss = loss_fn(output, target)
	loss.backward()
	return loss
	optimizer.step(closure)

Criterion

The criterion is our loss function, which is used to find loss. This function is used from the torch nn module.

Example:

Functions and objects which are required

Import torch
From torch.autagrad import Variable

And we need to define some data and assign them to variables in the following way

xdata=Variable(torch.Tensor([[1.0],[2.0],[3.0]]))
ydata=Variable(torch.Tensor([[2.0],[4.0],[6.0]]))

Following is the code which gives us prediction for training a complete regression model. It's just to understand how we implement the code and what function we used to train a regression model.

import torch 
from torch.autograd import Variable 
xdata = Variable(torch.Tensor([[1.0], [2.0], [3.0]]))
ydata = Variable(torch.Tensor([[2.0], [4.0], [6.0]])) 
class LRM(torch.nn.Module):
	def __init__(self): 
		super(LRM, self).__init__()  
		self.linear = torch.nn.Linear(1, 1)
	def forward(self, x): 
		ypred = self.linear(x)  
		return ypred  
ourmodel = LRM()
criterion = torch.nn.MSELoss(size_average = False) 
optimizer = torch.optim.SGD(ourmodel.parameters(), lr = 0.01) 
for epoch in range(500): 
	predy = our_model(xdata) 
	loss = criterion(predy, ydata)  
	optimizer.zero_grad() 
	loss.backward() 	
	optimizer.step() 
	print('epoch {}, loss {}'.format(epoch, loss.item()))
newvar = Variable(torch.Tensor([[4.0]]))
predy = ourmodel(newvar)
print("predict (after training)", 4, our_model(newvar).data[0][0])

Output:

epoch0,loss1.7771836519241333
epoch1,loss1.0423388481140137
epoch2,loss0.7115973830223083
epoch3,loss0.5608030557632446
.
.
.
.
epoch499,loss0.0003389564517419785
predict (after training) 4 tensor(7.9788)