Introduction to Linear Regression with Python

Linear regression is a fundamental machine learning algorithm used for predicting a quantitative response. In this tutorial, we'll walk through a step-by-step process of implementing linear regression using Python's scikit-learn library. We'll start with a simple example involving one independent variable and then expand to multiple variables. By the end of this tutorial, you'll understand how to prepare data, build a regression model, visualize results, and evaluate model performance.

Table of Contents

1. Understanding the Dataset
2. Importing Libraries
3. Loading and Exploring the Data
4. Data Preparation
   • Dropping Unwanted Fields
   • Handling Missing Values
   • Normalization
   • Outlier Cleaning
5. Visualizing Data with Seaborn
6. Simple Linear Regression with One Feature
   • Splitting the Data
   • Training the Model
   • Interpreting Coefficients
   • Making Predictions
   • Visualizing the Regression Line
   • Model Evaluation
7. Multiple Linear Regression with Multiple Features
   • Updating the Data
   • Retraining the Model
   • Model Evaluation
8. Conclusion
9. Jupyter Note

1. Understanding the Dataset

We'll use a synthetic dataset called advertising.csv, which contains data about advertising budgets and sales figures. The dataset includes the following features:

• TV: Advertising budget spent on TV (in thousands of dollars)
• Radio: Advertising budget spent on Radio
• Newspaper: Advertising budget spent on Newspaper
• Sales: Sales of the product (in thousands of units)

For the initial part, we'll use the TV budget to predict Sales.

2. Importing Libraries

Let's start by importing the necessary Python libraries.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn import metrics

Explanation:

• pandas: For data manipulation and analysis.
• numpy: For numerical operations.
• matplotlib.pyplot: For plotting graphs.
• seaborn: For statistical data visualization.
• sklearn.linear_model: Contains the LinearRegression class.
• sklearn.model_selection: For splitting data into training and testing sets.
• sklearn.metrics: For evaluating model performance.

3. Loading and Exploring the Data

We'll load the dataset into a pandas DataFrame and explore it.

# Assuming the CSV is available in the same directory
df = pd.read_csv('advertising.csv')

Explanation:

• pd.read_csv: Reads a CSV file into a DataFrame.
• 'advertising.csv': The CSV file containing our dataset.

Now, let's look at the first few rows of the data.

df.head()

Explanation:

• df.head(): Displays the first five rows of the DataFrame.

Let's get more information about the dataset.

df.info()

Explanation:

• df.info(): Provides a concise summary of the DataFrame, including data types and non-null values.

Let's check some statistical details.

df.describe()

Explanation:

• df.describe(): Generates descriptive statistics.

4. Data Preparation

Dropping Unwanted Fields

Suppose our dataset has an unnecessary column called 'Unnamed: 0'.

if 'Unnamed: 0' in df.columns:
    df = df.drop('Unnamed: 0', axis=1)

Explanation:

• df.drop(): Removes specified labels from rows or columns.
• axis=1: Specifies that we're dropping a column.

Handling Missing Values

Check for missing values.

df.isnull().sum()

Explanation:

• df.isnull(): Detects missing values.
• .sum(): Sums up the number of missing values per column.

Fill missing values if any.

df = df.fillna(df.mean())

Explanation:

• df.fillna(): Fills NA/NaN values.
• df.mean(): Calculates the mean for each column.

Normalization

For linear regression, normalization isn't strictly necessary, but it's good practice.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
df[['TV', 'Radio', 'Newspaper']] = scaler.fit_transform(df[['TV', 'Radio', 'Newspaper']])

Explanation:

• StandardScaler(): Standardizes features by removing the mean and scaling to unit variance.
• fit_transform(): Fits the scaler to the data and then transforms it.

Outlier Cleaning

Plotting boxplots to identify outliers.

sns.boxplot(data=df[['TV', 'Radio', 'Newspaper']])
plt.show()

Explanation:

• sns.boxplot(): Creates a boxplot to visualize the distribution.

For simplicity, we'll skip outlier removal in this tutorial.

5. Visualizing Data with Seaborn

Plotting a correlation heatmap.

corr = df.corr()
sns.heatmap(corr, annot=True)
plt.show()

Explanation:

• df.corr(): Computes pairwise correlation of columns.
• sns.heatmap(): Plots a heatmap of the correlation matrix.
• annot=True: Writes the data value in each cell.

6. Simple Linear Regression with One Feature

We'll use TV advertising budget to predict Sales.

Splitting the Data

X = df[['TV']]  # Independent variable
y = df['Sales']  # Dependent variable

# Split the data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Explanation:

• X: Features (TV budget).
• y: Target variable (Sales).
• traintestsplit(): Splits arrays or matrices into random train and test subsets.
• test_size=0.2: 20% of data is used for testing.
• random_state=42: Ensures reproducibility.

Training the Model

lr = LinearRegression()
lr.fit(X_train, y_train)

Explanation:

• LinearRegression(): Creates an instance of the linear regression model.
• fit(): Fits the model to the training data.

Interpreting Coefficients

print('Intercept:', lr.intercept_)
print('Coefficient:', lr.coef_)

Explanation:

• lr.intercept_: The y-intercept of the regression line.
• lr.coef_: The slope (coefficient) of the line.

Making Predictions

y_pred = lr.predict(X_test)

Explanation:

• predict(): Predicts the target values for the test set.

Visualizing the Regression Line

plt.scatter(X_test, y_test, color='blue', label='Actual')
plt.plot(X_test, y_pred, color='red', linewidth=2, label='Predicted')
plt.title('TV Advertising Budget vs Sales')
plt.xlabel('TV Advertising Budget (Standardized)')
plt.ylabel('Sales')
plt.legend()
plt.show()

Explanation:

• plt.scatter(): Plots the actual data points.
• plt.plot(): Plots the regression line.
• plt.legend(): Adds a legend to the plot.

Model Evaluation

Calculating error metrics.

from sklearn.metrics import mean_squared_error, r2_score

mse = mean_squared_error(y_test, y_pred)
r2 = r2_score(y_test, y_pred)

print('Mean Squared Error:', mse)
print('R^2 Score:', r2)

Explanation:

• meansquarederror(): Calculates the mean squared error.
• r2_score(): Calculates the R-squared score.

7. Multiple Linear Regression with Multiple Features

Now, we'll include Radio and Newspaper budgets as additional features.

Updating the Data

X = df[['TV', 'Radio', 'Newspaper']]  # Multiple features
y = df['Sales']

# Split the data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Retraining the Model

lr_multi = LinearRegression()
lr_multi.fit(X_train, y_train)

Interpreting coefficients.

print('Intercept:', lr_multi.intercept_)
print('Coefficients:', lr_multi.coef_)
print('Feature Names:', X.columns)

Explanation:

• lrmulti.coef: Array of coefficients for each feature.
• X.columns: Names of the features.

Making Predictions

y_pred_multi = lr_multi.predict(X_test)

Model Evaluation

mse_multi = mean_squared_error(y_test, y_pred_multi)
r2_multi = r2_score(y_test, y_pred_multi)

print('Mean Squared Error (Multiple Features):', mse_multi)
print('R^2 Score (Multiple Features):', r2_multi)

8. Conclusion

In this tutorial, we've:

• Explored and visualized the dataset.
• Performed data preparation steps.
• Built a simple linear regression model with one feature.
• Visualized the regression line.
• Evaluated the model using error metrics.
• Extended the model to include multiple features.
• Observed improvements in model performance.

Note: Remember to replace 'advertising.csv' with the actual path to your dataset in Google Colab.