🚀 Month 3 — AI & LLM Fundamentals (with code)

Additional Links:

ML Linear Regression Zero to Hero

Boston Kaggle

Tooling you’ll need this month

• Python 3.11+
• VS Code
• Create & use a virtualenv each week
• pip install -r requirements.txt for each project

Week 1 — Foundations for AI: NumPy, Pandas, Plotting, Data Thinking

Goals

• Understand arrays, vectors, matrices
• Load/clean/tabulate data with Pandas
• Make simple visualizations and basic EDA (exploratory data analysis)

File tree

week1-foundations/
├─ requirements.txt
├─ data.csv             # (we'll generate a tiny dataset in code if you don't have one)
└─ notebook.ipynb       # optional; code below also works as .py

requirements.txt

numpy==1.26.4
pandas==2.2.2
matplotlib==3.8.4

Starter code (foundations.py)

# Create venv, install reqs, then: python foundations.py
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pathlib import Path

# --- NumPy basics ---
a = np.array([1, 2, 3, 4], dtype=np.float32)
b = np.array([10, 20, 30, 40], dtype=np.float32)
print("Vector add:", a + b)
print("Dot product:", a @ b)  # 1*10 + 2*20 + ...

# Matrix ops
M = np.array([[1, 2], [3, 4]], dtype=np.float32)
v = np.array([5, 6], dtype=np.float32)
print("Matrix x vector:", M @ v)

# --- Pandas basics ---
# If there's no real CSV, generate one
csv_path = Path("data.csv")
if not csv_path.exists():
    rng = np.random.default_rng(42)
    study_hours = rng.uniform(0, 8, size=60).round(2)
    noise = rng.normal(0, 5, size=60)
    marks = (study_hours * 10 + 30 + noise).clip(0, 100).round(1)
    df = pd.DataFrame({"study_hours": study_hours, "marks": marks})
    df.to_csv(csv_path, index=False)

df = pd.read_csv(csv_path)
print("\nHead:\n", df.head())
print("\nDescribe:\n", df.describe())

# --- Plotting ---
plt.figure()
plt.scatter(df["study_hours"], df["marks"])
plt.title("Study Hours vs Marks")
plt.xlabel("Study Hours")
plt.ylabel("Marks")
plt.grid(True)
plt.tight_layout()
plt.savefig("scatter.png")
print("\nSaved scatter plot -> scatter.png")

Mini‑project

• Add columns: passed = marks >= 50
• Plot distribution (histogram) of marks
• Compute correlation between study_hours and marks
• Export a short EDA report (markdown or PDF)

Week 2 — Machine Learning Essentials (scikit‑learn)

Goals

• Train/test split & metrics
• Regression (predict numbers) and Classification (predict labels)
• Pipelines that combine preprocessing + model

File tree

week2-ml/
├─ requirements.txt
├─ regression_basic.py
├─ classification_basic.py
└─ pipeline_classification.py

requirements.txt

numpy==1.26.4
pandas==2.2.2
scikit-learn==1.5.1
matplotlib==3.8.4

Simple Regression (regression_basic.py)

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score, mean_absolute_error, mean_squared_error

# Synthetic "study hours -> marks" dataset
rng = np.random.default_rng(7)
hours = rng.uniform(0, 8, size=200)
noise = rng.normal(0, 5, size=200)
marks = (hours * 9 + 35 + noise).clip(0, 100)

X = hours.reshape(-1, 1)
y = marks

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=7)

model = LinearRegression()
model.fit(X_train, y_train)

pred = model.predict(X_test)
print("R^2:", round(r2_score(y_test, pred), 3))
print("MAE:", round(mean_absolute_error(y_test, pred), 2))
print("RMSE:", round(np.sqrt(mean_squared_error(y_test, pred)), 2))
print("Coef (slope):", model.coef_, "Intercept:", round(model.intercept_, 2))

Simple Classification (classification_basic.py)

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

data = load_iris()
X, y = data.data, data.target

X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)

clf = LogisticRegression(max_iter=1000)
clf.fit(X_tr, y_tr)

pred = clf.predict(X_te)
print("Accuracy:", round(accuracy_score(y_te, pred), 3))
print("\nClassification Report:\n", classification_report(y_te, pred, target_names=data.target_names))
print("\nConfusion Matrix:\n", confusion_matrix(y_te, pred))

Pipeline + Text (TF‑IDF) (pipeline_classification.py)

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

texts = [
    "I love this product", "Amazing quality", "Very good, recommended",
    "Terrible experience", "I hate it", "Worst ever", "Pretty decent", "Not bad", "Awful packaging",
    "Fantastic service", "Highly recommend", "Great value", "Works perfectly",
    "Very disappointing", "Broke quickly", "Waste of money", "Not worth it", "Disappointed",
    "Absolutely brilliant!", "So happy with this purchase", "Exceeded expectations", "Will buy again", "Top-notch performance",
    "Complete garbage", "Regret buying this", "Poor quality control", "Never again", "False advertising"
]
labels = [1,1,1,0,0,0,1,1,0, 1,1,1,1, 0,0,0,0,0, 1,1,1,1,1, 0,0,0,0,0]

X_tr, X_te, y_tr, y_te = train_test_split(texts, labels, test_size=0.3, random_state=42, stratify=labels)

pipe = make_pipeline(TfidfVectorizer(), LogisticRegression(max_iter=1000))
pipe.fit(X_tr, y_tr)
pred = pipe.predict(X_te)

print(classification_report(y_te, pred, zero_division=0))
print("Predict('This is fantastic!') ->", pipe.predict(["This is fantastic!"])[0])

Mini‑project

• Use any open dataset (or sklearn.datasets)
• Try two models on the same task (e.g., Logistic Regression vs RandomForest)
• Compare metrics; write a short “which one & why” note

Week 3 — Neural Networks & Deep Learning Intro

Goals

• Understand neurons, layers, activation, loss, optimizer, epochs
• Train a tiny neural network (vision or tabular)
• See overfitting/underfitting and how to fix (more data, regularization)

Option A — Quick start with scikit‑learn MLP (no heavy deps)

File: week3-dl/mlp_digits.py

week3-dl/
├─ requirements.txt
└─ mlp_digits.py

requirements.txt

numpy==1.26.4
scikit-learn==1.5.1

mlp_digits.py

from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import classification_report, accuracy_score

digits = load_digits()
X, y = digits.data, digits.target

X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2, random_state=0, stratify=y)

mlp = MLPClassifier(hidden_layer_sizes=(64,), activation="relu", solver="adam", max_iter=50, random_state=0)
mlp.fit(X_tr, y_tr)

pred = mlp.predict(X_te)

print("Accuracy:", round(accuracy_score(y_te, pred), 3))
 
print("\nReport:\n", classification_report(y_te, pred))

Option B — PyTorch (beginner‑friendly training loop)

File tree

week3-dl-pytorch/
├─ requirements.txt
└─ mnist_torch.py

requirements.txt

torch==2.9.0
torchvision==0.24.0
numpy==1.26.4

mnist_torch.py

import torch, torch.nn as nn
from torch.utils.data import DataLoader
from torchvision import datasets, transforms

# 1) Dataset & loaders (auto-download)
transform = transforms.Compose([transforms.ToTensor()])
train_ds = datasets.MNIST(root="data", train=True, transform=transform, download=True)
test_ds  = datasets.MNIST(root="data", train=False, transform=transform, download=True)

train_loader = DataLoader(train_ds, batch_size=64, shuffle=True)
test_loader  = DataLoader(test_ds, batch_size=256)

# 2) Model
class SimpleNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = nn.Sequential(
            nn.Flatten(),
            nn.Linear(28*28, 128),
            nn.ReLU(),
            nn.Linear(128, 10)
        )
    def forward(self, x):
        return self.net(x)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = SimpleNet().to(device)
loss_fn = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=1e-3)

# 3) Training loop
for epoch in range(3):
    model.train()
    total, correct, running_loss = 0, 0, 0.0
    for x, y in train_loader:
        x, y = x.to(device), y.to(device)
        opt.zero_grad()
        out = model(x)
        loss = loss_fn(out, y)
        loss.backward()
        opt.step()
        running_loss += loss.item() * x.size(0)
        preds = out.argmax(1)
        correct += (preds == y).sum().item()
        total += y.size(0)
    print(f"Epoch {epoch+1}: loss={(running_loss/total):.4f}, acc={(correct/total):.3f}")

# 4) Evaluation
model.eval()
total, correct = 0, 0
with torch.no_grad():
    for x, y in test_loader:
        x, y = x.to(device), y.to(device)
        preds = model(x).argmax(1)
        correct += (preds == y).sum().item()
        total += y.size(0)
print("Test accuracy:", correct/total)

Mini‑project

• Change network size (e.g., add another hidden layer)
• Try SGD vs Adam, different lr
• Plot a curve of loss/accuracy per epoch (save to PNG)

Week 4 — LLM Fundamentals (NLP → Embeddings → Calling an LLM)

Goals

• Learn tokenization & vectorization (Bag‑of‑Words, TF‑IDF)
• Understand embeddings (vectors that capture meaning)
• Call an LLM via API for summarization, Q&A, or chatbot

⚠️ You’ll need an API key if you call a hosted LLM. Keep your key in environment variables (never commit it).

File tree

week4-llm/
├─ requirements.txt
├─ tfidf_demo.py
├─ embed_and_search.py
└─ llm_chat.py

requirements.txt

numpy==1.26.4
scikit-learn==1.5.1
tqdm==4.66.4
openai==1.51.0          # If using OpenAI; or use your preferred provider's SDK
python-dotenv==1.0.1

1) Tokenization & TF‑IDF (tfidf_demo.py)

from sklearn.feature_extraction.text import TfidfVectorizer

docs = [
    "AI is transforming business",
    "Large language models power chatbots",
    "Transformers use attention",
    "Business analytics uses data and models",
]

tfidf = TfidfVectorizer()
X = tfidf.fit_transform(docs)

print("Vocabulary:", tfidf.vocabulary_)
print("Doc-term shape:", X.shape)
# Query similarity via dot product
from sklearn.metrics.pairwise import cosine_similarity
query = ["How do models help business?"]
Q = tfidf.transform(query)
scores = cosine_similarity(Q, X)[0]
print("Similarity scores:", scores)
best = scores.argmax()
print("Best match doc:", docs[best])

2) Tiny Embeddings + Search Demo (embed_and_search.py)

This simulates a mini knowledge base search using TF‑IDF embeddings.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

kb = [
    "Refund policy: You can request a refund within 14 days.",
    "Shipping: Orders ship within 2 business days.",
    "Support: Email support@example.com for technical issues.",
    "Pricing: Basic plan $9/month, Pro plan $29/month.",
]

vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(kb)

def search(query, top_k=2):
    qv = vectorizer.transform([query])
    sims = cosine_similarity(qv, X)[0]
    idx = sims.argsort()[::-1][:top_k]
    return [(kb[i], float(sims[i])) for i in idx]

print(search("How fast do you ship?"))
print(search("How to contact support?"))

3) Call an LLM (chat or summarization) — OpenAI SDK (llm_chat.py)

Put your key in .env as OPENAI_API_KEY=...

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# Basic chat completion
resp = client.chat.completions.create(
    model="gpt-4o-mini",   # pick a cost-effective model for beginners
    messages=[
        {"role": "system", "content": "You are a helpful tutor."},
        {"role": "user", "content": "Explain attention in simple terms."}
    ],
    temperature=0.7,
)
print(resp.choices[0].message.content)

# Summarization example
text = """Attention lets a model focus on the most relevant parts of the input.
It computes similarity between a query and keys, and uses those as weights over values."""
resp2 = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
        {"role":"system","content":"You are a concise assistant."},
        {"role":"user","content":f"Summarize this in 2 lines:\n{text}"}
    ]
)
print("\nSummary:\n", resp2.choices[0].message.content)

Swap the SDK/model if you prefer another provider. Always keep keys in env vars.

Mini‑project (LLM + Retrieval “RAG‑lite”)

• Use embed_and_search.py to fetch top‑k related snippets from your FAQ text file
• Concatenate the snippets + user question and send to the LLM as context

• Build a question‑answering CLI:

  • User asks → you search snippets → you call LLM with “context + question” → print answer

What interns submit each week

• Week 1: EDA report + scatter.png
• Week 2: Regression & classification scripts + metrics screenshots
• Week 3: MLP or PyTorch training script + test accuracy + a short “what I changed” note
• Week 4: LLM demo (CLI or small Flask app) with README explaining setup & how you protected the API key

Stretch goals (for ★★★ “Pro” track)

• Week 2: Try StandardScaler + SVC pipeline; compare to Logistic Regression
• Week 3: Add Dropout and early stopping; plot loss curves
• Week 4: Cache embeddings locally; add prompt guardrails (“If answer not in context, say ‘I don’t know’”)

Weekly Assignment & Project Sheet

Week 1 — Foundations: NumPy, Pandas, Plotting, EDA

Objective: Think in arrays/tables, perform basic EDA, and visualize data.

Assignments

1. NumPy warm‑up

   • Create vectors a,b and show: elementwise add/sub/mul, dot product, and matrix × vector.
   • Generate a 3×3 random matrix and compute transpose, determinant (optional), and inverse (optional).

2. Pandas basics

   • Load data.csv (or generate synthetic dataset if empty).
   • Show head(), info(), describe().

   • Create new columns:

     • passed = marks >= 50
     • hours_bucket = low/med/high by binning study_hours (e.g., pd.cut).

3. Visualization

   • Save scatter plot: study_hours vs marks → scatter.png.
   • Save histogram: distribution of marks → hist_marks.png.

4. EDA metrics

   • Compute correlation matrix (df.corr(numeric_only=True)).
   • Find top 3 insights (e.g., “hours correlates with marks ~0.78”).

Mini‑Project

EDA Report (Markdown or PDF)

• Include: data description, 2 plots, correlations, 3–5 bullet insights, and 2 action ideas (e.g., “collect study environment feature”). Deliverables: foundations.py (or notebook.ipynb), images, EDA_REPORT.md.

Stretch (★ Pro): Try Seaborn‑like styling using Matplotlib only; add a boxplot of marks by hours_bucket; handle missing values by imputation.

Submission checklist

• [ ] Code runs without errors
• [ ] scatter.png, hist_marks.png committed
• [ ] EDA_REPORT.md concise (≤1 page)

Week 2 — Machine Learning Essentials (scikit‑learn)

Objective: Train/test split, regression + classification, metrics & comparison.

Assignments

1. Regression

   • Use synthetic “study hours → marks” dataset.
   • Split 80/20; train LinearRegression.
   • Report R², MAE, RMSE, and print learned slope/intercept.

2. Classification

   • Use load_iris() or any simple dataset.
   • Split with stratify=y; train LogisticRegression.
   • Report accuracy, confusion matrix, and classification report.

3. Text classification pipeline

   • Tiny list of positive/negative sentences.
   • TfidfVectorizer + LogisticRegression (pipeline).
   • Show prediction for 2 custom sentences.

Mini‑Project

Model Compare

• Pick a dataset (Iris or similar). Train two models (e.g., LogisticRegression vs SVC/RandomForest) using a pipeline where needed.
• Compare at least two metrics; write Which model & Why (≤150 words). Deliverables: regression_basic.py, classification_basic.py, pipeline_classification.py, MODEL_COMPARISON.md.

Stretch (★ Pro): Add StandardScaler in a pipeline; use k‑fold cross‑validation; plot a simple learning curve (train size vs accuracy).

Submission checklist

• [ ] Reproducible random_state set
• [ ] Metrics printed & briefly interpreted
• [ ] Clear winner justified in MODEL_COMPARISON.md

Week 3 — Neural Networks Intro

Objective: Understand NN components and train a tiny network.

Track A (no heavy deps): Scikit‑learn MLP

• Train MLPClassifier on digits dataset (28×28 flattened).
• Experiment: change hidden_layer_sizes, max_iter, activation.
• Report accuracy and 2 observations about how hyperparameters affect results.

Track B (recommended if GPU available): PyTorch

• Train a simple FC network on MNIST.
• Show epoch loss & accuracy (print per epoch).
• Evaluate test accuracy.

Mini‑Project

Tuning Note

• Try at least two changes (optimizer, lr, hidden size, extra layer, dropout).
• Save a small table: config → test accuracy. Deliverables: mlp_digits.py (Track A) or mnist_torch.py (Track B), TUNING_NOTE.md.

Stretch (★ Pro):

• Plot loss/accuracy vs epoch to PNG.
• Add early stopping (manual: keep best, stop if no improvement N epochs).
• In PyTorch: add Dropout between layers and compare.

Submission checklist

• [ ] Code runs end‑to‑end
• [ ] Test accuracy printed
• [ ] TUNING_NOTE.md with at least 2 configs compared

Week 4 — LLM Fundamentals: Tokenization, Embeddings, API

Objective: Learn vector search basics and wire an LLM for chat/summarization.

Assignments

1. Tokenization/TF‑IDF

   • Build a TfidfVectorizer on 6–10 short docs.
   • Query with 2 questions; show cosine similarity scores & best match.

2. RAG‑lite search

   • Create a small FAQ knowledge base (8–12 lines of text).
   • Implement a function search(query, top_k) returning the top snippets.

3. Call an LLM (safe key handling)

   • Put your key in .env and load via python-dotenv.

   • Make two calls:

     • Chat: “Explain attention in simple terms.”
     • Summarize a 4–5 sentence paragraph in 2 lines.

Mini‑Project

CLI Q&A with Context

• Flow: user question → top‑k snippets from FAQ → compose a prompt with system + context + question → call LLM → print answer.
• Add a fallback: if similarity < threshold, reply “I don’t know” without calling the LLM (or call with a safety prompt). Deliverables: tfidf_demo.py, embed_and_search.py, llm_chat.py, README.md (how to set OPENAI_API_KEY).

Stretch (★ Pro):

• Cache vectorizer and matrix to disk (pickle).
• Add a simple Flask route /ask?q=... returning the answer.
• Add a “sources” section (print which snippets were used).

Submission checklist

• [ ] .env not committed
• [ ] CLI works for at least 3 diverse questions
• [ ] README shows setup & run commands clearly

🔎 Review & Scoring (lightweight)

• Week 1 (20%): EDA completeness + clarity of insights
• Week 2 (25%): Correct use of train/test + metrics + justified model choice
• Week 3 (25%): Successful training + meaningful tuning insights
• Week 4 (30%): Working RAG‑lite + clean prompt + safe key handling

📦 Submission Format (every week)

• GitHub repo per week (or single repo with week1/… week2/…).
• Include a short demo video (screen capture) where applicable.

• Each week must have a README.md with:

  • Setup (venv, pip install -r requirements.txt)
  • Run commands
  • Expected outputs (sample screenshots/metrics)

Grading Rubric

Week 1 — Foundations (NumPy, Pandas, Plotting, EDA)

Week 2 — Machine Learning Essentials (Regression, Classification, Pipelines)

Week 3 — Neural Networks Intro (MLP / PyTorch)

Week 4 — LLM Fundamentals (Tokenization, Embeddings, API)

✅ Final Evaluation (End of Month 3)

• Beginner (★) → Can run examples but lacks deeper understanding; needs guidance for projects.
• Intermediate (★★) → Can build end-to-end ML & LLM mini-projects independently with correct methods.
• Pro (★★★) → Goes beyond basics: experiments, explains results, and delivers polished, production-like outputs.

Suggested Weightage

• Week 1: 20%
• Week 2: 25%
• Week 3: 25%
• Week 4: 30%

📊 Scoring Example (out of 100):

• Week 1: 18/20
• Week 2: 20/25
• Week 3: 22/25
• Week 4: 25/30 Total = 85/100 → Intermediate-Pro