02 — Learning

Deep Learning Topics

A curated path exploring neural networks, CNNs, RNNs, and Transformers. Each topic includes core concepts, sample implementation, and a link to the full notebook.

Backpropagation

Understanding the core algorithm behind training neural networks by calculating gradients.

GradientsChain RuleOptimization

    
python
 def backward_pass(loss):
    loss.backward()
    optimizer.step()
 

View Notebook

Keras Functional and Subclassing API

Building complex neural network architectures using Keras Functional and Subclassing APIs.

Keras Functional APIModel SubclassingCustom Layers

    
python
 class CustomModel(tf.keras.Model):
    def __init__(self):
        super().__init__()
        self.dense = tf.keras.layers.Dense(32, activation="relu")
 

View Notebook

PyTorch Core Concepts

Fundamentals of PyTorch, including tensors, computational graphs, and basic operations.

TensorsAutogradComputational Graphs

    
python
 import torch
x = torch.tensor([1., 2.], requires_grad=True)
y = x.sum()
y.backward()
 

View Notebook

Dataset and DataLoader

Managing and batching data efficiently in PyTorch using Dataset and DataLoader utilities.

Custom DatasetDataLoaderBatchingShuffling

    
python
 from torch.utils.data import DataLoader
dataloader = DataLoader(dataset, batch_size=32, shuffle=True)
 

View Notebook

Dropout and Regularization

Preventing overfitting in deep neural networks using Dropout and other regularization techniques.

DropoutOverfittingRegularization

    
python
 import torch.nn as nn
model = nn.Sequential(
    nn.Linear(128, 64),
    nn.Dropout(p=0.5),
    nn.ReLU()
)
 

View Notebook

N-Gram Language Models (Part 1)

Introduction to statistical language modeling using N-grams.

Language ModelingN-GramsMarkov Assumption

    
python
 from nltk.util import ngrams
bigrams = list(ngrams(tokens, 2))
 

View Notebook

N-Gram Language Models (Part 2)

Advanced concepts in N-gram language models, including smoothing and perplexity.

SmoothingPerplexityProbability Distribution

    
python
 # Laplace Smoothing Example
prob = (count + 1) / (total_count + vocab_size)
 

View Notebook

Bengio MLP Language Model

Implementation of the classic Neural Probabilistic Language Model by Bengio et al.

Neural Language ModelsWord EmbeddingsMLP

    
python
 class BengioLM(nn.Module):
    def __init__(self, vocab_size, embed_dim):
        super().__init__()
        self.embeddings = nn.Embedding(vocab_size, embed_dim)
 

View Notebook

Multi-Layer Perceptron (MLP)

Building and training fully connected feed-forward neural networks.

MLPFeed-forwardActivation Functions

    
python
 mlp = nn.Sequential(
    nn.Linear(input_dim, hidden_dim),
    nn.ReLU(),
    nn.Linear(hidden_dim, output_dim)
)
 

View Notebook

CNN for Sentence Classification

Applying Convolutional Neural Networks to natural language processing and text classification.

Text CNN1D ConvolutionMax Pooling

    
python
 self.conv1d = nn.Conv1d(in_channels=embed_dim, out_channels=100, kernel_size=3)

View Notebook

Recurrent Neural Networks (RNN)

Working with sequential data using basic Recurrent Neural Networks.

RNNSequential DataHidden State

    
python
 self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
output, hidden = self.rnn(x)
 

View Notebook

Advanced RNN Applications

Applying Recurrent Neural Networks to real-world sequence modeling tasks.

Sequence to SequenceLSTMsTime-series

    
python
 self.lstm = nn.LSTM(input_size, hidden_size, num_layers=2, batch_first=True)

View Notebook

Sentiment Analysis with BERT

Fine-tuning a pre-trained BERT model for sentiment analysis tasks using Hugging Face Transformers.

TransformersBERTSentiment AnalysisFine-tuning

    
python
 from transformers import AutoModelForSequenceClassification
model = AutoModelForSequenceClassification.from_pretrained("google-bert/bert-base-uncased", num_labels=2)
 

View Notebook

Emotion Classification with BERT

Classifying text into emotion categories (happy, angry, sad, surprise) using BERT representations.

Emotion ClassificationMulti-class ClassificationClass Weights

    
python
 criterion = CrossEntropyLoss(weight=class_weights)
outputs = model(**inputs)
loss = criterion(outputs.logits, labels)
 

View Notebook

Instruction Fine-Tuning LLMs

Fine-tuning Large Language Models (like Phi-3.5) on instruction-following datasets using the Hugging Face Trainer.

LLMsInstruction TuningCausal LM

    
python
 from transformers import AutoModelForCausalLM, Trainer
model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-3.5-mini-instruct")
trainer.train()
 

View Notebook