Hands On Lab

Link for Ready code : https://colab.research.google.com/drive/1plh5ssx2KtKX_4piYAieHfmzw3T-CEv6?usp=sharing

Link for the dataset:  https://drive.google.com/file/d/1nWhfMnL61eHAYyeWQq9aG-Te1-JQL7cX/view?usp=drive_link

import torch

from transformers import BertTokenizer, BertForSequenceClassification

from torch.utils.data import DataLoader, Dataset

from sklearn.model_selection import train_test_split

from sklearn.metrics import classification_report

import torch: This imports PyTorch, a tool to build and train machine learning models.

from transformers import BertTokenizer, BertForSequenceClassification: These bring in tools from Hugging Face to use BERT for text tokenization and classification.

from torch.utils.data import DataLoader, Dataset: These help load and manage data for training models in an organized way.

from sklearn.model_selection import train_test_split: This splits the data into training and testing sets.

from sklearn.metrics import classification_report: This generates a report showing how well the model makes predictions.

import pandas as pd

# Load your dataset

df = pd.read_csv("/content/sentimental_dataset (1).csv")

import pandas as pd: This brings in Pandas, a library used to handle data easily like working with tables in Excel.

df = pd.read_csv("/content/sentimental_dataset (1).csv"): This line loads a CSV (spreadsheet-like) file into a Pandas DataFrame, which lets you view and work with your dataset conveniently.

X = df['review_text']

y = df['class_index']

X = df['review_text']: This selects the 'review_text' column from the dataset, which contains the text data (like product or movie reviews) for analysis.

y = df['class_index']: This selects the 'class_index' column, which holds the labels (like 0 for negative, 1 for positive) used to train the model to recognize sentiment.

train_texts, val_texts, train_labels, val_labels = train_test_split(df['review_text'], df['class_index'], test_size=0.2)

This line splits the dataset into training and validation sets:

train_texts: Contains 80% of the reviews used to train the model. val_texts: Contains 20% of the reviews to validate and check the model’s performance.

train_labels: The labels (like positive or negative) for the training set.

val_labels: The labels for the validation set. The test_size=0.2 means that 20% of the data is reserved for validation.

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

# Tokenize the data

train_encodings = tokenizer(list(train_texts), truncation=True, padding=True, max_length=512)

val_encodings = tokenizer(list(val_texts), truncation=True, padding=True, max_length=512)

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased'): This loads a pre-trained BERT tokenizer that converts text into numbers BERT can understand, using the uncased version (ignores letter case).

train_encodings = tokenizer(list(train_texts), truncation=True, padding=True, max_length=512): This converts the training texts into token IDs, making sure each one is cut off if it's too long (truncation) and fills shorter texts with extra tokens (padding) to ensure all are of the same length, up to 512 tokens.

val_encodings = tokenizer(list(val_texts), truncation=True, padding=True, max_length=512): This does the same for the validation texts, preparing them in a format BERT can use for prediction.

class SentimentDataset(Dataset):

    def __init__(self, encodings, labels):

        self.encodings = encodings

        self.labels = labels

    def __len__(self):

        return len(self.labels)

    def __getitem__(self, idx):

        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}

        item['labels'] = torch.tensor(self.labels[idx])

        return item

# Create datasets

train_dataset = SentimentDataset(train_encodings, train_labels.tolist())

val_dataset = SentimentDataset(val_encodings, val_labels.tolist())

class SentimentDataset(Dataset):: This defines a custom dataset class that helps organize the input data for BERT.

init(self, encodings, labels):: This method initializes the class by storing encodings (tokenized texts) and their corresponding labels (sentiment classes).

len(self):: This method returns the total number of samples in the dataset, which is useful for batching during training.

getitem(self, idx):: This retrieves a single data point (both text encoding and label) at the given index. It converts the encoding and label into tensors (used in PyTorch for model training).

train_dataset = SentimentDataset(train_encodings, train_labels.tolist()): This creates the training dataset by combining the tokenized training texts and their labels.

val_dataset = SentimentDataset(val_encodings, val_labels.tolist()): Similarly, this creates the validation dataset with tokenized validation texts and labels.

model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2)  # Change `num_labels` if needed

model = BertForSequenceClassification.from_pretrained('bert-base-uncased', num_labels=2): This loads a pre-trained BERT model from Hugging Face, specifically tailored for sequence classification tasks like sentiment analysis.

'bert-base-uncased': This version of BERT ignores the difference between uppercase and lowercase letters (uncased).

num_labels=2: This specifies the number of output classes. In this case, the model is configured for binary classification (like positive vs. negative sentiment). You can change num_labels if your task has more categories.

from torch.optim import AdamW

optimizer = AdamW(model.parameters(), lr=5e-5)

from torch.optim import AdamW: This imports the AdamW optimizer from PyTorch, a tool used to adjust the model’s parameters to improve its predictions.

optimizer = AdamW(model.parameters(), lr=5e-5): This initializes the optimizer, telling it to update the BERT model’s parameters during training with a learning rate of 5e-5 (a small step size to avoid overshooting while optimizing).

The AdamW optimizer is a variation of the Adam optimizer that helps with weight decay (regularization), preventing the model from overfitting.

train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True)

val_loader = DataLoader(val_dataset, batch_size=16)

train_loader = DataLoader(train_dataset, batch_size=16, shuffle=True): This creates a data loader that splits the training dataset into batches of 16 samples and shuffles them before each epoch to improve training.

val_loader = DataLoader(val_dataset, batch_size=16): This creates a validation data loader that also processes data in batches of 16, but without shuffling (since order doesn't matter for validation).

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')

model.to(device)

for epoch in range(3):  # Number of epochs

    model.train()

    total_loss = 0

    for batch in train_loader:

        batch = {key: val.to(device) for key, val in batch.items()}

        outputs = model(**batch)

        loss = outputs.loss

        total_loss += loss.item()

        optimizer.zero_grad()

        loss.backward()

        optimizer.step()

    print(f"Epoch {epoch + 1}, Loss: {total_loss / len(train_loader)}")

device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu'): This checks if a GPU (CUDA) is available; if not, it uses the CPU for computations. GPU speeds up training significantly.

model.to(device): Moves the BERT model to the selected device (GPU or CPU) for training.

for epoch in range(3):: This loop runs the training process for 3 epochs, meaning the model sees the entire dataset 3 times.

model.train(): Puts the model in training mode, enabling certain features like dropout for better generalization.

total_loss = 0: Initializes a variable to accumulate the total loss for the current epoch.

for batch in train_loader:: Iterates over each batch of training data.

batch = {key: val.to(device) for key, val in batch.items()}: Moves each part of the batch to the selected device (CPU or GPU).

outputs = model(**batch): Feeds the batch through the model to get predictions and calculate the loss.

loss = outputs.loss: Extracts the loss value from the model's output.

total_loss += loss.item(): Adds the loss from this batch to the total loss for the epoch.

optimizer.zero_grad(): Resets the gradients to prevent accumulation from previous batches.

loss.backward(): Computes gradients (how much each parameter needs to change) based on the loss.

optimizer.step(): Updates the model’s parameters to minimize the loss.

print(f"Epoch {epoch + 1}, Loss: {total_loss / len(train_loader)}"): Displays the average loss for the current epoch to track progress.

model.eval()

preds, true_labels = [], []

with torch.no_grad():

    for batch in val_loader:

        batch = {key: val.to(device) for key, val in batch.items()}

        outputs = model(**batch)

        logits = outputs.logits

        preds.extend(torch.argmax(logits, axis=1).cpu().numpy())

        true_labels.extend(batch['labels'].cpu().numpy())

print(classification_report(true_labels, preds))

model.eval(): This puts the model in evaluation mode, disabling certain features like dropout to get consistent predictions.

preds, true_labels = [], []: These lists will store the model’s predictions and the true labels for the validation data.

with torch.no_grad():: Disables gradient calculation to save memory and speed up evaluation, as no model updates are needed.

for batch in val_loader:: Iterates over each batch in the validation set.

batch = {key: val.to(device) for key, val in batch.items()}: Moves the batch data to the same device (CPU or GPU) as the model.

outputs = model(**batch): Feeds the batch through the model to get the predicted outputs.

logits = outputs.logits: Extracts the logits, which are raw prediction scores before applying softmax.

preds.extend(torch.argmax(logits, axis=1).cpu().numpy()): Converts the predicted logits into class predictions (e.g., 0 or 1 for binary classification) and stores them in the preds list.

true_labels.extend(batch['labels'].cpu().numpy()): Extracts the true labels from the batch and stores them in the true_labels list.

print(classification_report(true_labels, preds)): Prints a classification report, which shows metrics like precision, recall, and F1-score to evaluate the model’s performance.