Hands-on-Lab Practise

We used a dataset about Email analysis and the dataset link is here by.  The dataset has two attributes, both of which are objects datatypes. Hence, we employed a fine-tuned RoBERTa model to effectively classify email behaviors. You can find the dataset link here.:
LINK

The code is described step by step. Follow each steps properly.

if you encounter nay issue during copy paste,  please  find code in google colab from here : https://colab.research.google.com/drive/1g2w57Tm2BmjtUXAR7NmPdfJVLNR5Mf4w?usp=sharing

Code parts begins

Applications of these libraries 

transformers: Provides pre-trained models and tools for NLP tasks.

torch: A deep learning framework for building and training neural networks.

pandas: Offers powerful tools for data manipulation and analysis.

sklearn: Provides machine learning algorithms and utilities for model training and evaluation.

Now, reading the dataset in the notebook 

Import the library 

import pandas as pd

This will allow you to use the pandas library with the alias pd, which is commonly used in Python code. 

Import the library 

from sklearn.model_selection import train_test_split

The train_test_split function from scikit-learn is used to split a dataset into training and testing sets to evaluate model performance on unseen data. It allows you to specify the proportion of data for training and testing, ensuring random splitting and preventing overfitting. 

from transformers import RobertaTokenizer

The code imports the RobertaTokenizer from the Hugging Face transformers library, which is used to tokenize text for models like RoBERTa. It then initializes the tokenizer using the pre-trained 'roberta-base' model, allowing you to convert raw text into tokens that can be fed into a model for tasks like NLP.

import torch 

We use import torch to access PyTorch, a popular deep learning library, for building and training neural networks. It provides tools for tensor computations, automatic differentiation, and utilities for handling datasets, making it essential for machine learning tasks. 

This custom SpamDataset class code for PyTorch , that turns text encodings and labels into a format suitable for training. 

CODE

class SpamDataset(torch.utils.data.Dataset):

       def __init__(self, encodings, labels):

         self.encodings = encodings

         self.labels = labels

       def __getitem__(self, idx):

         item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}

         item['labels'] = torch.tensor(self.labels[idx])

         return item

       def __len__(self):

        return len(self.labels)

Before feeding into the model, we need to Formate the preprocessed text encodings and labels data with the help of SpamDataset class. This class ensures that the RoBERTa model receives data in the correct format for learning and evaluation. Following these code, we will obtain the dataset in the required format. .

from transformers import RobertaForSequenceClassification

The code from transformers import RobertaForSequenceClassification imports the RobertaForSequenceClassification model from the Hugging Face transformers library. This model is used specifically for text classification tasks, where it assigns predefined labels to input sequences.

Here, we load a RoBERTa model specialized for binary classification (RobertaForSequenceClassification) and set it up with 2 to num_labels. It builds on RoBERTa's pre-trained transformer architecture and adds a classification head on top to make predictions based on input sequences. 

CODE

model = RobertaForSequenceClassification.from_pretrained('roberta-base', num_labels=2)

Preparing model Trainer

from transformers import TrainingArguments 

TrainingArguments is used to specify various settings and hyperparameters for training models in the Hugging Face transformers library. It allows you to configure batch size, output directory, logging steps, and other important parameters to control the training process.

TrainingArguments primarily configures the settings for training a model using the transformers library. By using TrainingArguments function, we can customize our model learning phases. 

from transformers import Trainer

In NLP tasks, the `Trainer` manages the training and evaluation of a model. It handles tasks such as loading data, running training loops, calculating metrics, and saving model checkpoints, making it easier to train and assess the model's performance.

                        Code

trainer = Trainer(

     model=model,                    

     args=training_args,              

     train_dataset=train_dataset,    

     eval_dataset=test_dataset       

)

Model specifies the model to be trained, which has been previously defined and loaded [RobertaForSequenceClassification]. 

args Provides the training configuration that follows while training. 

train_dataset which dataset to be trained by the model.

eval_dataset Sets the dataset used for evaluating the model’s performance during training.

import numpy as np

from sklearn.metrics import accuracy_score, classification_report

We use NumPy to handle numerical operations like finding the predicted class using np.argmax. Scikit-learn's accuracy_score and classification_report are used to calculate model accuracy and provide a detailed performance report for each class.

Model Evaluation result

                        Code

predictions = trainer.predict(test_dataset)

preds = np.argmax(predictions.predictions, axis=-1)

print(f'Accuracy: {accuracy_score(test_labels, preds):.4f}')

print(classification_report(test_labels, preds, target_names= ['ham', 'Spam']))

The trained model is used to generate predictions on the `test_dataset`.  The accuracy of the model is then shown by comparing these predictions to the actual classes.

Finally our fine-tuned RoBERTa model produce 96.6% accuracy.