Hands-on-Lab Practise

The code is described step by step. Follow each steps properly.

Google Coolab accessing

Copy and paste the following link to open google colab

https://colab.research.google.com/notebooks/welcome.ipynb

Then click File --> New notebook after that Click Runtime-->Changes runtime to T4 GPU

If you are facing code indentation issues, you can visit the coolab directly 👉 LINK

Word Transformation Techniques Code

Import libraries

CODE

import nltk

from nltk.corpus import wordnet

from nltk.stem import WordNetLemmatizer

from sklearn.preprocessing import LabelEncoder, OneHotEncoder

import numpy as np

from gensim.models import Word2Vec

nltk.download('punkt')

nltk.download('wordnet')

nltk.download('averaged_perceptron_tagger')

Brings in tools for text processing (nltk), word definitions (wordnet), and lemmatization (converting words to their base form).

punkt : For splitting text into sentences/words.

wordent: For accessing the WordNet database

averaged_perceptron_tagger: Identifying parts of speech (e.g., nouns, verbs) in text.

WordNet

CODE

def get_wordnet_pos(word):

tag = nltk.pos_tag([word])[0][1][0].upper()

tag_dict = {"J": wordnet.ADJ,

"N": wordnet.NOUN,

"V": wordnet.VERB,

"R": wordnet.ADV}

return tag_dict.get(tag, wordnet.NOUN)

lemmatizer = WordNetLemmatizer()

def preprocess_text(text):

tokens = nltk.word_tokenize(text)

lemmatized_tokens = [lemmatizer.lemmatize(word, get_wordnet_pos(word)) for word in tokens]

return " ".join(lemmatized_tokens)

sample_text = "The striped bats are hanging on their feet for best."

preprocessed_text = preprocess_text(sample_text)

print("WordNet wise preprocessed text\n")

print(preprocessed_text)

The get_wordnet_pos function determines the correct part of speech (POS) for each word, which is necessary because the meaning and form of a word can change depending on its POS and then tokenizes the input text into individual words.

sample_text = "The striped bats are hanging on their feet for best."

output = "The strip bat be hang on their foot for best" This makes the text more standardized for further NLP tasks.

One-Hot Encodding

The one_hot_encode function converts each word in the input text into a binary vector, where each vector has a length equal to the vocabulary size.

CODE

vocabulary = ['bat', 'striped', 'hang', 'foot', 'best']

def one_hot_encode(text, vocab):

one_hot_vectors = []

for word in text.split():

one_hot_vector = np.zeros(len(vocab))

if word in vocab:

one_hot_vector[vocab.index(word)] = 1

one_hot_vectors.append(one_hot_vector)

return np.array(one_hot_vectors)

preprocessed_text = "The strip bat be hang on their foot for best"

encoded_text = one_hot_encode(preprocessed_text, vocabulary)

print(encoded_text)

From these code generated vectors all values are 0 except for a 1 at the index corresponding to the word's position in the vocabulary. If a word isn't in the vocabulary, its vector remains all zeros.

Distributed Representations using w2v

Sifmoid makes it ideal for output layers where the goal is to predict the probability of a particular class.

CODE

corpus = [

"The striped bats are hanging on their feet for best",

"The quick brown fox jumps over the lazy dog"

]

preprocessed_corpus = [preprocess_text(doc) for doc in corpus]

tokenized_corpus = [doc.split() for doc in preprocessed_corpus]

word2vec_model = Word2Vec(sentences=tokenized_corpus, vector_size=100, window=5, min_count=1, workers=4)

word_vector = word2vec_model.wv['bat']

print(word_vector)

The processed sentences are used to train a Word2Vec model, which learns vector representations for each word based on its context within the corpus. After training, the snippet retrieves the vector for the word "bat," which represents its learned semantic meaning in the context of the given sentences.

Word2Vec Subsampling

Tanh (hyperbolic tangent) activation function, which outputs values in the range of -1 to 1, is particularly effective in scenarios where the data is centered around zero. Additionally, tanh tends to work well when the network architecture benefits from non-linear transformations that can enhance the learning of complex patterns.

Code

corpus = [

"The striped bats are hanging on their feet for best",

"The quick brown fox jumps over the lazy dog"

]

preprocessed_corpus = [preprocess_text(doc) for doc in corpus]

tokenized_corpus = [doc.split() for doc in preprocessed_corpus]

word2vec_model_subsampling = Word2Vec(sentences=tokenized_corpus, vector_size=100, window=5, min_count=1, workers=4, sg=1, sample=1e-3)

word_vector_subsampling = word2vec_model_subsampling.wv['bat']

print(word_vector_subsampling)

Tanh Activation - Accuracy: 0.8264

ReLu Activation function

It transforms the input x to max⁡(0,x) which means it passes positive values through while setting negative values to zero. The non-linearity introduced by ReLU enables the network to learn complex patterns without the added computational cost of more complex activation functions.

CODE

model_relu = Sequential([

Embedding(input_dim=10000, output_dim=128, input_length=100),

Flatten(),

Dense(64, activation='relu'),

Dense(1, activation='sigmoid')

])

model_relu.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model_relu.fit(X_train, y_train, epochs=3, batch_size=64, validation_split=0.2, verbose=0)

y_pred_relu = (model_relu.predict(X_test) > 0.5).astype(int)

accuracy_relu = accuracy_score(y_test, y_pred_relu)

print(f"ReLU Activation - Accuracy: {accuracy_relu:.4f}")

ReLU Activation - Accuracy: 0.8017

GELU activation Function

GELU combines the advantages of both linear and non-linear activation functions. This design allows GELU to introduce a probabilistic non-linearity that can capture complex patterns more effectively.

CODE

model_gelu = Sequential([

Embedding(input_dim=10000, output_dim=128, input_length=100),

Flatten(),

Dense(64, activation=tf.keras.activations.gelu),

Dense(1, activation='sigmoid')

])

model_gelu.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])

model_gelu.fit(X_train, y_train, epochs=3, batch_size=64, validation_split=0.2, verbose=0)

y_pred_gelu = (model_gelu.predict(X_test) > 0.5).astype(int)

accuracy_gelu = accuracy_score(y_test, y_pred_gelu)

print(f"GELU Activation - Accuracy: {accuracy_gelu:.4f}")

Tanh Activation - Accuracy: 0.8264

These are crucial preprocessing steps for advancing NLP tasks.

Page updated

Google Sites

Report abuse