Hands-on-Lab Practise

Copy and paste the following link to open google colab 

https://colab.research.google.com/notebooks/welcome.ipynb

Then click File --> New notebook after that Click Runtime-->Changes runtime to T4 GPU

Install string                  to handle punctuation and whitespace .

import re                        useful for cleaning text data. 

from numpy import array, argmax, random, take

•  array: To create and manipulate arrays.

• argmax: Finds the index of the maximum value in an array.

• random: Generates random numbers.

• take: Selects elements from an array based on an index list.

from tensorflow.keras.models import Sequential:  Sequential is a linear stack of layers for building simple models. 

from tensorflow.keras.layers import Dense, LSTM, Embedding, RepeatVector       

• Dense: Fully connected layer.

• LSTM: Long Short-Term Memory layer, useful for sequential data.

• Embedding: Maps input tokens to dense vectors.

• RepeatVector: Repeats an input sequence for a specified number of timesteps.

from tensorflow.keras.preprocessing.text import Tokenizer:      A utility to convert text into sequences of tokens. 

from tensorflow.keras.callbacks import ModelCheckpoint:    to save the model at checkpoints during training. 

import pad_sequences:   to ensure they have the same length.

from tensorflow.keras import optimizers : to adjust model weights during training. 

def read_text(filename):

               file = open(filename, mode='rt', encoding='utf-8')

        text = file.read()

        file.close()

        return text

The  read_text(filename)  function opens a file, reads its content as a string, and then closes the file. It returns the text from the file for further use.

def to_lines(text):

    sents = text.strip().split('\n')

    sents = [i.split('\t') for i in sents]

    return sents

The to_lines(text) function splits the input text into lines, removes extra spaces, then splits each line by tab (\t). It returns a list of lists where each inner list contains tab-separated elements from each line. 

data = read_text("/content/drive/MyDrive/deu.txt")

german_eng = to_lines(data)

german_eng = array(german_eng)

german_eng = german_eng[:30000,:]

german_eng[:,0] = [s.translate(str.maketrans('', '', string.punctuation)) for s in german_eng[:,0]]

german_eng[:,1] = [s.translate(str.maketrans('', '', string.punctuation)) for s in german_eng[:,1]]

for i in range(len(german_eng)):

    german_eng[i,0] = german_eng[i,0].lower()

    german_eng[i,1] = german_eng[i,1].lower()

Explain: The code removes punctuation and converts both the German and English text to lowercase for the first 30,000 rows of the `german_eng` dataset. It processes the German in the first column and the English in the second column.

def tokenization(lines):

    tokenizer = Tokenizer()

    tokenizer.fit_on_texts(lines)

    return tokenizer

eng_tokenizer = tokenization(german_eng[:, 0])

eng_vocab_size = len(eng_tokenizer.word_index) + 1

eng_length = 8

print('English Vocabulary Size: %d' % eng_vocab_size)

german_tokenizer = tokenization(german_eng[:, 1])

german_vocab_size = len(german_tokenizer.word_index) + 1

german_length = 8

print('German Vocabulary Size: %d' % german_vocab_size)

def encode_sequences(tokenizer, length, lines):

    seq = tokenizer.texts_to_sequences(lines)

    seq = pad_sequences(seq, maxlen=length, padding='post', truncating='post')

    return seq

Explain: The code tokenizes the English and German text, calculates their vocabulary sizes, and sets a sequence length of 8 for both languages. The `encode_sequences` function converts text to sequences and pads them to the specified length.

from sklearn.model_selection import train_test_split

train, test = train_test_split(german_eng, test_size=0.2, random_state = 12)

trainX = encode_sequences(eng_tokenizer, eng_length, train[:, 0])

trainY = encode_sequences(german_tokenizer, german_length, train[:, 1])

testX = encode_sequences(eng_tokenizer, eng_length, test[:, 0])

testY = encode_sequences(german_tokenizer, german_length, test[:, 1])

Explain:  The code splits the `german_eng` dataset into training (80%) and testing (20%) sets. It then encodes the English and German text sequences for both the training and testing datasets using the `encode_sequences` function.

The `define_model` function creates a sequence-to-sequence model using LSTM layers with an embedding layer for input text and a dense output layer for generating predictions. It defines the model architecture with specified vocabulary sizes, time steps, and LSTM units, and initializes the model with 512 units. 

The code sets up a model checkpoint to save the best version of the model based on validation loss during training. It then trains the model on the training data for 30 epochs, using a batch size of 512 and validating on 20% of the data, while tracking the training history.

The code uses the trained model to predict outputs for the test data, reshaping the input as needed. It then extracts the most likely predicted indices by finding the position of the maximum value in the predictions for each sample.

The `get_word` function retrieves the corresponding word for a given index from the tokenizer. The code then converts predicted indices into readable German words, ensuring no consecutive duplicates or None values are included, and joins the words into complete sentences for each prediction.