make a lexer for the english lanuage in 16 lines of code in python that can learn new words and their parts of speech

First, we will need to import the json module because our data will be stored in the json format

import json

Next, we will need to ask the user for the word that they want to tokenize and open our json file:

y=input('enter a word')

y=y.split()

with open('dict.json', 'r+') as f:

dictionary=json.loads(f.read())

After that, we need to see if the word is in our dictionary and if it is not, we need to ask the user what kind of word it is(greeting, noun, verb, number, etc.) and store the value in a variable and then write the variable to the json file

try:

for word in y:

print("\n", dictionary[0][word])

except:

print('that word is not in the dictionary\n')

dtype=input('what type of word is that, like a greeting or question or something like that:\n')

dictionary[0][y]=dtype

with open('dict.json', 'r+') as f:

x=[dictionary[0]]

json.dump(x, f)

dictionary.pop()

starting contents of dict.json:

[{"hi":"hello"}]

full source for main.py:

import json


y=input("enter a word\n")

y=y.split()

with open('dict.json', 'r+') as f:

dictionary=json.loads(f.read())

try:

for word in y:

print("\n", dictionary[0][word])

except:

print('that word is not in the dictionary\n')

dtype=input('what type of word is that, like a greeting or question or something like that:\n')

dictionary[0][y]=dtype

with open('dict.json', 'r+') as f:

x=[dictionary[0]]

json.dump(x, f)

dictionary.pop()