First of all, we unify the encoding format and remove non-English characters of reviews. We convert characters with special unicode form to base form, also remove the diacritics (e.g., from café to cafe). Some reviews are in non-English languages, and some others contain special characters like emojis. Both are not target of our analysis targets*.
We unify user reviews to lowercase as well.
* This work focus on reviews in English.
We replace possible contractions (e.g., from won't to will not) and abbreviations (e.g., from ppl to people) with their original forms. The expansion list is summarized from both previous work~\cite{paper:app-love-users-ase15, website:nlptools-contractions-repo} and our dataset.
In this step, we correct the misspelled words in reviews using SymSpell~\cite{software:symspell}, an effective and efficient spelling correction tool. Some proper nouns in VR field are predefined and ignored to prevent overcorrection, like \textit{HMD}, \textit{PSVR} and \textit{SteamVR}. We also combine experience from literature~\cite{paper:keyatm-icse22} and observation on VR reviews, constructing a customized misspelled word list to correct accordingly (e.g., from \textit{oculas} to \textit{oculus}).
This is when user reviews are split to list of tokens. Punctuations are removed after tokenization since they provide little semantic information.
We transform the tokens in user reviews to their base forms, in order to combine all inflected forms of a same word and analyze as a single form together. We use SpaCy~\cite{paper:spacy-zenodo20} to perform both tokenization and lemmatization.
We remove stop words which appear frequently but bring little information. Specifically, we remove (1) general stop words in English provided by NLTK~\cite{software:nltk}, like \textit{a} and \textit{to}, and (2) a set of customized words, including those summarized by literature~\cite{paper:idea-online-app-review-icse18} and those observed in VR app reviews.
Users tend to repeat to emphasize their statements or feelings. The redundancy comes from both character-level and expression-level (phrases, sentences, etc.). Accordingly, we reduce words with abnormally repetitive characters to their dictionary forms, as well as reduce meaningless repetitive expressions.
We remove reviews with too few tokens ($<$3). Moreover, Remove short words (length $<= 2$) and extra-long words (length $> 20$).
We also use regular expressions to do some detailed cleaning, including URL replacement, digital replacement, special sign replacement, pattern recognition and substitution, etc.
Replace URLs with $<URL>$, since the words contained in URLs might influence the context after tokenization.
Replace special signs with corresponding meanings (e.g., from $\$$ to money and \&), to enrich their contribution to contextual semantics.
Replace all numbers with $<digit>$, as the specific values of numbers don't matter to our research questions.
Split words with pattern $<digits><alphabets>$ or $<alphabets><digits>$ to individual words (eg., from quest2 to quest 2).
Extract plain text from HTML. Unescape the special characters in HTML
1st
Remove empty characters
Remove punctuations