As an Analytical Linguist, I provided insights and recommendations for product feature development, data annotation management, and quality evaluations.Â
Below were some of my areas of expertise:
Taxonomies: user language behavior; black box model performance; error type categorization
Constructed error type categorization to understand black box model performance and to delineate the scope of different features
Decomposed components of inferred user personalization data for widespread feature adoption
Research: literature reviews; exploratory and pilot annotations; annotation experiments
Consolidated academic and business research to scope a new feature, execute proof-of-concept, and develop guidelines for large-scale annotation
Analysis: identify common patterns, edge cases, and potential user risks in business/professional language
Designing and managing large-scale corpus annotation
Collected model training dataset of 90,000 text pairs by collaborating with external vendors to manage over 2,000 hours of annotation
Developing synthetic data corpora for fine-tuning in-house LLMs
Human evaluations of model performance:
Intrinsic, against internal and external competitors, against previous model versions, and between multiple pre-release versions
Designed evaluation framework for ranking model and delivered optimization recommendations
Established new quality standard for evaluating Grammarly suggestions based on relative impact and user value
Human evaluations of training data:
Comparison of different data annotation pipelines
Quantitative and qualitative analysis to implement the optimal training data pipeline for pre-release model
Scripting to streamline evaluation data (pre-)processing and analysis
Developed reusable Jupyter notebooks for processing evaluation input data and analyzing output data
Confident tone - rewrites sentences to sound more confident
Sentence Clarity - rewrites sentences to be concise and unambiguous
Voice features for Grammarly Business - detects your unique voice profile and personalizes rewrites