Machine Learning 2023 Spring
Liyuan Geng, Jiayi Xu, Jinhong Xia
Understanding Cloud Organization(paper)(presentation)
Clouds play a huge role in determining the Earth’s climate. By classifying different cloud organizations accurately, people can have a better understanding of how clouds shape our future climates. Our project aimed at training a model that classifies different cloud organizations accurately, which may encounter difficulties in terms of the complex nature of clouds and the murky boundaries between different types. To solve this problem, we experimented with two models, Unet as the baseline model and PSPNet as a potentially improved model. We found out that the performance of PSPNet is slightly better than U-net,which is in our expectation.
Vivian Chen, Junjie Huang
StyleBank for Image Style Transfer and Fusion(paper)(presentation)
This project aims to realize image style tranfer, where we want to blend the texture of certain art works with the input image while preserving the content information of the latter. It allows one to appreciate new masterpieces created catering to one’s own taste, and we attempt to realize the objective with the flexible StyleBank model. Our model has basically managed to preserve the original content and represent certain styles. It is further equipped with new functions: resume training and style fusion.
Changlan Chen, Yuanzheng Han
German-English Translation with Seq2seq(paper)(presentation)
According to the Foreign Service Institute (FSI), German is one of the most difficult languages for English speakers among the European languages, thus English scholar Richard Porson once commented: “Das Leben ist zu kurz, um Deutsch zu lernen” (Life is too short to learn German). Due to complex grammar, verb conjugations, and changeable word order, German-English translation is a daunting task for humans. With Recurrent Neural Networks (RNN), we are eventually capable of approaching this problem from the perspective of machine translation and building up a model to translate for us. Our model performs exceptionally accurately in sentences where ”be” verbs are the primary verb with subordinate clauses of German grammar.
Chijie An, Zhihan Qin
Revolutionizing Image Caption Fine-Tuning with Adapters: A Parameter Efficient Approach(paper)(presentation)
The popularity of text-to-image models has led to discussions on reversing the typical direction of a generative text-to-image model. Instead of generating an image from a text prompt, our goal is to generate a reliable prediction of the text prompt given a generated image. AI painting has been an influential tool in the creative field, and similarly, the image-to-prompt field could establish a strong connection between computer vision and natural language processing, creating a new field of interest. However, the training and improvement of this kind of model require high model complexity, consume significant computational power, and demand high-quality labeled training datasets. These preconditions make it challenging to train the model with limited resources and investment.
In this paper, we address this problem by first implementing classic fine-tuning methods on a classic BLIP model to successfully fine-tune it on a highly correlated dataset. Next, we added an additional adaptor, a residual neural network, after the feed-forward neural network in the image and prompt encoder in the BLIP model to improve the output results. Our approach improved the fine-tuning results while consuming less computational power.
Haochen Zhai, Dingwen Song
KNN Method for Drug Molecules Classification(Active/Inactive)(paper)(presentation)
Drugs are very important substances in the healthcare system. Using traditional methods to produce new drugs to treat diseases in the market is time-consuming and expensive. More recently, the use of computer-aided drug design (CADD) has accelerated the drug design process. In drug design, biomolecules are responsible for producing new drugs. Therefore, molecular recognition is an important part of CADD.
In this study, we used the KNN method to classify the activity of the drug molecules. We selected three input features and one output feature(active/inactive) to build the KNN model. The KNN classifier shows a high prediction accuracy(90%) on the testing data set. We built the KNN classifier for drug molecules classification successfully.
Jerry Cao, Qiyong Wang
Machine Learning based Stock Price Change Predictor: Through Companies’ Growing Capacity(paper)(presentation)
In this study, we implement machine learning methods to predict the stock price change of a certain company based on its growth capacity. To meet this goal, we use ordinary linear regression, ridge regression and LSTM as our models, and use MAE as our loss function. We find that the LSTM can fit the true value in these three models, followed by ridge regression and then ordinary linear regression. Overall, the performance of our models are satisfactory. But there is still improvement to optimize our model.
Haotong Wu, Ragnor Wu
Personality Type Prediction using Natural Language Processing and Machine Learning(paper)(presentation)
To classify MBTI based on different styles of comments, this project used different machine learning models to train the dataset. Our project is a multi-classification problem. We compared the accuracy of KNN, SVM, logistic regression, XGBoost, FCNN, and other classifiers, and finally selected SVM and used cross-validation to find the most suitable parameters. We compared the results of 16 types and 4 types and found that 4 types were more accurate. Due to the imbalance of different categories of data, our accuracy is not very high, but it can be improved by reducing the categories.
Yuanhe Guo, Yiling Cao
Fine Tuning Stable Diffusion Model With LoRA(paper)(presentation)
We want the stable diffusion model to learn to generate images in a specific style. And our work is a downstream task of a diffusion model, which can do a specific mission. The difficulty lies in how to fine-tune a model with a large number of pre-trained parameters. We adopted the same LoRA method for fine-tuning the large language model, analyzed the structure of the stable diffusion model, and added new parameters to the cross-attention layer in the decoder for training. The LoRA model we trained can allow specific prompt text to guide the model to generate pictures of a certain style. After using scribble data for training, we successfully used the stable diffusion model to generate pictures stably in the style of ordinary people’s hand-painted scribbles. And the size of the LoRA model is only 3.3 MB, which is very easy to share and use.
Steven Zhang, Ziyun Yu
Leveraging Machine Learning for Automatic Product Matching in E-commerce(paper)(presentation)
The growth of e-commerce has led to increased competition among online retailers. Product matching is a way to offer competitive rates, but it’s challenging to identify similar products among the vast number listed. To address this, machine learning methods like ResNet, DenseNet, EfficientNet, and BERT models are being used to analyze product features and attributes. In this paper, we propose a novel approach that leverages ResNet and BERT models to improve accuracy and efficiency. Our model uses deep learning-based architecture to analyze multiple sources of product information and generate similarity scores. This can help retailers offer products at competitive rates and improve the customer experience on e-commerce platforms. Code is available in https://github.com/qingYzhang/Leveraging-Machine-Learning-for-Automatic-Product-Matching-in-E-commerce.
Kaiyan Zhan, Weichen Liu
Text Emotion Segmentation via LSTM Model(paper)(presentation)
Our team focus on the text emotion segmentation problem that has been troubling the Internet for a long period. Text emotion is a more intriguing question compared to simple emotion detection as textual information could easily cause ambiguity with little additional information. Our team, in an effort to address this issue, have designed a fine-tuning LSTM model upon the GoEmotion dataset to address the issue. The model has a more robust outcome compared to other baseline models and succeeds in experiments of predicting the emotions of given text prompts.
Xingjian Wu, Zekai Li, Xiaorong Tian, Cynthia Wang
Machine Learning Models in Distribution Network(paper)(presentation)
In this project, We want to leverage multiple machine learning models to optimize the supply chain from the perspective of logistic players in goods distribution. For the first part, we found out the main mismanagement comes from tedious transaction procedures, which originate from the misallocation of resources in regions and customers. In the second half of the project, we predict the sales with multiple regression models. We approached the problem by comparing the performance of our model with other traditional supply Chain optimization methods. This could probably help the manufacturers or the company to improve their supply chain efficiency and customer satisfaction.
Tianqi Zhan, Yuyang Wu
Applying CNN to Mandarin Tone Classification with Noisy Data(paper)(presentation)
This study investigates the performance of a convolutional neural network (CNN) model for Mandarin tone classification under noisy conditions. The model was initially trained on clean data, and its performance was evaluated on both clean and noisy data. To improve the model’s robustness to noise, it was retrained on a dataset containing a mix of clean and noisy data. Additionally, Cepstral Mean Variance Normalization (CMVN) was applied as a preprocessing step for noisy data. The results demonstrated that training the model on a diverse dataset and using CMVN significantly improved the testing accuracy for both clean and noisy data, highlighting the importance of incorporating noise during the training process and employing effective preprocessing techniques for real-world applications.
Ruiqi Xue, Yue Wang
Tweet Sentiment Analysis using BERT(paper)(presentation)
Twitter has been a popular platform for people to express their thoughts and mood or opinions towards some public events and news via tweets. Using machine learning methods, we perceive sentiments behind the texts, which attach additional value to those tweets. We design it as a 3-way classification problem and classify sentiments into ’positive’, ’neutral’ and ’negative’. We first use Naive Bayes method as a baseline model and then use BERT model to address this NLP problem. Naive Bayes approach nicely solved the problem with an accuracy of xx% and BERT gave a result of xx.
Di Li, Jiajun Wang, Yuhui Song
Quantitative Trading Model and Comparative Study of CSI 1000 Based on Machine Learning(paper)(presentation)
Based on the literature review and analysis, this article selects high-frequency trading related indicators of major market indexes from February 2, 2023 to May 5, 2023 as the input features of the price prediction model. The research sample is the CSI 1000 stock index futures with 1-minute high-frequency. The article constructs a price prediction model based on six machine learning algorithms: decision trees, random forests, support vector machine, XGBoost, neural networks, and stacking algorithms. The results show that the model constructed using neural networks has the smallest error in predicting stock index futures prices. In addition, the trading strategy constructed based on the results has a higher cumulative return rate under a certain level of risk, with more controllable risk, and is more suitable for predicting and investing in stock index futures prices.
Yujing Liu
Does Time Matter? Road Accident Severity Prediction and Temporal Factor Investigation(paper)(presentation)
This study analyzed the performance of widely used machine learning classifiers using a real-life road traffic accident (RTAs) dataset from the Great Britain Area of the United Kingdom. The study aimed to assess prediction model designs for RTAs’ severity prediction to assist transport authorities and policymakers. Decision Tree, AdaBoost, Random Forest, and Artificial Neutral Networks are evaluated using evaluation metrics, including weighted accuracy, precision, recall, and f1-score. The empirical results and analyses show that the Random Forest classifier, yielded the best performance when compared with the other classifiers. On the other hand, despite the fact that temporal factors show high feature importance, after further investigation, it’s shown that they don’t have additive prediction power to the models.
Yifei jiang, Haoyu Wang
Stock Price Prediction Using LSTM(paper)(presentation)
With the aim of providing a precise and logical prediction of a stock price in the market, the project creates a machine learning model using LSTM. To make our predictions logical, we selected 14 data features that financial analytics usually use in doing equity valuation. We did data standardization before inputting them into the model and within our model, we added dropout and L2 regularization to avoid model overfitting. Finally, the results of our stock price prediction model are surprisingly good, which can almost always get the right trend that the real stock price goes with the validation loss low and sustained.
Momoe Nomoto, Yuechen Wang
Stacking Classifier Approach for Music Genre Classification(paper)(presentation)
Automatic music genre classification is essential for efficient and reliable labeling of a large corpus of digitized music and for music recommendation systems, which rely on genre information to suggest similar music to listeners. This paper proposes an approach that utilizes multiple machine-learning classification algorithms (KNN, SVM, and Random Forest) and an ensemble learning technique to create an optimal meta-model that provides more accurate classifications. Our meta-model achieves an accuracy score of 0.69 in the test set, which exceeds the average accuracy of the models we have reviewed so far.
Qifu Wen
ATOMIC CLOCKS FREQUENCY OPTIMIZATION USING GENERALIZED LEAST SQUARES WITH A PYTHON(paper)(presentation)
This paper provides an overview of atomic clocks, specifically focusing on the definition of a second based on frequency ratios. The concept of timekeeping to the precision of atoms is explored, highlighting the importance of atomic clocks in modern technology and scientific research. The mathematical foundations of frequency ratio calculation, a key component of atomic clocks, are examined, with an emphasis on the application of least squares. Additionally, a Python implementation for calculating frequency ratios using these mathematical techniques is presented, providing practical insights for researchers and engineers working with atomic clocks.
Zhou Liu, Kexing Sheng
New York Rental Properties Pricing Prediction(paper)(presentation)
New York is renowned as one of the most expensive cities in the world. Analyzing rental property prices in relation to various factors such as location, room types, and days occupied can provide valuable insights. In recent years, machine learning has emerged as a crucial technique for forecasting rental property prices, especially with the availability of big data. Machine learning algorithms can effectively anticipate asset prices based on their quality, independent of relying solely on prior years' worth of data. Numerous studies have explored this issue and demonstrated the effectiveness of machine-learning approaches. These algorithms are trained to provide improved valuations by considering factors such as land price, location, and supply and demand dynamics (Truong et al., 2020). The results from these studies have been promising and consistent with contemporary findings, further validating machine learning as a powerful tool for price prediction in the real estate market (Jacob et al., 2023). By leveraging machine learning techniques, analysts and stakeholders can gain deeper insights into rental property prices, uncover hidden patterns, and make more informed decisions (Fields et al., 2023). This project aims to employ multiple machine learning models to uncover the underlying relationship between rental prices and New York properties. It encompasses comprehensive data analysis, and model selection, and ultimately draws conclusions from the findings.
Haojin Wu
RNN Based Stock Price Prediction(paper)(presentation)
Forecasting stock prices has always been an essential problem in the financial field. The standard method uses deep learning models, such as RNN, GRU and LSTM, to solve this problem. This article uses these three models to predict the highest price of Apple’s stock price from 2013 to 2018 and divides the training set and test set to compare the fitting and prediction effects of the three models.
During the experiment, we first divide the data set into a training set and test set and then use three models of RNN, GRU and LSTM to train and make predictions on the test set. The experimental results show that both LSTM and GRU models can fit the data well and predict the future stock price trend more accurately. In experiments, GRU works best, followed by LSTM. However, the RNN does not fit as well as expected and does not fit the data well. In contrast, GRU achieves best results. It uses only one gating regulator by simplifying the structure of LSTM, aiming to reduce the computational burden and memory footprint. Similar to LSTM, GRU also has two gates of forgetting and updating, but uses a new way to calculate, which makes GRU easier to train and optimize than LSTM.