Introduction & Motivation

In a world filled with brilliant entrepreneurs eager to make their ideas a reality, finding the funding necessary for such initiatives can be a challenging endeavour. Crowd-funding websites such as Kickstarter provide a way for savvy entrepreneurs to leverage the collective resources of the crowd to obtain the necessary capital to get their projects started. Kickstarter's core function is to act as a global crowd-funding platform where users (also known as backers) are offered tangible rewards or experiences in exchange for pledges of monetary support. While Kickstarter publishes many articles outlining best-practices for project creation, large scale analysis has shown that approximately half of all campaigns still fail.

We believe that meaningful indicators exist that can predict whether a project is more likely to get successfully funded as compared to others. By identifying these indicators, it is our goal to be able to provide insights into what types of decisions project creators should focus their scarce time and resources on. We believe that project creators across all categories would be able to use this information to inform their decisions and would both desire and benefit from these insights.

We feel that there are many ideas that exist in the world that, for one reason or another, do not have the financial resources or social visibility to be recognized by the public. It is our hope that by providing information to project creators about what features are most predictive of a successful campaign that all good ideas will be given equal representation and net social good will be promoted through the flourishing of novel innovations.

Dataset

The dataset for the project was obtained from web robots. It uses scraper robots to crawl all the Kickstarter projects and collects data in CSV format. The data consists of all projects from April 2009 to August 2019. Our dataset has a total of 169962 projects expanding across various categories out of which 96768 projects were successful. A successful project means that the project owners were able to crowdfund their desired goal amount. This amount is decided entirely by project owners. Each project has 37 features/attributes available.

Following is the description of each column:

• backers_count - number of people who participated in the crowdfunding for the project. This directly relates to success of the project. More the number of backers a project has, higher is the chance for it to be successful. As a result, we are not using this feature in our analysis.
• blurb - a short textual description of the project
• category - JSON object containing the category and sub-category of the project. We found a total of 15 unique categories and over 120 sub-categories. We are using category as a feature in our project. We extracted this information from the JSON object.
• converted_pledged_amount - the total amount gathered by a project converted to the original currency listed by the owner
• country - country the project owners are from
• created_at - Unix timestamp when the project was listed on Kickstarter by the owner
• creator - JSON object containing information about the creator of project. e.g. Kickstarter id number, name, etc.
• currency - original currency used to state the goal amount - e.g. USD, GBP, AUD, CAD, etc.
• currency_symbol - ASCII symbol of the original currency
• currency_trailing_code - boolean field indicating the currency trailing code
• current_currency - the currency the goal amount was converted to. Since all the goals were converted to USD, this field only has USD as entry
• deadline - Unix timestamp indicating the closure of crowdfunding period for the project
• disable_communication - whether or not a project owner has disabled communication on kickstarter. Since almost everyone opted to keep the communications open, this field mostly has FALSE as value
• friends - unclear about the purpose of this feature. It was either NULL or empty
• fx_rate - foreign exchange rate between the original currency and the current_currency
• goal - funding goal in USD
• id - unique identifier for the project
• is_backing - unclear about the purpose of this feature. It was either NULL or false
• is_starrable - boolean field indicating whether or not a project can be starred/liked/saved by the users. Again, most of the entries were FALSE
• is_starred - boolean field indicating whether a project was starred by any user. Most entries are NULL here
• launched_at - Unix timestamp indicating the date and time when the project was launched for funding on kickstarter
• location - JSON object containing the location information of the project. It includes fields like city name, state, country, town, etc.
• name - string indicating name of the project
• permissions - unclear about the purpose of this feature. It was either NULL or empty
• photo - JSON object containing information about the project's photo
• pledged - amount pledged in the original currency
• profile - JSON object containing details about the project's profile setting on the kickstarter website. It contains fields like the background colour, text colour,
• slug - hashtags for the project separated by hyphen ('-')
• source_url - url for the project's category
• spotlight - if a project is successful, it is spotlighted on the Kickstarter homepage. This boolean field indicates whether it was the case with current project - directly indicates success
• staff_pick - whether a project was highlighted by the Kickstarter when it was launched
• state - current state of the project - successful, failed, canceled, suspended or still live
• state_changed_at - Unix timestamp of date and time when a project's status was changed (same as the deadline for successful and failed projects)
• static_usd_rate - conversion rate between the original currency and USD
• urls - url to the project's page on Kickstarter
• usd_pledged - amount pledged in USD
• usd_type - whether the project is US based or not. The value is "domestic" for US based project and "international" otherwise

What do we want to answer?

Through group discussion and previous literature review, we have determined that there are 4 main questions that we wish to address through this work.

1. Perform sentiment analysis on both the project 'blurb' and 'name' sections of each project to determine if there is any relationship between the predicted sentiment and the success or failure of the project.
2. Determine if there is a specific time of year when projects can be launched where they are more likely to be funded.
3. Find out whether projects with "small" goal amounts are more likely to be successful as compared to projects with "large" goal amounts.
4. See if duration impacts whether a project will be successful or not. We want to study the impact of various durations like the time gap between the creation of project of Kickstarter and its official launch date - when the owners start accepting funds. We also study the impact of campaign length which is the difference between the launch date and the deadline - date on which owners stop accepting the funds.

Related Work

The recent popularity and success of crowdfunding platforms such as Kickstarter has lead to an exponential increase in research projects focused on predicting the success or failure of Kickstarter projects as well as outlining which features are most important to that prediction. Works such as [2], [5], and [9] use a combination of features extracted directly from the Kickstarter project page alongside other data collected from various social media in order to reach this prediction. [2] and [5] specifically are able to leverage these features with relative success. Other works such as [1] choose only to use information available directly from the Kickstarter project page and are able to achieve comparable results to the previously mentioned projects. Analysis of previous work informed our decision to use only features that were directly available from the Kickstarter project page.

A key aspect that we noticed was missing from all of the previously mentioned works is a modern application of natural language processing to the applicable Kickstarter project page features (specifically "name" and "blurb"). Works like [3] and [4] address certain aspects of natural language (notably persuasion theory) but stop short of in-depth application and analysis using the features. We decided to fill this gap in knowledge in our work by applying different variations and techniques of sentiment analysis on the applicable features.

Most of the previously mentioned works use supervised learning models as the driving force behind their predictive models. Though the creation of these models is useful, we chose to focus both on the creation of a model as well as highlight key features that were indicative of success alongside the specific aspects of these features that were most important. By doing this, we hoped to be able to provide actionable insights to real project creators about what aspects of their campaign to focus the most effort on rather than simply creating a classifier that could predict success or failure after the project was already created.

Data Preprocessing

We decided to drop some of the features that were not useful. The features discarded are :

• converted_pledged_amount - to be uniform in our analysis we decided to use USD as the currency. Since this information was made available by usd_pledged feature, we dropped this column
• creator - Since the information about the creator is not helpful in our problem statement, we dropped this field
• currency, currency_symbol, currency_trailing_code and current_currency - Since we are using USD for currency, we dropped all these fields
• disable_communication - Over 99% of the projects had this field set to FALSE and therefore we decided to remove this
• friends - All the values were either NULL or empty
• fx_rate - not a useful information for our task
• is_backing - Value was either NULL or FALSE
• is_starrable - Again, most of the entries were FALSE
• is_starred - Most entries were NULL here
• location - We decide to use country information instead of specific locations and hence dropped this one too
• permissions - We were unclear about the purpose of this feature. It was either NULL or empty
• photo - Image analysis was not in the scope of this project and therefore we ignored this one
• profile - Most of the information present here were already available via other columns. The colour of text and background were same for all the projects.
• source_url - We already extracted the project category and therefore this feature was redundant
• spotlight - Redundant field.The correlation between this and state field was 1.
• state_changed_at - this is just the timestamp of last change mostly done by kickstarter moderators. Since it was mostly done after the end of crowdfunding, it was not a contributing factor to project success.
• static_usd_rate - not related to our project goal
• urls - not useful for our project
• usd_type - We already are considering the country information as one of the features and therefore this information was not necessary

The following attributes were extracted from the existing data for further exploration:

• Extracted the general category a project belongs to. We found a total of 15 unique categories like art, comics, fashion, technology, etc. The number of sub-categories were over 120 and not all the projects had them.
• Extracted the launch month, year and day from the unix timestamps available in launched_at column
• Similar to the previous case, we extracted the deadline month, year and day from the field deadline

Data Exploration

Distribution of Projects over categories

Film and video, music, art, technology and publishing categories had the highest number of projects launched. However, categories such as comics, dance and music categories performed better in terms of success rate when compared to other categories.

Distribution & SUCCESS of Projects over time

Staff Pick

Trends in goal amounts over the years

The above graphs present the distribution of project goal amounts over the years for two cases: successful and failed projects together and only successful projects. The number of projects and their goal amounts follow a similar pattern in their growth and decay over the years. Mostly the goal amount for projects has consistently increased over the years (shown by the 75 percentile mark). The median goal amount has been nearly the same between the years 2015 and 2018 in both cases.

Trends in goal amounts over categories

On observing the goal amounts across all categories, 'technology' has projects with the highest goal amount. In fact, 75% of the projects set their goal amount as high as $50,000. But 50% of the successful projects have only $10000 as their goal amount.

Projects under categories such as 'dance', 'comics', 'arts' and 'crafts' set a much smaller goal amount. Under these categories, 75% of the projects have a goal amount of ~$10000. 50% of the successful projects have less than $5000 as their goal amount under these categories. Interestingly, if we look at the section on distribution of projects over categories, these four categories have a much higher success rate than the 'technology' category.

Trends in average amount pledged over the years

For successful and failed projects combined, the median of average amount pledged remains quite consistent over the years. The median lies between $45 to $60. For only successful projects, the median amount pledged is around $60. The 25 percentile lies around $40 for successful projects but fluctuates a lot when failed projects are also taken into consideration.

Trends in average amount pledged over categories

The average amount pledged for categories seems to mostly be in proportion to the goal amount set for the respective category (compared over only successful projects). The only exception seems to be the 'games' category. The average amount pledged for the 'games' category is one of the lowest despite its goal amount being amongst the top five highest.

Feature Engineering

There were additional features that we decided to consider for our model. Since one of our objective was to find relationship between the success of a project and its duration, we calculated a feature named campaign_days which is the difference between the deadline of the project and its launch date. We decided to use days as the granularity level because month was too big a duration to consider for a project and seconds/hours is too small. Number of days can easily be managed by product owners.

We also wanted to test if a longer waiting period between the official launch of project, for crowdfunding, and creation day negatively affects a project. To this effect, we included another feature titled creation_to_launch_days. Again for the reason mentioned above, we settled on days as the appropriate granularity.

Previous section explored the sentiment surrounding the project and and its description. We further wanted to test if there is a pattern in the length of these various textual information about the project. As a result we calculated following fields and included them in our dataset:

• slug_length - length of the hashtags
• blurb_length - length of the project description to find the ideal length that owners should use
• name_length - length of project name to see if lengthier name is desirable or a shorter one

Since our models require the data to be present as numbers, we used one-hot-encoding for the following fields - country, category, launch_month, launch_day, deadline_month and deadline_day. Moreover, since our existing features were all on different scale we decided to standardize them. To do this we decided to center the features around 0 and have variance in the same order. The following formula was used to achieve this:

z = (x - u) / s

where u is mean and s is standard deviation

Sentiment Features

One of the more complex features that we chose to engineer was the sentiment for both the project name and project blurb. The project name corresponded with a string that represented the project's displayed title and the project blurb corresponded with a string that encompassed a brief summary of the project. We chose to treat the sentiment analysis problem for this dataset as a classification problem. We decided upon positive, neutral, and negative as the 3 potential sentiment classes. Ultimately, our goal was to calculate the sentiment of both names and blurbs respectively using TF-IDF vectors in order to feed this feature into our final classifier. Note that TF-IDF was chosen as the means of representing the text data due to its ability to be computed efficiently and its previously measured success with tweet sentiment classification (both names and blurbs tend to be approximately similar in length to tweets) [7].

Our first attempt at engineering this feature was to use hand-labelled data to train a classifier and use that trained model to attempt to predict the sentiment labels for the remaining data. We randomly sampled 1,000 projects from our dataset and hand-labelled both the name and blurb for each. Due to the tendency of project creators to use very similar wording in both their name and blurb, oftentimes the labels would be the same for a given project. The result of training and testing upon the hand-labelled data were ultimately nonviable due to the poor accuracy scores. There are several potential reasons for this but the most apparent ones are the subjective nature of hand-labelled data (especially for this application) and our choice to randomly sample the data rather than due a stratified sampling. Due to time constraints of the project and the lack of guarantee that hand-labelling would yield a viable result we chose to attempt an automated process instead.

Of the many different paradigms of sentiment mining that exist, we felt that our problem was most in line with opinion mining [8]. SentiWordNet is a automated sentiment tagger that is able to assign positive, negative, and objective scores to a given statement [6]. We chose to use the scores assigned by SentiWordNet to classify a given name or blurb as either positive, negative, or neutral (where the objective score corresponded to our neutral classification). Though less labour intensive, this process ultimately resulted in poorly correlated labels. Hand analysis showed that the sentiment tags provided by SentiWordNet were generally not in line with human annotations. The most prominent deficiency was that it was unable to meaningfully identify and label neutral sentiment (which is sometimes difficult even for a human annotator).

After our two previous failed attempts at assigning representative labels to the sentiment based on a string, we chose to change our approach. Since we had data that confirmed whether or not a project was successful, we chose to use this information to assign the sentiment label of positive, neutral, or negative to respective name and blurb categories. Projects that were successful were deemed to have positive sentiment, projects that failed were deemed to have negative sentiment, and projects that raised 30% or more of their goal amounts were deemed to be neutral (while projects that raised less than 30% of their goal amount were deemed as negative). Based on previously observed trends, we chose to assign the same label to both name and blurb sentiment for each project (since this was what occurred with the vast majority of previous labelled cases). The intuition behind this method is that we wanted to exploit the already labelled class structure of "successful" and "failed" to create subsets within our data where we knew the language used in the "name" and "blurb" sections were indicative of success.

Once we had the newly labelled data, we split it into stratified training and test data that was used to train a random forest ensemble classifier. This classifier was able to achieve a 65% predictive accuracy across all unseen test data. The newly predicted labels were then fed into the final model which ultimately lead to an increase in predictive accuracy.

Proposed Models

A wide variety of factors contribute to the success or failure of a project. Some of these factors are able to be quantified, which allows for the construction of a model to predict whether or not a project will be successful. The models provide an insight to potential project creators on features that are important to make their project successful. This section details the models we experimented with. We started out with Vanilla Logistic Regression and moved onto experiment with SVMs, Decision Trees and Random Forests. During training, we used the state of the project (successful or failed) as the output of the models.

Logistic Regression

Logistic Regression is an appropriate regression analysis to conduct when the dependent variable is binary. It is widely used as a binary classifier to predict which of the two categories a given data point falls into. Our first model is a Logistic regression model with the default parameters.

Support vector machine(SVM)

Support Vector Machine is used for classification problems similar to Logistic Regression. While Logistic Regression focuses on maximizing the likelihood of the data, an SVM tries to find a separating hyper plane that maximizes the distance between the closest points to the margin. The focus of SVM is on making the right decisions rather than predicting the probability. SVM creates a more balanced boundary between the categories than Logistic Regression. For linearly separable data, logistic regression and SVM give similar performance. If the data is not linearly separable, SVM with non-linear kernels performs better than Logistic Regression. For our experiments, we used SVM with RBF kernel.

DECISION TREES

A decision tree is a widely used non-parametric supervised learning technique for classification and regression problems. The primary advantage of decision trees are that they are interpretative and can be easily visualized. They can effectively handle both numerical and categorical data.

RANDOM FORESTS

While decision trees are simple and intuitive, they sometimes create complex tree structures which fail to generalize well from the training data. Hence, decision trees are easily prone to over-fitting. This can be countered by using an ensemble of diverse learners which have very little correlation with each other. Random forests consists of a large number of individual decision trees where each tree gives out a class prediction and the model prediction is determined by majority voting. To ensure that the individual trees are not correlated, Random Forest uses the following methods:

• Bagging: Each learner is trained on a different random sample of the training data with replacement.
• Feature Randomness: During split at each node in the tree, not all features are considered, but only a random subset of features are considered.

Model objectives

• Predicting the success of a project based on the set of attributes that a project owner has control over. These includes attributes like campaign duration, length of project name and description as well as the sentiment features introduced in previous section.
• Upon inspecting the feature importance statistics (see next section) of our best performing random forest for the success prediction task, we found that getting "staff picked" is a big indicator of success. Over 80% of the projects flagged by kickstarter were successful. The relevant pie chart is available in data exploration section above. This result prompted us to look into how can we improve the likelihood of getting staff picked. Our second model achieves this purpose.

Results and Discussion

EVALUATION METRIC USED

For evaluating the prediction of project success, we used Accuracy of the classifiers as the evaluation metric. For evaluating the prediction of a project being staff-picked, we use the micro-averaged F1 score as there is a class imbalance problem for the staff-picked projects.

The above table summarizes the results of various models with and without the sentiment features. It is noted that including sentiment features improves our model performance (across all models) when deciding whether a project will be successful or not. Out of all models, Random Forest with 200 trees gave us the best performance. It had an accuracy of 77.25 % with the sentiment features.

For staff picked projects, we noticed that sentiment related features are not useful for our models. This is evident in the F1 scores we calculated for all our models. In fact, our Random Forest model trained without sentiment features performed the best with a score of 64.04%. We believe this might be because only ~13% of total projects are staff picked and ~90% of this is composed of successful projects. Our sentiment features are trained such that successful projects are associated with 'positive' sentiment label and 'failed' sentiment label. Therefore, for training our models, these sentiment features do not play a key role in deciding whether a project will be staff picked or not.

feature importance

IDEAL GOAL AMOUNT

IDEAL CAMPAIGN

Number of campaign days

Another duration related feature that we created was campaign days which is the duration between the deadline and official launch date. The data clearly indicates that the ideal campaign length is between 30-35 days.

When to Launch a project

sentiment factor

Please read up the Feature engineering section to understand how we trained our sentiment models to generate required features on the project name/title and its description (blurb). The two doughnut charts above clearly indicates a strong correlation between having a positive sentiment and success of project. We conclude that positive sentiment predictions tend to be indicative of successful projects.

TITLE LENGTH & Blurb LEnGTH

When it comes to length of the title and project description/blurb, we could not find great differentiating factors. It can be seen that the distribution of blurb length is quite similar for both the successful and failed projects. Majority of the projects had lengths between 100-135. The title length for successful projects were however a touch higher than the failed ones. The ideal title length for project is between 25-55 characters.

References

1. Kamath, R. S., and R. K. Kamat. "SUPERVISED LEARNING MODEL FOR KICKSTARTER CAMPAIGNS WITH R Mining." International Journal of Information Technology, Modeling and Computing (IJITMC) 4, no. 1 (2016).
2. Chen, Kevin, Brock Jones, Isaac Kim, and Brooklyn Schlamp. "Kickpredict: Predicting kickstarter success." Technical report, California Institute of Technology (2013).
3. Mitra, Tanushree, and Eric Gilbert. "The language that gets people to give: Phrases that predict success on kickstarter." In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing, pp. 49-61. ACM, 2014.
4. Kuppuswamy, Venkat, and Barry L. Bayus. "Crowdfunding creative ideas: The dynamics of project backers in Kickstarter." A shorter version of this paper is in" The Economics of Crowdfunding: Startups, Portals, and Investor Behavior"-L. Hornuf and D. Cumming (eds.) (2017).
5. Etter, Vincent, Matthias Grossglauser, and Patrick Thiran. "Launch hard or go home! Predicting the success of Kickstarter campaigns." In Proceedings of the first ACM conference on Online Social Networks (COSN'13), no. CONF, pp. 177-182. ACM, 2013.
6. Baccianella, Stefano, Esuli, Andrea and Sebastiani, Fabrizio. "SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining." In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2010), 2010.
7. Alsaeedi, Abdullah and Khan, Mohammad. "A Study on Sentiment Analysis Techniques of Twitter Data." In International Journal of Advanced Computer Science and Applications (IJACSA), Volume 10, Issue 2, 2019.
8. Guellil, Imene and Boukhalfa, Kamel. "Social big data mining: A survey focused on opinion mining and sentiments analysis." In 12th International Symposium on Programming and Systems (ISPS), 2015.
9. Zhou, Mi, Lu, Baozhou, Fan, Weiguo, and Wang, Alan. "Project description and crowdfunding success: an exploratory study." In Information Systems Frontiers, Volume 20, Issue 2, 2018.
10. Lewis Laura, "Using machine learning to predict Kickstarter success. Is Kickstarter right for your project? How can you optimise for success?", Blog, April 2019
11. Shweta Pai, "Predicting Success of a Kickstarter Campaign based on its campaign description", GitHub, Accessed November 2019