Trang chủ‎ > ‎IT‎ > ‎Data Mining‎ > ‎

Text Mining

Text mining, also referred to as text data mining, roughly equivalent to text analytics, is the process of deriving high-quality information from text. High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning. Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output. 'High quality' in text mining usually refers to some combination of relevancenovelty, and interestingness. Typical text mining tasks include text categorizationtext clusteringconcept/entity extraction, production of granular taxonomies, sentiment analysisdocument summarization, and entity relation modeling (i.e., learning relations between named entities).

Text analysis involves information retrievallexical analysis to study word frequency distributions, pattern recognitiontagging/annotationinformation extractiondata mining techniques including link and association analysis, visualization, and predictive analytics. The overarching goal is, essentially, to turn text into data for analysis, via application of natural language processing (NLP) and analytical methods.

A typical application is to scan a set of documents written in a natural language and either model the document set for predictive classification purposes or populate a database or search index with the information extracted.


Text mining, also referred to as text data mining, roughly equivalent to text analytics, is the process of deriving high-quality information from text. High-quality information is typically derived through the devising of patterns and trends through means such as statistical pattern learning. Text mining usually involves the process of structuring the input text (usually parsing, along with the addition of some derived linguistic features and the removal of others, and subsequent insertion into a database), deriving patterns within the structured data, and finally evaluation and interpretation of the output. 'High quality' in text mining usually refers to some combination of relevancenovelty, and interestingness. Typical text mining tasks include text categorizationtext clusteringconcept/entity extraction, production of granular taxonomies, sentiment analysisdocument summarization, and entity relation modeling (i.e., learning relations between named entities).

Text analysis involves information retrievallexical analysis to study word frequency distributions, pattern recognitiontagging/annotationinformation extractiondata mining techniques including link and association analysis, visualization, and predictive analytics. The overarching goal is, essentially, to turn text into data for analysis, via application of natural language processing (NLP) and analytical methods.

A typical application is to scan a set of documents written in a natural language and either model the document set for predictive classification purposes or populate a database or search index with the information extracted.


Resources:

StreamHacker has a series of tutorial articles on "Text Classification for Sentiment Analysis" using NLTK. 

You can begin it here: 
Text Classification for Sentiment Analysis – Naive Bayes Classifier

Here are the other articles from the tutorial series:

  1. Text Classification for Sentiment Analysis – Precision and Recall
  2. Text Classification for Sentiment Analysis – Stopwords and Collocations
  3. Text Classification for Sentiment Analysis – Eliminate Low Information Features

This post describes the implementation of sentiment analysis of tweets using Python and the natural language toolkit NLTK. The post also describes the internals of NLTK related to this implementation.
Twitter sentiment analysis using Python and NLTK


Here are two tutorials that use NLTK’s built in Naive Bayes classifier. You should be able to use their example code to get started.
Twitter sentiment analysis using Python and NLTK

Sentiment Analysis

You want to add your own emoticon dictionary to it? I found this open source project called Vader Sentiment Analysis on Github (cjhutto/vaderSentiment). I feel it would be a good starting point given the limited information provided by you.

Also NLTK's documentation (NLTK Book) is pretty good and once you have an outline on what processes you want to do like stemming, stop word removal etc you can always find snippets of code on Stack Overflow.