humanfactor
Why Human Layer in Sentiment Analysis is Useful ?
1)Cultural factors, linguistic nuances and differing contexts make it extremely difficult to turn a string of written text into a simple pro or con sentiment.
2)Seth Grimes, an analyst who runs the annual Social Analysis Symposium in San Francisco, says automated sentiment analysis tools out-of-the-box generally have a 50 to 60 percent accuracy level, as measured against how human beings would rate the same comments.
Tools have trouble with Subtlety
---------------------------------------
3)Granted, even human beings have difficulty with such subtleties. Grimes ran a Twitter poll where he asked people to evaluate the phrase, “I bought a Honda yesterday.” Of the 22 respondents, 45 percent rated the comment as positive; 55 percent as neutral. If people have trouble agreeing, it’s understandable why machines would often miss the mark. Beyond that, sentiment analysis tools have trouble with irony, humor, and subtleties of human speech, like how an emoticon such as -
can change the intent of blistering words.
Machines have trouble with identifying Spam
-----------------------------------------------------------------
4)That’s not the only issue with accurately classifying the tsunami of social media feelings. “In some areas, we’ve seen anywhere from 20 percent to close to 100 percent of comments in social media as spam,” Ramirez says. “When you search the key word ‘auto loan’ and find all these people asking, ‘Any suggestions on where to get an auto loan?’ that isn’t a conversation that’s really happening—it’s spam.” He says human beings can usually spot spam right off; machines often get fooled. And a large amount of spam may skew the data.
Human Layer of message classification can filter data such as spam identification, on which tools can work on.
5)Even with a relatively simple consumer product like a camera or smartphone, sentiment analysis software needs to be coupled with human analysis. For example, a human may set up categories and then train the software how to classify comments based on the categories.
6)“Some topics and conversations are easy to classify, some are complex,” Viswanathan says. “In any case, you always need humans to provide the context. There might be comments in a discussion forum about the amount of heat that a Dell laptop battery is generating. But only a human being can make the connection that heat in the context of a laptop battery is not a good thing. That is an aspect you need to teach the computer.”
Sentiment Analysis Limitations and Techniques to Improve Results
While they are getting better all the time, machines still face challenges when deciphering human sentiments in online statements.
Examples where sentiment analysis tools fall short:
Irony, humor and other subtleties of human speech, like how the emoticon can change the tone of an otherwise negative statement.
Spam-loaded conversations in social media that strike people as inauthentic.
False negatives, where the software sees a negative word like “crap” but doesn’t realize it’s positive in the overall context—”Holy crap! I loved this!”
Cultural differences, where some people from some countries might be more or less effusive in their use of language.
Techniques that help improve the effectiveness of sentiment analysis:
Picking a limited number of concrete product features to analyze
Pairing sentiment analysis tools with human analysts to examine contextual references
Use sentiment analysis as a starting point to identify issues for follow-up action
Connect sentiment analysis questions to a business problem
Going beyond the polarity of “positive” and “negative” to classify sentiment, and using more fine-grained categories like “angry,” “happy,” “frustrated,” and “sad.”
SMILEY Captcha helps to add human layer to sentiment Analysis in an automated way by the following the logic of crowdsourcing/human computation. It also helps in fine grained emotion Analysis such as assigning emotions such as - “angry,” “happy,” “frustrated,” and “sad" to data which is to be analysed.