Find solution
How to find Stakeholder analysis tools?

What are the best practices for doing Sentiment Analysis for Named Entities (rather than documents or sentences)?

  • Background I've been experimenting with using some open source text analytics tools (including NLTK and the Stanford tagger) and reading about others (like scikit-learn and pattern).  I'm particularly interested in: 1) Named Entity extraction. 2) Sentiment Analysis / Classification. It seems like NE extraction is relatively easy using NLTK and (as long as you take care with text encoding for non-ASCII characters).  Similarly, classification using Naive Bayes works well for sentiment analysis at a document level (e.g. for the movie review corpus). What if I want to find good guys and bad guys? Consider the following text: 'Quora is a great forum.  It has lots of good questions and answers. Many users are knowledgable. It is a shame that one of its users, Rahim, is an idiot.' What is a good technique to identify that the named entity 'Rahim' is attached to a negative sentiment?  I've considered: 1) Using the document sentiment.  This won't work well in instances like the one above, where the sentiment for a particular named entity is different from that for the document as a whole. 2) Using the sentence sentiment.  I could count a positive sentiment towards a Named Entity whenever their name appears in a sentence which is classified as positive. Does anyone know a better way?

  • Answer:

    A better method would be to: 1. parse the document or documents into scentences or phrases; 2. select out a random subset of these phrases; 3. classify these by hand according to the classification you want. [ie a phrase that is positive for the named entity you are looking for, then your class that as positive; if it is negative for that NE, then negative; relation to any other NE is irrelevant. You should know this from reading the phrase] 4.  use this training set to train a classifier (nltk has some) 5.  classify the rest of your phrases, then recombine the classified phrases into the documents they came from. 6.  if you want to classify for a different NE, then you will have to repeat 2 - 5. This method will provide a better measure across the document(s) of sentiment towards the thing you want to understand. The bow method from Jacob would give overall sentiment, including sentiment to other things.

Nigel Legg at Quora Visit the source

Was this solution helpful to you?

Other answers

In addition to sentence level sentiment analysis, also look into phrase level sentiment. Phrases can be identified using deep parsing or partial parsing (aka chunking),. Then to get the sentiment, you can use known keywords (generally adjectives). There's a few sentiment keyword lists at http://www.nd.edu/~mcdonald/Word_Lists.html or you can use sentiwordnet.

Jacob Perkins

Disclosure: I work for AlchemyAPI. You can use AlchemyAPI's text analysis service to extract entities and calculate the sentiment for each entity from any document or web page. It's actually all contained within a single API call, so you just send over the text or a URL and you get back an easy to parse JSON or XML response with each entity and its associated sentiment. To learn more about AlchemyAPI's entity extraction + sentiment analysis tool, check out the docs: http://www.alchemyapi.com/api/entity-extraction/

Steve Herschleb

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.