Choosing the best set of keywords?

Choosing the best set of keywords

• I have a dataset of tweets collected using twitter streaming API on a particular topic (say 'football') using around 40 keywords. Now if I'm going to track the same topic (football) in future how do I determine the best set of keywords to query. Any researchers working on these areas? How do I solve this keyword ambiguity? What are the sophisticated and state of the art techniques?

• Answer:

  In terms of a general response, there are definitely some excellent resources out there and lots of research going on in the space. I will focus first on topic models, since I used that in my previous work position and have done some research in this area. As a starting point, you might want to read an excellent paper https://www.cs.princeton.edu/~blei/papers/Blei2012.pdf by Dr. David Blei. As Dr. Blei notes in the article: "Topic models are algorithms for discovering the main themes that pervade a large and otherwise unstructured collection of documents." In the paper, he surveys a number of different approaches that can be used. This provides tremendous perspective. If following this, you are more interested in getting hands on with the specific technique of topic modelling, and are comfortable with say something like R, you could have a look at the following http://cran.uvigo.es/web/packages/topicmodels/vignettes/topicmodels.pdf. It provides a good mix of both theory and R code. If you want to go even farther, you could also read the package notes for http://cran.r-project.org/web/packages/topicmodels/vignettes/topicmodels.pdf. This work, as well as the others, should serve as a great starting point, while also reinforcing the fact that research and resources in this area are abundant (with lots of room for growth and new knowledge).