Find solution
What is metadata?

What are effective ways for recommendation engines to deal with totally novel but potentially important content for which you have no metadata?

  • Say you have an algorithm for recommending upcoming content for live TV. This works great for regular, recurring content like TV shows or sports. But say this were 1969 and the moon landing was about to be broadcast. Obviously, this is a very important show and you want your recommendation engine to handle it correctly. Before the event, you have no real metadata to speak of. At the same time, realizing that the moon landing was an important event after it's aired is totally useless to you. What are some strategies that recommendation engines use to deal with such phenomena in a scalable way?

  • Answer:

    Effectively what you end up with is an algorithm with a temporal component with a relatively low delta on the time.  The simple case is just picking out the most popular items (think trending topics on Twitter), but you can combine historical data to the model and just filter the results for a given time span (i.e. some function which weights items by recency).  In practice that's what the Facebook event stream does. For a slightly different case, e.g. if you're processing text and it's novel, but not popular item, you can do what Google News does and cluster topics based on content and then rank them as a group. However, at the end of the day, for a truly novel event, one that has no relation to previous topics of interest, a recommendations engine will never recommend it to the first person.  Recommendations engines are effectively by definition reactive rather than proactive.  Perhaps someday machine learning will advance to the point that it's able to gauge novelty by modeling the social implications of information, but I think we're still a ways out from such.

Scott Wheeler at Quora Visit the source

Was this solution helpful to you?

Other answers

If you do not have any metadata about the show itself in terms of how many people viewed it in the past, and what kind of people viewed it and so on like they do in a item-based or a user-based recommendation system, then you should look at regression to see if the show fits your list of shows for the day. Regression can look at the other kinds of attributes of the data rather than depending on user's past preferences, and so on.

Abhishek Shivkumar

I'm going to take a different approach. The recommendation engine I've worked on over the last couple of years uses analyses unstructured text to get a numerical high-level representation of its gist that can be used to compare against other documents for similarity. Part of what we do is take product descriptions (ordinary text) and compare them against people's social media output to see which (if any!) are suited. So one way would be to take the text description of a programme (assuming there is one which is entirely probable), and compare it against the recent news broadcasts. In this case, it would match up well with the plethora of news articles, letters, etc published about the moon landing in July 1969, and come to the conclusion that its semantic relatedness to current affairs makes it potentially relevant to a lot of people. A programme about the English Civil War, by contrast, would have less relevance because media would be less focused upon related topics. I've cheated here - instead of uses people's tweets as we can now, I've used what media was available (newspapers, letters, etc). It means the findings are not customised to individuals, but it gives us some idea about the programme's relevance to society in general. Perhaps we can compare against particular publications and identify their readerships to provide audience segments as further targeting.

Anonymous

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.