Is there any way to use Yahoo Pipes to remove RSS feed items that are covering the same news story?
-
e.g. if I combine the RSS feeds of Techcrunch and Yahoo: Techcrunch posts a story that company X just raised a $5m seed round of funding; Mashable post an article within the same hour also covering the same story. Is there any way to remove the Mashable article from the combined feed using Yahoo Pipes (or similar)?
-
Answer:
Unless the story has the exact same text, in which case you can just compare the contents and if they match by 90% (using RegEx) you decide which one to keep. If they are only similar articles, but written separately, you could technically still match them with RegEx, but it would be overly complex and prone to errors (false positives). You need much more complex solutions than a set of RegExes to do what Google does to news articles (group articles on the same topic).
Eddie Barcellos at Quora Visit the source
Other answers
Pipes offers a "Unique" module but it requires that you identify the fields that uniquely identify an item in the feed.
Greg Jarmiolowski
For your particular problem Yahoo Pipes might not be a complete solution, as building more complex filtering logic in Pipes is quite difficult, if not impossible. I have a option, but it is non-trivial. Consider https://github.com/ggaughan/pipe2py it's a program that will download and convert your Pipe logic into Python So you could have Yahoo Pipe setup with all the logic you expect you need for the aggregation of your input feeds. Insert a "null operation" of some sort before the output of your stream, look for that operation in the python output program. Create a filter module and replace the null operation with your filter, and run the subsequent python program, (perhaps on a cron job, that outputs an rss file to a local webserver? I'm not sure about that last mile bit) Using your already existing Pipe logic to bootstrap your own filter that you could get running on Google AppEngine. But be aware that neither Yahoo or the output Python program would have knowledge of articles in the past - only their current window of feeds - if a new article pops up days later than the rest of the pack, you'll probably still see it. Good luck. Yahoo Pipes is a great tool, but is limited in scope as to what it can do, also look into YQL, I don't think it will help you in your current endeavor, but its a good tool to know well.
David Hagler
Do not use Yahoo Pipes full stop period! A few years ago a build a set of Yahoo Pipes interfaces that I figured were pretty cool, they were deprecated without much notice. They are a waste of effort.
Bob Hooker
Related Q & A:
- How to save RSS-feed to a database?Best solution by feeds2mysql.com
- How to create RSS feed in Java?Best solution by Stack Overflow
- Wat is RSS n RSS FEED?Best solution by Yahoo! Answers
- How do I add an RSS Feed to My Yahoo?Best solution by Yahoo! Answers
- What exactly is an RSS feed and how/why would you use one?Best solution by Super User
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.