What exactly is URL scraping?

Screen-scraping laws in India

• Screen-scraping is a process by which content can be pulled off a website/ecommerce engine on the internet using robot/crawler scripts and can then be used on one's portal for end consumers to search/look on. This technique is useful in aggregating required information from many portals/ecommerce websites and presenting them to the end user as a more comprehensive search result. My questions are: 1) What do the Indian IT/IPR Laws say about screen-scraping or related technique s of content aggregation from multiple websites? 2) If the screenscraping portal acknowledges and references the URL of the sites/portals from which the content/information was pulled, does it minimize the risk of litigation? 3) Have there been past instances of cases/disputes arising due to screen-scraping / crawling-robots ? Also - Generous tip promised if all such cases in Europe/US can be compiled alongwith legal commentary, rulings and insights. I know that there have been many such litigations by airlines and ecommerce vendors there. Thanks!

• Answer:

  bajubakait-ga, Thanks for a most interesting question. As far as I know, there is little or no law anywhere in the world that directly addresses the practice of screen scraping. The internet is simply too new for legislation to have fully caught up to all the legal issues that it presents. And where legislation has been passed, it tends to focus on the major issues of cyber-security, privacy, pornography, and spam, rather than the important -- but less visible -- practice of screen scraping. This is not to say that there is no law that applies to the practice. All content, whether on the web or otherwise, is protected by copyright and other intellectual property law to various degrees, and a business is also generally protected against practices deemed blatantly unfair (though with the usual difference of opinion as to what is or is not fair practice). Cyber-law in India is a good case in point. But before proceeding any further, though, let me point out the disclaimer at the bottom of the page -- Google Answers is not a substitute for professional legal advice, so take everything here with the appropriate grains of salt. Back to India. The main law that directly address practices on the internet, and that describes various offences in cyberspace, is The Information Technology Act of 2000. You can find a full copy of the act here: http://unpan1.un.org/intradoc/groups/public/documents/APCITY/UNPAN002959.pdf though it might be easier, for starters, to make use of this overview of the law, written by one of India's pre-eminent authorities on cyberlaw: http://unpan1.un.org/intradoc/groups/public/documents/apcity/unpan002090.pdf India's Information Technology Act, 2000 ...India's first cyber law makes punishable cyber crimes like hacking, damage to computer source code, publishing of information which is obscene in the electronic form, breach of confidentiality and privacy, and publication of digital signature certificate false in certain particulars, says noted Supreme Court advocate Pavan Duggal. As the law and the summary show, the main offences defined in the Information Technology Act focus on tampering with source code, disruptive hacking, obscenity, and, in effect, forgery of digital signatures. There is nothing in the Act that specifically addresses screen scraping. By the way, the Information Technology Act in India was designed around a piece of model legislation developed by the UN, so the principles embodied in the act are commonly found in similar pieces of legislation around the world. In my searches, I did not find any screen-scraping legal cases that have gone to court in India. =============== Of course, the fact that screen scraping is not specifically addressed in the laws does not mean that controversies won't arise over the practice. One of the most active areas of dispute over screen-scraping involves the airlines, and their online fare offers. The airlines consider their fare-management systems to be part of their proprietary property. Outside firms that scrape the fares from an airlines site have been sued over the practice. As I said, I am not aware of any cases in India along these lines. There have been a few in the US, and since I am most familiar with US law, I have focused my review on a few key US cases. None of the case law that I am familiar with is at anything resembling a final stage. There have been preliminary decisions, appeals have been filed, and there have been several out-of-court settlements. However, there are not yet any firm rules of the road regarding screen scraping. One of the first major tests of screen scraping involved American Airlines, and a firm called FareChase: http://cyberlaw.stanford.edu/about/cases/american_airlines_v_farec.shtml American Airlines v. FareChase ...AA successfully obtained an injunction from a Texas trial court, stopping FareChase from selling software that enables users to comparison shop for webfares if it also searches American's website. The airline argued that FareChase's websearch software trespassed on American's servers when it collected the publicly available data. As you can see, the airlines won the first round, and convinced the court to issue an injunction temporarily preventing FareChase from further screen-scraping of their site. However, the injunction is being appealed, and the cases are still in motion. You can read FareChase's actual legal argument appealing the decision here: http://cyberlaw.stanford.edu/about/cases/FareChase%20Opening%20Brief.pdf BRIEF OF APPELLANT, FARECHASE, INC Southwest Airlines has also challenged screen-scraping practices, and has involved both FareChase and another firm, Outtask, in legal tussles: http://www.btnmag.com/businesstravelnews/headlines/frontpage_display.jsp?vnu_content_id=1000495420 ...Yet, given Southwest's legendary control over distribution, it is not happy with any access it has not explicitly authorized. "As far as we know, neither of them are screen scraping our site," Brown said of Galileo and Expedia, "but we will do everything we can to defend our rights and our property. We want to make sure that nothing we do lends itself to someone charging us fees down the road." He reiterated Southwest's insistence that no outside entity secure automated access to the carrier's fares and inventory. ...The airline continues a legal battle to prevent travel technology provider Outtask from accomplishing that objective. The parties are in the process of mediation, according to a Southwest spokesperson, and "dialogue has not been amicable." An Outtask motion to dismiss half of Southwest's claims was denied last month by the Texas judge overseeing the case. However, according to court documents, Outtask last month stopped scraping fares and inventory from southwest.com. ...Meanwhile, Southwest expects "a very peaceful agreement" with FareChase, which also was named in the Outtask lawsuit. "FareChase ceased screen scraping some time ago, so the ins and outs of mediation may not be necessary," said the Southwest spokesperson. It is interesting to look at Southwest Airlines v FareChase and Outtask in detail, as it lays out the many different legal complaints filed in the case. Unfortunately, the full text is not available online, but I was able to access it through a subscription database. In sum, Southwest Airlines is charging that the screen-scraping is illegal for any and all of the following reasons: The screen scraping is an example of "Computer Fraud and Abuse", and specifically, has led to "Damage and Loss", and to "Unauthorized Access" of Southwestern's site. It also constitues "Interference with Business Relations", "Trespass" and "Harmful Access by Computer". Furthermore, the screen-scraping consitutes what is legally known as Misappropriation and Unjust Enrichment, and is also a breach of the user agreement on the Southwest.com website. Whew!!! Outtask denies all this of course, and claims that the the prevailing law in this case should be US Copyright law, and that under copyright, the small bits of information being scraped would not be subject to copyright protection. The bottom line on all this is that the law is unsettled, in the US, and everywhere else in the world that I am aware of. So, screen-scrape at your own risk. It may turn out to be an acceptable practice over time. Or, it may turn out to be an enormous legal liability for those who undertake it. And even if the law is, eventually, on the side of the screen-scrapers, the practice can still lead to expensive litigation in both local jurisdictions, as well as across national borders. This last point is important. A computer in India, scraping another site in India, whose server is in New York, may find itself in a very complex, international legal entanglement: http://unpan1.un.org/intradoc/groups/public/documents/apcity/unpan008152.pdf Cyberlaws in Information Age ...One of the critical issues in the cyber era is a matter of jurisdiction, which is the authority of a court to hear a case and resolve a dispute within a sovereign territory. Because the legal environment of e-commerce has no geographical boundaries, it establishes immediate longdistance communications with any one who can access the web site. Usually an on line emerchant has no way of knowing exactly where the information on its site is being accessed. Hence, the jurisdiction issue is of primary importance in cyberspace. Acknowledging or otherwise identifying the source of any scraped information does not offer any legal protection that I am aware of. Either you have permission, or you don't. And if you don't, then proceed with caution! I trust this is the sort of information you were seeking. But if there's anything else I can do for you on this, don't hesitate to let me know. Just post a Request for Clarification, and I'm at your service. Cheers, pafalafa-ga search strategy -- Answer based on my knowledge of the topic, as well as bookmarked sites for intellectual property case law.