Is it possible to crawl through a database?

How do travel sites deploy meta search?Do they store all the data in their database or do they crawl other websites for information ?

  • For eg. Going from a place A to place B how does it provide all modes of transport i.e by road ,air and rail along with the ticket cost. Thanks in advance :)

  • Answer:

    Predominantly APIs and web scraping. They cannot store the data given its inherent dynamic nature.

Varun Khona at Quora Visit the source

Was this solution helpful to you?

Other answers

No serious meta search engine uses web scraping, it's a very bad and short-term solution that will fall apart quite easily. Maintaining such solution is a nightmare, especially for serious metasearch players (e.g. Kayak, Momondo, http://Atlastravelengine.com, etc.) with many providers integrated into their engines. The only long-term solution is to integrate the travel websites into the metasearch engine through API or feeds. Yes, metasearch engines store some (not all) data in their database. This is required for performance and flexibility in the service.

Elie Messo

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.