What is the best method to script the detection of domain and HTML info for a list of domain names.
-
My client is a domain broker and gets given a list of several thousands domain names by possible sales opportunities and my client wants me to write a reusable tool to read in the URLs and detect for each the following attributes: Domain registration status Domain registration date Dns records (A, MX) IP address of host Whois record (to insure ownership) Nslookup of IP or IP Whois Whois data Google page rank Google links to the URL Alexa rank HTML Meta tags Date of HTML last updated Search HTML for an array of keywords and tag where relevant (eg wordpress or <table>) What method or languages would you use to easily identify most of these for the least amount of effort. I had been thinking of Loading records into SQL and then stepping a python or php script to step through these ideally to a series of appropriate APIs. Or even a VB or autohotkey webscraper script? Horrid but could get it done.
-
Answer:
A webscraping script would be a quick and dirty way, but bombarding a lookup service with a huge number of requests might look suspicious.
Vivek Nagarajan at Quora Visit the source
Related Q & A:
- What is the best language for HTML parsing and web scraping?Best solution by Quora
- What is the cheapest method for shipping a domestic item?Best solution by csmonitor.com
- What are the top 5 most expensive domain names?Best solution by en.wikipedia.org
- What is the best studying method?Best solution by Yahoo! Answers
- What's the HTML code for a scroll box on top of a picture?Best solution by Yahoo! Answers
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.