Is it possible to build a stealth search engine (web crawling not web scraping) to target just one website online, without them knowing, and what coding skills or anonymity would be required?

Some things you should keep in mind: Use a reliable proxy that you can programmatically control(e.g., http://torproject.org/) Spoof user agent, a quick method is to handcraft 20-30 user agents that corresponds common web browsers then pick one randomly every time you crawl that website. Use a scheduling software to execute your crawling script regularly. For storage: go for NoSQL(MongoDB, etc.) over SQL databases(SQLite, MySQL, etc.) as it provides schema-less ways of storing data(i.e., key => value) which is, from previous experience, way better for storing inconsistent and huge data.

Use a crawler and use a user agent name similar to the browser and setup a periodic refresh on the pages you are indexing. You can setup a http collection on SearchBlox and do it for free.

Firstly, Yes it is possible to achieve what you are after fairly easily. Anonymity - A website identifies you with multiple parameters: HTTP Request Headers Cookies and other identity markers IP Address of your request Activity Pattern Every HTTP request carries with it a  payload of Headers. However, these headers are completely open for manipulation. So this is not your major concern. As suggested, you would want to spoof the User Agent String in there to appear normal and alter any other identifiable data. The IP Address is slightly more tricky. The way the web works, you will need to work hard to spoof this. For your scenario, basic spoofing will get you a long way. Proxies, VPN's and other similar tech are what you want to research. Bear in mind though, depending on the service that you use, you might end up getting the same or similar IP each time you use it thus creating patterns. Activity Patterns is an interesting, but unreliable technique to identify a user. So for instance, if you used a scheduler to run your crawler every 3 minutes using an IE10 User Agent, routing your request via a publicly available VPN service that has only a limited pool of widely known IP addresses, how difficult do you think it is going to be to track your activity after say a 100 connections? Fairly accurately! Add to that your pattern of only crawling certain keywords and there, you're bare naked in front of a smart webmaster. There are too many web crawling scripts available in the public domain that it doesn't warrant mentioning only a few here. Decide on a language that you are going to use, your coding abilities and pick a script accordingly. Have fun :)