Which should be used after messing up clustered index? Update statistics, Reindex or Reorganize?

Google won't index my site anymore.

• my website http://www.sohodecor.com was up and running since Oct 2003, during the Chrismas season, I have change the entire product line to be Chrismas ornament products. Google at that time was able to index my entire site about 400+ pages. I can use the product name that shows on title to find the specific page on Google. after Chrismas, I spend like 1.5 months to reconstruct entire site to be spring/summer product line. However, since the update, Google only index my top page http://store.sohodecor.com ( it really should be http://www.sohodecor.com although those two look the same but it is reading different from different site, I make it the same), and won't spider further to index my entire site about 728 pages. now I become helpless that all of those hard word of creating description for each product become useless because google won't go there... those pages that are still on the Google index are old pages that didn't exist anymore and were marked Supplemental Result . Few days ago I try to add my site to some garden related directory, and add their links to my infomation page at http://store.sohodecor.com/info.html , originally this page was on the index along with store.sohodecor.com , now it disapear. don't know if Google think I was doing link spam ? or I linked a bad site that decrease my ranking ? I know only keep those link that has high Pagerank on info page. but still, Google won't index it anymore. I have been trying to add my site to DMOZ, but still in waiting list. other paid inclusion site I used like inktomi are expensive. Does Yahoo directory worth to try? will it make help Google to index my entire site again and delete the old pages? Now Google seems visit my site every 2 days( from the date info in result page), should I manuly submit all the sub pages ? Thank you for the help it's greatly apperciated. Phil

• Answer:

  Hi Sohodecor ~ I can see that you're having problems with your listings on Google, and I know how frustrating that can be. Let's start with the double listings: There are two different listings, apparently with the same content. One is the yahoo store, http://store.sohodecor.com and the other is your domain, http://www.sohodecor.com. Both can be found listed in Google's index by searching for the URL: 1. Search for: store.sohodecor.com - ------------------------------------- "Showing web page information for store.sohodecor.com SohoDecor.com -- Ornaments for your Home and Garden -- Ornament | Garden Decor | Ceramic Ball Ornament | Holiday Bouncy Ornament | Bella Ornament Light Catcher Ornament | Rustic Ornament | Ornament Wind Chime ... Google can show you the following information for this URL: * Show Google's cache of store.sohodecor.com * Find web pages that are similar to store.sohodecor.com * Find web pages that link to store.sohodecor.com * Find web pages that contain the term "store.sohodecor.com" - ://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=store.sohodecor.com&btnG=Google+Search Clicking on "Find web pages that link to store.sohodecor.com" did not match any documents. - ://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=link:DGaYb35pinsJ:store.sohodecor.com/ Clicking on "Find web pages that contain the term "store.sohodecor.com" returns 257 pages, most of which are Winter and/or seasonal ornaments, and those pages all return to your first page. 2. Search for: www.sohodecor.com - ----------------------------------- "Showing web page information for www.sohodecor.com SohoDecor.com -- Ornaments for your Home and Garden -- Ornament | Garden Decor | Ceramic Ball Ornament | Holiday Bouncy Ornament | Bella Ornament Light Catcher Ornament | Rustic Ornament | Ornament Wind Chime ... Google can show you the following information for this URL: * Show Google's cache of www.sohodecor.com * Find web pages that are similar to www.sohodecor.com * Find web pages that link to www.sohodecor.com * Find web pages that contain the term "www.sohodecor.com" - ://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=www.sohodecor.com&btnG=Google+Search Clicking on "Find web pages that link to www.sohodecor.com shows no returns. - ://www.google.com/search?hl=en&lr=&ie=UTF-8&oe=UTF-8&q=link:GNGWS8ek52kJ:www.sohodecor.com/ Clicking on "Find web pages that contain the term "www.sohodecor.com" returns 9 pages. _________________________________ =================== Yahoo Search =================== Searching Yahoo's search results only shows one listing - which is for a PDF file which is no longer there: "http://www.sohodecor.com/PO_Form1.pdf www.sohodecor.com/PO_Form1.pdf - More pages from this site and clicking on "More pages from this site" does bring up your website. - http://search.yahoo.com/search?p=www.sohodecor.com&&fr=fp-pull-web-t&vst=0&vs=www.sohodecor.com Searching Yahoo's directory produced the following results: "We didn't find any Directory pages containing www.sohodecor.com" Your store URL, store.sohodecor.com, did produce results on Yahoo's search engine, although it is not listed in Yahoo's directory, either. ====================== All The Web Search ====================== Searching for your domain, www.sohodecor.com, produces no results: "No entries for www.sohodecor.com in our index." - http://alltheweb.com/search?avkw=fogg&cat=web&cs=utf-8&q=www.sohodecor.com&_sb_lang=pref Nor does your Yahoo store URL produce any results: "No entries for store.sohodecor.com in our index." ================ MSN Search ================ Your Yahoo store does snow up in MSN's Search results, or at least two pages, the Wind Chimes and Bird Feeder pages. - http://search.msn.com/pass/results.asp?RS=CHECKED&FORM=MSNH&v=1&q=&q=store.sohodecor.com&cp=1252 Your domain name, www.sohodecor.com, likewise shows up, - http://search.msn.com/pass/results.aspx?q=www.sohodecor.com&FORM=SMCRT ____________________________ So what does this mean to you? If this were simply a Google anomaly, the advice might be to wait; however, when the other major search engines are experiencing problems as well, then a closer look is in order. ==================================== Duplicate sites/Duplicate Content ==================================== This last Publishers Conference in Orlando said basically the same thing. For good placement in free search engine results, the best advice is still to follow Google's Guidelines. Important Google links will be included at the bottom of this answer. ===================== Duplicate Content ===================== You mentioned - and are right to be concerned about - duplicate content on the two URLs. A check with Whois show your domain is also hosted by Yahoo. I am a Web Designer/Developer and many of my clients use Yahoo Stores. I know Yahoo has the ability for its merchants to either permanently redirect their domain to their Yahoo Store, or to redirect the Yahoo store results so that it always looks like their domain, and the Yahoo store URLs do not show. You need to make a choice one way or the other and get Yahoo to fix it for you so that only one URL is showing. Most search engines will penalize you for duplicate content. Some either show only one domain, will 'penalize' you in lower placement, or may drop both listings altogether. The fact that you have made some design errors which literally hobble search engine crawlers doesn't help, either. You now have those same errors on two different websites. ============================ HTML - Back to the Basics ============================ When I first looked at your site, I thought it looked "odd". I have taken the liberty of doing a "quick and dirty" screen shot of what I see when I look at your page. By "quick and dirty", I mean that I did not optimize this graphic for anything other than to show you what I am seeing in several browsers. Take a look at http://209.151.82.182/soho/sohodecor.jpg ... I suspect that is NOT what you want visitors to see. This is what the page looks like, however, in Mozilla 1.6, Firebird, Opera, and IE 5.5. The site does look as you intended in IE 6.x, but more savvy users are starting to make choices other than IE as their browsers of choice. This led to taking a close look at the coding, and there were some important design elements missing. Shari Thurow, one of the leading authorities in web design and search engine optimization (and author of the book "Search Engine Visibility"), recently made the following observation with regard to HTML: "Clean HTML is absolutely imperative for search engine indexing. Browsers are extremely forgiving when it comes to displaying pages with "unclean" HTML (unclosed tags, no quotation marks, etc.). Search engine spiders are not so forgiving. Even something as simple as a missing quotation mark on the <.a href="page.html"> can cause a spider to not index text or a link." (See: Link Exchange Digest, July 3, 2003, "Clean HTML") - http://list.audettemedia.com/SCRIPTS/WA.EXE?A2=ind0307&L=led&D=1&T=0&H=1&O=D&F=&S=&P=282 She explained how errors in HTML can affect your ability to be indexed, and ultimately, ranked. Among those elements missing are: 1. DOCTYPE Declaration ---------------------- DOCTYPES are essential to the proper rendering and functioning of web documents in compliant browsers. It is also essential for the search engines to understand and follow the coding contained on your pages. DOCTYPE is explained and discussed further in "A List Apart", - http://www.alistapart.com/stories/doctype/ and in Web Design Group's article, "Choosing a DOCtype", - http://www.htmlhelp.com/tools/validator/doctype.html You do not have a DOCTYPE Declaration and you should add one to every page on your site. 2. No close Head </head> tag ---------------------------- There is no close of your head tag. Some search engines, in this case, IE., assumes it's there when it reaches the <body> tag; however, not all browsers make such assumptions, which is why there's that rather alarming blank place. In its Technical Guidelines, Google suggests downloading and using the Lynx browser, "Use a text browser such as Lynx to examine your site, because most search engine spiders see your site much as Lynx would. If fancy features such as Javascript, cookies, session ID's, frames, DHTML, or Flash keep you from seeing all of your site in a text browser, then search engine spiders may have trouble crawling your site." - ://www.google.com/webmasters/guidelines.html Lynx is only getting to the first mapped image with links. At the present time, this is basically what search engines are seeing" "[yhst-9901986642544_1779_573.jpg] [yhst-9901986642544_1779_39386.jpg] [yhst-9901986642544_1779_138486.gif] [USEMAP:yhst-9901986642544_1780_117595266.gif] pad [EMBED] SohoDecor Bird Feeder SohoDecor Bird House SohoDecor Charisma Rooters SohoDecor Welcome Sign SohoDecor Hummingbird Feeders SohoDecor Spring Garden Decor Collections ____________________________________________ Butterfly Feeders | Floating Candle | Door Decor | Wall Decor | Lantern | Garden Accessory Wind Chimes | Welcome Signs | Bird Feeders | Bird Houses | Votive Candle Holder Planter & Stake | Rooters | Shelf Sitters | Stepping Stones | Sun Catchers | Wine Bottle Charms SohoDecor.com - Your Source of Christmas Ornaments and Garden Decorations Copyright 2004 LivingSoho, Inc. & SohoDecor.com - All rights reserved. [botomicon.jpg]" You can download the Lynx browser here: - http://lynx.browser.org/ You will definitely want to correct the design deficiencies in order to make it more user friendly and search engine friendly. Remember, too, that the contents of your Flash cannot be indexed, which means that basically you're not giving much to the search engines to find and crawl. ========================================== Important Google Links For Your Reference ========================================== I am including important links from Google's site. This information will help you understand Google's goals and responsibility to the web searcher (and not the webmaster or site owner). Webmasters who follow the guidelines and avoid Google's "Thou shalt nots" usually have no problem getting listed and showing up under the search terms they desire. * Google Today (absolutely the best information you can read about the "how and why" of Google's results - ://www.google.com/corporate/today.html * How Do I Get My Site Listed on Google? - ://www.google.com/webmasters/1.html * My Web Pages Are Not Currently Listed (a good 'primer' on how and why Google works so well) - ://www.google.com/webmasters/2.html * PageRank Information (covers both Google's PageRank and SERPS) - ://www.google.com/webmasters/4.html * Webmaster Guidelines (contains both the dos and don'ts) - ://www.google.com/webmasters/guidelines.html * Google Facts & Fiction (can you buy your way to a high ranking in Google?) - ://www.google.com/webmasters/facts.html * Search Engine Optimizers (some good advice on what to look for if you're going to hire a Search Engine Optimizer) - ://www.google.com/webmasters/seo.html * Frequently Asked Questions (pretty much what it says, but definitely worth wading through) - ://www.google.com/webmasters/faq.html * Remove Content from Google's Index (just in case you feel a burning need to start all over again) - ://www.google.com/remove.html ================================ Other Sources of Information ================================ There is also good information from many of the top search engine optimization experts, such as * Detlev Johnson, Search Engine Guide - http://www.searchengineguide.com/detlev/ * Danny Sullivan, Search Engine Watch - http://www.searchenginewatch.com/ * Jill Whalen, High Rankings - http://www.highrankings.com/ * Shari Thurow, Web Pro News (and quoted all over the Internet) - www.webpronews.com/ who all have columns or newsletters to which you can subscribe and keep abreast of the best way to use good content for better positioning in search engine results. In addition, Webmaster World - http://www.webmasterworld.com/ - has discussion boards on most of the search engines. While some of the discussions are anecdotal and/or questions for information, there is usually enough discussion to keep abreast of what seems to be happening. There is an entire section devoted to Google at: - http://www.webmasterworld.com/forum3/ It never hurts to keep track of the discussions; but remember, trying to optimize for search engines only is like trying to hit a moving target. You'll notice among the more experienced contributors to the discussions - plus the SEO experts listed above - that there really is no substitute for content, relevant links and good HTML. =========== Summary =========== Your question was directed specifically at getting your pages indexed. In order to do so, you need to clean up the design. There really is no reason to use an image map for your left hand margins, when text on the blue would be easier for search engines to find and index. Besides the design problems, there are problems with the fact that you have duplicate content on two sites, even though they may be interlinked. Duplicate content is specifically seen as a method of spamming, and to keep from being penalized, it would be best to ask your Host to permanently redirect one of your sites to the other and then only try to list the actual site on the Web. You did not ask about ultimate search results placement for the search terms you want. Although it is beyond the scope of your question, it would be wise to remember that high placement requires a combination of links TO your site, relevant content on your page and good HTML to make your site both user friendly and search engine friendly. If the other search engines are still admitting that designing for Google's standards will ensure placement in their own search engines, it would behoove you to make sure your site meets the guidelines set out in the Google links pages I mentioned above. Search strategies =================== Other than the specific searches conducted and mentioned above, I relied on the material I use in the daily course of my own business as a Web designer/developer and search engine optimist. Making the changes to your site with Google's Guidelines and good HTML practices will greatly enhance the chances the search engine crawlers will find and index your site. Best wishes, Serenata Google Answers Researcher