Why Doesn’t My Site Show Up In The Search Engines
When it comes to marketing your website, the first place you are likely to look for traffic is the search engines. With the combined databases of Google, Yahoo and MSN being used for roughly 80% of all searches on the internet, a good position in the major search engines can mean the difference between lots or little traffic. Valuing the power of search engines to convert this traffic in to sales, site owners often ask themselves,
“Why doesn’t my site show up in the search engines?”
The following information hopes to answer this question by explaining how search engines work and what they look for in your site. For the purposes of this discussion, we will concentrate on the three major traffic bringing search engines of Google, Yahoo and MSN.
How Search Engines Work
Search engines use a computer program to prioritize websites in order of relevance to a search phrase. This means not only distinguishing that for example, PCWorld.com has more to do with “computer sales” than ToysRUs.com, but also which is more relevant to “computer sales” between PCWorld.com and Dell.com. The computer program that does this is called an algorithm and although each search engine aims to make its algorithm better than its competitors, they all use similar criteria to help reach the decision. These criteria are discussed further down.
The order in which a search engine lists web sites after a query for a specific search term is decided based on the information that is added to a search engine’s database by “bots” or “spiders”. These are computer programs that jump between web pages by following the links on those pages. They digest web page content as they go and add the information that they find to the search engine database.
Making your Website Search Engine Friendly
Before actively working on marketing your website, you should check that nothing about your website is disagreeable to search engines. Some web development techniques limit the ability of search engines to get and display information about your site. Check your website for the following:
Code Burden – Many web pages use additional code to perform specific tasks such as image roll-overs or fly-out menus. This code bulks up the file size and the length of code in the pages, giving search engine spiders more work before they reach the actual meat and potatoes of your website. This code can often be easily referenced from an outside page, instead of being embedded in the existing page, speeding up the download time of your pages and making sure that important content is not diluted.
Dynamic Urls – When reading pages that have been generated “on the fly” from information in a database, search engine spiders can get caught in an infinite loop causing bandwidth and cost increases. The chosen solution is to stop the spiders from indexing these pages. However this results in these sorts of sites performing poorly in the search engines. Dynamic urls usually end with a series of characters like this:
Everything after the “?” shows that this is a dynamic URL.
Although most search engines claim that they can deal with these sorts of pages there is strong evidence that search engine spiders do not crawl these pages as thoroughly and deeply as simple, static HTML pages. The result is less of your web pages being visible to the public through the search engines and a loss of valuable ranking material.
Flash – Macromedia Flash can be used tastefully to stylize a website but overuse can render its content almost invisible to the search engines. Sites that are constructed purely of Flash do not have the text content that search engine spiders feed on.
Framesets – Framesets or Frames can make spidering a web page difficult for a search engine, resulting in surfers being sent to a partial or limited page when clicking on a link to it from a search engine listing. Frames allow more than one web page to be displayed within the boundaries of a single browser window. Often a visitor will not even realize that he is viewing more than one page at once. A tell tale sign of framesets is when you scroll your content up or down, the navigation does not move but instead stays in exactly the same place.
Image Text – If the text on your website is embedded in an image instead of being typed HTML code, then it is almost useless to the search engines and a potentially valuable resource has been wasted.
Telling Search Engines What your Website is About
So your site is search engine friendly, there are no elements limiting search engine spiders but you are still not showing up for your keyword phrase. Is your site telling the search engines what it is about? The words on your website are an important factor in the search engine ranking algorithm so the right choice of phrase is imperative to gaining maximum exposure.
Keywords – Is your site targeting the right keywords? If you have not considered the keyword phrases that your visitors might type in to a search engine, then the answer is probably “no”. Keyword research is an important part of choosing the phrases around which your site is focused. Instead of concentrating on the phrase that you think is the most searched upon, look for the phrases that actually bring traffic. This might include some niche markets that you had not previously considered.
Content – The content of your site is one of the primary areas in which you need to target keyword phrases. Google likes an interlinking network of similarly themed pages so the more pages that you have mentioning your keyword, the more likely that you are to appear in the search engine results for that phrase.
Code – Streamlined code allows search engine spiders to quickly get to the important content. Code can also be used to highlight the important phrases in a page.
Meta Tags – Myth and truth – In the old days, code receptacles called Meta Tags were included in all web pages to help search engines to identify what a page was about. Unfortunately so much weight was put on these Meta Tags that they became subject to rampant abuse with site owners repeating their keywords over and over to gain top positioning or trying to capture traffic for non related phrases. Search engines devalued the power of Meta Tags and today they only account for a tiny part of what the search engine algorithm is looking for. However, every little bit helps so it is recommended that you employ both the Meta Description Tag and Meta Keywords Tag:
Meta Description – The most valuable aspect of this Meta tag is that search engines often use the contents of this tag in their listings. A well written Meta description can entice people to click on your listing. It also offers another opportunity to add keywords in to the page.
Meta Keywords – Search engines sporadically alter this tag’s relevance in their algorithm but as a general rule of thumb it is a good idea to include it in the site pages.
Competition – How much and how aggressive your competition is has a direct impact on how easy it is for your site to gain a good position in the search engines. Try a Google search for the phrase you want to appear under and then look at the number in the top bar. It will say “Results 1- 10 of about” and the following number is your competition. That number reflects the number of other web pages that are vying for ranking under that phrase. That number is often in the millions. Try this search
It shows somewhere in the 13 millions (at time of writing). Not only is that a lot of competition but the top websites are also aggressively marketing their websites. It is no surprise that it is portal websites with significant marketing budgets that fill the top ten and not single accommodation owners with a small fraction of the budget. The advice? Choose your battles wisely.
Submitting to search engines – Although submitting your site to the search engines does not hurt, it in no way guarantees inclusion or good positioning. Major search engines like Google, Yahoo and MSN prefer that they find your site by following links from another website to yours.
“ Submission isn’t necessary and does not guarantee inclusion in our index. Given the large number of sites submitting URLs, it’s likely that your pages will be found in an automatic crawl before they make it into our index through the URL submission form. ”
Regular resubmitting, contrary to the myth, does not improve search engine ranking and only serves to waste time that you could be working on other aspects of your search engine marketing. The only time that it is prudent to resubmit a website is when requesting re-inclusion in a search engine. A web page that has dropped from a search engine index may have done so because it has been identified as having problems. It is important to rectify technical and/or compliancy issues prior to the re-inclusion request.
Directory submissions – Directories are a good source of traffic and also provide the inbound links that search engines like to see. There are many large directories like Yahoo! Directory and LookSmart that offer a listing for a fee. There are also some free directories that can provide value. Look at DMOZ (http://www.dmoz.org/) as a first step.
Links – Websites without any incoming links from other websites are unlikely to gain inclusion let alone good positioning in the major search engines. Search engines value websites that are part of a themed network of unique sites and rank them accordingly. The value provided by the links is based on the following:
The quantity of links
The quality in terms of related theme
The quality in terms of linking page
This means that more links are better than less, that a link to www.goodhealthvitamins.ca from www.vitamins-nutrition.org is much more valuable than a link from www.carparts.com , and that a link from a well ranked article about vitamin supplements is better than a link from a poorly ranked page with hundreds of links to vitamin websites.
Orphaned networks, websites that interlink with each other but have no incoming links from outside the network, are not considered very valuable by the search engines, especially Google. Opting for speed rather than quality, a single webmaster can artificially create a supposedly popular website by pointing hundreds of his own websites to it. Developing incoming links takes time to research and contact and this means that exchanging links with every website is not as valuable for you as a one way link coming from another website.
Sandbox – You have great, unique content that targets niche keyword phrases, you have lots of links to your articles from similarly themed websites of good search engine standing and the Google Toolbar shows a good PageRank for your site but for some reason you just don’t seem to be able to show up in the top 100 yet alone 1000 for your search phrases in Google. What is going on? You just might have been caught in the Google “sandbox”. Despite following all the tried and tested search engine promotion practices, many new websites seem to be inhibited from progressing up the search engine ladder, leading many webmasters to claim that new websites are being deliberately treated differently to older ones. Newer websites are being placed on probation until they can prove themselves and this is something that seems to happen from 3 months to 2 years after the site launch. The fact that overnight, somewhat languishing websites suddenly jump up in search engines is cited as proof of this hypothesis. Although there is some debate over whether the sandbox exists, there is no doubt that many new sites have difficulty getting good positioning in Google and the only course of action is to continue developing good content and getting new links from websites, enjoy traffic from Yahoo and MSN and know that one day you will get a nice surprise with traffic from Google.
Time – Good search engine ranking does not happen overnight. If your business model is based on receiving instant traffic to your new website, it would be prudent to examine other methods of developing that traffic, such as paid listings in Google Adwords and Overture (Yahoo! Search Marketing), developing partnerships with lateral or vertical websites, and advertising the website through print media. It can take 3 months to 2 years before a new website is free standing in Google. In order to bring in the desired traffic during this time, you should be working on all the elements of your search engine campaign.