e-Business Guide - search engines DIY

There are also very cheap web based suppliers whose sites say they will submit your site for $10 a month, but provide no reporting, customer service or results. Choosing correct keyword phrases is also a job in itself and has a great effect on the end result but these routes offer no analysis.

The above are all ways of submitting your domain name only and offer no way of optimising the engines and creating prime results and traffic for your site. There are numerous companies that offer search engine positioning in the UK and unfortunately not all of them professional - the basic premise is the creation of gateway pages that are perfectly written for the keywords.

These are then submitted to the engines and get ranked high on those keyword searches- when an internet searcher clicks on the result they are invisibly and immediately redirected to the clients site.

The method can be very effective but the market is letting the clients down. Companies take clients money and do not provide the results, protecting their money by vague guarantees. You can of course go to an expensive agency but still the level of work is not high and the service is not bespoke, as implied. Reporting and feedback is usually absent from both these methods.The truth is that there are a lot of smoke screens in this market.

Searching, advanced searching & boolean logic
With the enormous growth of the Internet, search engines are indexing an ever-increasing amount of pages. Because of this we have to make search engines work harder to achieve the results that we expect. Gone are the days of a simple one word query which may (or may not) have found what you wanted - now we need to be specific in order to ensure the pages returned are entirely pertinent to our search strain. This can be achieved in part by using a phrase rather than one word to specify in more detail what we are looking for. It does not stop here, however, in order to get the very best results from any given search engine there are a host of advanced features which more and more people are using in order to make their searches more specific and controlled.

Using a search phrase rather than a single word will still search the web for the individual words involved - but will give higher relevancy to those words, which appear in the same order (and proximity) as the search phrase. (i.e. 'mobile phones' will return results for pages with only mobile or phones, but will score them lower than those with both, and those pages will score lower than those with both words in the right order). In order to search for a phrase AS A WHOLE without also searching for the individual words within - place "speech marks" around the search term and only pages with that exact phrase will be returned (a closed search). This can also be used in conjunction with an open search (i.e. "mobile phones" nokia). These are quite simple techniques that can yield vastly different results (and usually more specific) and as search engine databases become larger and larger, the need to learn how to search more effectively is very important.

Nearly all search engines will let you also search for specific title tags of pages by typing title:nokia phones (see below for which engines) in order to close the search further. Many will also let you use wildcards in order to search for an array of synonyms (i.e. 'mobile phon*' will search for both 'phone' and 'phones')

There are also characters known as operators that will influence the search result

• use + in front of a word means that word must be in pages for them to be a match. "mobile phones" + accessories if you are specifically looking for a nokia phone
• use - in front of a word means that any page that has that word is excluded from the list of matches "mobile phones" +accessories -US -$ if you want to exclude all pages that contain US references

Also you may have heard of Boolean commands, which can be used to further modify your searching techniques (although these can be viewed as excessive for the average user). Below are outlines of some popular Boolean commands:

• Use AND to join two terms that must both be present for a document to count as a match mobile AND phones AND nokia if you require all of these words in the results
• Use OR to join two terms if either one counts. "mobile phones" accessories OR covers if you want either/or accessories/covers
• Use AND NOT to join two terms if the first must be present and the second must NOT. "mobile phones" accessories AND NOT $ if you want to exclude all pages with $ references
• Use NEAR to join two terms if they both must appear and be within 10 words of each other. mobile phone accessories NEAR "hands free" if you want pages which have "hands free" as a phrase near the words mobile phone accessories.

These can be used to improve searches - but we must not over look the default settings of any given search engine which will often perform these tasks for us. By using two words without the + sign before them will still yeild results with both words, similarly any two words used will score higher relevancy when they are found within 10 words of each other (without using the NEAR operator)

Major types of algorithm used - key differences

All search engines have the basic parts described above, but there are differences in how these parts are tuned. That is why the same search on different search engines often produces different results.

Some of the significant differences between the major search engines are within their results algorithms that use different criteria with which to rank a page. Some engines have stopped reading the meta information contained within a page (the keywords etc) but lend more importance to words used in the body text, their ratio to other text and prominence in the page. Other criteria which is becoming necessary is the importance of your page, which is measured by the amount of links which any given search engine has within its index which points towards the site in question. Some engines favour this method very heavily (i.e. google) which makes its results differ considerably from those that do not. The key criteria are relevance throughout the page to determine a theme, and popularity through link boosting.

Spidering how often and how deep
There are three distinct processes which search engines go through when a page is submitted. The first of these is the spider - an automated program that visits the web page(s) which have been submitted to it, and will gather the information contained within. It will read the source code of a page, read its meta information (where applicable) and will follow links on that page to retrieve more information from different pages. Spiders (or crawlers) are supposed to return to sites regularly although with the massive volume of pages being submitted these days - we have seen a marked decline in the regularity of re-spidering.

Once spidered - the information collected is placed in the index (database) of the search engine and is then available to the end user who is searching for your page. Search engines have varying lead in times between spidering your page(s) and their availability in their index from as little as 2 weeks to as long as 6 months (which can be a very frustrating wait!!).

The third part is the ranking algorithm which takes the search term entered and then strips through its entire database - using its own criteria to find all of the pertinent pages and then to determine which pages are more relevant to the search term.

Submission quantity and frequency thresholds, manual vs automatic
The issues of submission frequency are open to interpretation as there have been some developments in search engines to combat spam which have had a knock on effect regarding the "allowed" frequency for any given engine. One thing is certain is that submitting the same page more than once in a day is not going to help at all - and might well hinder a great deal. Past this submitting two days in a row is not that advisable but can be useful to ensure that your previous submission was not dropped due to a particularly busy day at any given engine. There is no real point in repeatedly submitting the same page as it will not be indexed more than once.

You can submit multiple pages from the same domain (or subdomain) but you should make sure that they are different enough (file size, body text, keyword ratio/weight) so as not to look like spam, usually submitting a lot of pages from within your site is an effective way of achieving multiple listings - even more so if each pages title and description matches the individual subject of the page.

It is also advisable to submit such pages on different days also to ensure a ranking for each - and to secure the domain in use. If you wish you can resubmit your page weekly, but with many engines indexing monthly, this can have little effect. p>There are many stories of entirely innocent domains being dropped entirely for no apparent reason - it appears that the anti-spam algorithms are so stringent that those who are slightly over-zealous submitters - or who have many pages or subdomains - have experienced the consequences with no comeback. The best advise is to PLAY SAFE - if you are ranking well already then there is no need to resubmit many times, usually the best method is to alter the page in order to resubmit, or create other pages to submit. But as the engines change their algorithms (and definitions of illegal actions) the frequency and quantites which are allowed will change.

There's a heated debate among professionals regarding whether manual or automated submission ensures the best results. There are many other automated submission tools, some free and some not. Though they save time, the results they deliver can vary widely.

Many claim they will submit to hundreds or even thousands of "search engines." These sites are often simply just "free-for-all" link pages that exist primarily to capture the email addresses of people using them, which in turn are sold to bulk mailing firms.

It is true to say that over 90% of all of your traffic will come from the top ten or fifteen major search engines which you submit to. Submission programs and services are great for submitting to the hundreds of smaller "search engines" on the Web, but it's debatable whether submitting to these sites does anything other than add your email address to spam directories, and if not quite how much quality traffic will click through to your site.

Useful links:
Search Engine Home
FAQs
Search Engine Stats