-
1). The role of the robots.txt file to improve site ranking! By : Michael Kralj
Not many web master take the time to use a robots.txt file for their website. For search engine spiders that use the robots.txt to see what directories to search through, the robots.txt file can be very helpful in keeping the spiders indexing your actual pages and not other information, such as looking through your stats!
The robots.txt file is useful in keeping your spiders from accessing parts folders and files in your hosting directory that are totally unrelated to your actual web site content.
Article Related to: robots.txt, seo, search engine optimization, higher rankings
|
-
2). Search Engine Robots or Web Crawlers By : Susmita
Most of the common users or visitors use different available search engines to search out the piece of information they required. But how this information is provided by search engines? Where from they have collected these information? Basically most of these search engines maintain their own database of information. These database includes the sites available in the webworld which ultimately maintain the detail web pages information for each available sites.
Article Related to: seo, sem, search engine optimization, dynamic page, robots, robots.txt
|
|
|
