http://forums.thewebhostbiz.com/index.php/custom-research-paper-reviews/ | separation of mixtures lab report | monroe doctrine essay | essay peer pressure what is the importance of robot.txt file?
Results 1 to 2 of 2

Thread: what is the importance of robot.txt file?

  1. #1

    Default what is the importance of robot.txt file?

    what is the importance of robot.txt file?

  2. #2
    Join Date
    Jun 2013
    Location
    Forum
    Posts
    1,320

    Default

    Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
    Cheap VPS Hosting | $1 VPS Hosting
    Windows VPS Hosting | Windows with Remote Desktop Access
    Cheap Dedicated Servers | Linux Dedicated Server Hosting

Similar Threads

  1. Importance of Hosting
    By seojesica in forum Web Hosting and Related Requests Forum
    Replies: 0
    Last Post: 11-24-2018, 02:44 PM
  2. Which is better Meta Robot tags or robots.txt?
    By sinelogixtech in forum Business Tools
    Replies: 4
    Last Post: 09-26-2018, 01:49 AM
  3. What is the limit of a robot.txt file?
    By artijain1900 in forum Business Tools
    Replies: 3
    Last Post: 04-09-2018, 12:33 PM
  4. What is the limit of a robot.txt file?
    By ninadordev in forum Business Tools
    Replies: 4
    Last Post: 02-25-2018, 09:09 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •