What is the limit of a robot.txt file?

**shoppingswag** · 07-10-2019, 04:31 AM

**sinelogixtech** · 07-10-2019, 04:42 AM

Hi friends,
Yes, there is a limit to robots.txt file in terms of filesize 500 KB as Vipin points out. A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit of 500kb.

**MVMInfotech18** · 07-10-2019, 10:35 AM

Your robots.txt file must be smaller than 500KB.

**BangkokDinning** · 07-11-2019, 10:58 AM

Yes, there is a limit to robots.txt file in terms of filesize 500 KB as Vipin points out.

**RH-Calvin** · 07-12-2019, 06:52 AM

A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit of 500kb.

Thread: What is the limit of a robot.txt file?

Thread Tools

Display