What Robots.Txt Is & Why It Matters for SEO

A robots.txt file tells search engine bots which pages they should and shouldn’t crawl. Learn more.