Home / Intermediate

Blog

Teal office shot sam at his computer

What is Robots.txt? How is it Used for Search Engines?

27th February 2017

"Make use of the robots.txt file on your web server. This file tells crawlers which…