Okay that's about everything
This commit is contained in:
parent
023ad47ae4
commit
d094ff036f
19 changed files with 236 additions and 79 deletions
5
priv/static/robots-9e2c81b0855bbff2baa8371bc4a78186.txt
Normal file
5
priv/static/robots-9e2c81b0855bbff2baa8371bc4a78186.txt
Normal file
|
|
@ -0,0 +1,5 @@
|
|||
# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
|
||||
#
|
||||
# To ban all spiders from the entire site uncomment the next two lines:
|
||||
# User-agent: *
|
||||
# Disallow: /
|
||||
Loading…
Add table
Add a link
Reference in a new issue