Allow indexing by Yahoo! (there's no way they can afford AI right??)
This commit is contained in:
parent
2a41696244
commit
0b0d88dc45
|
@ -7,5 +7,14 @@ Disallow: /login
|
|||
Disallow: /logout
|
||||
Disallow: /human/
|
||||
|
||||
User-agent: Yahoo! Slurp
|
||||
Disallow: /cursors/
|
||||
Disallow: /images/
|
||||
Disallow: /fonts/
|
||||
Disallow: /moderator/
|
||||
Disallow: /login
|
||||
Disallow: /logout
|
||||
Disallow: /human/
|
||||
|
||||
User-agent: *
|
||||
Disallow: /
|
||||
|
|
Loading…
Reference in a new issue