File tree Expand file tree Collapse file tree
Discovery/Web-Content/trickest-robots-disallowed-wordlists Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -1523,10 +1523,10 @@ hp/video
15231523hpc/*/supported-applications.aspx?
15241524hpm
15251525hpmob
1526- http://bbc.com/robots.txt
15271526http://cdc.gov/robots.txt
1528- http://instagram.com/robots.txt
1529- http://slideshare.net/robots.txt
1527+ http://support.cloudflare.com/robots.txt
1528+ http://vimeo.com/robots.txt
1529+ http://web.archive.org/robots.txt
15301530hub?category=
15311531hz/contact-us/ajax/initiate-trusted-contact
15321532hz/help/contact/*/message/$
@@ -3206,7 +3206,7 @@ wiki/Wikipedia_Diskussion:Checkuser
32063206wiki/Wikipedia_Diskussion:Spam-Blacklist-Log
32073207wiki/Wikipedia_talk%3AAbuse_reports
32083208wiki/Wikipedia_talk%3AAbuse_response
3209- wiki/Wikipedia_talk%3AAdministrator_electionshttp://policies.google.com /robots.txt
3209+ wiki/Wikipedia_talk%3AAdministrator_electionshttp://goo.gl /robots.txt
32103210wiki/Wikipedia_talk%3AAdministrator_recall
32113211wiki/Wikipedia_talk%3AAdministrator_review
32123212wiki/Wikipedia_talk%3AAdministrators%27_noticeboard
You can’t perform that action at this time.
0 commit comments