[mythtv-commits] Ticket #11958: PATCH: Robots.txt for mythweb

[MythTV]

#11958: PATCH: Robots.txt for mythweb
---------------------------------------+------------------------
     Reporter:  skd5aner <skd5aner@…>  |      Owner:  kormoc
         Type:  Patch - Feature        |     Status:  new
     Priority:  minor                  |  Milestone:  unknown
    Component:  Plugin - MythWeb       |    Version:  0.27-fixes
     Severity:  medium                 |   Keywords:
Ticket locked:  0                      |
---------------------------------------+------------------------
 Over the years, several people accidentally leave mythweb exposed to the
 internet.  Even though strongly discouraged for a multitude of reasons,
 one of the biggest issues that occurs in this situation is that various
 search engines can cause havoc by crawling the site and hitting links that
 modify the system.  There are dozens of reports of bots like googlebot
 causing modifications to all recording rules, etc.

 At the very least, including a robots.txt file will keep the majority of
 valid bots from causing havoc when crawling the site.  It will not prevent
 bots that ignore robots.txt from causing issues.  This is not a magic
 bullet, but it's certainly a simple way to add a small layer of protection
 that could otherwise cause someone to have a real bad day.

 The attached patch simply creates a robots.txt file at the root of the
 mythweb directory, and disallows all user-agents from crawling the site.
 This patch can apply to master and also to 0.27 and 0.26.

--
Ticket URL: <http://code.mythtv.org/trac/ticket/11958>
MythTV <http://www.mythtv.org>
MythTV Media Center