<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 28/11/13 12:47, Paul Clark wrote:<br>
</div>
<blockquote
cite="mid:CAP5ACz1qn9wHhrhA=0yUyEFCVxQp4eDhtzsTU87wU+vaH2EHtw@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div>Just an idea (as someone who had this happen to them) but
shouldn't a dev add a directive to mythweb to tell the bots
not to crawl it? Not sure how these things work but there's a
page <a moz-do-not-send="true"
href="http://en.wikipedia.org/wiki/Robots_exclusion_standard">here</a>
that seems to explain it. </div>
</div>
</blockquote>
You want to stop everybody using surely?<br>
I can go a delete those pages just as well as a bot can.<br>
<br>
<pre class="moz-signature" cols="72">--
'ooroo
Stinga...(:)-)
---------------------------------------------------
Email: <a class="moz-txt-link-abbreviated" href="mailto:stinga+mythtv@wolf-rock.com">stinga+mythtv@wolf-rock.com</a> o
You need only two tools. o /////
A hammer and duct tape. If it /@ `\ /) ~
doesn't move and it should use > (O) X< ~ Fish!!
the hammer. If it moves and `\___/' \) ~
shouldn't, use the tape. \\\
---------------------------------------------------</pre>
</body>
</html>