[mythtv-users] CVS wget gzip problem

Nathan Elliott nelliott at gci.com
Tue May 11 17:34:43 EDT 2004


Yet another reply to myself  =)

I copied the post-file that gets created when setup tries to get the DD data
and re-ran the wget process using this post-file.

The result:  1 plain text file named test6, NOT in gzip format.

This seems to indicate my proxy server may be the culprit.  I'm a little
lost.  It seems like I can:

1)  Disable the proxy.  (I'd rather NOT do this, as it seems to do a good
job of keeping my kids out of the porn sites)

2)  Patch the CVS source to disable GZIP and recompile.  (I'd rather not do
this as it seems like a lame solution)

Ideas?

Nathan Elliott 


-----Original Message-----
From: Nathan Elliott [mailto:nelliott at gci.com]
Sent: Tuesday, May 11, 2004 11:16 AM
To: 'Discussion about mythtv'
Subject: RE: [mythtv-users] CVS wget gzip problem


Yes, DansGuardian is running on my IPCOP firewall...which I believe is a
proxy server?

I ran the same wget command with my own username and password, here is the
output:

---------
[root at Mythtv root]# wget --header='Accept-Encoding: gzip' --http-user='xxx'
--http-passwd='xxx' --post-file='/tmp/mythtpost7SM5W0'
http://datadirect.webservices.zap2it.com/tvlistings/xtvdService
--output-document=test6 --debug

DEBUG output created by Wget 1.9.1 on linux-gnu.

--08:55:44--
http://datadirect.webservices.zap2it.com/tvlistings/xtvdService
           => test6'
Resolving datadirect.webservices.zap2it.com... 206.18.98.160
Caching datadirect.webservices.zap2it.com => 206.18.98.160
Connecting to datadirect.webservices.zap2it.com[206.18.98.160]:80...
connected.
Created socket 4.
Releasing 0x8089848 (new refcount 1).
POST data file missing: /tmp/mythtpost7SM5W0
---request begin---
POST /tvlistings/xtvdService HTTP/1.0
User-Agent: Wget/1.9.1
Host: datadirect.webservices.zap2it.com
Accept: */*
Connection: Keep-Alive
Authorization: Basic bmVsbGlvdHQ6d2hhdDN2M3I=
Content-Type: application/x-www-form-urlencoded
Content-Length: 0
Accept-Encoding: gzip

---request end---
HTTP request sent, awaiting response... HTTP/1.0 401 Unauthorized
Date: Tue, 11 May 2004 16:55:43 GMT
Server: Orion/2.0.1
Content-Type: text/html
WWW-Authenticate: Digest realm="TMSWebServiceRealm",
nonce="PDEwODQyOTQ1NDQxNjEuNWViOGQ5ZGIzZWQzODBlMjAwNjg0ZTNjYTlkMDgyOTBAbWlja
2V5cz4=", opaque="5ef1e7a8a2f14748", algorithm=MD5, qop="auth"
X-Cache: MISS from Warden
Connection: close


Closing fd 4
Found datadirect.webservices.zap2it.com in host_name_addresses_map
(0x8089848)
Connecting to datadirect.webservices.zap2it.com[206.18.98.160]:80...
connected.
Created socket 4.
Releasing 0x8089848 (new refcount 1).
POST data file missing: /tmp/mythtpost7SM5W0
---request begin---
POST /tvlistings/xtvdService HTTP/1.0
User-Agent: Wget/1.9.1
Host: datadirect.webservices.zap2it.com
Accept: */*
Connection: Keep-Alive
Authorization: Digest username="xxx", realm="TMSWebServiceRealm",
nonce="PDEwODQyOTQ1NDQxNjEuNWViOGQ5ZGIzZWQzODBlMjAwNjg0ZTNjYTlkMDgyOTBAbWlja
2V5cz4=", uri="/tvlistings/xtvdService",
response="16f9c7ed0ad20246148335d7b19bcd8f", opaque="5ef1e7a8a2f14748"
Content-Type: application/x-www-form-urlencoded
Content-Length: 0
Accept-Encoding: gzip

---request end---
HTTP request sent, awaiting response... HTTP/1.0 500 Internal Server Error
Date: Tue, 11 May 2004 16:55:43 GMT
Server: Orion/2.0.1
Content-Type: text/html
X-Cache: MISS from Warden
Connection: close


Closing fd 4
08:55:45 ERROR 500: Internal Server Error.

---------
"Warden" is the name of my IPCOP firewall.  =)

I tried the wget command WITHOUT the "--header='Accept-Encoding: gzip'" and
it successfully downloaded an empty file named "test6"

I'll try disabling the DansGuardian on my firewall and see if makes a
difference.

Nathan Elliott 

-----Original Message-----
From: Brad Fritz [mailto:brad-mythtv at fritzfam.com]
Sent: Tuesday, May 11, 2004 6:56 AM
To: mythtv-users at mythtv.org
Subject: Re: [mythtv-users] CVS wget gzip problem


Nathan,

On Tue, May 11, 2004 at 04:23:06AM -0800, Nathan Elliott wrote:

> I am using wget 1.9.1.  I tried both installing wget from RPM, and
compiling
> it from source.    

[..]
> Connecting to datadirect.webservices.zap2it.com[206.18.98.160]:80...
> connected.
> HTTP request sent, awaiting response... 401 Unauthorized
> Connecting to datadirect.webservices.zap2it.com[206.18.98.160]:80...
> connected.
> HTTP request sent, awaiting response... 200 OK
> Length: unspecified [text/xml]
> 
>     [<=>
> ] 0             --.--K/s
> gzip: stdin: not in gzip format
>     [ <=>
> ] 33,580        --.--K/s
> 
> 
> Cannot write to `xtvdService' (Broken pipe).

Is there a proxy server between your mythtv backend and zap2it.com?
If so, it might be silently decoding the gzip data and confusing wget.

That's just speculation based on these two mailing list postings:

 
http://gossamer-threads.com/lists/mythtv/dev/21602?do=post_view_threaded#216
02
  http://lists.debian.org/debian-user/2003/07/msg00751.html

It would be interesting to compare wget debug output with HTTP headers
from your host with those in the mythtv-dev posting.

--Brad

_______________________________________________
mythtv-users mailing list
mythtv-users at mythtv.org
http://mythtv.org/cgi-bin/mailman/listinfo/mythtv-users


More information about the mythtv-users mailing list