[mythtv-users] Supporting schedulesdirect.org

[Daniel Kristjansson]

On Wed, 2007-08-08 at 11:40 -0700, Robert Current wrote:
> On 8/8/07, Jake <jakep_82 at hotmail.com> wrote:

> > They don't use the word
> > fraud, but it's not exactly hard to figure out what they're talking about.
> 
> yes, actually, it is hard to figure out.  They couldn't do it anymore
> because of "misuse" and "risk."  But that all goes away for a fee?
> But somehow, they don't want charge a fee?

TMS is a B2B operation, they do not have the infrastructure in place
to deal directly with consumers. I'm sure they evaluated the costs
of doing this and decided that it would cost more for them to create
the infrastructure than it would benefit them. There may also have
been the risk of alienating their many existing customers who may
be paying $20,000 to $40,000 a month for access to the data they
redistribute to their customers.

> I'm sorry, once again there seems to be something that someone isn't
> saying.  I honestly, truely, don't know what your talking about.  How
> can it be a risk and misuse in the past, but not now?  Can someone
> define what the risk was?  What the misuse was?

There are many things people aren't saying. Rest assured however that
Schedules Direct wants to tell you all kinds of things, but is under
an NDA that prevents us revealing a great deal of information that
we do know. Now that a contract is in place and TMS has made an
announcement we can tell you a lot more than we could yesterday.
We are still finalizing some of what we will be telling everyone
soon. First there will be an official announcement, then as soon
as it is ready we will post the Subscriber Agreement, then as soon
as it is ready we will post the Privacy Policy, then we will start
accepting members. We're talking days not weeks.

> I THOUGHT I figured it out, being the over zealous use of pulling
> data..  The hardware load on their system...  but, it didn't say that,
> it was just my wild guess.  If that was it, how will
> schedulesdirect.org prevent it?

In many ways. We've discussed a few internally and implemented some
of them. Honestly, on this front it is just a matter of limited time.
You may not be aware of this, but computer programming is a very labor
intensive process and it took a while to even negotiate programmer
access to the listings data. The Robert Eden and Chris Petersen have
been hard at work since 5 minutes after they got access to the data
and haven't had any time to write documentation.

> if that wasn't it, then I don't know what it was..  so I'm asking.
> 
> > Now that money is involved, I don't see fraud or violation of TOS being a
> > major problem.
> 
> I don't see how money solves the problem.  Because, to me, it's
> totally unclear what the problem was in the first place.
>
> Yes, loosing Zap2It is a problem, in the end.  But the problem that
> CAUSED them to stop is still a bit foggy to me.

Of course, violation of the Subscriber Agreement will be a problem
for Schedules Direct just like it was for TMS. I'm sure TMS thought
of some ways to combat abuse that they just could not financially
justify spending their limited and valuable programmer time on; we
came up with a half dozen ways to combat fraud in a brainstorming
session before ever talking to TMS about licensing the data.
Schedules Direct current ratio of programmers to non-programmers
is about 10:1. While this will change when we open to the public,
we may already have more programmer hours available to us than the
entire TMS subsidiary of Tribune.

BTW Schedules Direct is incorporated as a non-profit in the State of
Washington. Anyone can go to the state website and verify this. No
one working on Schedules Direct has been given any renumeration or
even reimbursement for filing and other fees they have paid out of
pocket. I can't promise that this will always be the case, but there
are laws that prevent board members of non-profits from unduly 
enriching themselves. None of us founded Schedules Direct as a get
rich quick scheme, we could have easily incorporated as a for-profit.
If in a few years Schedules Direct has hundreds of thousands of users,
I'm sure the Board of Directors will have to hire staff to take care
of day to day business. This is unlikely at the moment, we would be
happy to just have a viable solution in place for the MythTV and other
OSS and "freeware" users that currently use Labs, and hope to have
enough members and donations to make using MythTV to it's full
potential affordable for anyone with a computer.

Putting my MythTV hat back on, we have no intention of making MythTV
unfriendly to other means of getting listings data. I personally
wrote much of the EIT data collection code, even though I normally use
Labs for my listings data. This was done in part to prepare for labs
eventually going away, which I saw as a possibility long ago. I've
committed code for many other means of loading listings data into
MythTV, as have others. Many of the people with commit access to the
MythTV code base and other regular contributors are not part of
Schedules Direct and would surely mutiny if Schedules Direct Board
Members tried to be heavy handed in this area. Some of the MythTV
developers simply could not find any all encompassing source for
listings data and decided to seek out a service provider, and failing
that create one ourselves. None of the alternatives very easy to set up,
most are restricted to owners of certain hardware which has licensed
the data from TMS or GemStar via some intermediary. None have Canadian
listings so far as I can tell. The screen scrapers are all very limited
because whenever a bunch of people start hitting a site, the site
changes it's TOS to prevent scraping, so the scraper immediately
looses it's experienced developers and then small changes in the web
site break the scrapers for long periods of time. It would be completely
infeasible for a project of MythTV's size to use scrapers, both
legally and practically. Also in today's web 2.0 world it is much
easier to break scrapers than it used to be. But there still remain
other ways of getting listings data, you can manually input it if you
only watch a few channels, you can use EIT which works for many DVB-S
and ATSC services; In some cases EIT even carries data for analog
stations and other affiliates, and you may soon be able to get it
from CTpvr for a fee.

-- Daniel