[mythtv] Re: New idea for storing recordings to disks

[Robert Tsai]

Is your point that Myth should just not bother recording at all if it
thinks it might run out of space?

Otherwise, if we discuss something along the lines of "Myth should
record as scheduled, and simply gracefully abort recording if it runs
out of space mid-recording", then read on. (Whether this means
deleting the aborting recording, or just saving a partial recording is
a detail outside the scope of my discussion.)

On Wed, Jun 29, 2005 at 10:58:21AM -0400, Joseph A. Caputo wrote:
> On Wednesday 29 June 2005 10:33, Robert Tsai wrote:
> > On Wed, Jun 29, 2005 at 09:50:29AM -0400, Joseph A. Caputo wrote:
> > > > The point is that for MOST recording methods, Myth can't known
> > > > in advance whether a recording will fit in a given location.
> > > > Only PVR-x50 recordings seem to have predictable file size -
> > > > DVB, ATSC, RTJPEG etc do not. 

Not necessarily predictable, but there is an upper bound. HD is
20Mbps, SD is also some known maximal bitrate.

> > > But that's the case currently, anyway.  As it is now, we only
> > > have a setting that controls the minimum amount of free space
> > > that must be available before Myth will start a new recording.
> > > There's no guarantee that there is actually enough space to
> > > *finish* the recording... you just have to set the number high
> > > enough and hope for the best.
> > 
> > Doesn't auto-expire run every <n> minutes? It seems that if free
> > space dips below this threshold during the making of a recording,
> > then auto-expire should kick in to free up some more space.
> > 
> > If your bitrate is wildly variable, then it seems like the
> > solution is to simply decrease <n> (e.g., have auto-expire make
> > more frequent checks).
> > 
> > It also means that Myth doesn't need to know the eventual size of
> > the recording, since it's checking for free space every so often
> > anyway.
> 
> All well and good, but there are still special cases and
> considerations:
> 
> (1) if your auto-expire space threshold <t> is not at least as large
> as the max number of bytes a recording could consume in <n>, you
> could still run out of space (not too likely, but possible)

Yes; my implied response is that the polling interval <n> should be
decreased to some reasonably small interval. Even 4 HD tuners x 20Mbps
is only 600MB/minute, so the space threshold <t> could be as small as
1GB with <n> at 60 seconds.

Polling the amount of free space once per minute while recording (and
transcoding, and userjobs, and other myth disk-consuming jobs ...)
shouldn't be too bad.

There is a known upper bound on the rate at which disk space will be
consumed by new recordings. It is probably reasonable to assume that
transcoding and userjobs will not consume more space than the content
they are operating on, if you want to factor those in as well.

> (2) if auto-expire runs, but there's nothing that can be expired,
> you're hosed

You're hosed anyway. If nothing is auto-expirable and you're out of
space, you're out of space, no matter how fancy your storage
configuration.

> Bottom line is still, to *guarantee* you won't run out of space
> mid-recording, you must set the new recording space threshold high
> enough, figuring in factors like multiple tuners and transcoding
> jobs that may also consume space.

If you have insufficient auto-expirable content, then you will some
day run out of space mid-recording.

If there is sufficient auto-expirable content to make room for the new
recording(s), than running auto-expire checks sufficiently frequently
for a given space threshold will guarantee that your recordings can
finish successfully.

My points are:

	- You don't have to set an arbitrarily high free-space
	  threshold and "hope for the best" to avoid failed recordings
	  due to full disk space.

	- You can alleviate a huge space threshold by running
	  auto-expire more frequently, and that it is possible to
	  compute optimal values for space threshold and auto-expiry
	  polling, with knowledge of the content being recorded (HD
	  vs. SD, number of tuners, etc.).

	- Myth does *not* need to pro-actively know the eventual size
	  of its recordings. A re-active auto-expire thread running
	  sufficient frequently should take care of that.

My assumption is that auto-expire does run periodically while
recordings are recording, and is not only invoked once before each
recording. I could be wrong.

Anyway, I suspect we are simply in violent agreement. I don't have
anything more to contribute.

--Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mythtv.org/pipermail/mythtv-dev/attachments/20050629/083eeaa4/attachment.pgp