[mythtv-users] mythtv cli channel maintenance and python web services binding
Alistair Grant
akgrant0710 at gmail.com
Sat Feb 7 22:10:27 UTC 2015
Hi Everyone,
Before I launch in to the progress I've made I just want to clarify one of the
drivers for developing this...
Based on what I've seen (although hardly scientific) it appears that when
people get stuck maintaining Channel callsign, name, xmltvid and icon urls they
often fall back to using SQL. The MythTV developers, for good reason,
discourage the use of SQL as much as possible. These utilities will allow that
maintenance to be performed without resorting to SQL. It should be safer as
the changes that can be performed are limited, it will take advantage of any
input validation performed by the backend web services, and can perform
additional input validation (although admittedly there isn't much at the
moment).
On to the changes:
* Added a number of automated tests
* This is enough to give me confidence that I haven't broken anything.
* Added packaging
* Installation instructions for Ubuntu derivatives are provided at
https://github.com/akgrant43/mythtv_cli_extensions It should be
straightforward to translate to Fedora, etc.
* Added the Profile class. This returns a large amount of information about
the backend. You can see the information with:
mythtv_cli dump Myth ProfileText
The version number is now 0.2.0.
It will only install on python 3.4.0 and later versions. It should work on
3.2.5+ (the minimum version supported by suds-jurko), but I'm not able to test
it. If someone wants to modify setup.py and test it on an earlier version,
please let me know.
Thanks,
Alistair
On Fri, Feb 6, 2015 at 10:02 AM, Alistair Grant <akgrant0710 at gmail.com> wrote:
> Hi Everyone,
>
> There's been regular discussion about restoring XMLTVIDs and Icon URLs after
> performing a channel re-scan.
>
> I've been meaning to learn a bit more about the MythTV web services, and have
> also just set up MythTV in the Czech Republic using tv_grab_huro to get the
> program guide, and performed a few channel re-scans as part of getting it
> set up correctly.
>
> So, combining the two, I've created a couple of utilities that will assist
> with maintaining XMLTVIDs and Icon URLs, and may also eventually be a useful
> python binding for the MythTV web services.
>
> The code is at: https://github.com/akgrant43/mythtv_cli_extensions
>
> If there is enough interest in it, I'll put more effort in to expanding its
> features and improving error checking, automated tests, etc. (If
> there isn't any interest, I'll still use it, but it won't really ever be
> production quality).
>
> Note that this is version 0.0.0. This has been a learning experience for me
> with suds-jurko and mythtv web services, so I'm sure there's plenty of room
> for improvement. It also doesn't have any automated tests yet,
> hasn't been extensively tested, is missing lots of functionality, comments,
> etc. I'm just hoping to get some feedback on whether it may be useful for
> anyone else.
>
> It currently doesn't have any installation capability, so if you're not
> comfortable playing with python scripts, this probably isn't for you (yet).
>
> To install and use it, you'll need a recent Python3 interpreter (I'm using
> 3.4.2), and I'd suggest setting up a virtual environment, e.g. (assuming a
> bash shell or similar):
>
> $ pyenv-3.x --system-site-packages mythenv # This will depend on your
> particular environment
> $ cd mythenv
> $ source bin/activate
> $ pip install suds-jurko
> $ git clone https://github.com/akgrant43/mythtv_cli_extensions
> $ cd mythtv_cli_extensions
> $ bin/mythtv_chanmaint.py --help
>
> Hopefully at that point you're good to go.
>
> See below for more information (the README from github).
>
> Thanks,
> Alistair
>
>
>
> mythtv_cli_extensions
> =====================
>
> MythTV python CLI utilities and classes
>
> First: Many thanks to the developers of the MythTV project for making
> such a great piece of software available. The mythtv_cli_extensions are
> my personal work and any shortcomings, bugs, problems, etc. are mine and
> no reflection on the MythTV developers.
>
> mythtv_cli_extensions provides two utilities and a couple of python
> libraries for use with the MythTV DVR software. To find out more about
> MythTV please see http://mythtv.org.
>
> WARNING: This software is still in the early development stage, there's
> no guarantee of backward compatibility until it reaches V1.0.
>
> Utilities
> ---------
>
> mythtv_cli.py
> Provides a command line utility for:
> calling the MythTV web services and exploring the database a bit.
> updating the database. Only fields that are considered user maintainable
> can be modifed. Currently only the Channel class can be updated.
>
> mythtv_chanmaint.py
> Provides a utility for maintaining XMLTVIDs and (eventually)
> channel icons in MythTV.
> This is typically used to restore XMLTVIDs after a channel re-scan
> has been performed.
>
> The current functionality is enough to allow a simple shell script to be
> written that will allow the channel XMLTVIDs and Icon URLs to be reset to
> their correct values after a re-scan.
>
> The help text for each is included below.
>
> Python3 Libraries
> -----------------
>
> MythTVServiceAPI
> Provides low level access to the MythTV web services using the
> suds-jurko module. See https://bitbucket.org/jurko/suds for information
> about suds-jurko.
>
> MythTVQuerySet
> Provides a django like interface to query the backend.
>
> MythTVClass
> Provides a python object representation of the back end web service
> objects, taking care of naming inconsistencies, etc. Currently only
> Channel is supported. Look at the version number if you're wondering.
>
>
> An example of retrieving all channels with call sign "ABC" and updating
> the first record:
>
> from mythtvlib.object import MythTVQuerySet
>
> # Get a QuerySet on the Channel class
> query_set = MythTVQuerySet("MythTVChannel")
> # Filter records with CallSign equal to "ABC"
> query_set = query_set.filter(CallSign="^ABC$") # This is a
> regular expression, and we don't want callsigns containing "ABC"
> matching_records = query_set.all()
> print(matching_records)
> # Update the first record's name to be "DEF"
> query_set[0].ChannelName = "DEF"
> query_set[0].save()
>
> Dependencies
> ------------
>
> mythtv_cli_extensions depend on the suds-jurko module
> (https://bitbucket.org/jurko/suds). The simplest way to install
> suds-jurko is:
>
> [sudo] pip install suds-jurko
>
> ToDo:
> -----
>
> LOTS!
>
> - Add automated testing
> - Save and restore icon definitions
> - This can be done using mythtv_cli.py update, but must be manually
> maintained
> - Extend the library to handle all the classes defined by the web
> services
> - Extend filter() to do proper numeric comparisons
>
> mythtv_cli.py help
> ==================
>
> usage: mythtv_cli.py [-h] [--post] [--hostname HOSTNAME]
> [--server-port PORT]
> [-y] [--version]
> {dump,update} params [params ...]
>
> MythTV Web Services CLI
>
> positional arguments:
> {dump,update} Maintenance command, see below
> params Command parameter(s)
>
> optional arguments:
> -h, --help show this help message and exit
> --post Show POST operations with operation help
> --hostname HOSTNAME MythTV Backend hostname
> --server-port PORT MythTV Backend services port
> -y Execute updates without user confirmation
> --version show program's version number and exit
>
> mythtv_cli.py has 2 basic use cases:
>
> mythtv_cli.py dump <service> <operation> <key...>
> Print the results of the requested service/operation
> mythtv_cli.py update <class name> <filter field>
> <filter regex> <update field> <update value>
> Update the records matching the supplied regular expression in the
> requested class.
>
> Valid Services: Capture, Channel, Content, DVR, Frontend, Guide, Myth, Video
>
> Valid Class Names: Channel
>
> Additional Help:
>
> mythtv_cli.py dump <service> help # for help on
> individual services.
> mythtv_cli.py dump <service> <operation> help # for
> detailed parameter information
>
>
> MythTV Web Services Documentation: https://www.mythtv.org/wiki/Services_API
>
> mythtv_chanmaint.py help
> ========================
>
> usage: mythtv_chanmaint.py [-h] [--xmltv XMLTV] [--hostname HOSTNAME]
> [--server-port PORT] [--config CONFIG]
> [--create-config] [-y] [--version]
> {list,update_xmltvids} [params]
>
> MythTV Channel Maintenance
>
> positional arguments:
> {list,update_xmltvids}
> Maintenance command, see below
> params Command parameter
>
> optional arguments:
> -h, --help show this help message and exit
> --xmltv XMLTV XMLTV data file
> --hostname HOSTNAME MythTV Backend hostname (localhost)
> --server-port PORT MythTV Backend services port (6544)
> --config CONFIG Configuration data
> --create-config Create a new configuration data file
> -y Execute updates without user confirmation
> --version show program's version number and exit
>
> mythtv_chanmaint.py has 3 basic use cases:
>
> mythtv_chanmaint.py list xmltv --xmltv file.name
> List the channel and XMLTVID data contained in the xmltv file
> mythtv_chanmaint.py list channels
> List the channel data contained in the MythTV backend
> mythtv_chanmaint.py update_xmltvids --xmltv file.name [-y]
> Update Channel XMLTVIDs, see below
>
> Updating Channel XMLTVIDs
> -------------------------
>
> This option can be used to re-populate and correct XMLTVIDs in the Backend
> database.
>
> It reads the XMLTVIDs from the supplied file (--xmltv file.name), creates a
> mapping from all CallSign variations as defined in the users settings file
> (mythtv_chanmaint_settings.py) and reads and checks each channel in the
> backend. If -y is not supplied the proposed updates are listed and the user
> is asked for confirmation prior to updating the backend.
>
> Typical XMLTVID Workflow
> ------------------------
>
> The typical workflow is to recognise that you aren't getting EPG
> data for all
> your channels, or that you need to re-scan for whatever reason.
>
> 1. Get a copy of the XMLTV EPG data. Hopefully you know how this is done
> already. The program data isn't needed, just the channel data.
> For my system:
>
> $ tv_grab_huro --days 1 --offset 0 --output xmltv.xml --config
> ~/.mythtv/tv_grab.xmltv
>
> 2. List the xmltv callsigns:
>
> $ mythtv_chanmaint.py list xmltv --xmltv xmltv.xml
>
> 3. List the MythTV callsigns:
>
> $ mythtv_chanmaint.py list channels --host backend.host.name
>
> 4. Update the mapping table in mythtv_chanmaint_settings.py
> (XMLTV_CALLSIGNS).
>
> Note that the callsign as it exists in the xmltv file is the key
> (left of the colon), and the MythTV callsign is the value (right of the
> colon).
>
> See mythtv_chanmaint_example.py for an example configuration with some
> comments.
>
> 5. Run the update:
>
> $ mythtv_chanmaint.py update_xmltvids --host backend.host.name
>
> Check the proposed updates and confirm if you're happy. If not, go back
> to step 4 and check the mapping table.
>
> 6. Run mythfilldatabase
>
> You should then be able to see the correct / additional EPG data in the
> MythTV program guide.
More information about the mythtv-users
mailing list