MythNetTV release 1


I figured it was time for another release of MythNetTV, especially since I’ve fixed some bugs recently and rolled in code generously provided by George Nassas. New things in this release:

  • Initial work on transcoding smaller files to something else
  • Better error checking for MySQL configuration and accessibility
  • Fixed bug where MythNetTV was unhappy if there were no channels configured in the MySQL database
  • Found, and hopefully fixed, a bug where program dates were not always being tracked correctly. Also added a check to make sure this doesn’t happen again
  • Added –markread, which lets you mark old shows as already downloaded

The code has seen a fair bit of use now, so it should be fairly stable. Let me know if you find any problems…

    $ ./mythnettv --statistics
    Programs tracked: 735
      Google EngEdu: 350
      Scoble Show: 85 2007: 74
      Mobuzz TV: 82 32
      Net Cafe: 2
      Foo: 1
      ABC News: 2
      Australia: 1
      Play School: 1
      Diggnation: 72
    Programs still to download: 50
      Google EngEdu: 49
      Diggnation: 1
    Data transferred: 103 gb (111017264341 bytes)

Get it here if you care:


…and then there was a pause…


I guess we’re at a different phase of the Zap2IT guide data thing now. The story has broken, there’s been tons of email on the mythtv-users mailing list (remember, don’t panic!), and now things have quietened down. I think that’s probably a good thing, because it almost certainly means people are now no longer distracted from actually sitting down and working on the problem.

I imagine that journalists would make up some sensational thing to say to keep the story rolling during this quiet news cycle. I can’t think of any such thing to say, and it wouldn’t be ethical anyway.

I must say that I think
coverage like
isn’t always helpful.


Some quick further thoughts on the Zap2IT shutdown


The Zap2IT forums has a thread about the shutdown (you need to have a Zap2IT login to get to the forum). One of the posts on the thread about the shutdown expands on the reasons for the free guide data shutting down:

– Continued use of the service to support commercial products, in violation of the agreement.
– Commercial products continued to grow despite previous appeals that this activity stop.
– There are significant changes to the supporting data structure forthcoming and we could not devote resources to the continued upkeep and enhancements of the service.
– Maintenance of the service is impacting our resource pool for other projects.
– We sought alternative options but were unable to find a solution.
– We recognize the hardship this creates for the user community. We are open to alternative solutions and would consider proposals that met the needs of the user community and our company.
– We looked into options to turn this into a paid service however we do not have the infrastructure at this time.

So it seems that the folks selling MythTV based solutions which used the guide data, as well as possible other commercial uses (the was no real way to check what system was connecting to the guide data) are the main reasons for the service shutdown.

That seems reasonable to me… It’s inconvenient for MythTV users in the US, but there are other guide data sources. It is certainly disappointing that people we misusing the free service and caused it’s shutdown for everyone.


An initial analysis of the removal of Zap2IT guide data in the US


(I had some spare time while waiting for the kids to finish their breakfast, so this is posted earlier than I planned).

As mentioned briefly earlier today is reporting that Tribune Media Services (TMS) is turning off their free guide data service for MythTV on 1 September 2007. This is mainly because of claims of abuse — the theory on the MythTV users mailing list is that this abuse is vendors selling pre-configured MythTV systems for profit, which use the not-for-commercial-use guide data service from Zap2IT.

The announcement from TMS reads:


For several years we have offered a free TV listings service to hobbyists for their own personal, noncommercial use. In October of 2004 we posted here an open letter saying the future of Zap2it Labs was at risk because of certain growing misuses of the Zap2it Labs data. Unfortunately this misuse has continued and grown. These misuses, combined with other business factors have led to the decision to discontinue Zap2it Labs effective September 1, 2007.

We thank those users who have honoured the terms of the agreement, and we suggest you consider the many TV listings options offered by the commercial licensees of TMS TV listings data.

If you would like to discuss how to license TV listings for commercial use, please let us know by e-mail ( and include your company name, telephone number and the best time to reach you. We will respond to your inquiry as soon as possible.

We anticipate this decision will generate discussion and invite you to respond via the Zap2it Labs forum. We will do our best to respond to relevant questions posted on this forum in a timely manner.

There has been much consternation on the MythTV users mailing list about this announcement. There haven’t been a lot of helpful suggestions yet though. There is the assurance that the development team is looking into options, and some users are suggesting rounding up enough people to pay for a commercial license. The reality however is that this is likely not a big deal. I can think of a few possible outcomes:

  • Doom! The guide data stops, we all go home. This seems particularly unlikely though. It would take many people ignoring the problem, and just accepting that their MythTV systems are going to stop working.
  • Some sort of scraper using XMLTV is written. This is in fact how guide data is obtained for most other countries at the moment. This technique is discussed in my MythTV book (Amazon), and is well understood. I would expect new XMLTV grabbers for the US to start appearing in the next few days.
  • A conglomerate is formed to buy a commercial license to the guide data, and then sells that in XMLTV form to US based MythTV users. This option seems less likely to me, although not impossible. It would take someone being willing to buy the license for the data, on the assumption that enough people would actually pay for a XML feed. Lots of people have said they would, but there is already arguments over how much they would b willing to pay and it’s yet to be seen if those people will actually front up with the cash. It should also be noted that this sort of arrangement has been tried in other countries, and seems to invariably fail.
  • Another company steps up to provide guide data for free. There are lots of companies which have this data for their own uses now. Examples include the various TV guide web sites out there. They might chose to provide the data as a way of generating good will, as well as because providing the XML will cause less load on their systems than being automatically scraped to produce a third party XMLTV feed (the second option). This was the reason that the Zap2IT feed started being provided in the first place after all.

So, no doom yet. I’ll post more analysis as it appears.


Breaking: Zap2IT turning off guide data for the US?

Share is reporting that Tribune Media Services (TMS), the providers of the Zap2IT guide data used by MythTV in the US, will turn that guide data off on 1 September 2007. I haven’t had a lot of time to look into this further, mainly because I am giving a talk this morning. I’ll have a better look at lunch time (PDT) and write some more about this.

I think there’s no need for panic, as a new guide data source will be found, but this is unexpected.


Implementing parental filters with MythTV


The other morning I walked out to my five year old watching Robot Chicken. This isn’t what I would call the highlight of my parenting career, but it did leave me with an immediate need to come up with a solution to stopping the kids from watching inappropriate things now that they’re big enough to work their way through the MythTV user interface without our assistance. Its now quite common for me to wake up in the morning and discover that the kids have turned the monitor on in the living room, have found one of their shows, and are happily watching.

(I guess it says something about the ease of use of the MythTV user interface that my four and five year olds have no trouble navigating it without assistance. Specifically, the video thumbnail of the various recordings is a killer feature because it helps kids who can’t read yet find the shows they want.)

The animated thumbnail of the recording appears in the bottom right of this example image.

Back to my need to filter recordings. Unfortunately this isn’t covered in the MythTV book (Amazon), mainly because it didn’t seem to me to be a killer feature at the time of laying out the chapters, and because we wanted to focus on getting the important aspects of Myth working well. I’m going to make up for that by writing up everything I’ve learnt about it in the last couple of days here.

You’ll find the filtering setup, which can be used for both parental filtering and filtering based on your random other needs (like perhaps only wanting to display sports programs at the moment, or only documentaries when you’re trying to impress visitors), in Utilities / Setup -> Setup -> TV Settings -> Playback -> View Recordings (Recording Groups), which is the fourth screen of the playback settings wizard. That location might be slightly different depending on what theme you are using. The screen looks like this:

Enabling a filter

You can see here my super secret password, which is used to escape the currently active filter, which I will show you how to do in a moment. You can also see that I’ve selected a filter called “Children” as the default filter to use when Media Library -> Watch Recordings is selected. The relevant options on this screen include whether to prompt for a filter when entering the Media Library -> Watch Recordings screen, which I disabled as I want the default to be the Children filter to stop my kids from selecting another filter. You can also have the filter default to whatever it was last time, which doesn’t work for me because that would mean it was almost always set to showing all programs, but might be convenient to others. You can also choose to show the name of the filter instead of the words “All Programs” in the Media Library -> Watch Recordings view — this is useful for reminding you what filter you are currently using, although you should turn this option off if you don’t want people to know that you are using a filter at all.

The list of filters you can apply is based off the categories of the recordings currently in your MythTV database. These are stored in the category column of the recorded table, and for my back end the list is currently like this:

    mysql> select distinct(category) from recorded;
    | category                 |
    | banana                   |
    |                          |
    | Children                 |
    | Sitcom                   |
    | Holiday-children special |
    | Cooking                  |
    | Crime drama              |
    | Reality                  |
    | Documentary              |
    | House/garden             |
    | Talk                     |
    | Special                  |
    | Comedy                   |
    | Children-special         |
    | Animated                 |
    | Science fiction          |
    | Auto                     |
    | Docudrama                |
    | Travel                   |
    | Science                  |
    | Drama                    |
    | Home improvement         |
    | Fantasy                  |
    | Suspense                 |
    24 rows in set (0.00 sec)

The “banana” category is me testing to see if the statement above is really true. It must be, because after I changed a recording to have a category of banana, this was then presented as an option in the user interface:

Bananas make for good filters

The currently active filter is stored in the settings table of the MySQL database. Look for entries named “DisplayRecGroup” if you are inclined to poke around in your database.

This level of filtering is sufficient if you only want to show one category at a time in the Media Library -> Watch Recordings screen. However, there is no facility to filter based on the rating of the program, and no way to combine filters to display more than one category at once. For example, there is no way to display Children’s shows, and Children’s holiday specials at the same time. This is a bit of a shame. Conceivably you could implement custom filtering like this by changing the values for the category column in the recording table, but I wouldn’t recommend this approach.

Anyway, on to how to use the filter now that you have it set up. Go to the Media Library -> Watch Recordings screen with the filter set, and you’ll see that the view has changed to only include those things in your default category. You’ll be able to tell that the filter is on if you selected the option to display the category name instead of “All Programs”. My recordings screen looks like this when filtered to only include children’s recordings:

Filtering to only show Children’s recordings

As opposed to what it used to look like:

No filtering at all

To change the filter, press the menu key on your remote, or the M key on the keyboard. You’ll see this menu:

The recordings menu

Select “Change Group Filter” from the menu. If you configured a password to change between filters, then the user interface to enter that password is displayed:

Entering a password

Enter your super secret password, and then you’ll be prompted with a list of possible filters. If you didn’t enable a password, you will of course end up going directly to this menu:

Selecting a new filter

And you’ve escaped the tyranny of the filter!

It would be nice for the filtering to be more granular, and there have been some proposals in that direction, although that proposal isn’t a perfect solution either.


How to transcode video for the Nokia N800


I have a Nokia N800 that I haven’t had much time to play with, but I am about to go away on a trip which will involve a lot of time commuting on buses, so I figure I should start thinking about it again. Luckily for me, this page has pretty good instructions of what settings to use with mencoder to make it all look good on the N800. I haven’t tried them yet, but I will report back when I do.


Building a symlink tree for MythTV recordings


I wanted to build a directory of symlinks that pointed to my MythTV recordings, so I wrote a little python script to do it for me. I figure someone else might find this useful too…

    # Copyright (C) Michael Still ( 2007
    # Released under the terms of the GNU GPL
    import MySQLdb
    import os
    import re
    from socket import gethostname
    # Connect to the MythTV database based on the MythTV config
    config_values = {}
    home = os.environ.get('HOME')
    config = open(home + '/.mythtv/mysql.txt')
    for line in config.readlines():
      if not line.startswith('#') and len(line) > 5:
        (key, value) = line.rstrip('\n').split('=')
        config_values[key] = value
    db_connection = MySQLdb.connect(host = config_values['DBHostName'],
                                    user = config_values['DBUserName'],
                                    passwd = config_values['DBPassword'],
                                    db = config_values['DBName'])
    cursor = db_connection.cursor(MySQLdb.cursors.DictCursor)
    # Regexp for what is allowed in the symlink name
    unsafe = re.compile('[^a-zA-Z0-9\-\:_]+')
    # Find the recordings directory -- this assumes you haven't used an
    # identifier string for this machine...
    cursor.execute('select * from settings where value="RecordFilePrefix" and '
                   'hostname="%s";' % gethostname())
    row = cursor.fetchone()
    basedir = row['data']
    # Now find all the recordings we have at the moment
    cursor.execute('select title, subtitle, starttime, basename from recorded;')
    for i in range(cursor.rowcount):
      row = cursor.fetchone()
      title = row['title']
      subtitle = row['subtitle']
      if subtitle == '':
        subtitle = str(row['starttime'])
      title = title.replace(' ', '_')
      title = unsafe.sub('', title)
      subtitle = subtitle.replace(' ', '_')
      subtitle = unsafe.sub('', subtitle)
      extn = row['basename'].split('.')[1]
        os.symlink('%s/%s' %(basedir, row['basename']),
                   '%s/%s.%s' %(title, subtitle, extn))

This creates a tree of symlinks in the current directory that looks like this:

    $ find . -type l