Made by Mikal – Page 76

Weird paramiko problem

Post author:mikal
Post published:September 17, 2008
Post category:Python

I had a strange paramiko problem the other day. Sometimes executing a command through a channel (via the exec_command() call) would result in an exit code being returned, but no stdout or stderr. This was for a command I was absolutely sure always returns output, and it wasn't consistent -- I'd run batches of commands and about 10% of them would fail, but not always on the same machine and not always at the same time. I spent ages looking at my code, and the code for the command running at the other end of the channel. Then it occurred to me that this seemed a lot like a race condition. I started looking at the code for the paramiko Channel class, and ended up deciding that the answer was to check that the eof_received member variable was true before trying to close the channel. It turns out this just works. I've my code running commands for a couple of days now and have had zero more instances of the "no output, but did exit" error. So, there you go. Its a shame that member variable doesn't have accessors and isn't documented though. I guess that makes my code a…

Couldn’t determine the video directory?

Post author:mikal
Post published:September 15, 2008
Post category:Mythtv

A couple of people have contacted me in the last couple of days with an error where MythNetTV can't determine the directory to put videos in. The error would look something like this: Importing data/tekzilla--0050--tehbunniez--hd.h264.mp4 Traceback (most recent call last): File "./mythnettv", line 453, in main(sys.argv) File "./mythnettv", line 212, in main mythnettvcore.DownloadAndImport(db, guid, out=out) File "/home/mythbox/Scripts/mythnettv/mythnettvcore.py", line 92, in DownloadAndImport prog.Import(out=out) File "/home/mythbox/Scripts/mythnettv/program.py", line 472, in Import raise FilenameException(self.db, 'Could not determine the video ' program.FilenameException: Could not determine the video directory for this machine. Please report this to mythnettv@stillhq.com The stack trace is mostly irrelevant. The problem here is that MythNetTV couldn't decide what directory to put the video in once downloaded. Please execute the following SQL statements against your MythTV database: select * from storagegroup where groupname="MythNetTV"; select * from storagegroup where groupname="Default"; select * from settings where value="RecordFilePrefix"; This will dump all of the possible places MythNetTV will look for a video directory. Once you've dumped this information, perform some simple checks: Is there anything listed at all? If not, you need to configure storage groups with the MythTV user interface Is there anything listed with the hostname that MythNetTV is running on? If not, you…

Foundation and Empire

Post author:mikal
Post published:September 14, 2008
Post category:Book

This is the second book in the original foundation trilogy, which I am reading as part of the the extended Foundation series that I am working my way slowly though. This book contains two stories -- both of them Seldon crises, although one of them unpredicted by Hari. As Hari had always said in the series -- his techniques can only predict broad social trends, and the not the work on individuals. What happens if a single person who could not be predicted appears? This story covers that scenario. I found this book harder to read than the previous one, but that might have been because I've had a pretty distracted week. Once I actually sat down to read without too many interruptions, I enjoyed it. The comments from others on LibraryThing are fair though -- the character names are odd, and the writing does feel a little awkward. [isbn: 0553293370] [award: winner hugo 1946] (LibraryThing for some reason gets the ISBN mapping for this book wrong. The above link's ISBN is right, but this link goes to the right place).

Foundation

Post author:mikal
Post published:September 11, 2008
Post category:Book

Foundation is an interesting book, as its quite old and was originally written as a series of short stories (as much early science fiction was). Because I am reading the books of the extended Foundation series in the order that Asimov recommended towards the end of his life, I have read the two prequels to Foundation (Prelude to Foundation and Forward the Foundation) before Foundation itself. This means that the time line is a little inconsistent, specifically about how the Foundation project ends up on Terminus (Was it lobbying or exile? Did Hari go or not?). That's not too bad though, and the book is very good. [isbn: 0586010807]

Forward the Foundation

Post author:mikal
Post published:September 9, 2008
Post category:Book

This is a Foundation prequel, coming after Prelude to Foundation and before Foundation. The book is almost a series of short stories or novelettes -- there are several year gaps between these stories. That was a shame in a sense, because each of these separate stories has its won startup cost -- the time it takes me to get into what is happening. For some reason I don't find that as much of a problem with collections of short stories, possibly because I'm expecting it more. This technique meant Asimov could cover a lot of ground, but I found it jarring over all. I guess I'd say this book was ok, but not one of Asimov's best. [isbn: 0553565079]

Executing a command with paramiko

Post author:mikal
Post published:September 4, 2008
Post category:Python

I wanted to provide a simple example of how to execute a command with paramiko as well. This is quite similar to the scp example, but is nicer than executing a command in a shell because there isn't any requirement to do parsing to determine when the command has finished executing. #!/usr/bin/python # A simple command example for Paramiko. # Args: # 1: hostname # 2: username # 3: command to run import getpass import os import paramiko import socket import sys # Socket connection to remote host sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((sys.argv[1], 22)) # Build a SSH transport t = paramiko.Transport(sock) t.start_client() t.auth_password(sys.argv[2], getpass.getpass('Password: ')) # Start a cmd channel cmd_channel = t.open_session() cmd_channel.exec_command(sys.argv[3]) data = cmd_channel.recv(1024) while data: sys.stdout.write(data) data = cmd_channel.recv(1024) # Cleanup cmd_channel.close() t.close() sock.close()

Implementing SCP with paramiko

Post author:mikal
Post published:September 4, 2008
Post category:Python

Regular readers will note that I've been interested in how scp works and paramiko for the last couple of days. There are previous examples of how to do scp with paramiko out there, but the code isn't all on one page, you have to read through the mail thread and work it out from there. I figured I might save someone some time (possibly me!) and note a complete example of scp with paramiko... #!/usr/bin/python # A simple scp example for Paramiko. # Args: # 1: hostname # 2: username # 3: local filename # 4: remote filename import getpass import os import paramiko import socket import sys # Socket connection to remote host sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((sys.argv[1], 22)) # Build a SSH transport t = paramiko.Transport(sock) t.start_client() t.auth_password(sys.argv[2], getpass.getpass('Password: ')) # Start a scp channel scp_channel = t.open_session() f = file(sys.argv[3], 'rb') scp_channel.exec_command('scp -v -t %s\n' % '/'.join(sys.argv[4].split('/')[:-1])) scp_channel.send('C%s %d %s\n' %(oct(os.stat(sys.argv[3]).st_mode)[-4:], os.stat(sys.argv[3])[6], sys.argv[4].split('/')[-1])) scp_channel.sendall(f.read()) # Cleanup f.close() scp_channel.close() t.close() sock.close()

Robot Visions

Post author:mikal
Post published:September 4, 2008
Post category:Book

This was a pretty short read -- in fact I read it on the bus into work this morning. That's mainly because there are only three short stories in this book which aren't covered in one of Asimov's other robot short story collections. The three stories were good, but I am not sure they were worth owning the entire book for. [isbn: 0451450647]

The Belgariad

Post author:mikal
Post published:September 3, 2008
Post category:Book

The Belgariad is a fantasy series by David Eddings. There are five books in the series, although there is also a follow series called the Mallorean, as well as three tie in books. The series follows a small farm boy as he grows up, discovering along the way that his relatives are famous, he's important to the history of the world, and that he has to defeat an ancient evil. The five books in the main series are: 1982: Pawn of Prophecy 1982: Queen of Sorcery 1983: Magician's Gambit 1984: Castle of Wizardry 1984: Enchanters' End Game

Enchanters End Game

Post author:mikal
Post published:September 3, 2008
Post category:Book

This is the final book in the Belgariad. Its a good one, although knowing there is another series involving these characters makes it feel a little less final to me. I enjoyed it though. [isbn: 0345338715]