When the pandemic lock downs started, my local Linux User Group started looking at video conferencing tools we could use to continue presenting talks and other events to members. We ended up adopting BigBlueButton: as well as being Open Source, it's focus on education made it well suited for presenting talks. It has the concept of a presenter role, and built in support for slides (it sends them to viewers as images, rather than another video stream).

With a GLib implementation of the Python asyncio event loop, I can easily mix asyncio code with GLib/GTK code in the same thread. The next step is to see whether we can use this to make any APIs more convenient to use. A good candidate is APIs that make use of GAsyncResult. These APIs generally consist of one function call that initiates the asynchronous job and takes a callback. The callback will be invoked sometime later with a GAsyncResult object, which can be passed to a "finish" function to convert this to the result type relevant to the original call.

To help keep myself honest, I wanted to set up automated test runs on a few personal projects I host on Github. At first I gave Travis a try, since a number of projects I contribute to use it, but it felt a bit clunky. When I found Github had a new CI system in beta, I signed up for the beta and was accepted a few weeks later. While it is still in development, the configuration language feels lean and powerful.

As an evening project, I've been working on a small library that integrates the GLib main loop with Python's asyncio. I think I've gotten to the point where it might be useful to other people, so have pushed it up here: https://github.com/jhenstridge/asyncio-glib This isn't the only attempt to integrate the two event loops, but the other I found (Gbulb) is unmaintained and seems to reimplement a fair bit of the asyncio (e.

With my old ThinkPad, Lenovo provided BIOS updates in the form of Windows executables or ISO images for a bootable CD. Since I had wiped Windows partition, the first option wasn't an option. The second option didn't work either, since it expected me to be using the drive in the base I hadn't bought. Luckily I was able to just copy the needed files out of the ISO image to a USB stick that had been set up to boot DOS.

One of the projects I've been working on has been to improve aspects of the Ubuntu One Developer Documentation web site. While there are still some layout problems we are working on, it is now in a state where it is a lot easier for us to update. I have been working on updating our authentication/authorisation documentation and revising some of the file storage documentation (the API used by the mobile Ubuntu One clients).

One feature in recent versions of Python I hadn't played around with until recently is the ability to package up a multi-module program into a ZIP file that can be run directly by the Python interpreter. I didn't find much information about it, so I thought I'd describe what's necessary here. Python has had the ability to add ZIP files to the module search path since PEP 273 was implemented in Python 2.

This week I put out a new release of pygpgme: a Python extension that lets you perform various tasks with OpenPGP keys via the GPGME library. The new release is available from both Launchpad and PyPI. There aren't any major new extensions to the API, but this is the first release to support Python 3 (Python 2.x is still supported though). The main hurdle was ensuring that the module correctly handled text vs.

One of the features of Rygel that I found most interesting was the external media server support. It looked like an easy way to publish information on the network without implementing a full UPnP/DLNA media server (i.e. handling the UPnP multicast traffic, transcoding to a format that the remote system can handle, etc). As a small test, I put together a server that exposes the ABC's iView service to UPnP media renderers.

Last week, we released the source code to django-openid-auth. This is a small library that can add OpenID based authentication to Django applications. It has been used for a number of internal Canonical projects, including the sprint scheduler Scott wrote for the last Ubuntu Developer Summit, so it is possible you've already used the code. Rather than trying to cover all possible use cases of OpenID, it focuses on providing OpenID Relying Party support to applications using Django's django.

One of the nice features of Bazaar is the ability to send a bundle of changes to someone via email. If you use a supported mail client, it will even open the composer with the changes attached. If your client isn't supported, then it'll let you compose a message in your editor and then send it to an SMTP server. GMail is not a supported mail client, but there are a few work arounds listed on the wiki.

The gio library provides both synchronous and asynchronous interfaces for performing IO. Unfortunately, the two APIs require quite different programming styles, making it difficult to convert code written to the simpler synchronous API to the asynchronous one. For C programs this is unavoidable, but for Python we should be able to do better. And if you're doing asynchronous event driven code in Python, it makes sense to look at Twisted. In particular, Twisted's Deferred objects can be quite helpful.

I've been playing with OAuth a bit lately. The OAuth specification fulfills a role that some people saw as a failing of OpenID: programmatic access to websites and authenticated web services. The expectation that OpenID would handle these cases seems a bit misguided since the two uses cases are quite different: OpenID is designed on the principle of letting arbitrary OpenID providers talk to arbitrary relying parties and vice versa. OpenID is intentionally vague about how the provider authenticates the user.

Since my last article on integrating Storm with Django, I've merged my changes to Storm's trunk. This missed the 0.13 release, so you'll need to use Bazaar to get the latest trunk or wait for 0.14. The focus since the last post was to get Storm to cooperate with Django's built in ORM. One of the reasons people use Django is the existing components that can be used to build a site.

In my previous post about Django, I mentioned that I found the transaction handling strategy in Django to be a bit surprising. Like most object relational mappers, it caches information retrieved from the database, since you don't want to be constantly issuing SELECT queries for every attribute access. However, it defaults to commiting after saving changes to each object. So a single web request might end up issuing many transactions:

Yesterday, Thomas rolled the 0.13 release of Storm, which can be downloaded from Launchpad. Storm is the object relational mapper for Python used by Launchpad and Landscape, so it is capable of supporting quite large scale applications. It is seven months since the last release, so there is a lot of improvements. Here are a few simple statistics: 0.12 0.13 Change Tarball size (KB) 117 155 38 Mainline revisions 213 262 49 Revisions in ancestry 552 875 323 So it is a fairly significant update by any of these metrics.

I've been playing around with Django a bit for work recently, which has been interesting to see what choices they've made differently to Zope 3. There were a few things that surprised me: The ORM and database layer defaults to autocommit mode rather than using transactions. This seems like an odd choice given that all the major free databases support transactions these days. While autocommit might work fine when a web application is under light use, it is a recipe for problems at higher loads.

The Python standard library contains a function called thread.get_ident(). It will return an integer that uniquely identifies the current thread at that point in time. On most UNIX systems, this will be the pthread_t value returned by pthread_self(). At first look, this might seem like a good value to key a thread local storage dictionary with. Please don't do that. The value uniquely identifies the thread only as long as it is running.

Last week we moved psycopg from Subversion to Bazaar. I did the migration using Gustavo Niemeyer's svn2bzr tool with a few tweaks to map the old Subversion committer IDs to the email address form conventionally used by Bazaar. The tool does a good job of following tree copies and create related Bazaar branches. It doesn't have any special handling for stuff in the tags/ directory (it produces new branches, as it does for other tree copies).

Yesterday Federico released version 2.0.7 of psycopg2 (a Python database adapter for PostgreSQL). I made a fair number of the changes in this release to make it more usable for some of Canonical's applications. The new release should work with the development version of Storm, and shouldn't be too difficult to get everything working with other frameworks. Some of the improvements include: Better selection of exceptions based on the SQLSTATE result field.

As most developers know, Valgrind is an invaluable tool for finding memory leaks. However, when debugging Python programs the pymalloc allocator gets in the way. There is a Valgrind suppression file distributed with Python that gets rid of most of the false positives, but does not give particularly good diagnostics for memory allocated through pymalloc. To properly analyse leaks, you often need to recompile Python with pymalloc. As I don't like having to recompile Python I took a look at Valgrind's client API, which provides a way for a program to detect whether it is running under Valgrind.

Marc uploaded a new revision of the Python DB-API 2.0 Specification yesterday that documents the new two phase commit extension that I helped develop on the db-sig mailing list. My interest in this started from the desire to support two phase commit in Storm – without that feature there are far fewer occasions where its ability to talk to multiple databases can be put to use. As I was doing some work on psycopg2 for Launchpad, I initially put together a PostgreSQL specific patch, which was (rightly) rejected by Federico.

At Canonical, one of the approaches taken to accelerate development is to hold coding sprints (otherwise known as hackathons, hackfests or similar). Certain things get done a lot quicker face to face compared to mailing lists, IRC or VoIP. When collaborating with someone at one of these sprints the usual way to let others look at my work would be to commit the changes so that they could be pulled or merged by others.

Nicolas: Your metaclass example is a good example of when not to use metaclasses. I wouldn't be surprised if it is executed slightly different to how you expect. Let's look at how Foo is evaluated, starting with what's written: class Foo: __metaclass__ = FooMeta This is equivalent to the following assignment: Foo = FooMeta('Foo', (), {...}) As FooMeta has an __new__() method, the attempt to instantiate FooMeta will result in it being called.

Over the weekend, I spent a number of hours tracking down a bug caused by the cache in the Python urlparse module. The problem has already been reported as Python bug 1313119, but has not been fixed yet. First a bit of background. The urlparse module does what you'd expect and parses a URL into its components: >>> from urlparse import urlparse >>> urlparse('http://www.gnome.org/') ('http', 'www.gnome.org', '/', '', '', '') As well as accepting byte strings (which you'd be using at the HTTP protocol level), it also accepts Unicode strings (which you'd be using at the HTML or XML content level):

This week at the EuroPython conference, Gustavo Niemeyer announced the release of Storm and gave a tutorial on using it. Storm is a new object relational mapper for Python that was developed for use in some Canonical projects, and we've been working on moving Launchpad over to it. I'll discuss a few of the nice features of the package: Loose Binding Between Database Connections and Classes Storm has a much looser binding between database connections and the classes used to represent records in particular tables.

When at conferences and sprints, I often want to see what someone else is working on, or to let other people see what I am working on. Usually we end up pushing up to a shared server and using that as a way to exchange branches. However, this can be quite frustrating when competing for outside bandwidth when at a conference. It is possible to share the branch from a local web server, but that still means you need to work out the addressing issues.

While browsing the log of one of my Bazaar branches, I noticed that the commit messages were being recorded as occurring in the +0800 time zone even though WA switched over to daylight savings. Bazaar stores commit dates as a standard UNIX seconds since epoch value and a time zone offset in seconds. So the problem was with the way that time zone offset was recorded. The code in bzrlib that calculates the offset looks like this:

In my previous entry, I mentioned that Andrew was actually publishing the contents of all his Bazaar branches with his rsync script, even though he was only advertising a single branch. Yesterday I had a need to actually do this, so I thought I'd detail how to do it. As a refresher, a Bazaar repository stores the revision graph for the ancestry of all the branches stored inside it. A branch is essentially just a pointer to the head revision of a particular line of development.

Daylight saving started yesterday: the first time since 1991/1992 summer for Western Australia. The legislation finally passed the upper house on 21st November (12 days before the transition date). The updated tzdata packages were released on 27th November (6 days before the transition). So far, there hasn't been an updated package released for Ubuntu (see bug 72125). One thing brought up in the Launchpad bug was that not all applications used the system /usr/share/zoneinfo time zone database.

In my previous entry about using the Maemo obex-module on the desktop, Johan Hedberg mentioned that bluez-utils 3.7 included equivalent interfaces to the osso-gwconnect daemon used by the method. Since then, the copy of bluez-utils in Edgy has been updated to 3.7, and the necessary interfaces are enabled in hcid by default. Before trying to modify the VFS code, I thought I'd experiment a bit with the D-BUS interfaces via the D-BUS python bindings.

The Python developers have been looking for a new bug tracker, and essentially put out a tender for people interested in providing a bug tracker. Recently I have been working on getting Launchpad's entry ready, which mainly involved working on SourceForge import. The entry is now up, and our demonstration server is up and running with a snapshot of the Python bug tracker data. As a side effect of this, we've got fairly good SourceForge tracker import support now, which we should be able to use if other projects want to switch away from SF.

Emmanuel: if you are using a language like Python, you can let the language keep track of your state machine for something like that: def load_items(treeview, liststore, items): for obj in items: liststore.append((obj.get_foo(), obj.get_bar(), obj.get_baz())) yield True treeview.set_model(liststore) yield False def lazy_load_items(treeview, liststore, items): gobject.idle_add(load_items(treeview, liststore, item).next) Here, load_items() is a generator that will iterate over a sequence like [True, True, ..., True, False]. The next() method is used to get the next value from the iterator.

Back in January I started working on a new Python wrapper for the GPGME library. I recently put out the first release: http://cheeseshop.python.org/pypi/pygpgme/0.1 This library allows you to encrypt, decrypt, sign and verify messages in the OpenPGP format, using gpg as the backend. In general, it stays fairly close to the C API with the following changes: Represent C structures as Python classes where appropriate (e.g. contexts, keys, etc). Operations on those data types are converted to methods.

Anyone who has played with Zope 3 has probably seen the syntax used to declare what interfaces a particular class implements. It looks something like this: class Foo: implements(IFoo, IBar) ... This leads to the following question: how can a function call inside a class definition's scope affect the resulting class? To understand how this works, a little knowledge of Python metaclasses is needed. Metaclasses In Python, classes are instances of metaclasses.

The Python developers have been discussing a migration off CVS on the python-dev mailing list. During the discussion, Bazaar-NG was mentioned. A few posts of note: Mark Shuttleworth provides some information on the Bazaar roadmap. Importantly, Bazaar-NG will become Bazaar 2.0. Steve Alexander describes how we use Bazaar to develop Launchpad. This includes a description of the branch review process we use to integrate changes into the mainline. I'm going to have to play around with bzr a bit more, but it looks very nice (and should require less typing than baz .

One of the features added back in Python 2.2 was class methods. These differ from traditional methods in the following ways: They can be called on both the class itself and instances of the class. Rather than binding to an instance, they bind to the class. This means that the first argument passed to the method is a class object rather than an instance. For most intents and purposes, class methods are written the same way as normal instance methods.

Found out about The Python Challenge. While you don't need to use Python to solve most of the problems, a knowledge of the language certainly helps. While the initial problems are fairly easy, some of the later ones are quite difficult, and cover many topics. If you decide to have a go, here are a few hints that might help: Keep a log of what you do. Solutions to may provide insight into subsequent problems.

Tracing Python Programs I was asked recently whether there was an equivalent of sh -x for Python (ie. print out each statement before it is run), to help with debugging a script. It turns out that there is a module in the Python standard library to do so, but it isn't listed in the standard library reference for some reason. To use it, simply run the program like this: /usr/lib/python2.4/trace.py -t program.

While discussing unicode on IRC with owen, we ran into a peculiarity in Python's unicode handling. It can be tested with the following code: >>> s = u'\U00010001\U00010002' >>> len(s) >>> s[0] Python can be compiled to use either 16-bit or 32-bit widths for characters in its unicode strings (16-bit being the default). When compiled in 32-bit mode, the results of the last two statements are 2 and u'\U00010001' respectively. When compiled in 16-bit mode, the results are 4 and u'\ud800'.