Tuesday, May 3, 2011

Time to port your Python application to GObject introspection?

From time to time I realize that not all maintainers of GNOME applications written in Python have been following what has been happening upstream during the GNOME 3 development cycle.

Some of the noise we have been making about introspection and hackfests may have reached them, but for someone who has only time to review patches and make releases, this new stuff may have seemed as of secondary relevance.

For those that are asking themselves whether they should invest time now into moving away from PyGtk and the other static bindings, I would summarize the decision as to whether you want your application to run on Gtk+ 3 (and other recent APIs in GNOME 3) or not.

Besides that, there are other advantages such as reduced memory usage, fast startup and being able to use libraries that have no static bindings.

About distribution, most major distros have already packaged the whole of GNOME 3 and the rest will follow in the following months. As a rule of thumb, if you can develop for GNOME 3 in C in a particular distro version, you have already all you need to port your app to introspection.

As always, I want to remember that PyGObject (and gobject-introspection for that matter) has no dedicated developers. If it's progressing forward is because some application authors decided to jump and lend a hand, so if you are producing code that uses GNOME and Python, please consider dedicating some time to file bugs, write test cases and maybe writing a patch yourself.

10 comments:

Jon Dowland said...

I have a gtk2/python/glade app that currently refuses to run (it exposes a bug at some layer in the stack, not sure which, impacting my app's core functionality). I need to invest the time to replace not only pygtk with GI but also glade with gtkbuilder and possibly upgrade other bits and pieces. In other words a major, major job. OTOH, it's almost certainly not worth chasing down the show-stopper bug, since nobody upstream cares about the versions of the components of the framework.

I keep thinking, if I wrote it in C with motif or something similarly ancient and dusty, it would still run fine.

Anonymous said...

I primarmarily work on shoebot on windows and ubuntu, last time I checked gnome3 wasn't available on ubuntu, I'm sort of assuming gobject introspection will be on windows too, otherwise that would be a massive showstopper.
I'm pretty interested in looking at pypy at some point, but since this isn't python 3 compatible yet, I guess I can't use gobject introspection there either.
Still, porting is something I have been looking at, I'm especially curious as to whether it'll mean that gtk and Cairo bindings for pygi and jython start to look similar.

- stuaxo

ulrik sverdrup said...

I managed to port Kupfer (pygtk user) to gobject-introspection in major parts, and it was all working except for gtk widget style properties. Then I learn from upstream developers (who were very informative in their response to my bug report), that python + g-i + GTK2 is not supposed to be used at all.

Anyone know more about this? No surprise that GTK2 is dead maybe. How hard is it to port to GTK3 and g-i at the same time?

John Stowers said...

@ulrik: I/we ported gnome-sudoku from pygtk to gobject-introspection + gtk3 in about 2 weeks. In full-time equivalent terms, this was about 2 days work.

gnome-sudoku used quite a lot of custom widget drawing so was heavily impacted by the gtk+-2 -> gtk+-3 transition (draw signal, cairo stuff, stylecontext changes). So most of the work was due to gtk+-3 and not g-i.

However everything basically worked and gave me great confidence in the new technology.

John Stowers said...

@Jon Dowland

PyGTK should be pretty stable - bug fixes only from this point. Can you please file a bug and I will look into why it has stopped working.

John (wearing PyGTK maintainer hat)

dm said...

I think my answer to this is "not soon". Specifically, the big issue for my app, phraymd, is:

"It doesn’t fix up changed signatures (such as adding missing arguments which were default arguments in PyGTK, or the ubiquitous “user_data” argument for signal handlers)" http://www.piware.de/2011/01/na-zdravi-pygi/

phraymd relies extensively on default args and user_data in part because I decided to not use Glade/Builder (a mistake, I guess, in retrospect). The gtk3 changes will also cause me a few headaches, though probably not as big. (The other big issue is unintended sloppy programming on my part, but not much *I* can do about that)

@John Stowers, Tomeu Vizoso: Any tips on how to efficiently deal with these issues?

Bernie Innocenti said...

> Besides that, there are other advantages such as
> reduced memory usage, fast startup and [...]

Could you quantify these with some numbers? I'd heard that PyGI has performance regressions in method dispatching compared to PyGTK. Is this true?

Tomeu Vizoso said...

@dm:

I suggest to follow what is in http://live.gnome.org/PyGObject/IntrospectionPorting That page is maintained by application authors like you that wanted to share what they learnt.

Once you get stuck, use the communication channels mentioned in http://live.gnome.org/PyGObject#Support_and_Feedback

Tomeu Vizoso said...

@bernie: Don't have much time for that right now, but in practical terms, the biggest reduction will come from not having to import all wrappers for the whole of Gtk+

That means that a quick and rough way of getting an idea of the benefits is comparing how much private memory it takes to do:

import gtk

versus:

from gi.repository import Gtk

If you want something a bit more accurate, instrument your app to get a list of all the Gtk+ classes it uses, then reference them right after import. Note the difference in startup time in a slow machine.

About method invocation overhead, currently invoking through introspection should be far slower. Though there's work (J5 is leading it) under way to improve that several times.

But my own opinion is that if method invocation time is a significant factor in your application's performance, you are using Python wrong.

If you don't understand why, consider running a profiler such as sysprof and see how much time is spent in Python code vs. C code.

Jon Dowland said...

@John Stowers

Thanks for your offer. I just tried to reproduce the problem and couldn't -- which is either good or bad, depending on your POV :-)

I'm on my work laptop ATM, it's possibly I can (more reliably) reproduce it on my home PC. I'll have a go.