Docs hackfest: starting point

Have spent my trip to Brno undusting and rebasing the branch in gobject-introspection that uses to Mako templates to generate Mallard code from GIR files.

Here you can see how looks like the generated documentation for the Account class in the TelepathyGLib typelib for Python, and here is the one generated with gtk-doc.

As can be seen, there's still a long way to get even close to the gtk-doc level, but it's a very promising start.

Now moving for real

In January I tried to move from Sugar to other projects, but Collabora prevented it by hiring me to work on Sugar and Telepathy integration.

Nine months and a release cycle later, I'm doing it for real. During the next months I will be switching to work on totally different stuff, involving Clutter and WebKit hacking.

Also plan to keep hacking a bit on Telepathy, would be a pity stop using what I learned during most of this year.

Plans for the next cycle

So it's that time of the ½ year when one starts to think on what to spend his next months. Of course it will depend on what my employer thinks it's best, but this is what I'm thinking right now.

Port Sugar to telepathy-glib: right now Sugar uses Telepathy via python-dbus, which is a quite low level API. The code could get considerably shorter and simpler by using the higher level API in telepathy-glib. For that, we would need to use gobject-introspection because there are no plans for static bindings for Python.

Expose extended contact attributes through an official addition to telepathy-spec: Right now Sugar is using an extension to telepathy-salut and telepathy-gabble for discovering additional information about the online contacts. In our effort to make Sugar a regular Telepathy user and reducing the maintenance burden of such extensions, we should move to use only official additions to telepathy-spec.

Expose activities through an official addition to telepathy-spec: In Sugar you can discover which activities are being publicly shared by your contacts, join them and also send and receive private invites. This is basically announcing which applications running in your desktop have collaboration capabilities by means of IM channels. Same as above, we need to move to use interfaces of more general use.

Port Sugar to GNOME 3: Sugar's cycle is synchronized with GNOME, Fedora and Ubuntu so each release can be more easily packaged and delivered by distros. Though Gtk+2 and Gtk+3 are installable in parallel, the lamentable state in which the stable Python GNOME bindings are means nobody is there to maintain all the bindings and much less for wrapping new APIs such as GSettings. For the sake of moving along with our platform, we'll have to port Sugar and all the activities to GNOME 3 through with gobject-introspection. PyGObject is very close to have feature-complete support for introspection but still will take quite a bit of work to make it stable.

Telepathy and Ubuntu

As mentioned in my last post about the Ubuntu Developer Summit last week, there's interest in Ubuntu for using the Telepathy framework in more of their projects. There was discussion of using Telepathy in their telephony stack and also on some Ubuntu-specific changes in Empathy, but I was most interested in their effort to make easy and fun to code collaborative applications.

As part of the work on Sugar that Collabora is sponsoring, I also have a big interest in making as easy as possible to use the telepathy framework, be it in applications or in desktop components such as the Sugar shell.

For now, I'm going to help kickstart the process by implementing some toy-like API on top of Telepathy in the form of widgets for Quickly, here is the branch.

This is what we came up with in the brainstorming session about the "easy and fun" API:

class Client(Boo):
    def __init__(self):
        self.tube_handler = MessageTubeHandler("tubey")
        self.tube_handler.connect("handle-channel", self.on_handle_channel)

    def on_handle_channel(self, tube):
        self.image_tube = tube
        tube.connect("message-received", self.on_message_received)

    def on_message_received(self, tube, msg):
        if "img_url" in msg:
            self._download_image(msg["img_url"])
        
    def on_contact_selected(self, contact, account):
        self.image_tube = tp.MessageTube(contact, account)
        msg = {}
        msg["note"] = self.label1.get_text()
        self.image_tube.send(msg)
        
    def on_tube_initiated(self, tube):
        if "kenvandine" in tube.contact.props.identifier:
                return

As PyGI is not packaged yet for Ubuntu and for the sake of getting started quickly, I'm going to do the initial implementation with the old telepathy-python bindings, but I hope that whatever gets into the Maverick release uses the much nicer telepathy-glib API through introspection.

Also will be trying to keep an eye on the collaboration needs of Sugar activities so more people benefit from this effort.

Going to UDS-M

Canonical has been so nice to sponsor my attendance to the next Ubuntu Developer Summit in Brussels next week and Collabora is sponsoring my time there for Telepathy, Python and Sugar stuff. Thanks a lot to both companies.

The Ubuntu community has interest in making as easy as possible for regular users to create software and part of this effort is having a high-level API for having collaboration across the network. This of course will involve Telepathy and I have also a big interest in providing this experience to users of Sugar. Collabora are going to be flying as well some of their Telepathy developers to work on this and I look forward to finally meeting them.

Ubuntu has also committed heavily on Python for their development efforts, both on the desktop and in the server. For the desktop side of things, Python on GNOME is facing several challenges right now and I will be happy to help them make sure they don't lose any opportunities because of that. The challenges I have referred to are the obsolescence of static library bindings and the update to Python 3. On both issues Red Hat has contributed several developers to upstream and I will be happiest to discuss how we can also make Ubuntu contributors most welcome there. I'm also looking forward to finally meet Simon van der Linden in person after working together in PyGI for so long, he happens to live close to the venue so he'll jump to say hello and maybe do some hacking together (I have some patches in the queue that could use some review love).

There's also interest from Edubuntu in shipping Sugar, so I have registered to their blueprint and will be available to answer any questions they have regarding it. I have felt sad often when Ubuntu users have come asking how they could run Sugar on their machines and I had to point them to the instructions to build from the source, so it's great to have the possibility for Sugar to be accepted and supported by Ubuntu at last.

Using telepathy-glib in python through gobject-introspection

Until now, you had two options for writing telepathy clients in python: use the low level D-Bus API through dbus-python, or using telepathy-python which provided very little more for clients apart from registering the zillion of constants in Telepathy.

Telepathy-glib is a library that provides a higher level API for interacting with the Telepathy framework than the low level D-Bus API. No languages bindings exist currently for this library so only C apps such as Empathy used it, developers using languages such as Python and JavaScript were stuck with the D-Bus API and generally each app invented a more or less complete new API on top of it.

In parallel to a push for adding more high-level APIs to telepathy-glib, we are looking at what remains to be done in the gobject-introspection stack so we don't need to write static bindings for each language. After some weeks hacking on Gjs (Danielle Madeley), Vala (Travis Reitter) and PyGI (me) things are starting to take shape.

Following Danni's post on her progress, here comes an example observer written in Python and using PyGI to call telepathy-glib:

import gobject
gobject.threads_init()

from gi.repository import TelepathyGLib

#TelepathyGLib.debug_set_flags('all')

def observe_channels(observer, account, connection, channels,
                     dispatch_op, requests, context, user_data):
    print 'observe_channels'

    print 'account = %s' % account.get_object_path()
    print 'connection = %s' % connection.get_object_path()

    for channel in channels:
        print 'channel = %s' % channel.get_object_path()

    if dispatch_op is not None:
      print 'dispatch_op = %s' % dispatch_op.get_object_path()
    else:
      print 'dispatch_op = None'

    if requests is not None:
        for request in requests:
          print 'request = %s' % request.get_object_path()

    context.accept()

dbus = TelepathyGLib.DBusDaemon.dup()
observer = TelepathyGLib.SimpleObserver.new(dbus, True, 'PythonObserver', True,
                                            observe_channels, None)
observer.add_observer_filter({
  TelepathyGLib.PROP_CHANNEL_CHANNEL_TYPE: TelepathyGLib.IFACE_CHANNEL_TYPE_TEXT,
})

observer.register()

main_loop = gobject.MainLoop()
main_loop.run()

This needs the latest code in telepathy-glib, gobject-introspection and pygi, and also a patch to PyGI that hasn't been release yet.

Sugar with better Telepathy

After three weeks of little coding but much reading, a reward:



This shows the Chat activity talking to Empathy through link-local xmpp. This was already possible, but the difference now is that Chat is a normal telepathy client, and it has been activated by dbus as would be any other telepathy-enable application in GNOME, Meego or whatever.

The current code is very sketchy, but we are starting to get a good sense of the next challenges. If you are interested in knowing why this is important, see my previous post.

Progress can be followed in this page.

My focus during 0.90: Collaboration

From now on to the next Sugar release in about 6 months (0.90), I'm going to be working on improving the collaboration stack in Sugar, graciously sponsored by Collabora. Collabora employees developed almost all of the collaboration code in Sugar, and are the main force behind the Telepathy framework, on which Sugar's collaboration is based. Though pervasive collaboration is one of the major features of Sugar, it hasn't been funded for a long time and there's large room for improvement, also due to the new developments in Telepathy. My work will aim to make presence and collaboration more reliable and to put the pieces in place for enriching the experience with the new features in Telepathy.

Collabora has almost 5 years of experience in open source software so they know very well that for a project to survive, someone needs to take the role of software maintenance, and that's why they are going to be sponsoring me as well 1-2 days per week of maintenance work. Maintenance work includes tasks such as reviewing proposed changes, maintaining the bug database, participating in community discussions, welcoming new contributors, making new releases, etc, and is fundamental in keeping an open source project such as Sugar alive. That's why I would like to put a call for our many users of Sugar to help the other maintainers keep doing their work by supporting them financially. As of today the only Sugar maintainers that are paid anything for doing their job are Sayamindu and me, and this is not sustainable.

The grunt of the work will be dropping custom code that was developed ad-hoc in the early days and making Sugar use instead new developments in Telepathy. Many of the bugs we see today in collaboration are due to the several layers on which collaboration information is cached. By dropping the Presence Service and having activities call Telepathy directly, we'll improve the reliability of presence and collaboration.

There's also code in Telepathy that is exclusively for Sugar and that is redundant now due to new developments in other parts of Telepathy. I will be working as well in making Sugar don't need those specific mechanisms because the Telepathy maintainers will drop those pieces at some point and Sugar cannot be left behind. As well, Sugar depends on the server being configured in an specific way, breaking collaboration with non-Sugar specific servers, there will be work as well to remove this limitation.

By using the same pieces as other environments such as GNOME and MeeGo, Sugar will be closer to the development of those platforms and will be in a better position to take advantage from new features. We'll be reducing as well the effort required to maintain and further develop our collaboration stack, and we'll be sharing with others the burden of keeping our collaboration foundations solid.

Many of the changes will be under the hood and won't be seen nor by users nor by activity developers. Other changes will affect developers because API that they were using won't be there any more, I'm going of course to try to minimize this as much as possible. Other changes will impact the UI, mostly by adding new features, but sometimes will be changes required to make Sugar closer to the other Telepathy clients. I will be starting discussions very soon about all these changes so we can find the best solutions for each.

In summary, Sugar will improve in these ways:

- the collaboration code will be smaller and simpler,

- Sugar users will be able to collaborate through more networks, with more protocols, and with people not using Sugar,

- Sugar will be more compatible with future versions of Telepathy,

- we'll have a simpler path to acquire recent and upcoming features in Telepathy and related frameworks, such as VOIP and video calls.

Telepathy Developer's Manual

I'm skimming through it right now and this is REALLY good stuff:

http://people.collabora.co.uk/~davyd/telepathy-book/

Ideally, 90% of Sugar activity authors shouldn't need to get to work much with telepathy directly, because they most likely only want to share a data structure and interchange some messages or signals.

But for the people doing activities such as Chat or VideoChat, implementing collaboration frameworks such as Groupthink or messing with the presence service in Sugar, this will be invaluable and will hopefully lower the floor for new contributors.

Thanks, Collabora!