Thunar/GIO – Quick Status Report
I've started hacking on the migration of Thunar to GIO on April 9th. In about 61 commits, I've reduced the original number of ThunarVFS references in the Thunar source code dramatically. The most important and probably most time-consuming part of this work is only mentioned briefly on those pages: rewriting all recursive copy/move/trash/restore/chmod/chgrp/chown jobs -- by now most of the jobs have been rewritten based on GIO and the new ThunarJob framework ... and Thunar still works (for me at least)!
All in all, I suppose that about a third of the implementation work is done. Ok, maybe just a quarter, but an important one. I now have a very good overview over the source code and I'm almost done with one of the most critical parts of the migration.
If you want to know how that looks like for me, here's a screenshot:
These are the most important/big things that are still waiting for me:
- Replace ThunarVFSVolumeManager with GFileMonitor. Volume management is a large and complex subsystem of both ThunarVFS and GIO.
- Load ThunarFile objects asynchronously. This will be a pain. A lot of functions will have to be split up to fit into the asynchronous concept.
- Move the thumbnail related code into exo
These two are a bit out of scope but very important nonetheless:
- Integrate functionality similar to Gigolo (remote/virtual places management) into the file manager.
- Write GNOME-independent GIO extensions for volume management and the trash.
On a side node: Xubuntu 9.04 is available as of today! Go and grab it if you're interested in a nice distro based on Xfce. If you're interested in the Xubuntu development, you'll be able to meet Stephan, several Xubuntu folks and me at the Ubuntu Developer Summit in Barcelona from May 25th-29th. I'm very excited already!
Writing Your Own GIO Jobs
After having moved into a new apartment I'm now back at working on my thesis about Thunar. There are a few things which are solved very differently in GIO than in ThunarVFS. One of them is the way jobs are handled. A job basically is a task which may take a while to run and thus is executed in a separate thread (so it doesn't block the GUI).
ThunarVFS has a framework called ThunarVfsJob. It lets you create different kinds of jobs e.g. for changing file permissions recursively or for computing the total number of files and the total size of a directory. These jobs report back to the GUI thread using signals such as "new-files" (when new files are added and need to picked up by the GUI) or "progress".
GIO has something similar ... but it's not so obvious how it works. When I tried to figure out how to migrate ThunarVfsJob to GIO I thought: hey, GIO must have something like this already! It contains several job-like functions such as g_file_copy_async() after all.
So here's what I found out after spending some time on reading gfile.c and glocalfile.c: there is a job framework in GIO ... but it's hidden behind easy-to-use asynchronous functions. It's based on GCancellable, GAsyncResult and a bunch of callback types. It uses GIOScheduler internally to glue everything together to something that is actually pretty convenient (but still kinda tricky).
So, what do you need in orderto write your own jobs in the GIO style?
First of all, you need an example task. I picked counting files and computing the total size of a directory to understand how it works. What we want is an asynchronous function which does exactly that and uses a callback to report the progress back to the GUI thread ... just like g_file_copy_async() does.
First of all, you define the callback type and two functions for starting the job (sync and async version):
The Public API
typedef void (*GFileCountProgressCallback) (goffset current_num_files,
goffset current_num_bytes,
gpointer user_data);
static gboolean
g_file_deep_count (GFile *file,
GCancellable *cancellable,
GFileCountProgressCallback progress_callback,
gpointer progress_callback_data,
GError **error);
static void
g_file_deep_count_async (GFile *file,
int io_priority,
GCancellable *cancellable,
GFileCountProgressCallback progress_callback,
gpointer progress_callback_data,
GAsyncReadyCallback callback,
gpointer callback_data);
The Implementation
All the function g_file_deep_count_async() will do is to create a GSimpleAsyncResult, put
the callback information into it and then tell the GIOScheduler to run the job. Here's how
that looks like:
static void
g_file_deep_count_async (GFile *file,
int io_priority,
GCancellable *cancellable,
GFileCountProgressCallback progress_callback,
gpointer progress_callback_data,
GAsyncReadyCallback callback,
gpointer callback_data)
{
GSimpleAsyncResult *result;
DeepCountAsyncData *data;
g_return_if_fail (G_IS_FILE (file));
data = g_new0 (DeepCountAsyncData, 1);
data->file = g_object_ref (file);
data->progress_cb = progress_callback;
data->progress_cb_data = progress_callback_data;
result = g_simple_async_result_new (G_OBJECT (file),
callback,
callback_data,
g_file_deep_count_async);
g_simple_async_result_set_op_res_gpointer (result,
data,
(GDestroyNotify) deep_count_async_data_free);
g_io_scheduler_push_job (deep_count_async_thread,
result,
g_object_unref,
io_priority,
cancellable);
}
DeepCountAsyncData is a simple struct which needs no further explanation, I think.
First data with callback and user data information is added to the GSimpleAsyncResult
and then the job is added to the GIOScheduler. As you can see, there is another function
involved: deep_count_async_thread. This is the function which runs in a separate thread
and does most of the work (well, not quite ... but almost). Here's how it looks like:
static gboolean
deep_count_async_thread (GIOSchedulerJob *job,
GCancellable *cancellable,
gpointer user_data)
{
GSimpleAsyncResult *res;
DeepCountAsyncData *data;
gboolean result;
GError *error = NULL;
res = user_data;
data = g_simple_async_result_get_op_res_gpointer (res);
data->job = job;
result = g_file_deep_count (data->file,
cancellable,
data->progress_cb != NULL ? deep_count_async_progress_callback : NULL,
data,
&error);
if (data->progress_cb != NULL)
g_io_scheduler_job_send_to_mainloop (job, (GSourceFunc) gtk_false, NULL, NULL);
if (!result && error != NULL)
{
g_simple_async_result_set_from_error (res, error);
g_error_free (error);
}
g_simple_async_result_complete_in_idle (res);
return FALSE;
}
As you can see it runs the synchronous function g_file_deep_count() and makes sure the progress
callback is called at least once. It does one more thing though: it defines it's own progress
callback: deep_count_async_progress_callback. This is required for the real progress
callback to be called inside the GUI thread. This is the code for the internal callback:
static gboolean
deep_count_async_progress_in_main (gpointer user_data)
{
ProgressData *progress = user_data;
DeepCountAsyncData *data = progress->data;
data->progress_cb (progress->current_num_files,
progress->current_num_bytes,
data->progress_cb_data);
return FALSE;
}
static void
deep_count_async_progress_callback (goffset current_num_files,
goffset current_num_bytes,
gpointer user_data)
{
DeepCountAsyncData *data = user_data;
ProgressData *progress;
progress = g_new (ProgressData, 1);
progress->data = data;
progress->current_num_files = current_num_files;
progress->current_num_bytes = current_num_bytes;
g_io_scheduler_job_send_to_mainloop_async (data->job,
deep_count_async_progress_in_main,
progress,
g_free);
}
deep_count_async_progress_callback() is called from within the job thread. It then tells
the scheduler to call deep_count_async_progress_in_main from the GUI thread. And finally
deep_count_async_progress_in_main calls the real progress callback e.g. to update
the GUI.
Now you still haven't seen any code related to counting files and computing the total file size of a directory ... let's get to that now. Here's the synchronous deep count function which is called from within the job thread:
static gboolean
g_file_deep_count (GFile *file,
GCancellable *cancellable,
GFileCountProgressCallback progress_callback,
gpointer progress_callback_data,
GError **error)
{
ProgressData data = {
.data = NULL,
.current_num_files = 0,
.current_num_bytes = 0,
};
g_return_val_if_fail (G_IS_FILE (file), FALSE);
if (g_cancellable_set_error_if_cancelled (cancellable, error))
return FALSE;
return g_file_real_deep_count (file,
cancellable,
progress_callback,
progress_callback_data,
&data,
error);
}
Damn ... it still doesn't do any real work! Ok, but this time there's no big rat-tail
of nested function calls anymore, I promise. There's just one function left:
g_file_real_deep_count().
Before we can call it, however, g_file_deep_count() has to initialize the progress data.
After that we can call g_file_real_deep_count() recursively and do something useful.
Here we go:
static gboolean
g_file_real_deep_count (GFile *file,
GCancellable *cancellable,
GFileCountProgressCallback progress_callback,
gpointer progress_callback_data,
ProgressData *progress_data,
GError **error)
{
GFileEnumerator *enumerator;
GFileInfo *info;
GFileInfo *child_info;
GFile *child;
gboolean success = TRUE;
g_return_val_if_fail (G_IS_FILE (file), FALSE);
if (g_cancellable_set_error_if_cancelled (cancellable, error))
return FALSE;
info = g_file_query_info (file,
"standard::*",
G_FILE_QUERY_INFO_NOFOLLOW_SYMLINKS,
cancellable,
error);
if (g_cancellable_is_cancelled (cancellable))
return FALSE;
if (info == NULL)
return FALSE;
progress_data->current_num_files += 1;
progress_data->current_num_bytes += g_file_info_get_size (info);
if (progress_callback != NULL)
{
/* Here we call the internal callback */
progress_callback (progress_data->current_num_files,
progress_data->current_num_bytes,
progress_callback_data);
}
if (g_file_info_get_file_type (info) == G_FILE_TYPE_DIRECTORY)
{
enumerator = g_file_enumerate_children (file,
"standard::*",
G_FILE_QUERY_INFO_NOFOLLOW_SYMLINKS,
cancellable,
error);
if (!g_cancellable_is_cancelled (cancellable))
{
if (enumerator != NULL)
{
while (!g_cancellable_is_cancelled (cancellable) && success)
{
child_info = g_file_enumerator_next_file (enumerator,
cancellable,
error);
if (g_cancellable_is_cancelled (cancellable))
break;
if (child_info == NULL)
{
if (*error != NULL)
success = FALSE;
break;
}
child = g_file_resolve_relative_path (file, g_file_info_get_name (child_info));
success = success && g_file_real_deep_count (child,
cancellable,
progress_callback,
progress_callback_data,
progress_data,
error);
g_object_unref (child);
g_object_unref (child_info);
}
g_object_unref (enumerator);
}
}
}
g_object_unref (info);
return !g_cancellable_is_cancelled (cancellable) && success;
}
And that's it. We can now compute the number of files and the total size of a directory
recursively using a GCancellable and one or two callbacks. All of this is done using
threads, so you don't have to worry about blocking your GUI main loop.
If you want to see this in action, visit the job framework page in my thesis wiki and download deepcount.c and the Makefile.
libxfce4menu to be renamed to gdesktopmenu
Now that all Xfce dependencies have been removed from libxfce4menu, I'm planning to rename it to something more generic. Travis Watkins from Alacarte has expressed interest in helping with writing the XfceMenuEditor (err GDesktopMenuEditor) part. He is also working on Vala bindings for the library.
With a more complete API and polished code perhaps we can even move gdesktopmenu over to freedesktop.org ... who knows.
Xfce at FOSDEM 2009
FOSDEM is over and it was a blast! Finally, after about four years, the team met again. It was the first time for me and it was great to meet them all in person. Olivier, Yves-Alexis, Nick and I stayed at the same hotel. Stephan stayed in Gent and the others spread over several other hotels in Brussels.
I arrived early on Friday and relaxed in the hotel room for a while, going through my slides and listening to a couple having sex next door for hours. If you’re able to count the orgasms then it’s definitely too loud. So I left the hotel and walked around in the inner city of Brussels for a while until Yves-Alexis, and shortly after that, Nick and Stephan arrived. We had a few beers at a bar around the corner and later went to for Pasta and Pizza in an Italian restaurant close to the hotel where Jens joined us. After that we went to the FOSDEM beer event and stayed there until half past midnight, only interrupted by the arrival of Olivier whom we had to take to the hotel. The beer was nice and since the place was so crowded we had a hard time understanding each other … but we somehow managed it.
We got up at around 7am the next morning and of course the beer from the evening before didn’t really make things better. The nice thing about the hotel (Hotel Du Congres by the way) was that there was free breakfast included though. It was rather basic but it did it’s job. Shortly after that we left for FOSDEM. We arrived in time for the keynotes and met Jean-François and Maximilian on our way to Janson (the large auditorium). Later we went for some french fries and mainly spent our time in the X.org devroom. Some of us went to the LXDE lightning talk and were quite amused by their claim that LXDE is better than Xfce (what does better mean anyway?) and the huge number of slides showing photos of their team at different conferences. Later that day we went to a restaurant called Big Mama in the very heart of Brussels were we had a beer and nice food. Everyone went to bed afterwards, except Nick, who kept hacking on the panel.
Sunday was a big day for me – I had a talk about Xfce 4.6 and the future of Xfce scheduled at 11am. I didn’t really sleep well and was nervous the entire morning. It didn’t get any better when I had finally booted up my laptop and the microphone was turned on. I don’t know how long I talked but I have the feeling that I literally speed-talked through the slides. I forgot a few things I would’ve loved to mention but all in all I think it was ok. There were quite a few sceptical faces in the audience but also a few who looked very pleased by the features and plans I presented them. I definitely think it was ok for my first talk in English at a big conference but I know I can do better. So I’d be absolutely happy to do that again next year!
I felt very relieved afterwards. We then went out to take a few group pictures (the following one was shot by Jens). In the meantime, Samuel and Jelle de Jong had joined us (I have no idea where Jelle went afterwards, but Samuel stayed with us for the rest of the day).
After that we want back to one of the devrooms until Maximilian came back from his car with a number of sandwiches he had bought on the way. We had a beer at the FOSDEM bar, ate our sandwiches and then went back to the cross desktop room for Stephan’s talk about Xfce as a platform. I think he did a good job overall, especially considering that he had just finished his slides the night before. I would’ve loved to see more about how to use the different libraries and maybe more details about our plugin APIs but he covered a few very important things about Xfconf.
After that we had another beer at the bar and went to the last talks in Janson. Samuel dropped his full beer, I accidently kicked my empty one and Olivier and I left early for the FOSDEM bus back to Brussels-South. I managed to catch an earlier train to the airport, had a short flight, then had to wait for the bus from Hamburg to Lübeck for 90 minutes and finally arrived at around half past midnight.
I think it was a lot of fun and I’ll definitely go to FOSDEM again. It was nice to put a face to all the names of people I’ve been working with over the last few years. Thanks to everyone I met there, it was a real pleasure! Hope to see you all again! Thanks to all the people who went to our talks (oh, and to that guy who wanted that picture of him and me – could you send it to me please?) and asked questions (“Has the Xfce team ever thought about participating in Google SoC?”, “How good is the relationship of Xfce with Xubuntu?”, “Are you still aiming to be lightweight?” amongst others). It was also great to see several packagers, like Mark from Foresight, Landry from OpenBSD and Christoph from Fedora.
As I’ve mentioned before, I would really like to talk at conferences again. It’s a challenge but definitely a nice one. If you know any that could be interesting for Xfce, please let me know!
Oh, and by the way, if you’ve taken any pictures of us at FOSDEM, please let us know!
FOSDEM schedule and graphical 4.6 installers
The FOSDEM schedule has been published. You can find the Cross Desktop room schedule here. There will be two talks about Xfce. Short descriptions are linked from the schedule. We’re also mentioned in the press release.
We will be a fairly large group of people (around at dozen at least, with most of the core developers being present) so it’ll definitely rock!
On the Xfce mailing list people have been asking about graphical installers for 4.6. I’m currently preparing them and there should be installers for Xfce 4.6 RC1, RC2 and the final release. The main installer is already working except for one thing that’s bugging me. Should be able to fix it though.
Topics raised at UDS
Ok, this post is long overdue and has been around as a draft for more than a week now, so here it is. As readers of this weblog already know, I was at the Ubuntu Developer Summit from December 8th to 12th. We had quite a few discussions on topics I’m interested in outside the scope of the official sessions held at UDS. Let me just list them here in random order:
- Arnaud Quette told me about Buildbot which can be used to test your repository for compilation errors on various systems. I suppose it can be a real time saver if you want your project to compile on a variety of distributions and UNIX flavors and want immediate response as to whether compilation is going to succeed on these. That way you don’t have to wait for packagers and users to report compilation errors back to you and don’t have to enter that typical ping pong developer-reporter dialog. Buildbot not only catches compilation errors but is also capable of sending notification emails and provides detailed information about the compile log and so on. It’s implemented based on a server/client concept where each client tests compilation on one system and the server collects data from the clients. By using virtualization you can set up server and clients on the same machine. This would be very nice to have for Xfce but I suppose it would require another dedicated server just for running all these compilations. If anyone is interested in looking in to that, I’d be happy to establish contact to someone from Buildbot to provide you with more information.
As a funny coincidence, Frédéric Péters recently announced build.gnome.org which is a great example of Buildbot usage. - I talked to Christian Kellner about GIO/Gvfs a bit in order to get information on how remote/virtual filesystems work. One thing I’ve thought about is how to allow for a list of user-defined remote/virtual filesystems which show up in the side pane of Thunar. There are different approaches to that. One is to use bookmarks which seems to be what, according to Christian, Nautilus does right now. While this seems to work quite well it seems to confuse users a bit. Personally, I’d like to see them separated from bookmarks. There has been a proposal for Nautilus to make the side pane look better by using sections. One thing I could imagine would be to have a “Virtual Volumes” section listing the user-defined filesystems and provide some sort of GUI for creating/removing/editing these. From what I’ve heard there also is a third approach, which is to make remote filesystems to appear as fake volumes in HAL or DeviceKit. I’ll have to look into that in order to decide what the best way to go is, I guess.
- PulseAudio and GStreamer: We discussed the limited PulseAudio backend for GStreamer in a group of up to five people and agreed that it really needs improvement (as in more tracks have to be added to the GstMixer interface) if we don’t want users to be able to control PulseAudio through the mixer applications they know. PulseAudio-specific applications like pavucontrol are not really what we want them to use.
- Xubuntu remix for netbooks: During one of the Xubuntu sessions I talked to a member of the Ubuntu Mobile team about Xfce in general, the modularity of Xfce and the dependencies it has. He seemed to be very interested in it but he was also worried about Xfce becoming less lightweight, eating up more memory and disk space. It wasn’t the first time during UDS that someone mentioned LXDE and asked me about my opinion on it. Two main reasons for Xfce being lightweight always were the small number of dependencies and applications which you could call the Xfce stack. With more and more applications and libraries being added, Xfce naturally feel less lightweight, even if these applications are not considered a part of the core desktop. You always have to keep that in mind. It seems like the high modularity of Xfce compensates parts of that and is why people are interested in Xfce in the context of mobile devices. Despite being modular and somewhat lightweight, Xfce also has improved in terms of user experience lately. I think we can push this even further. Giving accessibility more attention might also be worth considering. Usability is something I don’t really see in LXDE (yet). That’s what I usually tell people if they ask me for my opinion on it.
Limitations of the PulseAudio backend for GStreamer
I’m at the Ubuntu Developer Summit this week and Emmet Hikory just approached me with a question about the PulseAudio support in xfce4-mixer. The problem he has is that when he wants to plug in in a bluetooth headset PulseAudio might act weird and there’s only one way to fix this from the user’s point of view, which is to use pavucontrol or one of the other PulseAudio tools. Unfortunately those tools don’t provide a good user experience. Controls are given technical names and too much of the internal technical stuff is being exposed to the user.
So, his idea is that there has to be a user-friendly way (like using xfce4-mixer) to e.g. control the bluetooth headset after it has been plugged in.
I just took a quick look at the PulseAudio code shipped with gst-plugins-good and to me it seems to be in a pretty bad shape. Unless I’m mistaken it only exposes one track through the GstMixer interface: Master. So if you have several devices capable of audio playback/recording, like a normal sound card and a bluetooth headset, you have no control over which of them is being muted, used for recording or whatever.
What I would expect is to either have a list of tracks (one for each sink – and please give them user-friendly names!) exposed through the GstMixer interface or to have a switch for choosing the sink you want to control with the Master track. Without this, no mixer application is able to provide a user-friendly way to control PulseAudio – unless it implements its own PulseAudio support.
One major reason for rewriting xfce4-mixer based on GStreamer has always been to get rid of the need to maintain our own audio system backends. The old mixer had its own backends for ALSA, OSS and BSD and it really sucked.
So I’m hoping that maybe someone steps up to implement a proper PulseAudio backend for GStreamer, with tracks for all available audio sinks and streams. I think I’d prefer tracks over a simple switch because they allow for a more fine-grained configuration of your devices. Thanks Emmet for pointing that out.
December plans and recent Xfce developments
I’m currently planning the last few things for my trip to the UDS Jaunty, taking place in Mountain View, California from December 8th to 12th. It looks like Brian will pick me up at San Francisco International on the day of my arrival. We’ll probably have something to eat before he drops me off at the hotel. What could possibly be more awesome than that? I’m very excited to say at least. I’ve also received my hotel reservation already and it looks like I’ll have a twin room on my own for half of my stay.
Ok, back to some Xfce related topics. Last week I submitted our FOSDEM devroom request. We’ll be notified before 2008-11-30 whether we the request is accepted or not. Five days left until we’ll know more – keep your fingers crossed if you haven’t already!
Nick and I have recently started to fix bugs in Thunar. So far we’ve managed to fix and close about a dozen bugs I think. We’re forced to do this due to Benny’s absence but I have to say it’s actually quite a lot of fun to fix bugs in other people’s code! In the end we’ll both have better understanding of how Thunar works. This will soon give us the possibility to push things forward. Not only am I planning to replace thunar-vfs with GIO/GVfs after 1.0 is released along with Xfce 4.6; there are more things that need to be done: adding support for xfconf might be worth considering as Thunar is now one of the last components that still stores its configuration using XfceRc. Also, we could think about using libxfce4menu for detecting installed applications which was actually one of the most important use cases of it that we had in mind before I started writing that library. And of course the list of possible features and cleanups doesn’t end here …
I also have some bad news. It looks like we’re having a bit of a problem with our 4.6 release schedule again. Due to the delay of the second beta, we were forced to delay the third beta as well. Maybe we can sort of fix that by leaving out the third release candidate but we still have to discuss how to handle this.
Second beta, German translations, FOSDEM
Just in case some of you are wondering already – yes, we’re late with releasing the second beta of Xfce 4.6 by a few days. It’ll probably be released this weekend, so we won’t fall behind the schedule too much.
On Wednesday we had a very productive IRC meeting with the most active German translators being present. Plans we made on that evening include to clean up the team’s wiki page so that it only contains rules and hints that are directly useful for everyone. We also decided that we need a better communication channel than private mails or the Xfce i18n mailinglist, so we’re probably going to have a mailinglist focused on German translations. Other teams might have the same need, so we’d like to do it public and encourage other teams to do the same, maybe officially hosted on foo-projects.org.
Another idea was to take a look at some of the distributions shipping Xfce. Do they have a translation system on their own? Are there potential translators among the users/developers of these distributions? How can we motivate more people to take part in the translation process so that everyone is happy with the result?
We’ve decided to have another German team meeting next Wednesday at 21:00 CET (20:00 UTC). Feel free to join if you’re interested and your German is good.
Today, we had a little discussion about the upcoming FOSDEM in February 2009. Several of us have expressed an interest to go there. With 4.6 out of the door and 4.8 ahead February is a pretty interesting date for Xfce as a project. I’m sure we could come up with several nice presentations and discussions about the future if we were to apply for a developer room. Xfce has been part of FOSDEM in 2005 already but since then a lot of things have changed and many new developers have joined the project, so it would be a great opportunity to finally meet everybody and have a few pints together.
We haven’t decided on whether Xfce is going to apply yet as we’re planning to get the second beta released before. Deadline for developer room applications is 11/22, so we still have some time. Even better, the deadline for the schedule itself is three weeks before the actual event. Enough time to prepare some decent speeches.
I’m pretty excited about all this. There’s a quite a lot of movement in the Xfce community at the moment (and also, related to this movement, a lot of exciting things are happening for me, with UDS and all that). I feel that things are improving constantly in terms of development as well as organization and communication. Now all we need is a proper goal – and FOSDEM might be exactly what we need to discuss things like that.
2AM, time for bed. Have a nice day (or night).
Devhelp assistant for VIM
Yesterday I came across a blog post by Richard Hult. He imported something called “assistant” into Devhelp in trunk. This assistant is a small window displaying documentation for one function, class/object or macro. He also wrote some code so Emacs would automatically update this window while the user is typing/coding. I thought it would be nice to have this for VIM, so I hacked on a small plugin for it. You can see a demo here. I haven’t uploaded the source code yet, but it’ll either end up in Devhelp or on vim.org as soon as it is finished.
Edit: After some positive feedback, I have now filed a feature request for the plugin to be added to devhelp. If you’re interested in testing, go grab it from there. Please let me know about your ideas or bugs you have found.
Edit: The code has been imported into devhelp trunk.