Ruby for Web Development
I recently started a new web dev project, and decided to use it to better learn Ruby. However, I don't want to use Rails. I'd like to keep it simple. I also prefer to know a lot about the inner workings of a particular technology before I go and use a large framework that hides a bunch of details from me.
However, I want to use ActiveRecord.
So, ActiveRecord. I install it on my laptop with "emerge ruby-activerecord", and there I go. One "require 'activerecord'" in my script later, and, awesome, it starts working.
Then I start working on my web host (DreamHost, if you're wondering). It can't find ActiveRecord. But it's obviously installed, because I know DH supports Rails out of the box, and I don't think you can have an install of Rails without ActiveRecord. So I poke, and then realise it might be installed as a Ruby "gem." Ok, so I put a "require 'rubygems'" above my activerecord require. Nice, now it works. Then I think, well, what if I put this somewhere that doesn't require rubygems? Not hard to work around automatically:
``begin require 'activerecord' rescue LoadError require 'rubygems' require 'activerecord' end``
Nice, ok, that works. It's probably a foolish micro-optimisation, but whatever.
Then I notice... ugh, this is super slow. Even on the web host, it can take a good two seconds for the "require 'activerecord'" statement to execute. Yeah, I know, ruby is kinda slow. But 2 extra seconds each time someone hits basically any page of the website? Ugh.
So.... FastCGI. I know DH supports it, so I head over to the control panel and enable it for the domain I'm working on, and start googling around to figure out how to use FastCGI in a ruby script.
Unfortunately, there aren't too many resources on this. Fortunately I found a couple sample dispatch scripts, one of which I ended up basing mine off of.
But then there was a problem. Inside my app, I use ruby's CGI class to access CGI form variables and other stuff. Since the FastCGI stuff overrides and partially replaces ruby's internal CGI class, there's a problem. Doing "cgi = CGI.new" inside a ruby script that's being served through FastCGI throws a weird exception. But I wanted to try to retain compatibility for non-FastCGI mode. And I couldn't figure out how to get the 'cgi' variable from the ruby dispatch script into my app's script, since I was using 'eval' to run my script. The dispatch script I saw used some weird Binding voodoo. Up at the top level we have:
``def getBinding(cgi, env) return binding end``
I had no idea what that was doing, so I looked it up. Apparently the built-in "binding" function returns a Binding object that describes the current execution context, including local variables and function/method arguments. Ok, that seems really powerful and cool. So I look down to the sample dispatch script's eval statement, and I see:
``eval File.open(script).read, getBinding(cgi, cgi.env_table)``
Ok, so it appears it tries to eval the script while providing an execution context that contains just 'cgi' and one of its member vars. I only sorta understand this. So I ditched the "cgi = CGI.new" line in my app's script. But I got a NameError when just trying to use 'cgi'. Huh? What's going on? So I get rid of the getBinding() call entirely, and just let it use the current execution context, and suddenly everything works right. Weird.
Well, sorta. Now, remember, I want to preserve compatibility with running as a normal CGI. So the normal CGI needs to create its own 'cgi' object, but the FastCGI one should just use the one from the dispatch script. So I came up with this:
``begin
if !cgi.nil?
mycgi = cgi
end
rescue NameError
require 'cgi'
mycgi = CGI.new
end``
Ok, that seemed to work ok. After that block, 'mycgi' should be usable as a CGI/FCGI::CGI object regardless of which mode it's running under.
So I play around a bit more, and suddenly notice that my POST requests have stopped working. I dig into it a bit, and realise that my POST requests are actually just fine. What's happening is that, somehow, the FCGI::CGI object completely ignores $QUERY_STRING on a POST request, while ruby's normal CGI object will take care of it and merge it with the POST data variables. You see, to make my URLs pretty, I have normal URLs rewritten such that the script sees "page=whatever" in the query string. So when I did a POST, the page= would get lost, and so the POST would end up fetching the root web page rather than the one that should be receiving the form variables. I'm not sure if this is "normal" behavior, or if the version of the fcgi ruby module on DreamHost has a bug. Regardless, we need a workaround. So I go back to my last code snippet, and hack something together:
``begin
if !cgi.nil?
if cgi.env_table['REQUEST_METHOD'] == 'POST'
CGI.parse(cgi.env_table['QUERY_STRING']).each do |k,v|
cgi.params[k] = v
end
end
mycgi = cgi
end
rescue NameError
require 'cgi'
mycgi = CGI.new
end``
Ick. But at least it works.
So far, I'm liking ruby quite a lot. It's a beautiful language, and seems well-suited for this kind of work, especially since I want to get something that works up and running relatively quickly.
We'll see, however, how many more gotchas I run into.
Ruby for Web Development
I recently started a new web dev project, and decided to use it to better learn Ruby. However, I don’t want to use Rails. I’d like to keep it simple. I also prefer to know a lot about the inner workings of a particular technology before I go and use a large framework that hides a bunch of details from me.
However, I want to use ActiveRecord.
So, ActiveRecord. I install it on my laptop with “emerge ruby-activerecord”, and there I go. One “require ‘activerecord’” in my script later, and, awesome, it starts working.
Then I start working on my web host (DreamHost, if you’re wondering). It can’t find ActiveRecord. But it’s obviously installed, because I know DH supports Rails out of the box, and I don’t think you can have an install of Rails without ActiveRecord. So I poke, and then realise it might be installed as a Ruby “gem.” Ok, so I put a “require ‘rubygems’” above my activerecord require. Nice, now it works. Then I think, well, what if I put this somewhere that doesn’t require rubygems? Not hard to work around automatically:
begin
require 'activerecord'
rescue LoadError
require 'rubygems'
require 'activerecord'
endNice, ok, that works. It’s probably a foolish micro-optimisation, but whatever.
Then I notice… ugh, this is super slow. Even on the web host, it can take a good two seconds for the “require ‘activerecord’” statement to execute. Yeah, I know, ruby is kinda slow. But 2 extra seconds each time someone hits basically any page of the website? Ugh.
So…. FastCGI. I know DH supports it, so I head over to the control panel and enable it for the domain I’m working on, and start googling around to figure out how to use FastCGI in a ruby script.
Unfortunately, there aren’t too many resources on this. Fortunately I found a couple sample dispatch scripts, one of which I ended up basing mine off of.
But then there was a problem. Inside my app, I use ruby’s CGI class to access CGI form variables and other stuff. Since the FastCGI stuff overrides and partially replaces ruby’s internal CGI class, there’s a problem. Doing “cgi = CGI.new” inside a ruby script that’s being served through FastCGI throws a weird exception. But I wanted to try to retain compatibility for non-FastCGI mode. And I couldn’t figure out how to get the ‘cgi’ variable from the ruby dispatch script into my app’s script, since I was using ‘eval’ to run my script. The dispatch script I saw used some weird Binding voodoo. Up at the top level we have:
def getBinding(cgi, env)
return binding
endI had no idea what that was doing, so I looked it up. Apparently the built-in “binding” function returns a Binding object that describes the current execution context, including local variables and function/method arguments. Ok, that seems really powerful and cool. So I look down to the sample dispatch script’s eval statement, and I see:
eval File.open(script).read, getBinding(cgi, cgi.env_table)Ok, so it appears it tries to eval the script while providing an execution context that contains just ‘cgi’ and one of its member vars. I only sorta understand this. So I ditched the “cgi = CGI.new” line in my app’s script. But I got a NameError when just trying to use ‘cgi’. Huh? What’s going on? So I get rid of the getBinding() call entirely, and just let it use the current execution context, and suddenly everything works right. Weird.
Well, sorta. Now, remember, I want to preserve compatibility with running as a normal CGI. So the normal CGI needs to create its own ‘cgi’ object, but the FastCGI one should just use the one from the dispatch script. So I came up with this:
begin
if !cgi.nil?
mycgi = cgi
end
rescue NameError
require 'cgi'
mycgi = CGI.new
endOk, that seemed to work ok. After that block, ‘mycgi’ should be usable as a CGI/FCGI::CGI object regardless of which mode it’s running under.
So I play around a bit more, and suddenly notice that my POST requests have stopped working. I dig into it a bit, and realise that my POST requests are actually just fine. What’s happening is that, somehow, the FCGI::CGI object completely ignores $QUERY_STRING on a POST request, while ruby’s normal CGI object will take care of it and merge it with the POST data variables. You see, to make my URLs pretty, I have normal URLs rewritten such that the script sees “page=whatever” in the query string. So when I did a POST, the page= would get lost, and so the POST would end up fetching the root web page rather than the one that should be receiving the form variables. I’m not sure if this is “normal” behavior, or if the version of the fcgi ruby module on DreamHost has a bug. Regardless, we need a workaround. So I go back to my last code snippet, and hack something together:
begin
if !cgi.nil?
if cgi.env_table['REQUEST_METHOD'] == 'POST'
CGI.parse(cgi.env_table['QUERY_STRING']).each do |k,v|
cgi.params[k] = v
end
end
mycgi = cgi
end
rescue NameError
require 'cgi'
mycgi = CGI.new
endIck. But at least it works.
So far, I’m liking ruby quite a lot. It’s a beautiful language, and seems well-suited for this kind of work, especially since I want to get something that works up and running relatively quickly.
We’ll see, however, how many more gotchas I run into.
Minimal word processors
I've just discovered two very interesting minimal word processors. They're designed by writers, for writers. They aim to get out of the way and let you just write, with no distractions.
PyRoom
First is PyRoom, which relies only on pygtk. It's really quite minimal and not distracting in the slightest, and easily themeable. I like it a lot. I created an ebuild, available here. Thank goodness for distutils!
WordGrinder
Second is WordGrinder, an even more minimal application that's entirely console-based. Unfortunately, it uses some weird freaky build system called PrimeMover, and it's scary. I asked one of my fellow developers who maintains the Lua package about it, but he's never heard of it. There aren't any eclasses for dealing with any Lua build system.
According to WordGrinder's Readme, the PrimeMover setup should be pretty simple. However, take a look at the pmfile itself. Man, that's ugly. It looks like three hours of judicious sed usage within the ebuild. I can't see any other way to alter it to something sanely installable to /usr.
If anyone has any tips, I'm all ears. I've got a skeleton ebuild for WordGrinder available in my repository, but it really needs fleshing out. So, who's willing to help?
Hardware: graphics shuffle redux
In other news, I had a really fun time getting my ATI X1950 Pro to work (again) with a silent aftermarket cooler (AC Accelero v1 rev2) and the latest bleeding-edge radeon, mesa, and libdrm packages. The hardware mods were fun, but the software . . . well, that's a long story for next time. 
Desktop
Oh yes, and this month's Xfce desktop. Token uncluttered version here. All those artists in that Thunar window are amazing. You should be listening to them right now.
icons: Meliae-dust (needed something reddish)
gtk+: Rezlooks L & D
xfwm4: Rezlooks-gtk (yes, it is confusingly named)
background: The Empire (from pixelgirlpresents)
Fix bitmap^Whorrible fonts
You have horrible fonts in Firefox and don't know how to fix it? It is fairy simple:cd /etc/fonts/conf.dAnd this is it. Restart X.
cp ../conf.avail/70-no-bitmaps.conf .
FOSDEM schedule and graphical 4.6 installers
The FOSDEM schedule has been published. You can find the Cross Desktop room schedule here. There will be two talks about Xfce. Short descriptions are linked from the schedule. We’re also mentioned in the press release.
We will be a fairly large group of people (around at dozen at least, with most of the core developers being present) so it’ll definitely rock!
On the Xfce mailing list people have been asking about graphical installers for 4.6. I’m currently preparing them and there should be installers for Xfce 4.6 RC1, RC2 and the final release. The main installer is already working except for one thing that’s bugging me. Should be able to fix it though.
FOSDEM schedule and graphical 4.6 installers
The FOSDEM schedule has been published. You can find the Cross Desktop room schedule here. There will be two talks about Xfce. Short descriptions are linked from the schedule. We're also mentioned in the press release.
We will be a fairly large group of people (around at dozen at least, with most of the core developers being present) so it'll definitely rock! On the Xfce mailing list people have been asking about graphical installers for 4.6. I'm currently preparing them and there should be installers for Xfce 4.6 RC1, RC2 and the final release. The main installer is already working except for one thing that's bugging me. Should be able to fix it though.
./jamendo-player
I was enough bored from my (small) music collection that I wanted to listen to something new, without actually caring what it is. I know Jamendo has a big load of music with artists uploading music under creative commons licence. So I went on their website to have a look, and they have "widgets". They are a small flash application not bigger than 200x300, but I actually didn't find any link to open such a widget inside an external window, except some lost links on some pages. In conclusion it was enough annoying to get an external player that I fired up vim and wrote a 20 lines C program.
/* gcc `pkg-config --cflags --libs webkit-1.0` main.c -o jamendo-player */
#include <gtk/gtk.h>
#include <webkit/webkit.h>
#define URL_FORMAT "http://widgets.jamendo.com/fr/playlist/?playertype=2008&playlist_id=%s"
int main (int argc, char *argv[])
{
GtkWidget *window;
GtkWidget *webview;
gchar *url;
gchar *playlist_id = argv[1];
if (argc < 2)
{
playlist_id = g_strdup ("65196");
g_message ("Loading the default playlist %s", playlist_id);
}
url = g_strdup_printf (URL_FORMAT, playlist_id);
gtk_init (&argc, &argv);
window = gtk_window_new (GTK_WINDOW_TOPLEVEL);
g_signal_connect (window, "delete-event", G_CALLBACK (gtk_main_quit), NULL);
gtk_window_set_icon_name (GTK_WINDOW (window), "multimedia-player");
gtk_window_set_title (GTK_WINDOW (window), "Jamendo Player");
gtk_window_set_resizable (GTK_WINDOW (window), FALSE);
webview = webkit_web_view_new ();
gtk_widget_set_size_request (webview, 200, 300);
gtk_container_add (GTK_CONTAINER (window), webview);
webkit_web_view_open (WEBKIT_WEB_VIEW (webview), url);
gtk_widget_show_all (window);
gtk_main ();
return 0;
}
The result is pleasing, the flash application has nice colors, and is something I would never be able to do with GTK+ :-)
If you like the code, feel free to extend it and send patches, I will enjoy any contribution. For now I will just leave a desktop file inside my applications directory to open it when I am bored again of my music :-p
Update: I hacked a little on the program to handle the links inside the Flash application. This means it is possible to change the playlist, which was one the thing that started to annoy me very much.
However, this update requires WebKit 1.0.3 and GLib 2.16.
This time I'm making the code available here instead of pasting it as it has grown up to 120 lines.
Topics raised at UDS
Ok, this post is long overdue and has been around as a draft for more than a week now, so here it is. As readers of this weblog already know, I was at the Ubuntu Developer Summit from December 8th to 12th. We had quite a few discussions on topics I’m interested in outside the scope of the official sessions held at UDS. Let me just list them here in random order:
- Arnaud Quette told me about Buildbot which can be used to test your repository for compilation errors on various systems. I suppose it can be a real time saver if you want your project to compile on a variety of distributions and UNIX flavors and want immediate response as to whether compilation is going to succeed on these. That way you don’t have to wait for packagers and users to report compilation errors back to you and don’t have to enter that typical ping pong developer-reporter dialog. Buildbot not only catches compilation errors but is also capable of sending notification emails and provides detailed information about the compile log and so on. It’s implemented based on a server/client concept where each client tests compilation on one system and the server collects data from the clients. By using virtualization you can set up server and clients on the same machine. This would be very nice to have for Xfce but I suppose it would require another dedicated server just for running all these compilations. If anyone is interested in looking in to that, I’d be happy to establish contact to someone from Buildbot to provide you with more information.
As a funny coincidence, Frédéric Péters recently announced build.gnome.org which is a great example of Buildbot usage. - I talked to Christian Kellner about GIO/Gvfs a bit in order to get information on how remote/virtual filesystems work. One thing I’ve thought about is how to allow for a list of user-defined remote/virtual filesystems which show up in the side pane of Thunar. There are different approaches to that. One is to use bookmarks which seems to be what, according to Christian, Nautilus does right now. While this seems to work quite well it seems to confuse users a bit. Personally, I’d like to see them separated from bookmarks. There has been a proposal for Nautilus to make the side pane look better by using sections. One thing I could imagine would be to have a “Virtual Volumes” section listing the user-defined filesystems and provide some sort of GUI for creating/removing/editing these. From what I’ve heard there also is a third approach, which is to make remote filesystems to appear as fake volumes in HAL or DeviceKit. I’ll have to look into that in order to decide what the best way to go is, I guess.
- PulseAudio and GStreamer: We discussed the limited PulseAudio backend for GStreamer in a group of up to five people and agreed that it really needs improvement (as in more tracks have to be added to the GstMixer interface) if we don’t want users to be able to control PulseAudio through the mixer applications they know. PulseAudio-specific applications like pavucontrol are not really what we want them to use.
- Xubuntu remix for netbooks: During one of the Xubuntu sessions I talked to a member of the Ubuntu Mobile team about Xfce in general, the modularity of Xfce and the dependencies it has. He seemed to be very interested in it but he was also worried about Xfce becoming less lightweight, eating up more memory and disk space. It wasn’t the first time during UDS that someone mentioned LXDE and asked me about my opinion on it. Two main reasons for Xfce being lightweight always were the small number of dependencies and applications which you could call the Xfce stack. With more and more applications and libraries being added, Xfce naturally feel less lightweight, even if these applications are not considered a part of the core desktop. You always have to keep that in mind. It seems like the high modularity of Xfce compensates parts of that and is why people are interested in Xfce in the context of mobile devices. Despite being modular and somewhat lightweight, Xfce also has improved in terms of user experience lately. I think we can push this even further. Giving accessibility more attention might also be worth considering. Usability is something I don’t really see in LXDE (yet). That’s what I usually tell people if they ask me for my opinion on it.
Topics raised at UDS
Ok, this post is long overdue and has been around as a draft for more than a week now, so here it is. As readers of this weblog already know, I was at the Ubuntu Developer Summit from December 8th to 12th. We had quite a few discussions on topics I'm interested in outside the scope of the official sessions held at UDS. Let me just list them here in random order:
Buildbot
Arnaud Quette told me about Buildbot which can be used to test your repository for compilation errors on various systems. I suppose it can be a real time saver if you want your project to compile on a variety of distributions and UNIX flavors and want immediate response as to whether compilation is going to succeed on these. That way you don't have to wait for packagers and users to report compilation errors back to you and don't have to enter that typical ping pong developer-reporter dialog. Buildbot not only catches compilation errors but is also capable of sending notification emails and provides detailed information about the compile log and so on. It's implemented based on a server/client concept where each client tests compilation on one system and the server collects data from the clients. By using virtualization you can set up server and clients on the same machine. This would be very nice to have for Xfce but I suppose it would require another dedicated server just for running all these compilations. If anyone is interested in looking in to that, I'd be happy to establish contact to someone from Buildbot to provide you with more information. As a funny coincidence, Frédéric Péters recently announced build.gnome.org which is a great example of Buildbot usage.
Thunar and GIO
I talked to Christian Kellner about GIO/GVfs a bit in order to get information on how remote/virtual filesystems work. One thing I've thought about is how to allow for a list of user-defined remote/virtual filesystems which show up in the side pane of Thunar. There are different approaches to that. One is to use bookmarks which seems to be what, according to Christian, Nautilus does right now. While this seems to work quite well it seems to confuse users a bit. Personally, I'd like to see them separated from bookmarks. There has been a proposal for Nautilus to make the side pane look better by using sections. One thing I could imagine would be to have a "Virtual Volumes" section listing the user-defined filesystems and provide some sort of GUI for creating/removing/editing these. From what I've heard there also is a third approach, which is to make remote filesystems to appear as fake volumes in HAL or DeviceKit. I'll have to look into that in order to decide what the best way to go is, I guess.
PulseAudio and GStreamer
PulseAudio and GStreamer: We discussed the limited PulseAudio backend for GStreamer in a group of up to five people and agreed that it really needs improvement (as in more tracks have to be added to the GstMixer interface) if we don't want users to be able to control PulseAudio through the mixer applications they know. PulseAudio-specific applications like pavucontrol are not really what we want them to use.
Xubuntu Remix for Netbooks
During one of the Xubuntu sessions I talked to a member of the Ubuntu Mobile team about Xfce in general, the modularity of Xfce and the dependencies it has. He seemed to be very interested in it but he was also worried about Xfce becoming less lightweight, eating up more memory and disk space. It wasn't the first time during UDS that someone mentioned LXDE and asked me about my opinion on it. Two main reasons for Xfce being lightweight always were the small number of dependencies and applications which you could call the Xfce stack. With more and more applications and libraries being added, Xfce naturally feel less lightweight, even if these applications are not considered a part of the core desktop. You always have to keep that in mind. It seems like the high modularity of Xfce compensates parts of that and is why people are interested in Xfce in the context of mobile devices. Despite being modular and somewhat lightweight, Xfce also has improved in terms of user experience lately. I think we can push this even further. Giving accessibility more attention might also be worth considering. Usability is something I don't really see in LXDE (yet). That's what I usually tell people if they ask me for my opinion on it.