Searching the desktop with Pinot and Catfish
I was looking around for desktop search frameworks today, specifically something with a gtk frontend and that required the fewest resources to run.
I discovered Pinot, a dbus-based file index/monitor/search tool. It even comes with a minimal gtk+ interface. I found few reviews on Pinot, and even fewer recent reviews comparing it to other search frameworks like Strigi, Tracker, and Beagle. I also discovered Catfish, a lightweight frontend to several different search services. There’s not much out there on integrating Catfish and Pinot, so I forged ahead and wrote my own code, then did some trial-and-error experiments.
All ebuilds are available on my overnight overlay. Instructions for adding the overlay are on the wiki.
Writing the ebuilds
The only ebuild I found for Pinot is sadly out-of-date, and is completely incorrect. Also, it depends on libtextcat, and I never found an ebuild for that.
So, I wrote my own ebuilds for the latest versions of Pinot and libtextcat.
Not content with Pinot’s minimal gtk+ interface, I decided to try Catfish, a PyGtk frontend for several different search engines, including Pinot. Catfish is made by the same developer of Midori, a well-respected lightweight WebKit browser. While Catfish’s development has been stalled for two years, I figured it was worth a shot, since its user interface is friendlier than Pinot’s.
Catfish, like Pinot and libtextcat, is not in Portage, but there is an open bug for its inclusion. However, the ebuild for the latest version needed updating, as it didn’t include Strigi or Pinot. So I rewrote it and added descriptive metadata.xml entries for Catfish’s and Pinot’s USE flags.
There’s still a bit of work left on the Catfish ebuild, since there’s a QA warning about not leaving precompiled Python object files in /usr/share/catfish. However, the application itself works perfectly. Just need to clean up the install process so that the bytecode doesn’t clutter up the filesystem.
Pinot
On first run, Pinot will take a long time to index your files. I pointed it at my user’s /home/ directory, which contains 51,000+ files, totaling 9.3GB on a Reiser3 filesystem with tail enabled. That operation took probably half an hour, and that’s on a fast SSD! All of Pinot’s indexes and databases take up 455MB, bringing my total /home/ usage to about 9.7GB. Pinot typically used about 50% of my CPU while doing so, sometimes dropping down to the 20s and 40s.
However, since Pinot is on a fast SSD, and it’s running off a 2.3Ghz dual-core Athlon backed by 4GB RAM, I didn’t notice any performance hit while indexing. I’m not running any special kernels or schedulers (like BFS) either; just vanilla-source-2.6.35.4. There was no noticeable lag or slowdown, despite viewing two Thunar windows, working with four terminals, and browsing nine Firefox tabs. My system was only laggy when compiling Pinot and its dependencies.
Once my /home/ was indexed, I searched around. Queries were pretty much instantaneous. There’s no easy way to measure the speed of each query, since it’s much too fast to time with a stopwatch. That’s probably mostly because of the SSD — as it is, without a desktop indexer/search app, most similar queries take less than a second. Once the initial filesystem index is complete, Pinot drops back to just monitoring directories if you’ve told it to do so, relying on the inotify feature in the kernel. That drops CPU and memory usage to zero, as near as I can tell. Nice!
Pinot’s greatest advantage on my system, at least, is not its speed, but its usefulness for easily finding deeply buried files and folders.
Interestingly, even though Pinot by default is not supposed to index Git, CVS, or SVN repositories, it seems to ignore that setting. Searching for “catfish” turns up a document named catfish tricks and all the ebuilds and git logs that have “catfish” in the title. Apparently Pinot’s regex filter isn’t very reliable. I probably need to add in another asterisk to disable searching or indexing of any files within a git directory.
Catfish
Catfish mostly works as expected, though it defaults to using “find” rather than “pinot” as its search engine. I haven’t yet found a way to set it to use Pinot as the default search provider. Catfish is quick to load, and its layout is fairly intuitive. Sometimes, however, it will just stop working with Pinot, and even though Pinot has indexed my entire home directory, Catfish won’t return any search results, though I can get those results by using Pinot’s interface. The rest of the time it works great.
Besides offering a friendlier UI for searches, Catfish’s real strengths are its useful options, both for presentation and for tying in with my desktop’s filemanager. With a couple of commandline switches, Catfish can display thumbnails of various filetypes, use larger icons in search results, use various wrappers for opening and working with files, or even use powerful regex search methods. No, it won’t have the awesome preview capabilities of Gloobus, but you also don’t have to install all of Gnome to get similar features.
Right out of the box, Catfish will allow you to open files and folders obtained from your search results just by clicking them. I don’t know if that works for all filemanagers, but it works with Thunar, which is all I ask.
I like to use Catfish in combination with another powerful feature of Thunar: custom actions. Since Thunar lacks a built-in search bar (aside from a rudimentary go-to alphabetical list when you press a key), how do you integrate a search utility? One way is by adding search functions to the right-click menu.
- Open a Thunar window, and go to Edit -> Configure custom actions.
- Click the plus icon: +. Give the action a helpful title, description, and icon. “Search” is pretty standard among icon sets, so there should always be one available even when you change themes.
- Add the action command:
catfish --path=%F - Now go to the Appearance Conditions tab. I left the file pattern as * and checked all boxes, so that no matter where I browse or click, I can launch a Catfish search.
- Save the new action and exit Thunar. The next Thunar window you launch will let you right-click anywhere in the browser to open a Catfish search.
You can add any commandline switch you like to the catfish command; just run catfish --help to see the available options.
Thunar’s custom action feature is pretty nifty; there are all kinds of things you can put in the context menu. It comes with an example to open a terminal in the current directory. You can create actions to launch applications with a root prompt, convert one image type into another, play media, print or email documents, and more. If you can script it, you can write a trigger for it and stick it in the context menu. Just read the custom actions documentation for many more examples of what you can do with Thunar. Neat!
Looking forward
So, will I keep using Pinot and Catfish? Possibly. While I am leery of any process like Pinot that writes so often to my SSD, and I’m not at all happy with its database size compared to my actual directory size, I do like that it’s fast, and responsive. It doesn’t seem to have the huge memory leaks or lag that Strigi/Nepomuk do in KDE. In fairness, KDE is trying to get us to believe in the power of the “semantic desktop,” while Pinot and Catfish just want to create an easy frontend for finding stuff, without worrying about associating them with various files or activities.
As long as the database doesn’t get too much larger, or the indexing/monitoring services use too many resources, I’ll keep it around. I’ve got five+ years of accumulated files in various folders, with more constantly being loaded to and from offline backups. Pinot and Catfish can help with my hard drive spring cleaning, and help me locate stuff that I’ve just plain forgotten about. The older you get, the less you remember, right?
What I’d really like is a search bar built-in to Thunar, maybe in the upper right corner, backed by Pinot. That’d place everything I need right up front, without having to drill down through right-click menus.
* * *
Speaking of Thunar:
Do you use Thunar? Do you use Dropbox? Xfce developer Mike Massonnet posted a message to the xfce-dev list this morning with a link to a new project: Thunar Dropbox. It integrates the Dropbox service right into your favorite lightweight filemanager. No longer do you have to run Nautilus just to use Dropbox easily. Now you can use it within Thunar.