Google TV and Native Libraries
The Google TV runs a fairly unusual flavor of Android (at least the 2nd-gen ARM-based devices). I have a Sony Internet Player (not the Blu-Ray version), so what I’m about to write applies to that device, but maybe not any other, though it stands to reason that the other ARM-based GTVs are the same.
Phone-and-tablet Android doesn’t look much like a Linux desktop or server system. It uses the Linux kernel, to be sure, but a lot of the userspace libraries are custom. It even does not use Glibc, but a C library that Google wrote called Bionic. It’s fairly stripped down and lightweight, and while it implements most things you might need out of a standard libc, it does not pretend to be POSIX compliant.
Due to this, a Native Development Kit (NDK) is not yet available for the GTV. So the question remains: can we hack one together that works? The answer is… sorta.
From some simple investigation, I’ve learned that the Sony GTV is running a EGlibc 2.12.2, and probably a mostly-unmodified version of it. Someone with an @google.com email address stated that the reason for this was that they couldn’t get Chrome running against the Honeycomb version of Bionic.
With this knowledge in hand, I built a relatively standard arm-linux-gnueabi toolchain using crosstool-ng. Then I ‘adb pull’-ed the contents of /system/lib from my GTV and merged them with the new toolchain’s sysroot, copied some headers out of a stock NDK, and ended up with a sysroot that approximates what you’d find in platforms/ in a stock NDK, just without Bionic, and with EGlibc.
I didn’t get to modifying the NDK’s build system (it would need to be changed to find the new toolchain), so I built my native library manually, and got a simple “hello world” type app with a native lib. (It just calls a native method that returns a string, and displays the string on a label.)
One annoying thing is that the ABI string in the Sony GTV is set to “none”, so you have to unpack the APK, rename lib/armeabi-v7a/ to lib/none/, and repack and resign it. All of this means that this would be strictly hobbyist for now: no chance that you could distribute something in the Play Store. Not only does Google have to release an officially-working NDK, but they need to decide on an ABI string, and get Sony (etc.) to push updates out to their customers that update build.prop on the devices with the new ABI string.
There’s also the possibility that Google doesn’t want to create and officially support that much native drift between phone-and-tablet Android and GTV Android, and will wait until manufacturers are running a more-stock Android 4.x on GTV (that uses the 4.x version of Bionic) before releasing an NDK that works… in which case we’re at the mercy of Sony for updates, unless XDA or CyanogenMod wants to take a crack at it. My money’s on this scenario, unfortunately.
One of the main things people have been screaming for is a version of XBMC that runs on GTV. I have been able to get it to build using my hacked-together toolchain, but not actually to run. I ran into problems with runtime linking: the built binaries depend on a shared libstdc++ and libgcc_s, neither of which appear to be included on the GTV’s filesystem. I tried including them in the APK, but, weirdly, when the GTV unpacks the native libs from the APK at install time, it discards those two libraries. Static linking of those two may not be possible since XBMC’s APK includes a bunch of native libs. A possible solution would be to build all of libxbmc.so’s dependencies as static libs, and then just make one big static library.
But I haven’t had time to work on this over the past couple weeks…
Google TV and Native Libraries
The Google TV runs a fairly unusual flavor of Android (at least the 2nd-gen ARM-based devices). I have a Sony Internet Player (not the Blu-Ray version), so what I’m about to write applies to that device, but maybe not any other, though it stands to reason that the other ARM-based GTVs are the same.
Phone-and-tablet Android doesn’t look much like a Linux desktop or server system. It uses the Linux kernel, to be sure, but a lot of the userspace libraries are custom. It even does not use Glibc, but a C library that Google wrote called Bionic. It’s fairly stripped down and lightweight, and while it implements most things you might need out of a standard libc, it does not pretend to be POSIX compliant.
From some simple investigation, I’ve learned that the Sony GTV is running a EGlibc 2.12.2, and probably a mostly-unmodified version of it. Someone with an @google.com email address stated that the reason for this was that they couldn’t get Chrome running against the Honeycomb version of Bionic.
Due to this, a Native Development Kit (NDK) is not yet available for the GTV. So the question remains: can we hack one together that works? The answer is… sorta.
With this knowledge in hand, I built a relatively standard arm-linux-gnueabi toolchain using crosstool-ng. Then I ‘adb pull’-ed the contents of /system/lib from my GTV and merged them with the new toolchain’s sysroot, copied some headers out of a stock NDK, and ended up with a sysroot that approximates what you’d find in platforms/ in a stock NDK, just without Bionic, and with EGlibc.
I didn’t get to modifying the NDK’s build system (it would need to be changed to find the new toolchain), so I built my native library manually, and got a simple “hello world” type app with a native lib. (It just calls a native method that returns a string, and displays the string on a label.)
One annoying thing is that the ABI string in the Sony GTV is set to “none”, so you have to unpack the APK, rename lib/armeabi-v7a/ to lib/none/, and repack and resign it. All of this means that this would be strictly hobbyist for now: no chance that you could distribute something in the Play Store. Not only does Google have to release an officially-working NDK, but they need to decide on an ABI string, and get Sony (etc.) to push updates out to their customers that update build.prop on the devices with the new ABI string.
There’s also the possibility that Google doesn’t want to create and officially support that much native drift between phone-and-tablet Android and GTV Android, and will wait until manufacturers are running a more-stock Android 4.x on GTV (that uses the 4.x version of Bionic) before releasing an NDK that works… in which case we’re at the mercy of Sony for updates, unless XDA or CyanogenMod wants to take a crack at it. My money’s on this scenario, unfortunately.
One of the main things people have been screaming for is a version of XBMC that runs on GTV. I have been able to get it to build using my hacked-together toolchain, but not actually to run. I ran into problems with runtime linking: the built binaries depend on a shared libstdc++ and libgcc_s, neither of which appear to be included on the GTV’s filesystem. I tried including them in the APK, but, weirdly, when the GTV unpacks the native libs from the APK at install time, it discards those two libraries. Static linking of those two may not be possible since XBMC’s APK includes a bunch of native libs. A possible solution would be to build all of libxbmc.so’s dependencies as static libs, and then just make one big static library.
But I haven’t had time to work on this over the past couple weeks…
Amazon MP3 Downloader on 64bit Gentoo Linux
(This is going to be a bit of a narrative. If you’re impatient, scroll to the “How-To” section at the bottom of the post.)
Today I decided to download an MP3 album from Amazon. I actually wanted all the songs on the particular album, and noted that you save a couple bucks by downloading the whole album vs. downloading each individual song. So click through, and find that it requires the Amazon Download Manager thingo.
Lame, I think. I click the download link, and it actually gives me some buttons to choose from that have to do with Linux. Huh. Nice work. However, I assume it’s closed-source proprietary crap. But hey, why not give it a try.
My choices are two .deb files, one for Ubuntu Jaunty, and one for Debian 5, or two .rpm files, one for Fedora 11, and one for OpenSuse somethingorother.
So I grab the Ubuntu one. I’m on Gentoo, so I “emerge dpkg”, and then run “dpkg -x amazonmp3.deb .” in an empty directory. I look in usr/bin that it created, and find a binary. But it’s a 32bit binary.
Crap.
I run ‘ldd’ on it, and of course it needs gtk and a bunch of other stuff that I don’t have 32bit libraries for. Like boost.
So I think, ok, I can install a 32bit chroot. So I download the stage3 Gentoo i686 tarball and unpack it in /opt/gentoo32. I also “emerge schroot” in my regular system, and set that up. I recently found schroot while using Ubuntu at work. It’s awesome. Eventually the stage3 tarball finishes up, so I schroot into my new 32bit environment and “emerge libboost”. After it’s finished, I note a problem. Amazon’s binary is expecting some weird version of boost that has “gcc42″ in the filename. Also, I just installed boost 1.35, and it wants 1.34. Out of desperation, I try to make symlinks to the correct file names, assuming it probably won’t work. I was right: it doesn’t work. Balls.
Then I think… well, why not install a chroot of 32bit Ubuntu Jaunty? I “emerge debootstrap” and pray… yep, it’s in portage, excellent. I install it, schroot into it, and apt-get a bunch of the libboost packages, and… holy crap, it works.
How-To
Here’s a step-by-step. This is for Gentoo, but it should work equally well on any system where you can install debootstrap and chroot. The following instructions assume you’ve installed them already.
As root on the “main” system:
# mkdir -p /opt/ubuntu-jaunty-32 # debootstrap --arch i386 jaunty /opt/ubuntu-jaunty-32 http://archive.ubuntu.com/ubuntu/ # cat >/etc/schroot/chroot.d/jaunty32.conf [ubuntu-jaunty-32] type=directory description=Ubuntu Jaunty (32-bit) location=/opt/ubuntu-jaunty-32 priority=3 users=$USER groups=$USER,root root-users=root,$USER root-groups=root,wheel personality=linux32 run-setup-scripts=true run-exec-scripts=true aliases=jaunty [hit ctrl+d] # cat >/etc/schroot/mount-defaults /proc /proc none rw,bind 0 0 /proc/bus/usb /proc/bus/usb none rw,bind 0 0 /sys /sys none rw,bind 0 0 /dev /dev none rw,bind 0 0 /dev/pts /dev/pts none rw,bind 0 0 /dev/shm /dev/shm none rw,bind 0 0 /home /home none rw,bind 0 0 /tmp /tmp none rw,bind 0 0 /var/run/dbus /var/run/dbus none rw,bind 0 0 /usr/portage /usr/portage none rw,bind 0 0 [hit ctrl+d] # schroot -c jaunty (32bit)# groupadd -r staff (32bit)# apt-get update (32bit)# apt-get install sudo curl libglademm-2.4-1c2a xdg-utils libboost-date-time1.34.1 libboost-filesystem1.34.1 libboost-iostreams1.34.1 libboost-regex1.34.1 libboost-signals1.34.1 libboost-thread1.34.1 (32bit)# dpkg -i $DOWNLOAD_DIR/amazonmp3.deb
Note that I didn’t just type all this verbatim… this was just my memory of what I did. So if something’s missing, please leave a note in the comments and I’ll update this.
Next you can try running “amazonmp3″ to test it. If that works, you can go back out to your “real” system (just type “exit” in that shell), and do this:
# mkdir -p /usr/local/bin # cat >/usr/local/bin/amazonmp3 #!/bin/bash schroot -c jaunty -p /usr/bin/amazonmp3 -- "$@" [hit ctrl+d] # chmod +x amazonmp3
This creates a script in your “real” system that will chain to the 32bit amazonmp3 binary in your Jaunty chroot. You can set up your browser of choice to automatically launch this script when you download .amz files. At this point you can try running “amazonmp3″ from your “real” system (outside the chroot) and see if that works.
There are also a few other things:
- If you are running a system that uses GDM to log in, you may need to add /var/run/gdm to the bind mounts in the mount-defaults file or else apps started in your chroot won’t be able to connect to your X server.
- If you’re using non-default gtk2 theme engines on your “real” desktop”, you’ll need to install them in the chroot (via apt-get) as well. Well, ok, you don’t have to, but if you don’t, the Amazon app will look uglier than it should.
- Remember to always pass the “-p” option to schroot. If you don’t, the environment in the chroot will get reset, which means you’ll lose your DISPLAY env var, and you won’t be able to connect to the X server without manually setting it.
- When I first ran the apt-get command in the chroot, I got some weird errors at the end of the process about some packages not being configured because another package (gtk, I think) that it depended on hadn’t been configured yet. Just running the same apt-get command a couple times fixed it, strangely.
- Make sure your “real” system has /usr/local/bin in the PATH.
Anyhow, that’s it. Works like a charm.
Amazon MP3 Downloader on 64bit Gentoo Linux
Update 2010/12/04: A commenter recommended clamz instead. I’ve installed it but not yet tried it, but it’s certainly easier than installing another OS in a chroot.
(This is going to be a bit of a narrative. If you’re impatient, scroll to the “How-To” section at the bottom of the post.)
Today I decided to download an MP3 album from Amazon. I actually wanted all the songs on the particular album, and noted that you save a couple bucks by downloading the whole album vs. downloading each individual song. So click through, and find that it requires the Amazon Download Manager thingo.
Lame, I think. I click the download link, and it actually gives me some buttons to choose from that have to do with Linux. Huh. Nice work. However, I assume it’s closed-source proprietary crap. But hey, why not give it a try.
My choices are two .deb files, one for Ubuntu Jaunty, and one for Debian 5, or two .rpm files, one for Fedora 11, and one for OpenSuse somethingorother.
So I grab the Ubuntu one. I’m on Gentoo, so I “emerge dpkg”, and then run “dpkg -x amazonmp3.deb .” in an empty directory. I look in usr/bin that it created, and find a binary. But it’s a 32bit binary.
Crap.
I run ‘ldd’ on it, and of course it needs gtk and a bunch of other stuff that I don’t have 32bit libraries for. Like boost.
So I think, ok, I can install a 32bit chroot. So I download the stage3 Gentoo i686 tarball and unpack it in /opt/gentoo32. I also “emerge schroot” in my regular system, and set that up. I recently found schroot while using Ubuntu at work. It’s awesome. Eventually the stage3 tarball finishes up, so I schroot into my new 32bit environment and “emerge libboost”. After it’s finished, I note a problem. Amazon’s binary is expecting some weird version of boost that has “gcc42” in the filename. Also, I just installed boost 1.35, and it wants 1.34. Out of desperation, I try to make symlinks to the correct file names, assuming it probably won’t work. I was right: it doesn’t work. Balls.
Then I think… well, why not install a chroot of 32bit Ubuntu Jaunty? I “emerge debootstrap” and pray… yep, it’s in portage, excellent. I install it, schroot into it, and apt-get a bunch of the libboost packages, and… holy crap, it works.
How-To
Here’s a step-by-step. This is for Gentoo, but it should work equally well on any system where you can install debootstrap and chroot. The following instructions assume you’ve installed them already.
As root on the “main” system:
# mkdir -p /opt/ubuntu-jaunty-32 # debootstrap --arch i386 jaunty /opt/ubuntu-jaunty-32 http://archive.ubuntu.com/ubuntu/ # cat >/etc/schroot/chroot.d/jaunty32.conf [ubuntu-jaunty-32] type=directory description=Ubuntu Jaunty (32-bit) location=/opt/ubuntu-jaunty-32 priority=3 users=$USER groups=$USER,root root-users=root,$USER root-groups=root,wheel personality=linux32 run-setup-scripts=true run-exec-scripts=true aliases=jaunty [hit ctrl+d] # cat >/etc/schroot/mount-defaults /proc/procnonerw,bind 00 /proc/bus/usb/proc/bus/usbnonerw,bind00 /sys/sysnone rw,bind00 /dev/devnonerw,bind00 /dev/pts/dev/ptsnonerw,bind00 /dev/shm/dev/shmnonerw,bind00 /home/homenonerw,bind00 /tmp/tmpnonerw,bind00 /var/run/dbus/var/run/dbusnonerw,bind00 /usr/portage/usr/portagenonerw,bind00 [hit ctrl+d] # schroot -c jaunty (32bit)# groupadd -r staff (32bit)# apt-get update (32bit)# apt-get install sudo curl libglademm-2.4-1c2a xdg-utils libboost-date-time1.34.1 libboost-filesystem1.34.1 libboost-iostreams1.34.1 libboost-regex1.34.1 libboost-signals1.34.1 libboost-thread1.34.1 (32bit)# dpkg -i $DOWNLOAD_DIR/amazonmp3.deb
Note that I didn’t just type all this verbatim… this was just my memory of what I did. So if something’s missing, please leave a note in the comments and I’ll update this.
Next you can try running “amazonmp3” to test it. If that works, you can go back out to your “real” system (just type “exit” in that shell), and do this:
# mkdir -p /usr/local/bin # cat >/usr/local/bin/amazonmp3 #!/bin/bash schroot -c jaunty -p /usr/bin/amazonmp3 -- "$@" [hit ctrl+d] # chmod +x amazonmp3
This creates a script in your “real” system that will chain to the 32bit amazonmp3 binary in your Jaunty chroot. You can set up your browser of choice to automatically launch this script when you download .amz files. At this point you can try running “amazonmp3” from your “real” system (outside the chroot) and see if that works.
There are also a few other things:
-
If you are running a system that uses GDM to log in, you may need to add /var/run/gdm to the bind mounts in the mount-defaults file or else apps started in your chroot won’t be able to connect to your X server.
-
If you’re using non-default gtk2 theme engines on your “real” desktop”, you’ll need to install them in the chroot (via apt-get) as well. Well, ok, you don’t have to, but if you don’t, the Amazon app will look uglier than it should.
-
Remember to always pass the “-p” option to schroot. If you don’t, the environment in the chroot will get reset, which means you’ll lose your DISPLAY env var, and you won’t be able to connect to the X server without manually setting it.
-
When I first ran the apt-get command in the chroot, I got some weird errors at the end of the process about some packages not being configured because another package (gtk, I think) that it depended on hadn’t been configured yet. Just running the same apt-get command a couple times fixed it, strangely.
-
Make sure your “real” system has /usr/local/bin in the PATH.
Anyhow, that’s it. Works like a charm.