Amazon's Kindle new Firmware 2.3

Amazon released firmware 2.3 a few days ago.

Before you can update to firmware 2.3 ; you *must* first uninstall all hacks previously installed ; in particular the screen saver hack and savory.

Other than the advertised new features such as PDF native reader ; what's more interesting (at least for me) is that the same kind of security that got added with Firmware 2.2 on the Kindle 2 International is now found on all Kindle model.

Each software installed on the Kindle, must be signed using Amazon's RSA key; which obviously we don't have.

Luckily, the same trick I did for the Kindle 2 International works fine with the 2.3 firmware...

You can find the new packager there:
http://www.avenard.org/kindle2/packager-0.7.tar.gz

I have changed the options format, so it's more in line with other unix command. Make sure to read the README.txt file included in the archive for the syntax.
In short, for firmware 2.3 ; you will want to use the --sign option (which will sign the packages using our own key) and the --ex (which will add auto-jailbreaking to the package)

For Kindle 2 US only users ; once you have updated your Kindle to firmware 2.3 ; the various hacks you enjoyed won't work anymore. You need to get new versions of those

The most popular hack is the screen saver one ; you can find an updated version that works with 2.3 (all kindle model) here:
http://www.avenard.org/kindle2/screensaverhack-0.1.tar.gz

USB network hack:
http://www.avenard.org/kindle2/usbnetwork23-0.9.tar.gz

Kindle 2 International packager; now even easier...

Well, curiosity got the better of me (and gone is my night of sleep).

And here is version 0.5 of the Kindle packager; almost fully rewritten.
http://www.avenard.org/kindle2/packager-0.5.tar.gz

What's new:
*Can now create packages for the Kindle 2 International that will install properly, even without having installed the previous freekindle package.

For that use the -k2iex flag.
-k2i will still generate packages requiring the freekindle package to have been installed first.

I suspect that developers would use the -k2i for testing (with the freekindle package installed), and when they are 100% confident and ready for release, us the -k2iex.

*Do not leave junk behind (like .dat, .sig files etc)... We now only use temporary files that get automatically deleted...

All other packages (usbnetwork, screen saver hack, savory) got repackaged using -k2iex : so installation is now much easier.
And are available as always:
http://www.avenard.org/kindle2/

Savory: Native PDF and mobile EPUB conversion for the Kindle 2 International

I've adapted Savory to work on the Kindle International.

Savory gives you the ability to convert PDF, OTF, HTML and other type directly on the Kindle.

It will soon be available on Savory main page as well as the google code page; but in the mean time you can get it there:
http://www.avenard.org/kindle2/Savory-0.07.tar.gz

Also included are the version for Kindle 2 and Kindle DX, though I haven't checked them (I'm fairly confident it will work fine)

What's new:
-New installer / uninstaller: now you can update directly to a new version without having to first uninstall the older version with the uninstaller that came with it. The uninstaller will also properly restore any changes done by the installer, even if you have installed other programs later on.

-The EXT3 binary image has been renamed to savory-image-007.ext3 in order to match the version of the package and to prevent complaining about issues because they were using an old disk image

How to use: make sure you read the official Savory page:
http://blog.fsck.com/2009/04/savory.html

To use: copy the file savory-image-007.ext3 in the system folder on your kindle.
Then copy the appropriate update_Savory-0.07 bin file for your kindle and go in Settings -> "Update Your Kindle"

To get Savory to convert your files, copy any 'epub','lit','odt','rtf','oebzip','fb2',or 'pdf' file in the document folder.
To start the native conversion ; you must unplug the USB cable from the Kindle.

Let me know if you have any issues...

Edit 2009-11-02: savory has been updated on the Google code site:
http://code.google.com/p/savory/

How to create packages for Kindle 2 international

---

Hi there.

Finally got it working.

As you may (or may not) ; loading software on the Kindle 2 International didn't work.

Using ;debugOn and ;dumpMessages ; I looked at what was going on:

091024:105149 EXT3 FS on mmcblk0p1, internal journal
091024:105150 system: I _otaupexec:def:processing update /mnt/us/update_test-k2i.bin
091024:105150 system: I _otaupexec:def:version is "FC02"
091024:105150 system: I _otaupexec:def:update image checksum OK
091024:105150 system: E _otaupexec:def:signature does not exist for "install.sh"
091024:105150 system: E _otaupexec:def:signature verification failed

The difference with earlier Kindle is that now each file in the package must be signed.
The signature is then checked against the RSA key installed in the ROM.

Unfortunately, there's no way to bypass that check .

So what we first do is install an extra RSA key in the device.
that's what the update file update_freekindle.bin is for.

update_freekindle.bin uses a "feature" (some would call it a flaw) of the tar compressor. Kindle binary package are just gzipped tar files slightly encrypted as discovered by Igor Skochinsky. When extracting the content of the update, the Kindle simply call tar on it.

Tar follows simlinks ; and we use this to write where we're not supposed to.

Copy the file update_freekindle.bin on the Kindle USB disk ; then:
go to HOME
MENU -> Settings
MENU -> Update Your Kindle
DO NOT SELECT RESTART, or you will brick your Kindle (though there's a way to get out of that infinite reboot)

That update will fail ! (Error 004). This is normal.

What this package does is add our key to the list of keys available. So next time the Kindle will check the signatures of the files we are installing, the verification process will succeed.

You only have to do this once...

In the future, installing packages will be just like with previous version of the Kindle.

Attached is the new Kindle packager kindle_update_tool.py. It has one extra flag for generating Kindle 2 International packages: -k2i. When creating an update file for the Kindle 2 International, now it will create the appropriate signatures.

Important points:
OpenSSL must be installed ; and the openssl binary must be in your path (This is the default on Linux or MacOS).

The file freekindle.pem is the RSA key used to sign and check the validity of the packages. Do not modify it. The file must be present in your current directory, along with the kindle_update_tool.py file.

(That's until I find a way to generate all the internal signature files within python rather than calling openssl directly.)

For safety purposes, here is the MD5 checksum of the keys, and of the package installing the keys:
MD5 (freekindle.pem) = ce9584e222f0f910d15b7b20bbaa6a34
MD5 (update_freekindle-k2i.bin) = f1da587b3f4b7feafbc2785f0b022620

Only packages signed with the freekindle.pem RSA key will work with the update_freekindle-k2i package.


If you look at this site:
http://www.avenard.org/kindle2/

You'll see some programs I've repackaged for the Kindle 2 International:
-usbnetwork tool, compiled for Kindle 2, Kindle DX and Kindle International... The source code also shows how to easily make such packages. This tool let you connect to your kindle via telnet or ssh, as well as tether the PC internet connection from the Kindle (check my previous blog entry to know more)
-The screen saver hack ; replace the default screen saver images with your own.
-Savory ; a program allowing you to read PDF file natively on your Kindle (make sure you read the author web page: http://blog.fsck.com/2009/04/savory.html)

Hope you like it, that was exciting !

Kindle's USB tethering part #2

Following my previous post.

here is a new version, much easier to setup.
No need to edit any files on the device anymore.

Get the file here:
http://www.mobileread.com/forums/showpost.php?p=617068&postcount=114

To install:
-----
Copy update_usbnetwork-k2.bin (for Kindle K2) or update_usbnetwork-dx.bin (for Kindle DX) to the root of the Kindle drive.

Menu -> Settings
Menu -> Update your Kindle

Warning: Do not reboot, if you do, you'll need to put the Kindle in recovery mode as explained here :
http://blogkindle.com/2009/03/kindle-2-recovery-mode/
------

To enable USB tethering, two possibilities.
1- Create a file AUTO on the Kindle drive in the usbnet directory.
Reboot the device (Menu -> Settings then Menu -> Reboot)

2- using ;DebugOn ; `usbNetwork. Unplug/Replug USB device

Once either of those option is done, telnet to the device (default IP is 192.168.2.2)
then type:
# /mnt/us/usbnet/bin/usbnet-enable

This command will set the DNS ot use OpenDNS, disable WAN access for browsing the Amazon Store
You'll need to reboot to be able to use the Kindle Store...

-----

To disable USB tethering, and revert to USB disk mode.
telnet to the device
# rm /mnt/us/usbnet/AUTO
# reboot

-----
To re-enable 3G access while keeping USB tethering:
telnet to the device
# /mnt/us/usbnet/bin/usbnet-disable
# reboot

-----
To re-enable 3G access and restore USB disk mode:
telnet to the device
# /mnt/us/usbnet/bin/usbnet-disable
# rm /mnt/us/usbnet/AUTO
# reboot

that's it... Enjoy

Kindle USB network tethering (Vista/Windows 7 version)

Getting it to work under Vista is very similar to getting it to work on the mac... though a tad more complicated.

My machine (Windows 7 beta) didn't come with RNDIS driver.
The easiest way to get those is installing the Windows Mobile drivers found here:
http://www.microsoft.com/windowsmobile/en-us/downloads/microsoft/device-center-download.mspx

(if using XP, download ActiveSync 4.5)

For the Kindle.
Get the binary file to re-enable usb networking there


Copied the file update_usbnetwork-k2.bin to the root of the Kindle mounted disk.

Then:
Home -> Menu -> Settings
Menu -> Update your Kindle

After the reboot, I edited the file usbnet/usbnetwork uncommenting the line:
route add default gw ${HOST_IP}
(remove the # in front of it).

For editing, make sure you are using a text editor that preserve the UNIX line-ending, I use the fantastic UltraEdit but there are many other options. Just do not use Windows' Notepad or Write. They will mess up the file and it won't work.

Eject the kindle, and unplug the USB cable.

Put the Kindle in debug mode:
Home.
Type any letter on the keyboard to start the Search mode.
Press DEL to blank the line.
type:
;debugOn
(you get the ; character in the SYM menu).

Type any letter on the keyboard to start the Search mode.
Press DEL to blank the line.
type:
`usbNetwork
(you get the ` character in the SYM menu, 2nd last character on the 3rd line).

Plug the USB cable.

Let Windows install the drivers for the newly found peripheral.

On my machine, the new ethernet connection was created as "Local Area Connection 3" and shows up as "DIRECTV HDPC20". YMMV

Go into Start -> Control Panel -> Network & Sharing Center.
Click on "Change Adapter Settings"

Do a right click on your main network adapter, click on the "Sharing" tab
Check the "Allow other network users to connect through this computer's Internet Connection"

Click "OK"
You will get prompted that the adapter interface will be changed to "192.168.0.1". That's okay, confirm.

Now go and edit the other network adapter "DIRECTV HDPC20" (or whatever it's called on your machine)
Click on the "Internet Protocol Version 4" and "Properties"
change the IP address from 192.168.0.1 to 192.168.2.1 , subnet mask 255.255.255.0.

Validate with OK.
---

Open a command line (windows + R, cmd)
(Note Telnet isn't activated by default in Vista / Windows 7, go into the Control Panel and select "Programs and Features", click "Turn Windows Features on or Off" and check "Telnet Client").
Another alternative is HyperTerminal from Hilgraeve...

telnet 192.168.2.2
[root@kindle root]#

Remount the root partition in read-write mode:
# mntroot rw

I configured the default DNS to use the OpenDNS servers:
# vi /etc/resolv.d/resolv.conf.default

nameserver 208.67.222.222
nameserver 208.67.220.220

(to enter edit mode in vi, press 'i', to save and quit: ':wq!')

# vi /opt/amazon/ebook/config/framework.mario.conf
change the line:
USE_WAN : true
into:
USE_WAN : false

create the file /etc/init.d/usbnet with:
--- Cut below
#!/bin/sh

_FUNCTIONS=/etc/rc.d/functions
[ -f ${_FUNCTIONS} ] && . ${_FUNCTIONS}

US_ENABLE=/mnt/us/usbnet/AUTO

case "$1" in

start)
if [ -f ${US_ENABLE} ]; then
/mnt/us/usbnet/usbnetwork
else
msg "not usbnet" I
fi
;;

stop)
;;
*)
msg "Usage: $0 {start|stop}" W >&2
exit 1
;;
esac

exit 0
--- Cut above

Make the file executable
# chmod +x /etc/init.d/usbnet

Create the link to active the init script
# ln -s /etc/init.d/usbnet /etc/rcS.d/S72usbnet


Now create the file AUTO in the usbnet folder:
# touch /mnt/us/usbnet/AUTO

reboot your Kindle (either using the reboot command, or by doing Menu -> Settings -> Restart.

That's it...

To revert the changes and make the Kindle seen as a USB storage device once again do the following:
telnet 192.168.2.2
# rm /mnt/us/usbnet/AUTO
# reboot

To activate it again, simply create a file AUTO in the usbnet folder. The file system is case insensitive. So AUTO or auto for the name of the file doesn't matter.
Then either reboot ; or enter debug mode (see above); and start `usbNetwork again

That's it... Enjoy.

Kindle USB network tethering

I got a Kindle last week.

So sweet, that unit alone made me wish I lived in the US... They have some pretty cool stuff over there.

If like me you live outside the US, you will miss some of the great features provided by the Kindle and its "Whispernet" wireless network.
I then read that there was a debugging option on the Kindle 2 letting you share the internet connection of your PC using the USB cable.
Unfortunately, I had upgraded my Kindle to firmware 2.03 which removed the ability to tether over USB.

Luckily, some bright people were at work and created a work around it...
I merely collected information found in forums into one easy (well I hope) tutorial.
Most of it came from this forum:
http://www.mobileread.com/forums/showthread.php?t=49350

So here are the steps for Mac OS (10.5 and 10.6).. Instructions for Windows Vista/7 to follow.

Get the binary file to re-enable usb networking there:
[url]http://www.mobileread.com/forums/showpost.php?p=498421&postcount=14[/url]

Copy the file update_usbnetwork-k2.bin to the root of the Kindle mounted disk.

Then:
Home -> Menu -> Settings
Menu -> Update your Kindle

After the reboot, I edited the file usbnet/usbnetwork uncommenting the line:
route add default gw ${HOST_IP}
(remove the # in front of it).

Eject the kindle, and unplug the USB cable.

Put the Kindle in debug mode:
Home
Type any letter on the keyboard to start the Search mode.
Press DEL to blank the line.
type:
;debugOn
(you get the ; character in the SYM menu).

Type any letter on the keyboard to start the Search mode.
Press DEL to blank the line.
type:
`usbNetwork
(you get the ` character in the SYM menu, 2nd last character on the 3rd line).

Plug the USB cable.

I'm on a Mac, using 10.6 (snow Leopard)
The interface wasn't added automatically, so I went into the System Preferences -> Network

I press the + button just above the lock
Selected the Interface "RNDIS/Ethernet Gadget" , I named it "Kindle"

I set the interface as follow:
Configure IPv4: Manually
IP Address: 192.168.2.1
Subnet mask: 255.255.255.0
Click Apply.

Then I went in Network Preferences -> Sharing
Configured the Internet Sharing
To share my Airport connection to "RNDIS/Ethernet Gadget" (I had two listed there, for me the first one worked)

Then I opened a Terminal (use in the Finder Command-Shift-U to get to the utility folder)
telnet 192.168.2.2
I was welcomed by the friendly prompt:
[root@kindle root]#

Remount the root partition in read-write mode:
# mntroot rw

I configured the default DNS to use the OpenDNS servers:
# vi /etc/resolv.d/resolv.conf.default

nameserver 208.67.222.222
nameserver 208.67.220.220

(to enter edit mode in vi, press 'i', to save and quit: ':wq!')

# vi /opt/amazon/ebook/config/framework.mario.conf
change the line:
USE_WAN : true
into:
USE_WAN : false

create the file /etc/init.d/usbnet with:
--- cut below
#!/bin/sh

_FUNCTIONS=/etc/rc.d/functions
[ -f ${_FUNCTIONS} ] && . ${_FUNCTIONS}

US_ENABLE=/mnt/us/usbnet/AUTO

case "$1" in

start)
if [ -f ${US_ENABLE} ]; then
/mnt/us/usbnet/usbnetwork
else
msg "not usbnet" I
fi
;;

stop)
;;
*)
msg "Usage: $0 {start|stop}" W >&2
exit 1
;;
esac

exit 0
--- cut above

Make the file executable
# chmod +x /etc/init.d/usbnet


Create the link to active the init script
# ln -s /etc/init.d/usbnet /etc/rcS.d/S72usbnet

Now create the file AUTO in the usbnet folder:
# touch /mnt/us/usbnet/AUTO

reboot your Kindle (either using the reboot command, or by doing Menu -> Settings -> Restart.

That's it...
Now whenever I plug my Kindle to my mac, I enjoy all the internet goodness.

To revert the changes and make the Kindle seen as a USB storage device once again do the following:
telnet 192.168.2.2
# rm /mnt/us/usbnet/AUTO
# reboot

To activate it again, simply create a file AUTO in the usbnet folder. The file system is case insensitive. So AUTO or auto for the name of the file doesn't matter.
Then either reboot ; or enter debug mode (see above); and start `usbNetwork again

ZFS: FreeBSD vs Solaris

Following up on the previous benchmark ; I wanted to check on the difference between FreeBSD and Solaris.
Sun being the inventor of ZFS and had plenty of time to fine tune Solaris for it, they had to have it better.

But by how much...

So I installed OpenSolaris 2009.06... Installation was smooth as, very easy... A tad lost after that however as this is very different to FreeBSD or Linux which I'm more familiar with.

Being able to read the ZFS created in FreeBSD was a bit complicated as for some weird reasons, Solaris was convinced the pool was using different partitions (even though the pool was created using entire disks).

So I ran iometer, and here are the results...
On the left FreeBSD 8.0-current (compiled without debugging option to speed things up)
on the right, OpenSolaris 2009.06
Write Sequential: 279MB/s - 289MB/s
Read Sequential: 320MB/s - 362MB/s
Write Random: 249MB/s - 282MB/s
Read Random: 181MB/s - 176MB/s

The use of memory caches is an interesting topic. Solaris seems to make much better use of the memory cache for longer as seen on the graph below, but FreeBSD has faster access to it. FreeBSD however didn't seem to use more than 1GB for write cache, even though 6GB was available. Overall when it comes to accessing the disks directly, Solaris is definitely faster.

Write:


Read:


No matter what, the motherboard I'm using has dual-gigabit NIC only, so the difference in speed isn't that much relevant... Being much more familiar with BSD, this is what I will continue to use, and no doubt, ZFS support will keep improving.

ZFS / RAIDZ Benchmarks

At work we needed a new storage/file server as our existing is starting to show sign of aging (it's been running for over 5 years now).

ZFS looked promising on paper. But I had to try.

So I got the following setup.

Lian Li PC343B Cube case, which can host up to 30GB.
Intel 77 920 processor, 6GB DDR3 in triple-channel configuration
Motherboard Gigabyte EX58-UD5P
6 x 2TB Western Digital RE4 drives (RE stands for Raid Edition)

OS Is FreeBSD 8.0-BETA2 (with debugging off) and disks are set in a RAIDZ configuration.

I have to say that so far it's runs quite well, but not as well as I would have thought.
over 275MB/s sequential write, and 310MB/s sequential read...
Invidual disk benchmarks showed that the Western Digital 2TB RE4 could achieve 100MB/s quite easily on its own. Yet, over RAIDZ, that's an average of 45MB/s per disk only.

Typical file server profile with file bench shows in excess of 180MB/s of random access (file server storage). I do not however fully rely on filebench as it had never been ported to FreeBSD and it took me over an hour to get it working on FreeBSD (filebench is a Sun package)







Here is a comparison with our previous file server, which was state of the art 5 years ago (3Ware 9500-S RAID controller,8x74GB 10,000rpm raptor disks, AMD Opteron 248) 98MB/s physical speed






ZFS has passed every attempts to crash it... However, FreeBSD doesn't handle hotplug properly.. So any change of drive due to failure will have to be done with the machine offline unfortunately.