Linux SD Card Boot for RK3399 Firefly SBC

The Firefly-RK3399 can boot from either onboard eMMC, or a TF-Card Slot (microSDHC). The microSDHC is the more flexible for my usage.

I’m using a Kingston 128GB microSDHC card, this makes a good sized root filesystem for the Firefly-RK3399. However, it was not easy to find info on how to set this up with an updated Linux kernel and Ubuntu distro. I’ll want to update the kernel and device tree in future.

Updating Linux Kernel when booting from SD Card for Firefly-RK3399 SBC.

Firefly provide instructions on how to update the kernel on the internal flash, but none for SD Card boot.

From a discussion forum I could find an SD Card image for Ubuntu 16.04 image here, Firefly-rk3399_xubuntu1604_SD_201705151725.img.

Disk Layout

The SD Card has one partition with the root filesystem. The kernel is not included on that partition, it is located earlier in the disk.

fdisk -l Firefly-rk3399_xubuntu1604_SD_201705151725.img
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 34C89256-CF83-4475-A7CF-48F7BDA34490

Device                                           Start     End Sectors  Size Type
Firefly-rk3399_xubuntu1604_SD_201705151725.img6 114688 4560862 4446175  2.1G Microsoft basic data

The kernel files that need to be updated are resource.img and kernel.img. By looking for the magic numbers (RSCE and KRNL) these could be found on disk.

For reference these can be extracted from the image with these commands:_

resource.img at 0x100_0000 (locate 4 byte ID: RSCE )
kernel.img at 0x200_0000 (locate 4 byte ID: KRNL )
root filesystem at 0x380_0000 (from fdisk, 114688*512)

DD uses a block size of 512 bytes by default, so the images can be extracted with these commands:

dd if=Firefly-rk3399_xubuntu1604_SD_201705151725.img skip=32768 of=old-resource.img count=32768
dd if=Firefly-rk3399_xubuntu1604_SD_201705151725.img skip=65536 of=old-kernel.img count=40960
dd if=Firefly-rk3399_xubuntu1604_SD_201705151725.img skip=114688 of=old-rootfs.img

Write New Images

Update the SD Card, assuming it is mapped to /dev/sdX (e.g. /dev/sdb):

Write loader

The original image can be used to write the boot loader and GPT partition table.

sudo dd if=Firefly-rk3399_xubuntu1604_SD_201705151725.img count=32768 of=/dev/sdX
sudo fdisk -l /dev/sdX

Write the Kernel Image

The kernel images (resource.img and kernel.img) compiled from source are written to the next region.

sudo dd if=resource.img seek=32768 of=/dev/sdX
sudo dd if=kernel.img seek=65536 of=/dev/sdX         
sync

Setup the root filesystem

The root partition size in the original image is only a few GB. gparted can be used to resize it.

sudo gparted /dev/sdX

In the gparted GUI format /dev/sdX6, I used ext4 and resize to to fill the remaining SD Card size.

sudo mkdir /media/SD Card
sudo mount /dev/sdX6 /media/SD Card

Ubuntu 18.04 Root FS

Using the base distro image ubuntu-base-18.04.1-base-arm64.tar.gz. and following instructions such as here.

wget http://cdimage.ubuntu.com/ubuntu-base/releases/18.04/release/ubuntu-base-18.04.1-base-arm64.tar.gz
sudo tar xvzf <path to >/ubuntu-base-18.04.1-base-arm64.tar.gz /media/SD Card

Install modules.

cd <path to>/firefly-kernel
export ARCH=arm64
export CROSS_COMPILE=aarch64-linux-gnu-
make modules
sudo make modules_install INSTALL_MOD_PATH=/media/SD Card
find /media/SD Card/lib/modules/ -name source -or -name build -type l | xargs sudo rm -f

The results:

ls /media/SD Card/lib/modules/4.4.77/
kernel         modules.alias.bin  modules.builtin.bin  modules.dep.bin  modules.order    modules.symbols
modules.alias  modules.builtin    modules.dep          modules.devname  modules.softdep  modules.symbols.bin

Install firmware:

make firmware
sudo make modules_install firmware_install INSTALL_FW_PATH=/media/SD Card/lib/firmware/

Install headers:

sudo make headers_install firmware_install INSTALL_HDR_PATH=/media/SD Card/lib/firmware/

Copy the Firmware Images from the Rockchip/Firefly Image

The original SD Card image includes firmware for SoC modules and on board components. (e.g. the wireless lan). Copy those to the same location on the new root filesystem.

The wireless lan driver won’t start unless this is done.

mkdir rootfs-loop
sudo mount -o loop,offset=0x3800000 Firefly-rk3399_xubuntu1604_SD_201705151725.img rootfs-loop/
sudo cp -prf rootfs-loop/system /media/SD Card/

Setup QEMU chroot User Space Emulation

Using QEMU User Space Emulation and the Linux Binfmt Misc feature we can mount and run the arm64 root filesystem. QEMU will translate syscalls from emulated arm64 user space into the host linux.

sudo apt-get install qemu binfmt-support qemu-user-static

Check the user format is registered.

update-binfmts --display | grep aarch64
interpreter = /usr/bin/qemu-aarch64-static

That file needs to be available inside the chroot environment.

sudo cp /usr/bin/qemu-aarch64-static /media/SD Card/usr/bin/qemu-aarch64-static

Ubuntu 18.04 Root FS Update

The root filesystem can be modified locally and updated for remote login. Qemu allows us to chroot to the arm64 system.

sudo chroot /media/SD Card/
# uname -a
Linux xxxxxxxxxx 4.15.0-36-generic #39-Ubuntu SMP Mon Sep 24 16:19:09 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux

Setup the root and user account.

passwd root
useradd -G sudo -m -s /bin/bash xxxx
passwd xxxx

Setup the nameserver (using google for the example here) and update the apt database, install some essentials. (This allows chroot to resolve network addresses..)

echo nameserver 8.8.8.8 > /etc/resolv.conf
echo Firefly-RK3399 > /etc/hostname

Setup packages for basic no-gui system with remote access (again with thanks to here).

apt-get update
apt-get upgrade
apt-get install dialog perl # stop warnings during install
apt-get install locales
locale-gen "en_US.UTF-8
update-locale LC_ALL="en_US.UTF-8""
apt-get install sudo ifupdown net-tools ethtool udev wireless-tools iputils-ping resolvconf wget apt-utils wpasupplicant initramfs-tools
apt-get install openssh-server # so we can ssh in
apt-get install htop # This board has 6 cores, I'd like to see them
apt-get install emacs nano less

Exit the chroot

exit

Edit the network interface, /mount/SD Card/etc/network/interfaces

iface eth0 inet dhcp
 
allow-hotplug wlan0
auto wlan0
 
 
iface wlan0 inet dhcp
        wpa-ssid "ssid"
        wpa-psk "password"

Unmount the rootfs.

sudo umount /media/SD Card
sync

Place the microSDHC Card in the Firefly-RK3399 TF Card Slot and Boot

The wlan0 connects and the device can be ssh’ed to.

xxxx@Firefly-RK3399:~$ uname -a
Linux Firefly-RK3399 4.4.77 #3 SMP Thu Oct 11 22:43:17 ACDT 2018 aarch64 aarch64 aarch64 GNU/Linux
xxxx@Firefly-RK3399:~$ cat /proc/device-tree/model
Rockchip RK3399 Firefly Board (Linux Opensource)xxxx@Firefly-RK3399:~$
xxxx@Firefly-RK3399:~$ cat /proc/device-tree/name
xxxx@Firefly-RK3399:~$ cat /etc/debian_version
buster/sid
xxxx@Firefly-RK3399:~$ df -h .
Filesystem      Size  Used Avail Use% Mounted on
/dev/root       115G  879M  108G   1% /
xxxx@Firefly-RK3399:~$ dmesg | grep "Linux version"
[    0.000000] Linux version 4.4.77 (xxxx@xxxx-AERO-15WV8) (gcc version 7.3.1 20180425 [linaro-7.3-2018.05 revision d29120a424ecfbc167ef90065c0eeb7f91977701] (Linaro GCC 7.3-2018.05) ) #3 SMP Thu Oct 11 22:43:17 ACDT 2018