A part of my desk, with some of my benchmarking installation, banshee is the one in the back left
A part of my desk, with some of my benchmarking installation, banshee is the one in the back left, you can also have an idea of what is my next card on the benchmark list / Photo © JG Perrin

Following the success of my article on benchmarking microSD cards on the Raspberry Pi, I wanted to describe the methodology I used. It may sound boring to some, but it can be useful to others and it will be interesting to compare results based on a consistent approach and methodology.

Processes

I will now describe the processes I used in this benchmark. I will:

  • Write an image from my MacBook Air to the microSD card, this process includes getting the right image, writing it to the SD card.
  • Prepare the Raspberry Pi system, mainly the drive.
  • Run the tests.

Writing an Image

This test is about writing the image of the Raspbian Operating System to the microSD card. I did that using my MacBook Air.

Full specs:

MacBook Air (13-inch, Early 2015), MacBookAir7,2
Processor Name: Intel Core i7
Processor Speed: 2.2 GHz
Number of Processors: 1
Total Number of Cores: 2
L2 Cache (per Core): 256 KB
L3 Cache: 4 MB
Memory: 8 GB

MacOS X El Capitan
Version: 10.11.3

Or, if you prefer:
$ uname -a
Darwin jgp-MacBook-Air.local 15.3.0 Darwin Kernel Version 15.3.0: Thu Dec 10 18:40:58 PST 2015; root:xnu-3248.30.4~1/RELEASE_X86_64 x86_64

Step 1: Getting the Raspbian image

Get the latest release of Raspbian at https://www.raspberrypi.org/downloads/raspbian/
If you are a little geekier, you can try this on a terminal on your Mac (do not type the $, I leave them in the example to identify the command-line prompt):
$ url="https://downloads.raspberrypi.org/raspbian_latest";filename=$(curl -sIL $url | grep -oE 'Location: .*$' | tail -1 | sed 's/.*\///' | sed s'/.$//');curl -o "$filename" -#L $url
It will get you the latest version of Raspbian and name it appropriately. This step will only have to be done once, regardless of the number of microSD cards you want to benchmark.

Step 2: Unzip the Image

The image comes as a zip file. You need to unzip it to use it.

$ unzip 2016-02-26-raspbian-jessie.zip
Archive: 2016-02-26-raspbian-jessie.zip
  inflating: 2016-02-26-raspbian-jessie.img

This step will only have to be done once, regardless of the number of microSD cards you want to benchmark.

Step 3: Identifying the Disk #

Now, we need to prepare the drive for the benchmark. Let’s prepare the disk:

  • Insert the microSD in its adapter, then on the Mac.
  • In a terminal, then run: diskutil list
$ diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *500.3 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:          Apple_CoreStorage Macintosh HD            499.4 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1 (internal, virtual):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:                  Apple_HFS Macintosh HD           +499.1 GB   disk1
                                 Logical Volume on disk0s2
                                 05326665-4271-4911-AD28-D6293E83AA8B
                                 Unencrypted
/dev/disk6 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *62.6 GB    disk6
   1:               Windows_NTFS                         62.6 GB    disk6s1
  • Identify the disk (not partition) of your SD card e.g. disk6 (not disk6s1). It might not be disk6 on your system.

Note:

  • You will not have to do it several times in the same session. However, if you reboot or do some system management, the disk # may change.

Step 4: Prepare the Disk and Copy the Image

  • Unmount your SD card by using the disk identifier to prepare copying data to it: diskutil unmountDisk /dev/disk<disk# from diskutil>
$ diskutil unmountDisk /dev/disk6
Unmount of all volumes on disk6 was successful
  •  Copy the data to your SD card:sudo dd bs=1m if=image.img of=/dev/rdisk<disk# from diskutil>. We will time the operation, to get a first sense of the performance.
$ sudo time dd bs=1m if=2016-02-26-raspbian-jessie.img of=/dev/rdisk6

Notes:

  • This may result in an dd: invalid number ‘1m’ error if you have GNU coreutils installed. In that case you need to use 1M:sudo dd bs=1M if=image.img of=/dev/rdisk<disk# from diskutil>
  • This will take a few minutes, depending on the image file size. You can check the progress by sending a SIGINFO signal pressing Ctrl+T.
  • If this command still fails, try using disk instead of rdisk:
    e.g. `sudo dd bs=1m if=2016-02-09-raspbian-jessie.img of=/dev/disk6` or e.g. `sudo dd bs=1M if=2016-02-09-raspbian-jessie.img of=/dev/disk6`

Result should be something like:

Password:
 3936+0 records in
 3936+0 records out
 4127195136 bytes transferred in 186.948352 secs (22076660 bytes/sec)
      187.25 real         0.01 user         3.36 sys

We keep track of this measured time to make sure it is consistent. The time given on line 4 should be about the same as the real time on line 5.

Finally, unmount the drive so you can use it.

$ diskutil unmountDisk /dev/disk6
Unmount of all volumes on disk6 was successful

You can safely remove the card now.

You can start this operation with another card to test.

Preparing the Raspberry Pi Drive

For this benchmark, I used a Raspberry Pi 2, Model B. I nicknamed it “banshee”, as what is going to happen to it is not really nice.

Full specs:

Raspberry Pi 2, Model B
Processor Name: ARM Cortex-A7
Processor Speed: 900MHz
Total Number of Cores: 4
Memory: 1GB

Step 1: Full Update of the Pi

Before we start using the Pi, I will perform a full upgrade of the system. Your display might be different.

Update the system’s package information.

$ sudo apt-get update
Get:1 http://archive.raspberrypi.org jessie InRelease [13.2 kB]
Get:2 http://mirrordirector.raspbian.org jessie InRelease [15.0 kB]
Get:3 http://archive.raspberrypi.org jessie/main armhf Packages [139 kB]
Get:4 http://mirrordirector.raspbian.org jessie/main armhf Packages [8,963 kB]
Hit http://archive.raspberrypi.org jessie/ui armhf Packages
Ign http://archive.raspberrypi.org jessie/main Translation-en_GB
Ign http://archive.raspberrypi.org jessie/main Translation-en
Ign http://archive.raspberrypi.org jessie/ui Translation-en_GB
Ign http://archive.raspberrypi.org jessie/ui Translation-en
Get:5 http://mirrordirector.raspbian.org jessie/contrib armhf Packages [37.5 kB]
Get:6 http://mirrordirector.raspbian.org jessie/non-free armhf Packages [70.3 kB]
Get:7 http://mirrordirector.raspbian.org jessie/rpi armhf Packages [1,356 B]
Ign http://mirrordirector.raspbian.org jessie/contrib Translation-en_GB
Ign http://mirrordirector.raspbian.org jessie/contrib Translation-en
Ign http://mirrordirector.raspbian.org jessie/main Translation-en_GB
Ign http://mirrordirector.raspbian.org jessie/main Translation-en
Ign http://mirrordirector.raspbian.org jessie/non-free Translation-en_GB
Ign http://mirrordirector.raspbian.org jessie/non-free Translation-en
Ign http://mirrordirector.raspbian.org jessie/rpi Translation-en_GB
Ign http://mirrordirector.raspbian.org jessie/rpi Translation-en
Fetched 9,239 kB in 28s (323 kB/s)
Reading package lists... Done

Update the distribution.

$ sudo apt-get dist-upgrade
Reading package lists... Done

Building dependency tree

Reading state information... Done

Calculating upgrade... Done

0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

Upgrade the system and installed packages.

$ sudo apt-get upgrade
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
The following packages will be upgraded:
  raspi-config
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 14.3 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://archive.raspberrypi.org/debian/ jessie/main raspi-config all 20160210 [14.3 kB]
Fetched 14.3 kB in 0s (33.3 kB/s)
(Reading database ... 125978 files and directories currently installed.)
Preparing to unpack .../raspi-config_20160210_all.deb ...
Unpacking raspi-config (20160210) over (20160209) ...
Processing triggers for systemd (215-17+deb8u3) ...
Setting up raspi-config (20160210) ...

Step 2: Tell the Raspberry Pi to use the Full Disk Space

The image we downloaded is designed to fit on a 4GB microSD card. Therefore, when you use bigger cards, you need to tell the system to expand the file system to use all the space available. This can be specially useful when you are running benchmarks with large files.

I have not found this operation in command line (I have not looked that hard), so here is the graphical version:

Run:

$ sudo raspi-config
Raspberry Pi Configuration Tool (raspi-config) - Welcome Screen
Raspberry Pi Configuration Tool (raspi-config) – Welcome Screen
Raspberry Pi Configuration Tool (raspi-config) - Partition has been extended, confirmation screen
Raspberry Pi Configuration Tool (raspi-config) – Partition has been extended, confirmation screen
Raspberry Pi Configuration Tool (raspi-config) - Reboot
Raspberry Pi Configuration Tool (raspi-config) – Reboot

If you have have not rebooted at this point, reboot using:

$ sudo reboot
Connection to 10.0.100.167 closed by remote host.
Connection to 10.0.100.167 closed.

 

As a side note, you can see that, behind the scene, raspi-config is running fdisk and some things are being done…

Welcome to fdisk (util-linux 2.25.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): Disk /dev/mmcblk0: 59.5 GiB, 63864569856 bytes, 124735488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x3e7fd08e

Device         Boot  Start     End Sectors  Size Id Type
/dev/mmcblk0p1        8192  131071  122880   60M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      131072 8060927 7929856  3.8G 83 Linux


Command (m for help): Partition number (1,2, default 2):
Partition 2 has been deleted.

Command (m for help): Partition type
p   primary (1 primary, 0 extended, 3 free)
e   extended (container for logical partitions)
Select (default p): Partition number (2-4, default 2): First sector (2048-124735487, default 2048): Last sector, +sectors or +size{K,M,G,T,P} (131072-124735487, default 124735487):
Created a new partition 2 of type 'Linux' and of size 59.4 GiB.

Command (m for help): Disk /dev/mmcblk0: 59.5 GiB, 63864569856 bytes, 124735488 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x3e7fd08e

Device         Boot  Start       End   Sectors  Size Id Type
/dev/mmcblk0p1        8192    131071    122880   60M  c W95 FAT32 (LBA)
/dev/mmcblk0p2      131072 124735487 124604416 59.4G 83 Linux


Command (m for help): The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Device or resource busy

The kernel still uses the old table. The new table will be used at the next reboot or after you run partprobe(8) or kpartx(8).

After rebooting, you can check that the operation went well:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        59G  3.4G   53G   6% /

Step 3: Install the Test Tools

To test the drives, I will use fio. The tool is included in the Raspbian repository, but it is a rather old version and it is not reliable (I got some Bus Errors). Therefore, I suggest to install an updated version of fio. I also wrote a separate article on installing fio on a Raspberry Pi. Here is a summary.

$ mkdir -p /tmp/fio-testing/data
$ cd Documents
$ git clone --branch fio-2.6 https://github.com/axboe/fio.git fio-2.6
$ cd fio-2.6
$ ./configure
$ make
$ sudo make install
$ /usr/local/bin/fio -v
$ cd ..

The output of the version should be:

fio-2.6

Step 4: Download the Test Configuration Files

We are now ready to get the configuration files for the tests. I have saved them on a public github repository.

$ git clone https://github.com/JGPnet/net.jgp.benchmark.git
$ cd net.jgp.benchmark/fio4pi

Let’s look at a couple of files to understand how they work.

$ cat random-1reader-test.fio
; random read of 128mb of data

[random-read]
rw=randread
size=128m
directory=/tmp/fio-testing/data
  • The ; as the first character of the line is the marker for comments.
  • The name between square bracket ([…]) is the name of the process.
  • rw= indicates the type of process.
  • size=128MB (131072 bytes).
  • directory= working directory.

Run the Tests

You can now run the read tests:

$ /usr/local/bin/fio random-1reader-test.fio
$ /usr/local/bin/fio random-2readers-test.fio
$ /usr/local/bin/fio random-4readers-test.fio
$ /usr/local/bin/fio random-6readers-test.fio
$ /usr/local/bin/fio random-8readers-test.fio

…and the write tests:

$ /usr/local/bin/fio random-1writer-test.fio
$ /usr/local/bin/fio random-2writers-test.fio
$ /usr/local/bin/fio random-4writers-test.fio

Writing tests are a little more slower for 2 reasons: writing is always slower than reading on NAND and the cache is hit more frequently. Raspbian caches writes and  I did not want to find & deactivate the cache system. So I decided to use significantly bigger data set to test writes – this is why it makes them slower and longer to execute.

In the latest iteration of the test tools, I added a simple shell to automate & archive the test result:

$ ./autorun.sh {drive id} {platform}

Where:

  • Drive id is way I identify drives, you should see it associated to all benchmarks.
  • The platform code is rpi2 for Raspberry Pi 2 and rpi3 for Raspberry Pi 3.

It is then easy to add it to the git repository by:

$ git commit
$ git push

 

Further reading:

2 thoughts on “Benchmarking microSD on the Raspberry Pi, a Detailed Methodology

Leave a Reply