benchmarking security tokens speed

[Antoine Beaupré]

Hi,

I'm in the process of reviewing performance of various security tokens
(the Yubikeys, the FST-01, Nitrokey), and I am getting somewhat
interesting (if not surprising) results:

-------------- next part --------------
A non-text attachment was scrubbed...
Name: results-16b-all.png
Type: image/png
Size: 24808 bytes
Desc: not available
URL: </pipermail/attachments/20170825/da0f46d1/attachment-0001.png>
-------------- next part --------------

Times are in seconds.

It looks like the Yubikey 4 is the fastest, being (only?) 10 times
slower than the CPU (i3-6100U). That slows down another order of
magnitude with 4096 keys. The NEO is as slow in 2048 as the 4 is in
4096, and of course doesn't support 4096 at all. The FST-01 is the
slowest of the bunch, taking more than a full second to kick decryption
in 2048bit RSA and 8 seconds in 4096 bits.

I'm looking for feedback on the results and the test procedure, which is
a Python script, attached. I'm aware of the limitations of the script,
namely that it treats the *whole* GPG decryption process as a blackbox,
which includes AES and all sorts of stuff. In my tests, GPG chooses
AES-256 which is why I chose a 16 bytes filesize. Since the timings
seems to be fairly consistent, I am assuming the delays are consistent.

Also, I was thinking of removing the file altogether and pipe
pseudo-random bytes in to remove possible disk contention issues, but my
test results are fairly consistent so I don't think that's necessary
either.

I'm also looking at getting my hands on Nitrokey hardware and adding
elleptic curve support to complete the test suite.

Any comments and help would of course be very welcome.

A.
-- 
Antoine Beaupr?
LWN.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bench-tokens.py
Type: text/x-python
Size: 14837 bytes
Desc: not available
URL: </pipermail/attachments/20170825/da0f46d1/attachment-0001.py>