--weak-digest SHA1 causes significant slowdown in --check-trustdb (2.1.10)

[Daniel Kahn Gillmor]

On Wed 2016-01-06 00:05:32 -0800, Werner Koch <wk at gnupg.org> wrote:
> On Tue,  5 Jan 2016 17:42, dkg at fifthhorseman.net said:
>
>> Hm, the sig-cache definitely speeds things up, but it seems orthogonal
>> to the --weak-digest SHA1 situation:
>
> Thanks.  This is consistent with my tests although my test setting is
> simpler.  I need to setup a test similar to yours.

fwiw, my test setup has a fair number of local (non-exportable)
certifications as well.  (i haven't counted how many of them are made
with SHA1) I don't know whether that has any relevance to the tests,
though.

here's a separate test that i ran using keys from the debian-keyring
package:

---------------
export GNUPGHOME="$(mktemp -d)"
otrust="$GNUPGHOME/otrust.txt"
gpgconf="$GNUPGHOME/gpg.conf"
echo no-auto-check-trustdb > "$gpgconf"
cat /usr/share/keyrings/* | gpg2 --import
add_trust() {
  count="$1"
  level="$2"

  for fpr in $(gpg2 --fingerprint --with-colons | \
               awk -F:  '/^fpr/{ print $10 }' | \
               sort -R | head -n "$count"); do
    printf '%s:%d:\n' "$fpr" "$level"
  done
}
echo '# randomly-generated ownertrust' > "$otrust"
add_trust 1 6    >> "$otrust" # one ultimate
add_trust 2 5    >> "$otrust" # two full
add_trust 3 4    >> "$otrust" # three marginal
add_trust 4 3    >> "$otrust" # four never
add_trust 5 128  >> "$otrust" # five disabled keys
try() {
 rm -f "$GNUPGHOME/trustdb.gpg"
 gpg2 --import-ownertrust < "$otrust" 2>/dev/null
 printf "==> gpg2 %s --check-trustdb <==\n" "$*"
 time gpg2 "$@" --check-trustdb
}
try
try --weak-digest SHA1
try --no-sig-cache
try --no-sig-cache  --weak-digest SHA1
---------------

interestingly, this shows severe performance degradation (and a lot of
kernel-space activity) the --no-sig-cache --weak-digest SHA1 case, but
doesn't show any degradation in the plain --weak-digest SHA1 case for
me.

The random otrust.txt i generated was:

--------
2517B724C5F6CA9953296E612FF9CD59612616B5:6:
F962EF7FBF67B18C96108BFAE7E6F782283D6300:5:
B82CDC351FC8191C3B75290DBE5CF687DC5AB7C2:5:
9A303F9907A03230D188B75A2EF5710914C6A712:4:
35E876FAB4D3732E93B4D237631DE7553BE8AFD4:4:
D95C32042EE54FFDB25EC3489F2733F40928D23A:4:
6791403B68AE2690517C42EAE6FFF1E38DC968B0:3:
5919FC3BDC30CE4135F1DC070F5E1ED4664196E2:3:
AC297E5C46B9D0B61C717681D6D09BE48405BBF6:3:
4C8F6B0D121EB15F076FEB1704EE131AE6D621BE:3:
8A7F208C6D9E73291657414D2135D123D8C19BEC:128:
AC297E5C46B9D0B61C717681D6D09BE48405BBF6:128:
4D1B7796C03E0831991BDD423F490DEB871EF9FA:128:
BBBD45EA818AB86FF67E7285D3E17383CFA7FF06:128:
276D7B77B69B756C7CB68669DD29F88442839ED3:128:
--------

and, also interestingly, i note that --weak-digest SHA1 --check-trustdb
thinks it can reach one fewer key with --no-sig-cache  is set :/

==> gpg2 --weak-digest SHA1 --check-trustdb <==
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: Note: signatures using the SHA1 algorithm are rejected
gpg: depth: 0  valid:   1  signed:   9  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   9  signed: 283  trust: 9-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2016-01-15


==> gpg2 --no-sig-cache --weak-digest SHA1 --check-trustdb <==
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: Note: signatures using the SHA1 algorithm are rejected
gpg: Note: signatures using the MD5 algorithm are rejected
gpg: depth: 0  valid:   1  signed:   9  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   9  signed: 284  trust: 9-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2016-01-15

sorry for just reporting weird corner cases here without helping to fix
them.

thanks for looking into it,

     --dkg