TOFU code available

[Neal H. Walfield]

Hi!

I now have an almost complete version of the TOFU code available for
testing.  I've pushed it to the neal/next branch.  I'd appreciate if
people would try it.

  $ git clone git://git.gnupg.org/gnupg.git
  $ git checkout neal/next


The patch adds two new trust models: tofu and tofu+pgp (alias:
tofu+wot).  The first trust model just uses TOFU for determining
whether a key is trusted.  The second combines TOFU and the WoT.  The
basic idea is that we evaluate the key's trust in both models and then
if either deems the key to be untrusted (NEVER or EXPIRE) that is
returned.  Otherwise, the maximum of the two is returned (UNKNOWN <
UNDEFINED < MARGINAL < FULLY < ULTIMATE).


I've implemented TOFU as follows: when we observe a new binding (<key,
email>), we add it to the TOFU database.  If there is no other binding
with the email address, then there is no conflict and we just silently
add the binding to the TOFU database with the "auto" policy.
Consider:

  $ gpg2 --trust-model=tofu --verify EE37CF96-1.txt 
  gpg: Signature made Fri 18 Sep 2015 02:05:37 PM CEST using RSA key ID EE37CF96
  gpg: TOFU: New binding <testing (insecure!), 362D3527F53AAD1971AAFDE658859975EE37CF96>, no conflict.
  gpg: TOFU: Set trust policy for binding <362D3527F53AAD1971AAFDE658859975EE37CF96, testing (insecure!)> to auto.
  gpg: DBG: TOFU: Saving signature <362D3527F53AAD1971AAFDE658859975EE37CF96, testing (insecure!), oyOLc2uXxiTbOdZwSqlo03fxBX8TAhE8iNJkBlEaYso>
  gpg: Good signature from "Testing (insecure!)" [full]
  gpg: TOFU: auto (default trust: auto) binding: <362D3527F53AAD1971AAFDE658859975EE37CF96, testing (insecure!)>
  gpg: DBG: Already observed the signature <key: 362D3527F53AAD1971AAFDE658859975EE37CF96, testing (insecure!), time: 0x55fbfe11, sig: oyOLc2uXxiTbOdZwSqlo03fxBX8TAhE8iNJkBlEaYso, unknown>

There are 5 policies in all: good, unknown, bad, auto and ask.  It is
possible to manually set the policy as follows:

  $ gpg2 --tofu-policy good 2183839A
  gpg: Setting TOFU policy for 2183839A to good
  gpg: TOFU: Set trust policy for binding <96DCD49DA38D389927687ECB3C5A915F2183839A, testing (insecure!)> to good.

When a key's trust level is checked, the binding's policy is retrieved
and translated to a trust level.  There are three main scenarios:

  - If the policy is good, unknown or bad, then FULLY, UNKNOWN or
    NEVER is returned.

  - If the policy is auto, then we use the value of the
    --tofu-default-policy option as the policy.  (In other words, the
    default policy is determined lazily.)  This option can also either
    be good, unknown, bad, auto or ask.  If it is also auto (the
    default), then the good policy is used, i.e., FULLY is returned.

  - If the binding's policy is ask, then we ask the user how to
    continue.  If batch mode is enabled, then we act as if the unknown
    policy is in effect.

--tofu-default-policy is a powerful knob.  A common concern among
security sensitive users is that TOFU is too weak, because it
automatically trusts everyone.  But, TOFU can detect man-in-the-middle
attacks.  Although a careful use of the WoT can also prevent such
attacks, the WoT imposes a large overhead: secure communication is
often not possible until a physical meeting has occured and the user
must spend a lot of time not only collecting signatures, but also
curating their trusted introducers (gpg --key-edit KEYID; trust).
(Anecdotally, even those people who actively sign keys don't realize
they have to do this.)  By setting --tofu-default-policy to unknown,
we only use the TOFU data for negative assertions (i.e., conflicts)
and rely on the WoT for positive assertions.  Thus, TOFU can help even
the most paranoid without exposing them to additional risk.

When the TOFU policy detects a conflict (a new binding has the same
email address as some known binding), then we show a dialog to the
user describing the problem as well as some statistics about messages
verified with the keys.  Here's an example:

  $ gpg2 --trust-model=tofu --verify BC15C85A-1.txt 
  gpg: Signature made Fri 18 Sep 2015 02:06:17 PM CEST using RSA key ID BC15C85A
  gpg: TOFU: Set trust policy for binding <439D954F18F79CC4F71BED91CACED996BC15C85A, testing (insecure!)> to ask.
  gpg: DBG: TOFU: Saving signature <439D954F18F79CC4F71BED91CACED996BC15C85A, testing (insecure!), IupTST8BFF8+YL4ZTaQNyIZOXqcLbdwqNVaU2DubeNU>
  gpg: Good signature from "Testing (insecure!)" [unknown]
  Please indicate whether you believe the binding <testing (insecure!), 439D954F18F79CC4F71BED91CACED996BC15C85A> is legitimate (good) or a forgery (bad).
  
  Known user ids associated with this key:
    Testing (insecure!) (policy: ask)
  
  Statistics for keys with the email `testing (insecure!)':
    439D954F18F79CC4F71BED91CACED996BC15C85A (this key):
      1 message 13 days ago.
    362D3527F53AAD1971AAFDE658859975EE37CF96, which was judged to be ask:
      1 message 0 minutes ago.
      14 messages 13 days ago.
  
  Normally, there is only a single key associated with an email
  address.  However, people sometimes generate a new key if
  their key is too old or they think it might be compromised.
  Alternatively, a new key may indicate a man-in-the-middle attack!
  Before accepting this key, you should talk to or call the person
  to make sure this new key is legitimate.
  
  (G)ood/(A)ccept once/(U)nknown/(R)eject once/(B)ad? 

If gpg is in batch mode, then the dialog is not shown.  Instead, the
ask policy is associted with the binding and UNKNOWN is returned as
the trust value.

In addition to marking the current key as suspicious, the TOFU code
also changes the policy of any conflicting bindings to 'ask' if their
policy was 'auto'.  In this way, the next time the user verifies a
signature created by that key or encrypts to that key, they will be
presented with the above dialog.  This can be a bit confusing for a
key that has been apparently good for a long time.  It is probably a
good idea to indicate why the key's policy was changed to ask.
Currently, I don't do this.


It is possible to retrieve the current policy by listing the key in
--with-colons mode.  Note: this is only shown when the trust model is
TOFU or TOFU+PGP.  The policy is shown in the 18th field.  Since the
binding is between the key and email address, this is only relevant
for uid records.

  $ gpg2 --trust-model=tofu --list-keys --with-colons 2183839A
  gpg: TOFU: Known binding <96DCD49DA38D389927687ECB3C5A915F2183839A, testing (insecure!)>'s policy: good
  gpg: TOFU: Known binding <96DCD49DA38D389927687ECB3C5A915F2183839A, testing (insecure!)>'s policy: good
  tru:t:1:1442583132:1474113640:3:1:5
  pub:f:1024:1:3C5A915F2183839A:1442583126:1474119126::u:::scESC:::::::
  uid:f::::1442583126::82414861BDCC320E1812DDD1DCAD506DFCDC1347::Testing (insecure!)::::::::good:
->                                                                                          ****
  sub:f:1024:1:6C63BFF012636FCF:1442583126:1474119126:::::e::::::


In summary, the following new options are available:

  # Change the trust model.
  --trust-model=tofu|tofu+pgp|togu+wot
  # Determine the policy for bindings whose policy is auto.
  --tofu-default-policy=good|unknown|bad|auto|ask
  # Set the policy for one or more keys
  --tofu-policy=good|unknown|bad|auto|ask KEYID [KEYID...]

The TOFU DB is used when verifying (gpg --verify message.txt) and when
encrypting (gpg -e -r EE37CF96).


Note: the tofu data is stored in .gnupg/tofu.d.  There is one DB per
email address and one per fingerprint.  This will hopefully make
synchronizing TOFU data between multiple machines using something like
unison possible.


The main missing bit is the mechanism to normalize email addresses.
Currently, I simply ASCII lower-case the local part.  However, I want
to use the locale independent case folding data [1] and specially
handle gmail addresses (e.g., dots are simply ignored).  If you are
aware of existing code that would help me do either of these things,
I'd appreciate it if you'd let me know.  I'm also happy to hear
additional suggestions.  Now is the time since the interface can still
be easily changed.


Thanks!

:) Neal

[1] http://www.unicode.org/Public/UNIDATA/CaseFolding.txt