Thoughts on GnuPG and automation

[Hans of Guardian]

On Feb 27, 2015, at 1:19 PM, Bjarni Runar Einarsson wrote:

> Hi Hans-Christoph!
> 
> Hans-Christoph Steiner <hans at guardianproject.info> wrote:
>> With all the recent attention to GnuPG and Werner's work, I have begun to
>> think about things differently.  GnuPG has an amazing security track record.
>> It has had few serious security bugs, nothing even close to heartbleed that I
>> know of, and yet it is core to providing security to GNU/Linux distros, as
>> well as protecting people like Laura Poitras and Edward Snowden. So instead
>> of complaining about the difficulties, I now try to think about whether such
>> difficulties might actually be related to what makes GnuPG so solid.
> 
> Some of the more jaded will call this Stockholm syndrome. :-P
> 
> I don't agree with the voices that want to discard PGP and start from
> scratch. There is valuable experience and maturity in this project,
> which is why we care enough to complain when it is hard to work with.
> 
>> anyone interested in providing usable security needs to think hard about this.
>> Sure we can make things easier to use, but it is a very slippery slope
>> towards reducing security.
> 
> I really disagree with this. If a security tool is too hard to
> understand, whether for a developer or user, then insecurity will be the
> inevitable result: people will use it wrong. This is only in part
> GnuPG's resposibility, most of the complexity is inherited from OpenPGP
> and the fact that public/private key crypto and key management are just
> very complex topics.
> 
> This is the one point where I agree with the voices calling for
> abandoning OpenPGP entirely. It can well be argued that the whole
> cryptoscheme has been field tested and proven too complex for humans to
> use correctly. That's not exactly GnuPG's problem either, but these
> voices are becoming louder and, increasingly, there is finally
> competition in this space. The project will get marginalized if
> usability is ignored completely.
> 
> Mailpile's attempt to make OpenPGP easy to use is us stubbornly trying
> to prove that it can be done. But I'm only somewhat optimistic that
> we'll succeed, and we'll only do so if we face reality and drop a large
> number of the features that make PGP what it is - in particular the web
> of trust and default trust model, and who knows what else. I don't mind
> if that code exists inside GnuPG, but Mailpile is absolutely not going
> to be using it.

I think if you actually disagree, you are missing my essential point.  But my guess is that we agree more than disagree.  Of course, I entirely agree that bad usability can also make a technically secure system actually very insecure.  This point is not mutually exclusive with what I said, and indeed both must be taken fully into account.

So for example, we could make PGP email really easy if we consider only a mass surveillance threat model.  Just make all the processes transparent in the background: passphrases, key generation, downloading public keys, etc.  But that would produce a system that would not work for the highly targeted threat models like Edward Snowden and Laura Poitras.  They would need to use a specialized system, and that specialized system might then be a marker of suspicion (for example, lots of governments, including the NSA, already mark all PGP messages as suspicious).  That then makes it a lot harder for people who suddenly realize that they might be under scrutiny.

There is a big tide of thinking about usability of security tools.  This is a great thing, and we need lots of contributions. But what it happening far too often these days is that the new generation are trumpeting "ease of use" above all else.  We are seeing systems like keybase.io that make things really easy, but also expect users to upload their _private_ key to some alpha web service.  That is terrible security practice.

I've also been arguing that we need to make encryption much easier to do.  But we are failing as UX designers if we do not deeply understand the systems that are trying to make easier to use.  And that is why I advocate thinking long and hard about GnuPG, and what makes it hard to use, and what makes it secure.  Only then can we really make solid security usable.


>> I also have to call out that part of the problem that mailpile is continuing:
>> it is generally more fun to write code, rather than figure out someone else's
>> library. That is especially true when its a complicated thing like GnuPG.
>> But in order to have shared maintenance and work, we all need to take
>> responsibility and try to build upon the work of others whenever possible.
>> Mailpile did not do that, and instead wrote yet another incomplete
>> python API for GnuPG.
> 
> Fair enough. We were in a hurry, and we probably did make a mistake
> here. There is a reason why we haven't broken our library out and
> published separately though: we do hope to tear it out and replace it
> with something more standard down the line.
> 
> However, having done the work, I can now state with confidence that the
> complexity of our library is not because GnuPG is doing complex things.
> It is because the GnuPG command line interfaces for automation are
> incomplete and very hard to work with. Libraries might be able to hide
> this fact, but that doesn't make the problem go away - sysadmins and
> scripters everywhere have to deal with this all the time. It's a real
> burden and the source of many, many posts to this list and others.
> 
> It's easy for developers to lose sight of the fact that no matter how
> many libraries exist, most people first encounter gpg at the command
> line. Slowly expanding on that and automating what you have learned is
> the Unix way, and it's a wonderful thing. Whatever the reason, GnuPG is
> not very good at this today.
> 
> Unfortunately lots of existing code depends on these things staying
> unchanged, quirks and all. So it may be too late to fix, realistically
> speaking. Missing flags could be added, but cleaning up the stuff that
> already exists may be impossible. If that's Werner's verdict, then I
> totally understand and promise to stop complaining about it.

I strongly support an effort to make a clean, modern method of interacting with GnuPG.  Then we can make flexible, maintainable, and native-feeling GnuPG libraries.  I think that `gpg --json` encapsulates the idea quite well.  Then --with-colons can remain as is for compatibility.

The big question here is: who is going to do the work?  And I don't think it is a valid question to just punt and say "GnuPG has all this money now".  It is bigger than that.


>> Another possibility is making ASSUAN, the internal protocol between GnuPG
>> components, the API instead of `gpg --json`. This only works on GnuPG 2.1, as
>> far as I understand it, since in 2.1, even commands like gpg communicate with
>> gpg-agent using ASSUAN, and it is actually gpg-agent that does all the work.
> 
> FWIW, I took a lok at Assuan a while back, and I really like it.
> Replacing Assuan with JSON might help the project interface with the
> rest of the world, but that's the only argument in its favour; Assuan is
> definitely more suited to the things that GPG has to do. There is also a
> nice little Python library for interacting with it, so if gpg-agent ends
> up exposing an Assuan interface which can do all the stuff people do
> with GPGME, then I'll be very happy to switch to that.
> 
> Very happy! :-)

Werner wasn't too keen in the past, but maybe there are some new thoughts on this.  `gpg --json` seems like a worthwhile fallback for public ASSUAN.


>> Contrary to the mailpile write-ups, I think that having all the work
>> happen in gpg-agent makes sense, as long as there is a good API to it.
> 
> I think you misunderstood my complaint. I don't mind if the agent is a
> persistance daemon that provides GPG-related services, that's all well
> and good. It's good process separation and I have no problem with that.
> 
> My gripe with the agent, is the agent is controlling the UI of
> authentication. This breaks Mailpile, and this is one of the key areas
> where GnuPG crosses the imaginary line between library/utility and
> "application". Fixing this was point 1. in my list of suggestions and explaining why it was necessary was the bulk of the post.
> 
> I took a look at the kludge Werner directed me towards in his previous
> mail - as far as I can tell, I can't use it unless I run my own
> dedicated gpg-agent with a custom configuration and custom keyring.
> Because gpg-agent controls the UI.
> 
> I can do that. I can run a dedicated gpg-agent just for Mailpile and
> have a dedicated keyring. That'll work fine for most people and some may
> even prefer it. Only the folks on this list and folks that already have
> PGP keys will be unhappy that suddenly they have two keyrings to worry
> about, not just one... and for people wanting to keep their keychains in
> sync, we will have made key management even harder than it already is.
> 
> As far as I can tell, those are the options available to me at the
> moment, unless I just stick with gnupg 1.4.x.

My guess is that you can achieve whatever you want to achieve using a custom pinentry.  But my memory is rusting, and Abel Luck, the other main dev on the Android port, did the bulk of that work.

.hc