Thoughts on GnuPG and automation

[Bjarni Runar Einarsson]

Hi Hans-Christoph!

Hans-Christoph Steiner <hans at guardianproject.info> wrote:
> With all the recent attention to GnuPG and Werner's work, I have begun to
> think about things differently.  GnuPG has an amazing security track record.
> It has had few serious security bugs, nothing even close to heartbleed that I
> know of, and yet it is core to providing security to GNU/Linux distros, as
> well as protecting people like Laura Poitras and Edward Snowden. So instead
> of complaining about the difficulties, I now try to think about whether such
> difficulties might actually be related to what makes GnuPG so solid.

Some of the more jaded will call this Stockholm syndrome. :-P

I don't agree with the voices that want to discard PGP and start from
scratch. There is valuable experience and maturity in this project,
which is why we care enough to complain when it is hard to work with.

> anyone interested in providing usable security needs to think hard about this.
> Sure we can make things easier to use, but it is a very slippery slope
> towards reducing security.

I really disagree with this. If a security tool is too hard to
understand, whether for a developer or user, then insecurity will be the
inevitable result: people will use it wrong. This is only in part
GnuPG's resposibility, most of the complexity is inherited from OpenPGP
and the fact that public/private key crypto and key management are just
very complex topics.

This is the one point where I agree with the voices calling for
abandoning OpenPGP entirely. It can well be argued that the whole
cryptoscheme has been field tested and proven too complex for humans to
use correctly. That's not exactly GnuPG's problem either, but these
voices are becoming louder and, increasingly, there is finally
competition in this space. The project will get marginalized if
usability is ignored completely.

Mailpile's attempt to make OpenPGP easy to use is us stubbornly trying
to prove that it can be done. But I'm only somewhat optimistic that
we'll succeed, and we'll only do so if we face reality and drop a large
number of the features that make PGP what it is - in particular the web
of trust and default trust model, and who knows what else. I don't mind
if that code exists inside GnuPG, but Mailpile is absolutely not going
to be using it.

> I also have to call out that part of the problem that mailpile is continuing:
> it is generally more fun to write code, rather than figure out someone else's
> library. That is especially true when its a complicated thing like GnuPG.
> But in order to have shared maintenance and work, we all need to take
> responsibility and try to build upon the work of others whenever possible.
> Mailpile did not do that, and instead wrote yet another incomplete
> python API for GnuPG.

Fair enough. We were in a hurry, and we probably did make a mistake
here. There is a reason why we haven't broken our library out and
published separately though: we do hope to tear it out and replace it
with something more standard down the line.

However, having done the work, I can now state with confidence that the
complexity of our library is not because GnuPG is doing complex things.
It is because the GnuPG command line interfaces for automation are
incomplete and very hard to work with. Libraries might be able to hide
this fact, but that doesn't make the problem go away - sysadmins and
scripters everywhere have to deal with this all the time. It's a real
burden and the source of many, many posts to this list and others.

It's easy for developers to lose sight of the fact that no matter how
many libraries exist, most people first encounter gpg at the command
line. Slowly expanding on that and automating what you have learned is
the Unix way, and it's a wonderful thing. Whatever the reason, GnuPG is
not very good at this today.

Unfortunately lots of existing code depends on these things staying
unchanged, quirks and all. So it may be too late to fix, realistically
speaking. Missing flags could be added, but cleaning up the stuff that
already exists may be impossible. If that's Werner's verdict, then I
totally understand and promise to stop complaining about it.

> Another possibility is making ASSUAN, the internal protocol between GnuPG
> components, the API instead of `gpg --json`. This only works on GnuPG 2.1, as
> far as I understand it, since in 2.1, even commands like gpg communicate with
> gpg-agent using ASSUAN, and it is actually gpg-agent that does all the work.

FWIW, I took a lok at Assuan a while back, and I really like it.
Replacing Assuan with JSON might help the project interface with the
rest of the world, but that's the only argument in its favour; Assuan is
definitely more suited to the things that GPG has to do. There is also a
nice little Python library for interacting with it, so if gpg-agent ends
up exposing an Assuan interface which can do all the stuff people do
with GPGME, then I'll be very happy to switch to that.

Very happy! :-)

> Contrary to the mailpile write-ups, I think that having all the work
> happen in gpg-agent makes sense, as long as there is a good API to it.

I think you misunderstood my complaint. I don't mind if the agent is a
persistance daemon that provides GPG-related services, that's all well
and good. It's good process separation and I have no problem with that.

My gripe with the agent, is the agent is controlling the UI of
authentication. This breaks Mailpile, and this is one of the key areas
where GnuPG crosses the imaginary line between library/utility and
"application". Fixing this was point 1. in my list of suggestions and explaining why it was necessary was the bulk of the post.

I took a look at the kludge Werner directed me towards in his previous
mail - as far as I can tell, I can't use it unless I run my own
dedicated gpg-agent with a custom configuration and custom keyring.
Because gpg-agent controls the UI.

I can do that. I can run a dedicated gpg-agent just for Mailpile and
have a dedicated keyring. That'll work fine for most people and some may
even prefer it. Only the folks on this list and folks that already have
PGP keys will be unhappy that suddenly they have two keyrings to worry
about, not just one... and for people wanting to keep their keychains in
sync, we will have made key management even harder than it already is.

As far as I can tell, those are the options available to me at the
moment, unless I just stick with gnupg 1.4.x.

Thanks for the comments,
 - Bjarni

-- 
Sent using Mailpile, Free Software from www.mailpile.is