python bindings for gpgme

[Ben McGinnes]

On Mon, Jun 06, 2016 at 05:20:47PM +0200, Justus Winter wrote:
> Hello Bernhard,
> 
> Quoting Bernhard Reiter (2016-06-01 17:54:43)
>> Am Mittwoch, 1. Juni 2016 16:15:04 schrieb Justus Winter:
>>>> so we probably should evaluate this.
>>>
>>> I just did, and as Werner noted, we are very well beyond the
>>> evaluation phase.
>> 
>> your evaluation looks a bit biased because you are defending
>> what you have currently implemented.
> 
> What else would I be defending?
> 
>> 
>>> SWIG requires *only* language-specific code.  In case you are curious,
>>> it is here:
>>> 
>> http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gpgme.git;a=blob;f=lang/python/gpgme.i;hb=HEAD
>> 
>> I still don't see that defying my argument that SWIG as an additional layer 
>> creates some overhead (learning python's c api and swig).
> 
> It defies your argument that using SWIG only pays off if you create
> bindings for more than one language, which can only be the case if
> there is a language-independend part to it, which there isn't.
> 
>>>> My argument is: First I have to understand python's c api with SWIG I
>>>> have to understand python's c api and SWIG (and its generated code).
>>>
>>> I disagree.
>>>
>>> With SWIG, you write functions translating c types to python types and
>>> back.  Any function provided by the c library is then automagically
>>> wrapped, with the data being converted by the functions you wrote
>>> *once*.  Add a new function to gpgme, and you likely don't need to do
>>> anything in pyme.
>> 
>> When debugging you need to fully understand the resulting c-code
>> and it was more complicated with SWIG (last time I've tried).
>> 
>>> With pygpgme, you write code translating the types forth and back
>>> *explicitly for every function you want to make available to python*.
>>> It is a huge overhead, and likely a source of many errors as that code
>>> is likely copied over and over again, without being properly
>>> understood.
>> 
>> I haven't checked how it is implemented in pygpgme, there will be typical C 
>> methods to avoid too much code duplication.
> 
> One of such typical methods is code generation, which is exactly what
> SWIG does.  In fact, pyme and pygpgme are very similar, they employ a
> compiled module at the core, but pyme uses SWIG to generate large
> parts of that code.
> 
>>> Let's look at the sizes:
>>>
>>> % sloccount pygpgme/src
>>> [...]
>>> 3037    src             ansic=3037
>>> % sloccount lang/python/helpers.* lang/python/gpgme.i lang/python/pyme
>>> 544     top_dir         ansic=544
>>> 497     pyme            python=497
>>>
>>> Based on that I gather that there must be three to six times as many
>>> bugs in pygpgme.
>> 
>> If it is mostly duplicated code, your estimation does not hold.
> 
> I doubt that.  In fact, I believe that copying code around increases
> the chance of errors, and that it leads to cargo-cult-style
> programming.  It also makes fixing these bugs hard.
> 
>>>> Comparing the operation and the results, pygpgme combination
>>>> into one call seems more pythonic to me. Over time this will add up to a
>>>> lot of more code.
>>>
>>> So let's combine these operations in pymes python shim.
>> 
>> Before the release? Or do we have two ways to achieve the same thing then?
> 
> I'm thinking of providing joined operations,
> op_decrypt_verify+op_decrypt_result+op_verify_result =>
> decrypt_verify.  So there would be a second set of operations.
> 
>>>> My main concern is the python API.
>>>
>>> I understand that, and I believe that providing a more pythonic api is
>>> easier with pyme, for reasons I repeatedly stated.
>> 
>> If it is done with SWIG or later with CFFI, I don't care.
> 
> Ok.
> 
>> We should look at the API ideas of pygpgme, I guess.
> 
> What ideas?  I'm tired of "make it more pythonic" with noone having a
> clear idea what that actually means.

Whereas my view of remaking things with Python following an overhaul
of GPGME was not so much "make it more Pythonic" (though if that
happened as well I wouldn't object) as much as it was "make it more
modular" and "make it kind of REST-like" (with all the relevant
caveats).

For example; using JSON for feeding in data types and receiving the
responses.  I wanted something which would be familiar enough to all
those web-weenies, Javanauts, Node-velopers and Mono-thoughts; at
least in relation to the data types they fed it and received from it
that they'd have to actually want to code an insecure thing to do
anything else.

Having something that could also be imported into other Python 3 code
is just a functional effect of using the language, not the end goal.
If it were the end goal then it'd be no different from any other
Python binding.

> I fear that it means something different for everyone.  So until
> someone here actually provides a constructive critique of the
> current pyme API,

Almost certainly.  Which is why I took my cue from what Werner
described last year as the intended aim of having an official API for
the API.

> I'm going to sand-off the edges that would annoy me the most.

Fair enough.

> I plan to extend this to every object implementing the buffer
> interface.  I also want to provide the compound operations, and work a
> little on the documentation.

Cool.  I think that currently the biggest impediment to pyme adoption
is the lack of usable documentation from Python coders' perspectives.
The documentation that does exist is essentially just the existing
GPGME documentation with a terse introduction and a couple of
examples.  It doesn't help that the examples don't really demonstrate
the two most common use cases in code: encrypting a thing and
decrypting that same thing.

There's a lot to be said for a gentler introductory document with some
basic examples in it, at least so people can wrap their brains around
what they're looking at.  The current code and documentation is
basically saying, "if you didn't start your coding career with C then
you're screwed."  So rather than learn C, people go and grab something
which might not be quite so secure (like all those awful javascript
"implementations" of OpenPGP ... HaneWIN, for instance).

Now python-gnupg has many flaws, but it does have fairly coherent
documentation with easily replicable code examples throughout the
entire thing.  By contrast, pyme has the GPGME documentation plus a
handful of introductory text paragraphs near the start with one
example inside it and no guarantees on what can be found in the
examples directory itself on account of every single example being
written by people who were *not* the original author (and said
original author washed his hands of the thing over a decade ago).

Meeting that particular "unspoken requirement" of most developers
these days is most likely to gradually lead to an ever increasing
uptake of GPGME based code.  So I guess I'd better pitch in with the
documentation efforts.  ;)


Regards,
Ben


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 630 bytes
Desc: not available
URL: </pipermail/attachments/20160608/a8aa30ec/attachment-0002.sig>