Python bindings HOWTO proof reader request

[Tobias Mueller]

hi,

On Mon, 2018-03-19 at 14:35 +1100, Ben McGinnes wrote:
> On Sat, Mar 17, 2018 at 11:24:47PM +0100, Tobias Mueller wrote:
> > I understand that the system's gpgme.h is needed for generating the
> > python bindings. But is that not very similar to other python
> > bindings
> > to (system wide) libraries on PyPI? What's making gpgme more special
> > than those?
> 
> Essentially it boils down to the dynamic generation of that header
> file when GPGME is compiled
But that's before someone installed the python bindings, right?

>  in conjunction with the dynamic generation
> of the SWIG bindings.  If gpgme.h shipped statically with the source
> then there'd be no problem, but that isn't the case here.
> 
But in case of source distribution, gpgme.h is on the target system, no?
And for a binary distribution, gpgme.h doesn't matter.

> > You don't really have to ship a gpgme.h, do you?
> 
> Yeah, actually we do since that's what GPGME itself depends upon.
wait. what?  Of course, gpgme itself does not depend on an existing
gpgme.h.  The python bindings do.

The source distribution of the Python bindings can probably pick the
gpgme.h up that is already present on the system.

One problem with that approach: When the python bindings make use of
features not present in the system's GPGME and thus try to access fields
or functions that do not exist.
But even then you could argue that having the source up on PyPI helps
people finding and installing the library.
A binary distribution would neatly solve this problem, assuming gpgme
itself being able to deal with older version of gnupg (which it
generally does).


> > Neither for source distribution (you pick the system's header) nor
> > for
> > binary distribution (the header is not needed at runtime, is it?). 
> > That's at least how to packages that I know do it.
> 
> You've just hit the nail on the head; these bindings don't behave the
> same way as the ones you're used to do.
> 
I still don't see why it wouldn't.


> > For binary distribution, you don't have to provide things for each
> > and every OS on each and every architecture under the sun. If you
> > provide only a small subset, you've helped already someone
> > installing the library.
> 
> No, but we are already aware of the range of architectures which
> actively use GnuPG components
Right. I assume that people on most of these architectures are happy to
compile their own stuff.  That's unlike people on consumer hardware.

> 
> Although there is another aspect to that, of course, and that is that
> with crypto projects there is a general preference to not ship
> binaries at all.
Sure.  Notice how I started my last email with "discussing the technical
bits".
Whether you (the general you, not you, personally) actually want to
distribute binaries via PyPI is a whole other set of questions. (hint:
I'm in favour of doing that)


> > That is to say, PyPI is quite happy to accept packages without
> > having binaries compiled for HURD on s/390 or BSD on a toaster. So
> > there is no technical limitation in providing only, say, Linux
> > binaries for x86 and x64.
> 
> As Jakob already pointed out, it's more accurate to say that those are
> the only options available for Linux binaries on PyPI.
You see what I did there.



> So the question then becomes, is that usage enough to justify a
> special binary built just for Linux x86-64 users or not?
I don't think it's needs to be special.  Otherwise, I think your
assessment is correct.
My naïve thinking is that the those users benefit the most from a binary
distribution on PyPI. Many others are comfortable with a binary
distribution via other channels (I'm thinking Windows here), source
distribution via PyPI (probably the BSD and other folks who are happy to
compile their own stuff), or source distribution via other channels
(MacOS probably).

And the hassle should be really small.  For a release, you probably do a
sdist and a bdist anyway to check whether everything builds correctly.
Publishing these is a "twine upload" away.




> > > By the way, each architecture there would mean *both* software
> > > (operating system) and hardware.
> > 
> > I guess it depends on what you want to achieve.  If you want to
> > cater for the HURD user with their s/390 server or the single
> > remaining person running Solaris on an UltraSPARC then yes, you'll
> > have trouble providing that.
> 
> And yet GnuPG has an extraordinarily wide customer base.  Recent posts
> to gnupg-devel and gnupg-users alone indicates there are still plenty
> of people using Solaris and GPG.
I appreciate that.
As by my last argument, these people are happy to provide the necessary
infrastructure for building from source.
The casual developer for Ubuntu apps, let alone their users, are
probably not.


> > PEP-513 is quite clear about what it expects and while producing
> > such a manylinux wheel is a bit of an effort, the number of
> > potential consumers might make up for it.
> 
> Yes, it might very well do so at that.  We'll certainly consider it
nice.

Thanks for this fruitful discussion,
  Tobi