[gpgme] generate a wheel of the python bindings

[Matt]

Thanks for the detail answer.

> With GPGME as a dependency ... Claws or Mutt/Neomutt?
Nope, just "alot" :) https://github.com/pazz/alot

Your mail cleared some of my doubts: Nixos is a source based
distribution with binary cache. I was trying to package the python
bindings separately than gpgme because it seemed cleaner. At the end
of the day though I just overcomplicated things. Looking into more
details building a wheel is not necessary either so I will just use
the upstream `make install`.

python seems to run some tests during the build but I couldn't find
the flag to prevent these ? I found
"--disable-gpgconf-test" "--disable-gpg-test" "--disable-gpgsm-test"
"--disable-g13-test" but  none disable the python tests.

Ideally I would make the tests run too (nixos rules don't make it
mandatory but it's encouraged) but iI wonder if the paths
"/build/gpgme-1.10.0/lang/python/tests/" are ok ? (it's unusual to see
a path starting with "/build" ).

=====
gcc -pthread -shared -lgcc_s -g -O2 -Wall -Wcast-align -Wshadow
-Wstrict-prototypes -Wformat -Wno-format-y2k -Wformat-security -W
-Wextra -Wbad-function-cast -Wwrite-strings
-Wdeclaration-after-statement -Wno-missing-field-initializers
-Wno-sign-compare -Wpointer-arith
python2-gpg/temp.linux-x86_64-2.7/python2-gpg/gpgme_wrap.o
python2-gpg/temp.linux-x86_64-2.7/python2-gpg/helpers.o
-L../../src/.libs
-L/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/lib
-lpython2.7 -o python2-gpg/lib.linux-x86_64-2.7/gpg/_gpgme.so
-L/nix/store/3m9br2anary4yssv6ag2kjh35i0qnhim-gpgme-1.10.0/lib -lgpgme
-L/nix/store/drlbifsp8ivs4fb3pk8sf2i58s82456y-libassuan-2.5.1/lib
-lassuan -L/nix/store/y92c6dahh8i7wjhlh71mj5c1225a5073-libgpg-error-1.27/lib
-lgpg-error
running build_py
copying gpg/core.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
copying gpg/results.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
copying gpg/util.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
copying gpg/__init__.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
copying gpg/callbacks.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
copying gpg/errors.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/validity.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/create.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/status.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/sigsum.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/event.py -> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/pk.py -> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/md.py -> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/protocol.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/import.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/keysign.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
copying gpg/constants/__init__.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants
creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/data
copying gpg/constants/data/encoding.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/data
copying gpg/constants/data/__init__.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/data
creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/keylist
copying gpg/constants/keylist/mode.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/keylist
copying gpg/constants/keylist/__init__.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/keylist
creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
copying gpg/constants/sig/notation.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
copying gpg/constants/sig/mode.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
copying gpg/constants/sig/__init__.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/tofu
copying gpg/constants/tofu/__init__.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/tofu
copying gpg/constants/tofu/policy.py ->
python2-gpg/lib.linux-x86_64-2.7/gpg/constants/tofu
make[4]: Leaving directory '/build/gpgme-1.10.0/lang/python'
Making all in tests
make[4]: Entering directory '/build/gpgme-1.10.0/lang/python/tests'
echo no-force-v3-sigs > ./gpg.conf
echo ignore-invalid-option agent-program >> ./gpg.conf
echo "agent-program `which gpg-agent`|--debug-quick-random" >> ./gpg.conf
/nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/bash:
which: command not found
echo pinentry-program /build/gpgme-1.10.0/tests/gpg/pinentry >gpg-agent.conf
gpgconf --kill all
/nix/store/cb3slv3szhp46xkrczqw7mscy5mnk64l-coreutils-8.29/bin/mkdir
-p ./private-keys-v1.d
for k in ../../../tests/gpg/13CD0F3BDF24BE53FE192D62F18737256FF6E4FD
../../../tests/gpg/76F7E2B35832976B50A27A282D9B87E44577EB66
../../../tests/gpg/A0747D5F9425E6664F4FFBEED20FBCA79FDED2BD
../../../tests/gpg/13CBE3758AFE42B5E5E2AE4CED27AFA455E3F87F
../../../tests/gpg/7A030357C0F253A5BBCD282FFC4E521B37558F5C; do \
          cp $k private-keys-v1.d/${k#../../../tests/gpg/}.key; \
        done
echo x > ./private-keys-v1.d/gpg-sample.stamp
gpg --batch --no-permission-warning \
               --import ../../../tests/gpg/pubdemo.asc
gpg: keybox '/build/gpgme-1.10.0/lang/python/tests/pubring.kbx' created
gpg: /build/gpgme-1.10.0/lang/python/tests/trustdb.gpg: trustdb created
gpg: key 2D727CC768697734: public key "Alfa Test (demo key)
<alfa at example.net>" imported
gpg: failed to start agent '|--debug-quick-random': No such file or directory
gpg: can't connect to the agent: No such file or directory
gpg: key FE180B1DA9E3B0B2: public key "Bob (demo key)" imported
gpg: failed to start agent '|--debug-quick-random': No such file or directory
gpg: can't connect to the agent: No such file or directory
....
gpg: Total number processed: 26
gpg:               imported: 26
make[4]: *** [Makefile:630: pubring-stamp] Error 2
make[4]: Leaving directory '/build/gpgme-1.10.0/lang/python/tests'
make[3]: *** [Makefile:471: all-recursive] Error 1
....
make: *** [Makefile:449: all] Error 2

2018-03-04 8:39 GMT+09:00 Ben McGinnes <ben at adversary.org>:
> On Sun, Mar 04, 2018 at 02:50:52AM +0900, Matt wrote:
>> Hi,
>>
>> I've been trying to package gpgme python bindings for nixos
>> (www.nixos.org) since it's a dependency of the mail reader I use
>> (alot) but I haven't succeeded yet.
>
> Okay.
>
> With GPGME as a dependency ... Claws or Mutt/Neomutt?
>
>> I manage to compile the python 2.7 bindings and to build a "wheel"
>> (as required by nixos, a wheel is a zip file replacing the older
>> "egg' binaries) but for some reason my wheel is empty ;/ (it has
>> egg-info but not the gpg package).
>
> This is pretty much always going to run into trouble with these
> bindings due to the nature of what they are.  Unlike other wrappers
> (e.g. python-gnupg) they're a bit more than simply wrapping the
> command line executables with subprocess invocations.
>
> The GPGME Python bindings are built against the version of GPGME it's
> released with; functions are linked to their C counterparts with SWIG
> via the gpgme.h header file for that version.  In order to have
> functional bindings in the python module, you must also have a
> functional copy of GPGME built at the same time, for the same platform
> (architecture and OS).
>
> Which means providing a separate, precompiled python binary like a
> wheel will almost always introduce unexpected errors and failures.
>
>> The -meager- result of my efforts is visible here
>> https://github.com/NixOS/nixpkgs/pull/30429#issuecomment-370126211 and
>
> Interesting.  I can definitely answer one part of it, though:
>
> The reason you ended up with an "empty" module was because you were
> just trying to build the module as a stand alone Python module.
> Without the rest of the GPGME C API to which it's bound, it is
> effectively useless.  With the two together, however, it provides a
> pythonic interface to the entire GnuPG suite of software and
> libraries.
>
> When GPGME is compiled it produces the gpgme.h file most relevant to
> that host (from src/gpgme.h.in) and once that's done the building and
> installation of the various bindings in lang/ are performed against
> that gpgme.h file.
>
>> Also I believe that if one wants to be able to run "setup.py install",
>> he will need a patch similar to:
>> https://github.com/teto/gpgme/compare/7da01c7352d41eb33e445968b248544d301588f9...nix
>
> No chance.  It'd introduce too many points of failure and an
> expectation that these bindings will behave the same way as PyNaCl or
> cryptography.py.
>
> Also, your repo there looks like an old release, somewhere in the beta
> period before 1.10.0 was released.  There have definitely been
> significant updates to both GPGME and the python bindings since then.
>
>> I've tried asking on #python but couldn't find a solution.
>
> Yeah, I saw that, but it seems I missed you by a couple of hours or
> so.
>
>> I really some skilled user can help solve that as I miss my
>> mailreader :)
>
> I'm not really familiar with NixOS, but from what I've read it's
> another Linux distribution with its own take on package management.
> If that system can handle both binary packages and compiling packages
> from source, like many other package managers, then it should be
> achievable.
>
> If it only has the facility to provide binary packages then you will
> need to build those binaries for every single architecture and OS type
> Nix supports in order to do so.  Either that or go bloat happy and try
> to support as many as possible.  This is very much *not* recommended,
> however, because it is quite possible that incorporating something for
> one set of architecture and system version could very easily cause
> problems with others.
>
> There are certainly other package managers which are already quite
> capable of correctly building GPGME and the Python bindings.  I build
> it regularly on OS X with a slightly mixed approach.  I use MacPorts
> to grab all the library dependencies and build from source (actually
> most of my ports are built from source, but not quite all); the gnupg2
> (i.e. GPG 2.2.x) has a slightly modified configuration (compared to
> the standard portfile) and then built from source.
>
> Then I build GPGME manually while referencing the dependencies in
> /opt/local, but installing to /usr/local (so it finds my preferred
> installation of Python 3 instead of the MacPorts one).  After running
> autogen.sh I use this configure command:
>
>     ./configure --prefix=/usr/local --exec-prefix=/usr/local \
>     --with-libgpg-error-prefix=/opt/local \
>     --with-libassuan-prefix=/opt/local --enable-maintainer-mode
>
> Followed by the usual make, make check, sudo make install (though you
> can use gmake instead if there's a difference on your system).
>
> I also adjust the portfile for Neomutt to use my manually built GPGME
> in /usr/local instead of the one kicking around in /opt/local (which
> is just there to tick the dependency boxes for other packages).
>
>
> Regards,
> Ben