[gpgme] generate a wheel of the python bindings

[Matt]

All problems disappeared with latest source. So everything is ok :)
Thank you once again

2018-03-06 3:15 GMT+09:00 Matt <mattator at gmail.com>:
> Thanks for the detail answer.
>
>> With GPGME as a dependency ... Claws or Mutt/Neomutt?
> Nope, just "alot" :) https://github.com/pazz/alot
>
> Your mail cleared some of my doubts: Nixos is a source based
> distribution with binary cache. I was trying to package the python
> bindings separately than gpgme because it seemed cleaner. At the end
> of the day though I just overcomplicated things. Looking into more
> details building a wheel is not necessary either so I will just use
> the upstream `make install`.
>
> python seems to run some tests during the build but I couldn't find
> the flag to prevent these ? I found
> "--disable-gpgconf-test" "--disable-gpg-test" "--disable-gpgsm-test"
> "--disable-g13-test" but  none disable the python tests.
>
> Ideally I would make the tests run too (nixos rules don't make it
> mandatory but it's encouraged) but iI wonder if the paths
> "/build/gpgme-1.10.0/lang/python/tests/" are ok ? (it's unusual to see
> a path starting with "/build" ).
>
> =====
> gcc -pthread -shared -lgcc_s -g -O2 -Wall -Wcast-align -Wshadow
> -Wstrict-prototypes -Wformat -Wno-format-y2k -Wformat-security -W
> -Wextra -Wbad-function-cast -Wwrite-strings
> -Wdeclaration-after-statement -Wno-missing-field-initializers
> -Wno-sign-compare -Wpointer-arith
> python2-gpg/temp.linux-x86_64-2.7/python2-gpg/gpgme_wrap.o
> python2-gpg/temp.linux-x86_64-2.7/python2-gpg/helpers.o
> -L../../src/.libs
> -L/nix/store/7b2z0vfbs9539ga4pxx5gmli47rz5y3n-python-2.7.14/lib
> -lpython2.7 -o python2-gpg/lib.linux-x86_64-2.7/gpg/_gpgme.so
> -L/nix/store/3m9br2anary4yssv6ag2kjh35i0qnhim-gpgme-1.10.0/lib -lgpgme
> -L/nix/store/drlbifsp8ivs4fb3pk8sf2i58s82456y-libassuan-2.5.1/lib
> -lassuan -L/nix/store/y92c6dahh8i7wjhlh71mj5c1225a5073-libgpg-error-1.27/lib
> -lgpg-error
> running build_py
> copying gpg/core.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
> copying gpg/results.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
> copying gpg/util.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
> copying gpg/__init__.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
> copying gpg/callbacks.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
> copying gpg/errors.py -> python2-gpg/lib.linux-x86_64-2.7/gpg
> creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/validity.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/create.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/status.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/sigsum.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/event.py -> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/pk.py -> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/md.py -> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/protocol.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/import.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/keysign.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> copying gpg/constants/__init__.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants
> creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/data
> copying gpg/constants/data/encoding.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/data
> copying gpg/constants/data/__init__.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/data
> creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/keylist
> copying gpg/constants/keylist/mode.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/keylist
> copying gpg/constants/keylist/__init__.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/keylist
> creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
> copying gpg/constants/sig/notation.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
> copying gpg/constants/sig/mode.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
> copying gpg/constants/sig/__init__.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/sig
> creating python2-gpg/lib.linux-x86_64-2.7/gpg/constants/tofu
> copying gpg/constants/tofu/__init__.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/tofu
> copying gpg/constants/tofu/policy.py ->
> python2-gpg/lib.linux-x86_64-2.7/gpg/constants/tofu
> make[4]: Leaving directory '/build/gpgme-1.10.0/lang/python'
> Making all in tests
> make[4]: Entering directory '/build/gpgme-1.10.0/lang/python/tests'
> echo no-force-v3-sigs > ./gpg.conf
> echo ignore-invalid-option agent-program >> ./gpg.conf
> echo "agent-program `which gpg-agent`|--debug-quick-random" >> ./gpg.conf
> /nix/store/zqh3l3lyw32q1ayb15bnvg9f24j5v2p0-bash-4.4-p12/bin/bash:
> which: command not found
> echo pinentry-program /build/gpgme-1.10.0/tests/gpg/pinentry >gpg-agent.conf
> gpgconf --kill all
> /nix/store/cb3slv3szhp46xkrczqw7mscy5mnk64l-coreutils-8.29/bin/mkdir
> -p ./private-keys-v1.d
> for k in ../../../tests/gpg/13CD0F3BDF24BE53FE192D62F18737256FF6E4FD
> ../../../tests/gpg/76F7E2B35832976B50A27A282D9B87E44577EB66
> ../../../tests/gpg/A0747D5F9425E6664F4FFBEED20FBCA79FDED2BD
> ../../../tests/gpg/13CBE3758AFE42B5E5E2AE4CED27AFA455E3F87F
> ../../../tests/gpg/7A030357C0F253A5BBCD282FFC4E521B37558F5C; do \
>           cp $k private-keys-v1.d/${k#../../../tests/gpg/}.key; \
>         done
> echo x > ./private-keys-v1.d/gpg-sample.stamp
> gpg --batch --no-permission-warning \
>                --import ../../../tests/gpg/pubdemo.asc
> gpg: keybox '/build/gpgme-1.10.0/lang/python/tests/pubring.kbx' created
> gpg: /build/gpgme-1.10.0/lang/python/tests/trustdb.gpg: trustdb created
> gpg: key 2D727CC768697734: public key "Alfa Test (demo key)
> <alfa at example.net>" imported
> gpg: failed to start agent '|--debug-quick-random': No such file or directory
> gpg: can't connect to the agent: No such file or directory
> gpg: key FE180B1DA9E3B0B2: public key "Bob (demo key)" imported
> gpg: failed to start agent '|--debug-quick-random': No such file or directory
> gpg: can't connect to the agent: No such file or directory
> ....
> gpg: Total number processed: 26
> gpg:               imported: 26
> make[4]: *** [Makefile:630: pubring-stamp] Error 2
> make[4]: Leaving directory '/build/gpgme-1.10.0/lang/python/tests'
> make[3]: *** [Makefile:471: all-recursive] Error 1
> ....
> make: *** [Makefile:449: all] Error 2
>
> 2018-03-04 8:39 GMT+09:00 Ben McGinnes <ben at adversary.org>:
>> On Sun, Mar 04, 2018 at 02:50:52AM +0900, Matt wrote:
>>> Hi,
>>>
>>> I've been trying to package gpgme python bindings for nixos
>>> (www.nixos.org) since it's a dependency of the mail reader I use
>>> (alot) but I haven't succeeded yet.
>>
>> Okay.
>>
>> With GPGME as a dependency ... Claws or Mutt/Neomutt?
>>
>>> I manage to compile the python 2.7 bindings and to build a "wheel"
>>> (as required by nixos, a wheel is a zip file replacing the older
>>> "egg' binaries) but for some reason my wheel is empty ;/ (it has
>>> egg-info but not the gpg package).
>>
>> This is pretty much always going to run into trouble with these
>> bindings due to the nature of what they are.  Unlike other wrappers
>> (e.g. python-gnupg) they're a bit more than simply wrapping the
>> command line executables with subprocess invocations.
>>
>> The GPGME Python bindings are built against the version of GPGME it's
>> released with; functions are linked to their C counterparts with SWIG
>> via the gpgme.h header file for that version.  In order to have
>> functional bindings in the python module, you must also have a
>> functional copy of GPGME built at the same time, for the same platform
>> (architecture and OS).
>>
>> Which means providing a separate, precompiled python binary like a
>> wheel will almost always introduce unexpected errors and failures.
>>
>>> The -meager- result of my efforts is visible here
>>> https://github.com/NixOS/nixpkgs/pull/30429#issuecomment-370126211 and
>>
>> Interesting.  I can definitely answer one part of it, though:
>>
>> The reason you ended up with an "empty" module was because you were
>> just trying to build the module as a stand alone Python module.
>> Without the rest of the GPGME C API to which it's bound, it is
>> effectively useless.  With the two together, however, it provides a
>> pythonic interface to the entire GnuPG suite of software and
>> libraries.
>>
>> When GPGME is compiled it produces the gpgme.h file most relevant to
>> that host (from src/gpgme.h.in) and once that's done the building and
>> installation of the various bindings in lang/ are performed against
>> that gpgme.h file.
>>
>>> Also I believe that if one wants to be able to run "setup.py install",
>>> he will need a patch similar to:
>>> https://github.com/teto/gpgme/compare/7da01c7352d41eb33e445968b248544d301588f9...nix
>>
>> No chance.  It'd introduce too many points of failure and an
>> expectation that these bindings will behave the same way as PyNaCl or
>> cryptography.py.
>>
>> Also, your repo there looks like an old release, somewhere in the beta
>> period before 1.10.0 was released.  There have definitely been
>> significant updates to both GPGME and the python bindings since then.
>>
>>> I've tried asking on #python but couldn't find a solution.
>>
>> Yeah, I saw that, but it seems I missed you by a couple of hours or
>> so.
>>
>>> I really some skilled user can help solve that as I miss my
>>> mailreader :)
>>
>> I'm not really familiar with NixOS, but from what I've read it's
>> another Linux distribution with its own take on package management.
>> If that system can handle both binary packages and compiling packages
>> from source, like many other package managers, then it should be
>> achievable.
>>
>> If it only has the facility to provide binary packages then you will
>> need to build those binaries for every single architecture and OS type
>> Nix supports in order to do so.  Either that or go bloat happy and try
>> to support as many as possible.  This is very much *not* recommended,
>> however, because it is quite possible that incorporating something for
>> one set of architecture and system version could very easily cause
>> problems with others.
>>
>> There are certainly other package managers which are already quite
>> capable of correctly building GPGME and the Python bindings.  I build
>> it regularly on OS X with a slightly mixed approach.  I use MacPorts
>> to grab all the library dependencies and build from source (actually
>> most of my ports are built from source, but not quite all); the gnupg2
>> (i.e. GPG 2.2.x) has a slightly modified configuration (compared to
>> the standard portfile) and then built from source.
>>
>> Then I build GPGME manually while referencing the dependencies in
>> /opt/local, but installing to /usr/local (so it finds my preferred
>> installation of Python 3 instead of the MacPorts one).  After running
>> autogen.sh I use this configure command:
>>
>>     ./configure --prefix=/usr/local --exec-prefix=/usr/local \
>>     --with-libgpg-error-prefix=/opt/local \
>>     --with-libassuan-prefix=/opt/local --enable-maintainer-mode
>>
>> Followed by the usual make, make check, sudo make install (though you
>> can use gmake instead if there's a difference on your system).
>>
>> I also adjust the portfile for Neomutt to use my manually built GPGME
>> in /usr/local instead of the one kicking around in /opt/local (which
>> is just there to tick the dependency boxes for other packages).
>>
>>
>> Regards,
>> Ben