python-gpgme test suite failures [was: Re: begging for pyme name change]

Daniel Kahn Gillmor dkg at fifthhorseman.net
Thu Oct 27 01:47:11 CEST 2016 

Hi James and others interested in python GnuPG bindings--

On Wed 2016-10-19 06:36:37 -0400, James Henstridge wrote:
> The test suite covers most of the module's API, but in parts depends
> on the behaviour of GPG itself (in the past I've had to patch it after
> gpg upgrades).  So a good approach would be to check that it passes
> with current gpg/gpgme

I started looking into this, and i find that it doesn't actually work
with the current gpg 2.1 :/

Running from the current bzr head ("Allow building with gpgme versions <
1.4.") against gnupg 2.1.15-6 and libgpgme11 1.7.0 (in debian unstable),
i see a bunch of errors (below).  some similar ones have also been
reported on the debian build systems here:

    https://bugs.debian.org/835174

I fixed a few of these test suite discrepancies back in February
(https://bugs.debian.org/797776) but it seems that more have arisen
since then.

This makes it difficult for me to even start with the rest of the
prospective API unification process.

Are these errors things that you might be up for fixing upstream?

If not, I'm starting to think that python-gpgme2 (along with deprecation
of all the other bindings) would be the simplest approach toward
cleaning up the python ecosystem here.

    --dkg

test suite failures below:

0 dkg at alice:~/src/python-gpgme/pygpgme$ env -i make check < /dev/null
python setup.py build_ext -i
running build_ext
GPG_AGENT_INFO= python test_all.py -v
test_armor (tests.test_context.ContextTestCase) ... ok
test_constructor (tests.test_context.ContextTestCase) ... ok
test_include_certs (tests.test_context.ContextTestCase) ... ok
test_keylist_mode (tests.test_context.ContextTestCase) ... ok
test_passphrase_cb (tests.test_context.ContextTestCase) ... ok
test_progress_cb (tests.test_context.ContextTestCase) ... ok
test_protocol (tests.test_context.ContextTestCase) ... ok
test_set_engine_info (tests.test_context.ContextTestCase) ... ok
test_textmode (tests.test_context.ContextTestCase) ... ok
test_key1 (tests.test_keys.KeyTestCase) ... ok
test_key2 (tests.test_keys.KeyTestCase) ... ok
test_revoked (tests.test_keys.KeyTestCase) ... ok
test_signonly (tests.test_keys.KeyTestCase) ... ok
test_list_by_email (tests.test_keylist.KeylistTestCase) ... ok
test_list_by_email_substring (tests.test_keylist.KeylistTestCase) ... ok
test_list_by_name (tests.test_keylist.KeylistTestCase) ... ok
test_list_secret (tests.test_keylist.KeylistTestCase) ... ok
test_listall (tests.test_keylist.KeylistTestCase) ... ok
test_import_concat (tests.test_import.ImportTestCase) ... FAIL
test_import_empty (tests.test_import.ImportTestCase) ... ok
test_import_file (tests.test_import.ImportTestCase) ... ok
test_import_secret_file (tests.test_import.ImportTestCase) ... FAIL
test_import_stringio (tests.test_import.ImportTestCase) ... ok
test_import_twice (tests.test_import.ImportTestCase) ... ok
test_export_by_email (tests.test_export.ExportTestCase) ... ok
test_export_by_fingerprint (tests.test_export.ExportTestCase) ... ok
test_export_by_name (tests.test_export.ExportTestCase) ... ok
test_delete_non_existant (tests.test_delete.DeleteTestCase) ... ok
test_delete_public_key (tests.test_delete.DeleteTestCase) ... ok
test_delete_public_key_with_secret_key (tests.test_delete.DeleteTestCase) ... ok
test_delete_secret_key (tests.test_delete.DeleteTestCase) ... ok
test_sign_clearsign (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_sign_detatch (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_sign_normal (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_sign_normal_armor (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_verify_bad_signature (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_verify_clearsign (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_verify_detached (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_verify_multiple_sigs (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_verify_no_signature (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_verify_normal (tests.test_sign_verify.SignVerifyTestCase) ... ok
test_decrypt (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... ok
test_decrypt_verify (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... ok
test_encrypt (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... ok
test_encrypt_armor (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... ok
test_encrypt_sign (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... ok
test_encrypt_symmetric (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... ERROR
test_encrypt_to_signonly (tests.test_encrypt_decrypt.EncryptDecryptTestCase) ... FAIL
test_sign_with_passphrase_cb (tests.test_passphrase.PassphraseTestCase) ... ERROR
test_sign_without_passphrase_cb (tests.test_passphrase.PassphraseTestCase) ... FAIL
test_sign_with_progress_cb (tests.test_progress.ProgressTestCase) ... ok
test_edit_ownertrust (tests.test_editkey.EditKeyTestCase) ... ERROR
test_edit_quit (tests.test_editkey.EditKeyTestCase) ... ERROR
test_edit_sign (tests.test_editkey.EditKeyTestCase) ... ERROR
test_invalid_parameters (tests.test_genkey.GenerateKeyTestCase) ... ok

======================================================================
ERROR: test_encrypt_symmetric (tests.test_encrypt_decrypt.EncryptDecryptTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_encrypt_decrypt.py", line 136, in test_encrypt_symmetric
    ctx.encrypt(None, 0, plaintext, ciphertext)
GpgmeError: (7, 11, u'Bad passphrase')

======================================================================
ERROR: test_sign_with_passphrase_cb (tests.test_passphrase.PassphraseTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_passphrase.py", line 65, in test_sign_with_passphrase_cb
    new_sigs = ctx.sign(plaintext, signature, gpgme.SIG_MODE_CLEAR)
GpgmeError: (5, 303, u'Unknown error code')

======================================================================
ERROR: test_edit_ownertrust (tests.test_editkey.EditKeyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_editkey.py", line 63, in test_edit_ownertrust
    gpgme.editutil.edit_trust(ctx, key, trust)
  File "/home/dkg/src/pygpgme/pygpgme/gpgme/editutil.py", line 75, in wrapper
    ctx.edit(key, edit_callback, output)
GpgmeError: (32, 1, u'General error')

======================================================================
ERROR: test_edit_quit (tests.test_editkey.EditKeyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_editkey.py", line 48, in test_edit_quit
    ctx.edit(key, self.edit_quit_cb, output)
GpgmeError: (0, 32779, u'Bad file descriptor')

======================================================================
ERROR: test_edit_sign (tests.test_editkey.EditKeyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_editkey.py", line 80, in test_edit_sign
    gpgme.editutil.edit_sign(ctx, key, check=0)
  File "/home/dkg/src/pygpgme/pygpgme/gpgme/editutil.py", line 75, in wrapper
    ctx.edit(key, edit_callback, output)
GpgmeError: (32, 1, u'General error')

======================================================================
FAIL: test_import_concat (tests.test_import.ImportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_import.py", line 108, in test_import_concat
    self.assertEqual(result.imported_rsa, 1)
AssertionError: 0 != 1

======================================================================
FAIL: test_import_secret_file (tests.test_import.ImportTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_import.py", line 75, in test_import_secret_file
    None, gpgme.IMPORT_NEW | gpgme.IMPORT_SECRET))
AssertionError: Tuples differ: (u'E79A842DA34A1CA383F64A1546B... != ('E79A842DA34A1CA383F64A1546BB...

First differing element 2:
1
17

- (u'E79A842DA34A1CA383F64A1546BB55F0885C65A4', None, 1)
?  -

+ ('E79A842DA34A1CA383F64A1546BB55F0885C65A4', None, 17)
?                                                     +


======================================================================
FAIL: test_encrypt_to_signonly (tests.test_encrypt_decrypt.EncryptDecryptTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_encrypt_decrypt.py", line 185, in test_encrypt_to_signonly
    self.assertEqual(exc.args[0], gpgme.ERR_SOURCE_UNKNOWN)
AssertionError: 7 != 0

======================================================================
FAIL: test_sign_without_passphrase_cb (tests.test_passphrase.PassphraseTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dkg/src/pygpgme/pygpgme/tests/test_passphrase.py", line 43, in test_sign_without_passphrase_cb
    self.assertEqual(exc.args[0], gpgme.ERR_SOURCE_GPGME)
AssertionError: 5 != 7

----------------------------------------------------------------------
Ran 55 tests in 49.933s

FAILED (failures=4, errors=5)
Makefile:7: recipe for target 'check' failed
make: *** [check] Error 1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 930 bytes
Desc: not available
URL: </pipermail/attachments/20161026/6ddc44f0/attachment-0001.sig>

More information about the Gnupg-devel mailing list