From guidovranken at gmail.com Thu Sep 3 14:19:22 2020 From: guidovranken at gmail.com (Guido Vranken) Date: Thu, 3 Sep 2020 14:19:22 +0200 Subject: gcry_mpi_invm succeeds if the inverse does not exist In-Reply-To: <87d06jmdzh.fsf@iwagami.gniibe.org> References: <87zhbeh921.fsf@iwagami.gniibe.org> <51fc5b09-7b2f-ce2e-bb8c-f653ba907446@iki.fi> <87tv0k9vwj.fsf@jumper.gniibe.org> <6f38bc98-85fe-12b1-9cce-8e96e699e378@iki.fi> <87d06jmdzh.fsf@iwagami.gniibe.org> Message-ID: The following inputs to gcry_mpi_invm(): 36fb5bdb5daa9864113ad8a49a41722fc7003a40b02a13daca6997859c2d8534192ff6c02447 25c88352cfa171fc728503df037c355a6d5588b22e3510b08f10848ad7c0980b400 produces the number: 66CAF1A9A03478A288760C2E05E237F11432BA70BECEE56D942ACCD337470E5D77 But this is incorrect (another library reports the modular inverse does not exist). ---------- The following inputs to gcry_mpi_invm(): 12cf3a8ca3d97bea2f080362600cee355 1c3fddf62aee0be2f6dc2ef8471f1be2e produces the number: 60A6520F494E6EE6EE436283FB34B945 but it should produce: 1339462644931fd624528ea6b3fb1f985 On Mon, Jun 1, 2020 at 9:39 AM NIIBE Yutaka wrote: > Jussi Kivilinna wrote: > > Cryptofuzz is reporting another heap-buffer-overflow issue in > > _gcry_mpi_invm. I've attached reproducer, original from Guido and > > as patch applied to tests/basic.c. > > My fix of 69b55f87053ce2494cd4b38dc600f867bc4355be was not enough. > I just push another change: > > 6f8b1d4cb798375e6d830fd6b73c71da93ee5f3f > > Thank you for your report. > -- > > _______________________________________________ > Gcrypt-devel mailing list > Gcrypt-devel at gnupg.org > http://lists.gnupg.org/mailman/listinfo/gcrypt-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tianjia.zhang at linux.alibaba.com Fri Sep 18 11:39:55 2020 From: tianjia.zhang at linux.alibaba.com (Tianjia Zhang) Date: Fri, 18 Sep 2020 17:39:55 +0800 Subject: https://git.gnupg.org/ website cannot be accessed Message-ID: <64117fe0-681a-516f-f22d-c03441fd4189@linux.alibaba.com> Hi, As the title indicates, I don?t know if it?s a problem with my own network, but I can still access it some time ago. Thanks, Tianjia From jussi.kivilinna at iki.fi Fri Sep 18 17:14:17 2020 From: jussi.kivilinna at iki.fi (Jussi Kivilinna) Date: Fri, 18 Sep 2020 18:14:17 +0300 Subject: [PATCH] rijndael-aesni: tweak x86_64 AES-NI for better performance on AMD Zen2 Message-ID: <20200918151417.1512754-1-jussi.kivilinna@iki.fi> * cipher/rijndael-aesni.c (do_aesni_enc_vec8, do_aesni_dec_vec8): More first round key xoring and last round out to caller. (do_aesni_ctr_4): Change low 8-bit counter overflow check to 8-bit addition to low-bits and detect overflow from carry flag; Adjust slow path to restore counter. (do_aesni_ctr_8): Same as above; Interleave first round key xoring and first round with CTR generation on fast path; Interleave last round with output xoring. (_gcry_aes_aesni_cfb_dec, _gcry_aes_aesni_cbc_dec): Add first round key xoring; Change order of last round xoring and output xoring (shorten the dependency path). (_gcry_aes_aesni_ocb_auth): Add first round key xoring and last round handling. -- Benchmark on Ryzen 7 3700X: Before: AES | nanosecs/byte mebibytes/sec cycles/byte CBC dec | 0.113 ns/B 8445 MiB/s 0.407 c/B CFB dec | 0.114 ns/B 8337 MiB/s 0.412 c/B CTR enc | 0.112 ns/B 8505 MiB/s 0.404 c/B CTR dec | 0.113 ns/B 8476 MiB/s 0.405 c/B After (CBC-dec +21%, CFB-dec +24%, CTR +8% faster): AES | nanosecs/byte mebibytes/sec cycles/byte CBC dec | 0.093 ns/B 10277 MiB/s 0.334 c/B CFB dec | 0.092 ns/B 10372 MiB/s 0.331 c/B CTR enc | 0.104 ns/B 9209 MiB/s 0.373 c/B CTR dec | 0.104 ns/B 9192 MiB/s 0.373 c/B Performance remains the same on Intel Skylake. Signed-off-by: Jussi Kivilinna --- cipher/rijndael-aesni.c | 313 +++++++++++++++++++++++----------------- 1 file changed, 183 insertions(+), 130 deletions(-) diff --git a/cipher/rijndael-aesni.c b/cipher/rijndael-aesni.c index b26449a7..747ef662 100644 --- a/cipher/rijndael-aesni.c +++ b/cipher/rijndael-aesni.c @@ -788,17 +788,7 @@ do_aesni_dec_vec4 (const RIJNDAEL_context *ctx) static ASM_FUNC_ATTR_INLINE void do_aesni_enc_vec8 (const RIJNDAEL_context *ctx) { - asm volatile ("movdqa (%[key]), %%xmm0\n\t" - "pxor %%xmm0, %%xmm1\n\t" /* xmm1 ^= key[0] */ - "pxor %%xmm0, %%xmm2\n\t" /* xmm2 ^= key[0] */ - "pxor %%xmm0, %%xmm3\n\t" /* xmm3 ^= key[0] */ - "pxor %%xmm0, %%xmm4\n\t" /* xmm4 ^= key[0] */ - "pxor %%xmm0, %%xmm8\n\t" /* xmm8 ^= key[0] */ - "pxor %%xmm0, %%xmm9\n\t" /* xmm9 ^= key[0] */ - "pxor %%xmm0, %%xmm10\n\t" /* xmm10 ^= key[0] */ - "pxor %%xmm0, %%xmm11\n\t" /* xmm11 ^= key[0] */ - "movdqa 0x10(%[key]), %%xmm0\n\t" - "cmpl $12, %[rounds]\n\t" + asm volatile ("movdqa 0x10(%[key]), %%xmm0\n\t" "aesenc %%xmm0, %%xmm1\n\t" "aesenc %%xmm0, %%xmm2\n\t" "aesenc %%xmm0, %%xmm3\n\t" @@ -808,6 +798,7 @@ do_aesni_enc_vec8 (const RIJNDAEL_context *ctx) "aesenc %%xmm0, %%xmm10\n\t" "aesenc %%xmm0, %%xmm11\n\t" "movdqa 0x20(%[key]), %%xmm0\n\t" + "cmpl $12, %[rounds]\n\t" "aesenc %%xmm0, %%xmm1\n\t" "aesenc %%xmm0, %%xmm2\n\t" "aesenc %%xmm0, %%xmm3\n\t" @@ -920,14 +911,6 @@ do_aesni_enc_vec8 (const RIJNDAEL_context *ctx) "movdqa 0xe0(%[key]), %%xmm0\n" ".Ldeclast%=:\n\t" - "aesenclast %%xmm0, %%xmm1\n\t" - "aesenclast %%xmm0, %%xmm2\n\t" - "aesenclast %%xmm0, %%xmm3\n\t" - "aesenclast %%xmm0, %%xmm4\n\t" - "aesenclast %%xmm0, %%xmm8\n\t" - "aesenclast %%xmm0, %%xmm9\n\t" - "aesenclast %%xmm0, %%xmm10\n\t" - "aesenclast %%xmm0, %%xmm11\n\t" : /* no output */ : [key] "r" (ctx->keyschenc), [rounds] "r" (ctx->rounds) @@ -940,16 +923,7 @@ do_aesni_enc_vec8 (const RIJNDAEL_context *ctx) static ASM_FUNC_ATTR_INLINE void do_aesni_dec_vec8 (const RIJNDAEL_context *ctx) { - asm volatile ("movdqa (%[key]), %%xmm0\n\t" - "pxor %%xmm0, %%xmm1\n\t" /* xmm1 ^= key[0] */ - "pxor %%xmm0, %%xmm2\n\t" /* xmm2 ^= key[0] */ - "pxor %%xmm0, %%xmm3\n\t" /* xmm3 ^= key[0] */ - "pxor %%xmm0, %%xmm4\n\t" /* xmm4 ^= key[0] */ - "pxor %%xmm0, %%xmm8\n\t" /* xmm8 ^= key[0] */ - "pxor %%xmm0, %%xmm9\n\t" /* xmm9 ^= key[0] */ - "pxor %%xmm0, %%xmm10\n\t" /* xmm10 ^= key[0] */ - "pxor %%xmm0, %%xmm11\n\t" /* xmm11 ^= key[0] */ - "movdqa 0x10(%[key]), %%xmm0\n\t" + asm volatile ("movdqa 0x10(%[key]), %%xmm0\n\t" "cmpl $12, %[rounds]\n\t" "aesdec %%xmm0, %%xmm1\n\t" "aesdec %%xmm0, %%xmm2\n\t" @@ -1072,14 +1046,6 @@ do_aesni_dec_vec8 (const RIJNDAEL_context *ctx) "movdqa 0xe0(%[key]), %%xmm0\n" ".Ldeclast%=:\n\t" - "aesdeclast %%xmm0, %%xmm1\n\t" - "aesdeclast %%xmm0, %%xmm2\n\t" - "aesdeclast %%xmm0, %%xmm3\n\t" - "aesdeclast %%xmm0, %%xmm4\n\t" - "aesdeclast %%xmm0, %%xmm8\n\t" - "aesdeclast %%xmm0, %%xmm9\n\t" - "aesdeclast %%xmm0, %%xmm10\n\t" - "aesdeclast %%xmm0, %%xmm11\n\t" : /* no output */ : [key] "r" (ctx->keyschdec), [rounds] "r" (ctx->rounds) @@ -1204,8 +1170,8 @@ do_aesni_ctr_4 (const RIJNDAEL_context *ctx, */ asm volatile (/* detect if 8-bit carry handling is needed */ - "cmpb $0xfb, 15(%[ctr])\n\t" - "ja .Ladd32bit%=\n\t" + "addb $4, 15(%[ctr])\n\t" + "jc .Ladd32bit%=\n\t" "movdqa %%xmm5, %%xmm0\n\t" /* xmm0 := CTR (xmm5) */ "movdqa 0*16(%[addb]), %%xmm2\n\t" /* xmm2 := be(1) */ @@ -1217,9 +1183,10 @@ do_aesni_ctr_4 (const RIJNDAEL_context *ctx, "paddb %%xmm0, %%xmm4\n\t" /* xmm4 := be(3) + CTR (xmm0) */ "paddb %%xmm0, %%xmm5\n\t" /* xmm5 := be(4) + CTR (xmm0) */ "movdqa (%[key]), %%xmm1\n\t" /* xmm1 := key[0] */ - "jmp .Lstore_ctr%=\n\t" + "jmp .Ldone_ctr%=\n\t" ".Ladd32bit%=:\n\t" + "movdqa %%xmm5, (%[ctr])\n\t" /* Restore CTR. */ "movdqa %%xmm5, %%xmm0\n\t" /* xmm0, xmm2 := CTR (xmm5) */ "movdqa %%xmm0, %%xmm2\n\t" "pcmpeqd %%xmm1, %%xmm1\n\t" @@ -1265,8 +1232,9 @@ do_aesni_ctr_4 (const RIJNDAEL_context *ctx, "pshufb %%xmm6, %%xmm4\n\t" /* xmm4 := be(xmm4) */ "pshufb %%xmm6, %%xmm5\n\t" /* xmm5 := be(xmm5) */ - ".Lstore_ctr%=:\n\t" "movdqa %%xmm5, (%[ctr])\n\t" /* Update CTR (mem). */ + + ".Ldone_ctr%=:\n\t" : : [ctr] "r" (ctr), [key] "r" (ctx->keyschenc), @@ -1428,30 +1396,50 @@ do_aesni_ctr_8 (const RIJNDAEL_context *ctx, */ asm volatile (/* detect if 8-bit carry handling is needed */ - "cmpb $0xf7, 15(%[ctr])\n\t" - "ja .Ladd32bit%=\n\t" + "addb $8, 15(%[ctr])\n\t" + "jc .Ladd32bit%=\n\t" - "movdqa %%xmm5, %%xmm0\n\t" /* xmm0 := CTR (xmm5) */ - "movdqa 0*16(%[addb]), %%xmm2\n\t" /* xmm2 := be(1) */ - "movdqa 1*16(%[addb]), %%xmm3\n\t" /* xmm3 := be(2) */ - "movdqa 2*16(%[addb]), %%xmm4\n\t" /* xmm4 := be(3) */ - "movdqa 3*16(%[addb]), %%xmm8\n\t" /* xmm8 := be(4) */ - "movdqa 4*16(%[addb]), %%xmm9\n\t" /* xmm9 := be(5) */ - "movdqa 5*16(%[addb]), %%xmm10\n\t" /* xmm10 := be(6) */ - "movdqa 6*16(%[addb]), %%xmm11\n\t" /* xmm11 := be(7) */ - "movdqa 7*16(%[addb]), %%xmm5\n\t" /* xmm5 := be(8) */ "movdqa (%[key]), %%xmm1\n\t" /* xmm1 := key[0] */ - "paddb %%xmm0, %%xmm2\n\t" /* xmm2 := be(1) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm3\n\t" /* xmm3 := be(2) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm4\n\t" /* xmm4 := be(3) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm8\n\t" /* xmm8 := be(4) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm9\n\t" /* xmm9 := be(5) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm10\n\t" /* xmm10 := be(6) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm11\n\t" /* xmm11 := be(7) + CTR (xmm0) */ - "paddb %%xmm0, %%xmm5\n\t" /* xmm5 := be(8) + CTR (xmm0) */ - "jmp .Lstore_ctr%=\n\t" + "movdqa 16(%[key]), %%xmm7\n\t" /* xmm7 := key[1] */ + + "movdqa %%xmm5, %%xmm0\n\t" /* xmm0 := CTR (xmm5) */ + "movdqa %%xmm5, %%xmm2\n\t" /* xmm2 := CTR (xmm5) */ + "movdqa %%xmm5, %%xmm3\n\t" /* xmm3 := CTR (xmm5) */ + "movdqa %%xmm5, %%xmm4\n\t" /* xmm4 := CTR (xmm5) */ + "paddb 0*16(%[addb]), %%xmm2\n\t" /* xmm2 := be(1) + CTR */ + "paddb 1*16(%[addb]), %%xmm3\n\t" /* xmm3 := be(2) + CTR */ + "paddb 2*16(%[addb]), %%xmm4\n\t" /* xmm4 := be(3) + CTR */ + "pxor %%xmm1, %%xmm0\n\t" /* xmm0 ^= key[0] */ + "pxor %%xmm1, %%xmm2\n\t" /* xmm2 ^= key[0] */ + "pxor %%xmm1, %%xmm3\n\t" /* xmm3 ^= key[0] */ + "pxor %%xmm1, %%xmm4\n\t" /* xmm4 ^= key[0] */ + "aesenc %%xmm7, %%xmm0\n\t" + "aesenc %%xmm7, %%xmm2\n\t" + "aesenc %%xmm7, %%xmm3\n\t" + "aesenc %%xmm7, %%xmm4\n\t" + "movdqa %%xmm5, %%xmm8\n\t" /* xmm8 := CTR (xmm5) */ + "movdqa %%xmm5, %%xmm9\n\t" /* xmm9 := CTR (xmm5) */ + "movdqa %%xmm5, %%xmm10\n\t" /* xmm10 := CTR (xmm5) */ + "movdqa %%xmm5, %%xmm11\n\t" /* xmm11 := CTR (xmm5) */ + "paddb 3*16(%[addb]), %%xmm8\n\t" /* xmm8 := be(4) + CTR */ + "paddb 4*16(%[addb]), %%xmm9\n\t" /* xmm9 := be(5) + CTR */ + "paddb 5*16(%[addb]), %%xmm10\n\t" /* xmm10 := be(6) + CTR */ + "paddb 6*16(%[addb]), %%xmm11\n\t" /* xmm11 := be(7) + CTR */ + "pxor %%xmm1, %%xmm8\n\t" /* xmm8 ^= key[0] */ + "pxor %%xmm1, %%xmm9\n\t" /* xmm9 ^= key[0] */ + "pxor %%xmm1, %%xmm10\n\t" /* xmm10 ^= key[0] */ + "pxor %%xmm1, %%xmm11\n\t" /* xmm11 ^= key[0] */ + "aesenc %%xmm7, %%xmm8\n\t" + "aesenc %%xmm7, %%xmm9\n\t" + "aesenc %%xmm7, %%xmm10\n\t" + "aesenc %%xmm7, %%xmm11\n\t" + + "paddb 7*16(%[addb]), %%xmm5\n\t" /* xmm5 := be(8) + CTR */ + + "jmp .Ldone_ctr%=\n\t" ".Ladd32bit%=:\n\t" + "movdqa %%xmm5, (%[ctr])\n\t" /* Restore CTR. */ "movdqa %%xmm5, %%xmm0\n\t" /* xmm0, xmm2 := CTR (xmm5) */ "movdqa %%xmm0, %%xmm2\n\t" "pcmpeqd %%xmm1, %%xmm1\n\t" @@ -1512,44 +1500,50 @@ do_aesni_ctr_8 (const RIJNDAEL_context *ctx, "psubq %%xmm1, %%xmm5\n\t" ".Lno_carry%=:\n\t" - "movdqa (%[key]), %%xmm1\n\t" /* xmm1 := key[0] */ + "movdqa (%[key]), %%xmm1\n\t" /* xmm1 := key[0] */ + "movdqa 16(%[key]), %%xmm7\n\t" /* xmm7 := key[1] */ "pshufb %%xmm6, %%xmm2\n\t" /* xmm2 := be(xmm2) */ "pshufb %%xmm6, %%xmm3\n\t" /* xmm3 := be(xmm3) */ "pshufb %%xmm6, %%xmm4\n\t" /* xmm4 := be(xmm4) */ - "pshufb %%xmm6, %%xmm5\n\t" /* xmm5 := be(xmm5) */ + "pxor %%xmm1, %%xmm0\n\t" /* xmm0 ^= key[0] */ + "pxor %%xmm1, %%xmm2\n\t" /* xmm2 ^= key[0] */ + "pxor %%xmm1, %%xmm3\n\t" /* xmm3 ^= key[0] */ + "pxor %%xmm1, %%xmm4\n\t" /* xmm4 ^= key[0] */ + "aesenc %%xmm7, %%xmm0\n\t" + "aesenc %%xmm7, %%xmm2\n\t" + "aesenc %%xmm7, %%xmm3\n\t" + "aesenc %%xmm7, %%xmm4\n\t" "pshufb %%xmm6, %%xmm8\n\t" /* xmm8 := be(xmm8) */ "pshufb %%xmm6, %%xmm9\n\t" /* xmm9 := be(xmm9) */ "pshufb %%xmm6, %%xmm10\n\t" /* xmm10 := be(xmm10) */ "pshufb %%xmm6, %%xmm11\n\t" /* xmm11 := be(xmm11) */ + "pxor %%xmm1, %%xmm8\n\t" /* xmm8 ^= key[0] */ + "pxor %%xmm1, %%xmm9\n\t" /* xmm9 ^= key[0] */ + "pxor %%xmm1, %%xmm10\n\t" /* xmm10 ^= key[0] */ + "pxor %%xmm1, %%xmm11\n\t" /* xmm11 ^= key[0] */ + "aesenc %%xmm7, %%xmm8\n\t" + "aesenc %%xmm7, %%xmm9\n\t" + "aesenc %%xmm7, %%xmm10\n\t" + "aesenc %%xmm7, %%xmm11\n\t" - ".Lstore_ctr%=:\n\t" + "pshufb %%xmm6, %%xmm5\n\t" /* xmm5 := be(xmm5) */ "movdqa %%xmm5, (%[ctr])\n\t" /* Update CTR (mem). */ + + ".align 16\n\t" + ".Ldone_ctr%=:\n\t" : : [ctr] "r" (ctr), [key] "r" (ctx->keyschenc), [addb] "r" (bige_addb) : "%esi", "cc", "memory"); - asm volatile ("pxor %%xmm1, %%xmm0\n\t" /* xmm0 ^= key[0] */ - "pxor %%xmm1, %%xmm2\n\t" /* xmm2 ^= key[0] */ - "pxor %%xmm1, %%xmm3\n\t" /* xmm3 ^= key[0] */ - "pxor %%xmm1, %%xmm4\n\t" /* xmm4 ^= key[0] */ - "pxor %%xmm1, %%xmm8\n\t" /* xmm8 ^= key[0] */ - "pxor %%xmm1, %%xmm9\n\t" /* xmm9 ^= key[0] */ - "pxor %%xmm1, %%xmm10\n\t" /* xmm10 ^= key[0] */ - "pxor %%xmm1, %%xmm11\n\t" /* xmm11 ^= key[0] */ - "movdqa 0x10(%[key]), %%xmm1\n\t" - "cmpl $12, %[rounds]\n\t" - "aesenc %%xmm1, %%xmm0\n\t" - "aesenc %%xmm1, %%xmm2\n\t" - "aesenc %%xmm1, %%xmm3\n\t" - "aesenc %%xmm1, %%xmm4\n\t" - "aesenc %%xmm1, %%xmm8\n\t" - "aesenc %%xmm1, %%xmm9\n\t" - "aesenc %%xmm1, %%xmm10\n\t" - "aesenc %%xmm1, %%xmm11\n\t" - "movdqa 0x20(%[key]), %%xmm1\n\t" + asm volatile ("movdqa 0x20(%[key]), %%xmm1\n\t" + "movdqu 0*16(%[src]), %%xmm12\n\t" /* Get block 1. */ + "movdqu 1*16(%[src]), %%xmm13\n\t" /* Get block 2. */ + "movdqu 2*16(%[src]), %%xmm14\n\t" /* Get block 3. */ + "movdqu 3*16(%[src]), %%xmm15\n\t" /* Get block 4. */ + "movdqu 4*16(%[src]), %%xmm7\n\t" /* Get block 5. */ "aesenc %%xmm1, %%xmm0\n\t" "aesenc %%xmm1, %%xmm2\n\t" "aesenc %%xmm1, %%xmm3\n\t" @@ -1558,6 +1552,7 @@ do_aesni_ctr_8 (const RIJNDAEL_context *ctx, "aesenc %%xmm1, %%xmm9\n\t" "aesenc %%xmm1, %%xmm10\n\t" "aesenc %%xmm1, %%xmm11\n\t" + "cmpl $12, %[rounds]\n\t" "movdqa 0x30(%[key]), %%xmm1\n\t" "aesenc %%xmm1, %%xmm0\n\t" "aesenc %%xmm1, %%xmm2\n\t" @@ -1664,38 +1659,33 @@ do_aesni_ctr_8 (const RIJNDAEL_context *ctx, ".Lenclast%=:\n\t" : : [key] "r" (ctx->keyschenc), - [rounds] "r" (ctx->rounds) + [rounds] "r" (ctx->rounds), + [src] "r" (a) : "cc", "memory"); - asm volatile ("movdqu 0*16(%[src]), %%xmm12\n\t" /* Get block 1. */ - "movdqu 1*16(%[src]), %%xmm13\n\t" /* Get block 2. */ - "movdqu 2*16(%[src]), %%xmm14\n\t" /* Get block 3. */ - "movdqu 3*16(%[src]), %%xmm15\n\t" /* Get block 4. */ - "movdqu 4*16(%[src]), %%xmm7\n\t" /* Get block 5. */ - "pxor %%xmm1, %%xmm12\n\t" /* block1 ^= lastkey */ - "aesenclast %%xmm12, %%xmm0\n\t" - "movdqu 5*16(%[src]), %%xmm12\n\t" /* Get block 6. */ + asm volatile ("pxor %%xmm1, %%xmm12\n\t" /* block1 ^= lastkey */ "pxor %%xmm1, %%xmm13\n\t" /* block2 ^= lastkey */ - "aesenclast %%xmm13, %%xmm2\n\t" - "movdqu 6*16(%[src]), %%xmm13\n\t" /* Get block 7. */ "pxor %%xmm1, %%xmm14\n\t" /* block3 ^= lastkey */ - "aesenclast %%xmm14, %%xmm3\n\t" - "movdqu 7*16(%[src]), %%xmm14\n\t" /* Get block 8. */ "pxor %%xmm1, %%xmm15\n\t" /* block4 ^= lastkey */ + "aesenclast %%xmm12, %%xmm0\n\t" + "aesenclast %%xmm13, %%xmm2\n\t" + "aesenclast %%xmm14, %%xmm3\n\t" "aesenclast %%xmm15, %%xmm4\n\t" - "movdqu %%xmm0, 0*16(%[dst])\n\t" /* Store block 1 */ - "pxor %%xmm1, %%xmm7\n\t" /* block5 ^= lastkey */ - "aesenclast %%xmm7, %%xmm8\n\t" - "movdqu %%xmm0, 0*16(%[dst])\n\t" /* Store block 1 */ - "pxor %%xmm1, %%xmm12\n\t" /* block6 ^= lastkey */ - "aesenclast %%xmm12, %%xmm9\n\t" + "movdqu 5*16(%[src]), %%xmm12\n\t" /* Get block 6. */ + "movdqu 6*16(%[src]), %%xmm13\n\t" /* Get block 7. */ + "movdqu 7*16(%[src]), %%xmm14\n\t" /* Get block 8. */ + "movdqu %%xmm0, 0*16(%[dst])\n\t" /* Store block 1. */ "movdqu %%xmm2, 1*16(%[dst])\n\t" /* Store block 2. */ - "pxor %%xmm1, %%xmm13\n\t" /* block7 ^= lastkey */ - "aesenclast %%xmm13, %%xmm10\n\t" "movdqu %%xmm3, 2*16(%[dst])\n\t" /* Store block 3. */ + "movdqu %%xmm4, 3*16(%[dst])\n\t" /* Store block 4. */ + "pxor %%xmm1, %%xmm7\n\t" /* block5 ^= lastkey */ + "pxor %%xmm1, %%xmm12\n\t" /* block6 ^= lastkey */ + "pxor %%xmm1, %%xmm13\n\t" /* block7 ^= lastkey */ "pxor %%xmm1, %%xmm14\n\t" /* block8 ^= lastkey */ + "aesenclast %%xmm7, %%xmm8\n\t" + "aesenclast %%xmm12, %%xmm9\n\t" + "aesenclast %%xmm13, %%xmm10\n\t" "aesenclast %%xmm14, %%xmm11\n\t" - "movdqu %%xmm4, 3*16(%[dst])\n\t" /* Store block 4. */ "movdqu %%xmm8, 4*16(%[dst])\n\t" /* Store block 8. */ "movdqu %%xmm9, 5*16(%[dst])\n\t" /* Store block 9. */ "movdqu %%xmm10, 6*16(%[dst])\n\t" /* Store block 10. */ @@ -1910,7 +1900,9 @@ _gcry_aes_aesni_cfb_dec (RIJNDAEL_context *ctx, unsigned char *iv, for ( ;nblocks >= 8; nblocks -= 8) { asm volatile - ("movdqu %%xmm6, %%xmm1\n\t" /* load input blocks */ + ("movdqa (%[key]), %%xmm0\n\t" + + "movdqu %%xmm6, %%xmm1\n\t" /* load input blocks */ "movdqu 0*16(%[inbuf]), %%xmm2\n\t" "movdqu 1*16(%[inbuf]), %%xmm3\n\t" "movdqu 2*16(%[inbuf]), %%xmm4\n\t" @@ -1925,30 +1917,50 @@ _gcry_aes_aesni_cfb_dec (RIJNDAEL_context *ctx, unsigned char *iv, "movdqa %%xmm3, %%xmm13\n\t" "movdqa %%xmm4, %%xmm14\n\t" "movdqa %%xmm8, %%xmm15\n\t" + + "pxor %%xmm0, %%xmm1\n\t" /* xmm1 ^= key[0] */ + "pxor %%xmm0, %%xmm2\n\t" /* xmm2 ^= key[0] */ + "pxor %%xmm0, %%xmm3\n\t" /* xmm3 ^= key[0] */ + "pxor %%xmm0, %%xmm4\n\t" /* xmm4 ^= key[0] */ + "pxor %%xmm0, %%xmm8\n\t" /* xmm8 ^= key[0] */ + "pxor %%xmm0, %%xmm9\n\t" /* xmm9 ^= key[0] */ + "pxor %%xmm0, %%xmm10\n\t" /* xmm10 ^= key[0] */ + "pxor %%xmm0, %%xmm11\n\t" /* xmm11 ^= key[0] */ : /* No output */ - : [inbuf] "r" (inbuf) + : [inbuf] "r" (inbuf), + [key] "r" (ctx->keyschenc) : "memory"); do_aesni_enc_vec8 (ctx); asm volatile ( - "pxor %%xmm12, %%xmm1\n\t" + "pxor %%xmm0, %%xmm12\n\t" + "pxor %%xmm0, %%xmm13\n\t" + "pxor %%xmm0, %%xmm14\n\t" + "pxor %%xmm0, %%xmm15\n\t" + "aesenclast %%xmm12, %%xmm1\n\t" + "aesenclast %%xmm13, %%xmm2\n\t" + "aesenclast %%xmm14, %%xmm3\n\t" + "aesenclast %%xmm15, %%xmm4\n\t" + "movdqu 4*16(%[inbuf]), %%xmm12\n\t" - "pxor %%xmm13, %%xmm2\n\t" "movdqu 5*16(%[inbuf]), %%xmm13\n\t" - "pxor %%xmm14, %%xmm3\n\t" "movdqu 6*16(%[inbuf]), %%xmm14\n\t" - "pxor %%xmm15, %%xmm4\n\t" "movdqu 7*16(%[inbuf]), %%xmm15\n\t" + "pxor %%xmm0, %%xmm12\n\t" + "pxor %%xmm0, %%xmm13\n\t" + "pxor %%xmm0, %%xmm14\n\t" + "pxor %%xmm0, %%xmm15\n\t" + + "aesenclast %%xmm12, %%xmm8\n\t" + "aesenclast %%xmm13, %%xmm9\n\t" + "aesenclast %%xmm14, %%xmm10\n\t" + "aesenclast %%xmm15, %%xmm11\n\t" - "pxor %%xmm12, %%xmm8\n\t" "movdqu %%xmm1, 0*16(%[outbuf])\n\t" - "pxor %%xmm13, %%xmm9\n\t" "movdqu %%xmm2, 1*16(%[outbuf])\n\t" - "pxor %%xmm14, %%xmm10\n\t" "movdqu %%xmm3, 2*16(%[outbuf])\n\t" - "pxor %%xmm15, %%xmm11\n\t" "movdqu %%xmm4, 3*16(%[outbuf])\n\t" "movdqu %%xmm8, 4*16(%[outbuf])\n\t" @@ -2070,7 +2082,9 @@ _gcry_aes_aesni_cbc_dec (RIJNDAEL_context *ctx, unsigned char *iv, for ( ;nblocks >= 8 ; nblocks -= 8 ) { asm volatile - ("movdqu 0*16(%[inbuf]), %%xmm1\n\t" /* load input blocks */ + ("movdqa (%[key]), %%xmm0\n\t" + + "movdqu 0*16(%[inbuf]), %%xmm1\n\t" /* load input blocks */ "movdqu 1*16(%[inbuf]), %%xmm2\n\t" "movdqu 2*16(%[inbuf]), %%xmm3\n\t" "movdqu 3*16(%[inbuf]), %%xmm4\n\t" @@ -2084,31 +2098,50 @@ _gcry_aes_aesni_cbc_dec (RIJNDAEL_context *ctx, unsigned char *iv, "movdqa %%xmm3, %%xmm14\n\t" "movdqa %%xmm4, %%xmm15\n\t" + "pxor %%xmm0, %%xmm1\n\t" /* xmm1 ^= key[0] */ + "pxor %%xmm0, %%xmm2\n\t" /* xmm2 ^= key[0] */ + "pxor %%xmm0, %%xmm3\n\t" /* xmm3 ^= key[0] */ + "pxor %%xmm0, %%xmm4\n\t" /* xmm4 ^= key[0] */ + "pxor %%xmm0, %%xmm8\n\t" /* xmm8 ^= key[0] */ + "pxor %%xmm0, %%xmm9\n\t" /* xmm9 ^= key[0] */ + "pxor %%xmm0, %%xmm10\n\t" /* xmm10 ^= key[0] */ + "pxor %%xmm0, %%xmm11\n\t" /* xmm11 ^= key[0] */ + : /* No output */ - : [inbuf] "r" (inbuf) + : [inbuf] "r" (inbuf), + [key] "r" (ctx->keyschdec) : "memory"); do_aesni_dec_vec8 (ctx); asm volatile - ("pxor %%xmm5, %%xmm1\n\t" /* xor IV with output */ + ( + "pxor %%xmm0, %%xmm5\n\t" /* xor IV with key */ + "pxor %%xmm0, %%xmm12\n\t" /* xor IV with key */ + "pxor %%xmm0, %%xmm13\n\t" /* xor IV with key */ + "pxor %%xmm0, %%xmm14\n\t" /* xor IV with key */ + "pxor %%xmm0, %%xmm15\n\t" /* xor IV with key */ - "pxor %%xmm12, %%xmm2\n\t" /* xor IV with output */ - "movdqu 4*16(%[inbuf]), %%xmm12\n\t" + "aesdeclast %%xmm5, %%xmm1\n\t" + "aesdeclast %%xmm12, %%xmm2\n\t" + "aesdeclast %%xmm13, %%xmm3\n\t" + "aesdeclast %%xmm14, %%xmm4\n\t" - "pxor %%xmm13, %%xmm3\n\t" /* xor IV with output */ + "movdqu 4*16(%[inbuf]), %%xmm12\n\t" "movdqu 5*16(%[inbuf]), %%xmm13\n\t" - - "pxor %%xmm14, %%xmm4\n\t" /* xor IV with output */ "movdqu 6*16(%[inbuf]), %%xmm14\n\t" - - "pxor %%xmm15, %%xmm8\n\t" /* xor IV with output */ "movdqu 7*16(%[inbuf]), %%xmm5\n\t" - "pxor %%xmm12, %%xmm9\n\t" /* xor IV with output */ + "pxor %%xmm0, %%xmm12\n\t" /* xor IV with key */ + "pxor %%xmm0, %%xmm13\n\t" /* xor IV with key */ + "pxor %%xmm0, %%xmm14\n\t" /* xor IV with key */ + + "aesdeclast %%xmm15, %%xmm8\n\t" + "aesdeclast %%xmm12, %%xmm9\n\t" + "aesdeclast %%xmm13, %%xmm10\n\t" + "aesdeclast %%xmm14, %%xmm11\n\t" + "movdqu %%xmm1, 0*16(%[outbuf])\n\t" - "pxor %%xmm13, %%xmm10\n\t" /* xor IV with output */ "movdqu %%xmm2, 1*16(%[outbuf])\n\t" - "pxor %%xmm14, %%xmm11\n\t" /* xor IV with output */ "movdqu %%xmm3, 2*16(%[outbuf])\n\t" "movdqu %%xmm4, 3*16(%[outbuf])\n\t" "movdqu %%xmm8, 4*16(%[outbuf])\n\t" @@ -3452,8 +3485,14 @@ _gcry_aes_aesni_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, "pxor %%xmm5, %%xmm3\n\t" "pxor %%xmm0, %%xmm5\n\t" + "movdqa (%[key]), %%xmm0\n\t" "pxor %%xmm5, %%xmm4\n\t" + "pxor %%xmm0, %%xmm1\n\t" /* xmm1 ^= key[0] */ + "pxor %%xmm0, %%xmm2\n\t" /* xmm2 ^= key[0] */ + "pxor %%xmm0, %%xmm3\n\t" /* xmm3 ^= key[0] */ + "pxor %%xmm0, %%xmm4\n\t" /* xmm4 ^= key[0] */ + "pxor %%xmm7, %%xmm8\n\t" "pxor %%xmm5, %%xmm8\n\t" @@ -3465,13 +3504,27 @@ _gcry_aes_aesni_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, "pxor %%xmm14, %%xmm5\n\t" "pxor %%xmm5, %%xmm11\n\t" + + "pxor %%xmm0, %%xmm8\n\t" /* xmm8 ^= key[0] */ + "pxor %%xmm0, %%xmm9\n\t" /* xmm9 ^= key[0] */ + "pxor %%xmm0, %%xmm10\n\t" /* xmm10 ^= key[0] */ + "pxor %%xmm0, %%xmm11\n\t" /* xmm11 ^= key[0] */ : - : + : [key] "r" (ctx->keyschenc) : "memory" ); do_aesni_enc_vec8 (ctx); - asm volatile ("pxor %%xmm2, %%xmm1\n\t" + asm volatile ( + "aesenclast %%xmm0, %%xmm1\n\t" + "aesenclast %%xmm0, %%xmm2\n\t" + "aesenclast %%xmm0, %%xmm3\n\t" + "aesenclast %%xmm0, %%xmm4\n\t" + "aesenclast %%xmm0, %%xmm8\n\t" + "aesenclast %%xmm0, %%xmm9\n\t" + "aesenclast %%xmm0, %%xmm10\n\t" + "aesenclast %%xmm0, %%xmm11\n\t" + "pxor %%xmm2, %%xmm1\n\t" "pxor %%xmm3, %%xmm1\n\t" "pxor %%xmm4, %%xmm1\n\t" "pxor %%xmm8, %%xmm1\n\t" -- 2.25.1 From tianjia.zhang at linux.alibaba.com Mon Sep 21 15:01:49 2020 From: tianjia.zhang at linux.alibaba.com (Tianjia Zhang) Date: Mon, 21 Sep 2020 21:01:49 +0800 Subject: [PATCH] tests: Fix typo in comment Message-ID: <20200921130149.90824-1-tianjia.zhang@linux.alibaba.com> * tests/basic.c: Fix typo in comment. -- Signed-off-by: Tianjia Zhang --- tests/basic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/basic.c b/tests/basic.c index fdc5df4b..1d12c4a2 100644 --- a/tests/basic.c +++ b/tests/basic.c @@ -13036,7 +13036,7 @@ verify_one_signature (gcry_sexp_t pkey, gcry_sexp_t hash, } -/* Test the public key sign function using the private ket SKEY. PKEY +/* Test the public key sign function using the private key SKEY. PKEY is used for verification. */ static void check_pubkey_sign (int n, gcry_sexp_t skey, gcry_sexp_t pkey, int algo) @@ -13149,7 +13149,7 @@ check_pubkey_sign (int n, gcry_sexp_t skey, gcry_sexp_t pkey, int algo) } -/* Test the public key sign function using the private ket SKEY. PKEY +/* Test the public key sign function using the private key SKEY. PKEY is used for verification. This variant is only used for ECDSA. */ static void check_pubkey_sign_ecdsa (int n, gcry_sexp_t skey, gcry_sexp_t pkey) -- 2.19.1.3.ge56e4f7 From stefbon at gmail.com Sun Sep 27 07:22:36 2020 From: stefbon at gmail.com (Stef Bon) Date: Sun, 27 Sep 2020 07:22:36 +0200 Subject: Generate ECDH shared key - NIST P256 In-Reply-To: <518964648.31228.1592829871223.JavaMail.administrator@n7.nabble.com> References: <518964648.31228.1592829871223.JavaMail.administrator@n7.nabble.com> Message-ID: Do you have found a solution yet? Stef From jussi.kivilinna at iki.fi Sun Sep 27 20:08:25 2020 From: jussi.kivilinna at iki.fi (Jussi Kivilinna) Date: Sun, 27 Sep 2020 21:08:25 +0300 Subject: [PATCH 1/4] rijndael: tidy do_setkey little bit Message-ID: <20200927180828.744564-1-jussi.kivilinna@iki.fi> * cipher/rijndael.c (do_setkey): Reduce number of ifdefs by using function pointer for accelerated key-setup. -- Signed-off-by: Jussi Kivilinna --- cipher/rijndael.c | 26 ++++++++------------------ 1 file changed, 8 insertions(+), 18 deletions(-) diff --git a/cipher/rijndael.c b/cipher/rijndael.c index 3e9bae55..d81ed809 100644 --- a/cipher/rijndael.c +++ b/cipher/rijndael.c @@ -353,6 +353,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, { static int initialized = 0; static const char *selftest_failed = 0; + void (*hw_setkey)(RIJNDAEL_context *ctx, const byte *key) = NULL; int rounds; int i,j, r, t, rconpointer = 0; int KC; @@ -432,6 +433,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, #ifdef USE_AESNI else if (hwfeatures & HWF_INTEL_AESNI) { + hw_setkey = _gcry_aes_aesni_do_setkey; ctx->encrypt_fn = _gcry_aes_aesni_encrypt; ctx->decrypt_fn = _gcry_aes_aesni_decrypt; ctx->prefetch_enc_fn = NULL; @@ -466,6 +468,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, #ifdef USE_SSSE3 else if (hwfeatures & HWF_INTEL_SSSE3) { + hw_setkey = _gcry_aes_ssse3_do_setkey; ctx->encrypt_fn = _gcry_aes_ssse3_encrypt; ctx->decrypt_fn = _gcry_aes_ssse3_decrypt; ctx->prefetch_enc_fn = NULL; @@ -486,6 +489,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, #ifdef USE_ARM_CE else if (hwfeatures & HWF_ARM_AES) { + hw_setkey = _gcry_aes_armv8_ce_setkey; ctx->encrypt_fn = _gcry_aes_armv8_ce_encrypt; ctx->decrypt_fn = _gcry_aes_armv8_ce_decrypt; ctx->prefetch_enc_fn = NULL; @@ -507,6 +511,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, #ifdef USE_PPC_CRYPTO_WITH_PPC9LE else if ((hwfeatures & HWF_PPC_VCRYPTO) && (hwfeatures & HWF_PPC_ARCH_3_00)) { + hw_setkey = _gcry_aes_ppc8_setkey; ctx->encrypt_fn = _gcry_aes_ppc9le_encrypt; ctx->decrypt_fn = _gcry_aes_ppc9le_decrypt; ctx->prefetch_enc_fn = NULL; @@ -529,6 +534,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, #ifdef USE_PPC_CRYPTO else if (hwfeatures & HWF_PPC_VCRYPTO) { + hw_setkey = _gcry_aes_ppc8_setkey; ctx->encrypt_fn = _gcry_aes_ppc8_encrypt; ctx->decrypt_fn = _gcry_aes_ppc8_decrypt; ctx->prefetch_enc_fn = NULL; @@ -557,26 +563,10 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, /* NB: We don't yet support Padlock hardware key generation. */ - if (0) + if (hw_setkey) { - ; + hw_setkey (ctx, key); } -#ifdef USE_AESNI - else if (ctx->use_aesni) - _gcry_aes_aesni_do_setkey (ctx, key); -#endif -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - _gcry_aes_ssse3_do_setkey (ctx, key); -#endif -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - _gcry_aes_armv8_ce_setkey (ctx, key); -#endif -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - _gcry_aes_ppc8_setkey (ctx, key); -#endif else { const byte *sbox = ((const byte *)encT) + 1; -- 2.25.1 From jussi.kivilinna at iki.fi Sun Sep 27 20:08:27 2020 From: jussi.kivilinna at iki.fi (Jussi Kivilinna) Date: Sun, 27 Sep 2020 21:08:27 +0300 Subject: [PATCH 3/4] rijndael: clean-up generic bulk functions In-Reply-To: <20200927180828.744564-1-jussi.kivilinna@iki.fi> References: <20200927180828.744564-1-jussi.kivilinna@iki.fi> Message-ID: <20200927180828.744564-3-jussi.kivilinna@iki.fi> * cipher/rijndael.c (_gcry_aes_cfb_enc, _gcry_aes_cbc_enc) (_gcry_aes_ctr_enc, _gcry_aes_cfb_dec, _gcry_aes_cbc_dec) (_gcry_aes_ocb_crypt, _gcry_aes_ocb_auth, _gcry_aes_xts_crypt): Remove calls to hardware accelerated AES bulk functions. -- Patch removes accelerated implementation selection from generic bulk functions. These are no longer used/needed as setkey returns accelerated bulk functions for all callers and all callers have been updated to used those bulk functions. Signed-off-by: Jussi Kivilinna --- cipher/rijndael.c | 542 ++++++++++------------------------------------ 1 file changed, 118 insertions(+), 424 deletions(-) diff --git a/cipher/rijndael.c b/cipher/rijndael.c index ac70ba75..bf9c0050 100644 --- a/cipher/rijndael.c +++ b/cipher/rijndael.c @@ -974,60 +974,19 @@ _gcry_aes_cfb_enc (void *context, unsigned char *iv, unsigned char *outbuf = outbuf_arg; const unsigned char *inbuf = inbuf_arg; unsigned int burn_depth = 0; + rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_cfb_enc (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - _gcry_aes_ssse3_cfb_enc (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_cfb_enc (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - _gcry_aes_ppc9le_cfb_enc (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - _gcry_aes_ppc8_cfb_enc (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO*/ - else - { - rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - - if (ctx->prefetch_enc_fn) - ctx->prefetch_enc_fn(); + if (ctx->prefetch_enc_fn) + ctx->prefetch_enc_fn(); - for ( ;nblocks; nblocks-- ) - { - /* Encrypt the IV. */ - burn_depth = encrypt_fn (ctx, iv, iv); - /* XOR the input with the IV and store input into IV. */ - cipher_block_xor_2dst(outbuf, iv, inbuf, BLOCKSIZE); - outbuf += BLOCKSIZE; - inbuf += BLOCKSIZE; - } + for ( ;nblocks; nblocks-- ) + { + /* Encrypt the IV. */ + burn_depth = encrypt_fn (ctx, iv, iv); + /* XOR the input with the IV and store input into IV. */ + cipher_block_xor_2dst(outbuf, iv, inbuf, BLOCKSIZE); + outbuf += BLOCKSIZE; + inbuf += BLOCKSIZE; } if (burn_depth) @@ -1049,69 +1008,28 @@ _gcry_aes_cbc_enc (void *context, unsigned char *iv, const unsigned char *inbuf = inbuf_arg; unsigned char *last_iv; unsigned int burn_depth = 0; + rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_cbc_enc (ctx, iv, outbuf, inbuf, nblocks, cbc_mac); - return; - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - _gcry_aes_ssse3_cbc_enc (ctx, iv, outbuf, inbuf, nblocks, cbc_mac); - return; - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_cbc_enc (ctx, iv, outbuf, inbuf, nblocks, cbc_mac); - return; - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - _gcry_aes_ppc9le_cbc_enc (ctx, iv, outbuf, inbuf, nblocks, cbc_mac); - return; - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - _gcry_aes_ppc8_cbc_enc (ctx, iv, outbuf, inbuf, nblocks, cbc_mac); - return; - } -#endif /*USE_PPC_CRYPTO*/ - else - { - rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - - if (ctx->prefetch_enc_fn) - ctx->prefetch_enc_fn(); - - last_iv = iv; + if (ctx->prefetch_enc_fn) + ctx->prefetch_enc_fn(); - for ( ;nblocks; nblocks-- ) - { - cipher_block_xor(outbuf, inbuf, last_iv, BLOCKSIZE); + last_iv = iv; - burn_depth = encrypt_fn (ctx, outbuf, outbuf); + for ( ;nblocks; nblocks-- ) + { + cipher_block_xor(outbuf, inbuf, last_iv, BLOCKSIZE); - last_iv = outbuf; - inbuf += BLOCKSIZE; - if (!cbc_mac) - outbuf += BLOCKSIZE; - } + burn_depth = encrypt_fn (ctx, outbuf, outbuf); - if (last_iv != iv) - cipher_block_cpy (iv, last_iv, BLOCKSIZE); + last_iv = outbuf; + inbuf += BLOCKSIZE; + if (!cbc_mac) + outbuf += BLOCKSIZE; } + if (last_iv != iv) + cipher_block_cpy (iv, last_iv, BLOCKSIZE); + if (burn_depth) _gcry_burn_stack (burn_depth + 4 * sizeof(void *)); } @@ -1131,66 +1049,25 @@ _gcry_aes_ctr_enc (void *context, unsigned char *ctr, unsigned char *outbuf = outbuf_arg; const unsigned char *inbuf = inbuf_arg; unsigned int burn_depth = 0; + union { unsigned char x1[16] ATTR_ALIGNED_16; u32 x32[4]; } tmp; + rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_ctr_enc (ctx, ctr, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - _gcry_aes_ssse3_ctr_enc (ctx, ctr, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_ctr_enc (ctx, ctr, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - _gcry_aes_ppc9le_ctr_enc (ctx, ctr, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) + if (ctx->prefetch_enc_fn) + ctx->prefetch_enc_fn(); + + for ( ;nblocks; nblocks-- ) { - _gcry_aes_ppc8_ctr_enc (ctx, ctr, outbuf, inbuf, nblocks); - return; + /* Encrypt the counter. */ + burn_depth = encrypt_fn (ctx, tmp.x1, ctr); + /* XOR the input with the encrypted counter and store in output. */ + cipher_block_xor(outbuf, tmp.x1, inbuf, BLOCKSIZE); + outbuf += BLOCKSIZE; + inbuf += BLOCKSIZE; + /* Increment the counter. */ + cipher_block_add(ctr, 1, BLOCKSIZE); } -#endif /*USE_PPC_CRYPTO*/ - else - { - union { unsigned char x1[16] ATTR_ALIGNED_16; u32 x32[4]; } tmp; - rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - - if (ctx->prefetch_enc_fn) - ctx->prefetch_enc_fn(); - - for ( ;nblocks; nblocks-- ) - { - /* Encrypt the counter. */ - burn_depth = encrypt_fn (ctx, tmp.x1, ctr); - /* XOR the input with the encrypted counter and store in output. */ - cipher_block_xor(outbuf, tmp.x1, inbuf, BLOCKSIZE); - outbuf += BLOCKSIZE; - inbuf += BLOCKSIZE; - /* Increment the counter. */ - cipher_block_add(ctr, 1, BLOCKSIZE); - } - wipememory(&tmp, sizeof(tmp)); - } + wipememory(&tmp, sizeof(tmp)); if (burn_depth) _gcry_burn_stack (burn_depth + 4 * sizeof(void *)); @@ -1388,58 +1265,17 @@ _gcry_aes_cfb_dec (void *context, unsigned char *iv, unsigned char *outbuf = outbuf_arg; const unsigned char *inbuf = inbuf_arg; unsigned int burn_depth = 0; + rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_cfb_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - _gcry_aes_ssse3_cfb_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_cfb_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - _gcry_aes_ppc9le_cfb_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - _gcry_aes_ppc8_cfb_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO*/ - else - { - rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - - if (ctx->prefetch_enc_fn) - ctx->prefetch_enc_fn(); + if (ctx->prefetch_enc_fn) + ctx->prefetch_enc_fn(); - for ( ;nblocks; nblocks-- ) - { - burn_depth = encrypt_fn (ctx, iv, iv); - cipher_block_xor_n_copy(outbuf, iv, inbuf, BLOCKSIZE); - outbuf += BLOCKSIZE; - inbuf += BLOCKSIZE; - } + for ( ;nblocks; nblocks-- ) + { + burn_depth = encrypt_fn (ctx, iv, iv); + cipher_block_xor_n_copy(outbuf, iv, inbuf, BLOCKSIZE); + outbuf += BLOCKSIZE; + inbuf += BLOCKSIZE; } if (burn_depth) @@ -1460,69 +1296,28 @@ _gcry_aes_cbc_dec (void *context, unsigned char *iv, unsigned char *outbuf = outbuf_arg; const unsigned char *inbuf = inbuf_arg; unsigned int burn_depth = 0; + unsigned char savebuf[BLOCKSIZE] ATTR_ALIGNED_16; + rijndael_cryptfn_t decrypt_fn = ctx->decrypt_fn; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_cbc_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - _gcry_aes_ssse3_cbc_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_cbc_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - _gcry_aes_ppc9le_cbc_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - _gcry_aes_ppc8_cbc_dec (ctx, iv, outbuf, inbuf, nblocks); - return; - } -#endif /*USE_PPC_CRYPTO*/ - else - { - unsigned char savebuf[BLOCKSIZE] ATTR_ALIGNED_16; - rijndael_cryptfn_t decrypt_fn = ctx->decrypt_fn; - - check_decryption_preparation (ctx); - - if (ctx->prefetch_dec_fn) - ctx->prefetch_dec_fn(); + check_decryption_preparation (ctx); - for ( ;nblocks; nblocks-- ) - { - /* INBUF is needed later and it may be identical to OUTBUF, so store - the intermediate result to SAVEBUF. */ + if (ctx->prefetch_dec_fn) + ctx->prefetch_dec_fn(); - burn_depth = decrypt_fn (ctx, savebuf, inbuf); + for ( ;nblocks; nblocks-- ) + { + /* INBUF is needed later and it may be identical to OUTBUF, so store + the intermediate result to SAVEBUF. */ - cipher_block_xor_n_copy_2(outbuf, savebuf, iv, inbuf, BLOCKSIZE); - inbuf += BLOCKSIZE; - outbuf += BLOCKSIZE; - } + burn_depth = decrypt_fn (ctx, savebuf, inbuf); - wipememory(savebuf, sizeof(savebuf)); + cipher_block_xor_n_copy_2(outbuf, savebuf, iv, inbuf, BLOCKSIZE); + inbuf += BLOCKSIZE; + outbuf += BLOCKSIZE; } + wipememory(savebuf, sizeof(savebuf)); + if (burn_depth) _gcry_burn_stack (burn_depth + 4 * sizeof(void *)); } @@ -1539,39 +1334,7 @@ _gcry_aes_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, const unsigned char *inbuf = inbuf_arg; unsigned int burn_depth = 0; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - return _gcry_aes_aesni_ocb_crypt (c, outbuf, inbuf, nblocks, encrypt); - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - return _gcry_aes_ssse3_ocb_crypt (c, outbuf, inbuf, nblocks, encrypt); - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - return _gcry_aes_armv8_ce_ocb_crypt (c, outbuf, inbuf, nblocks, encrypt); - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - return _gcry_aes_ppc9le_ocb_crypt (c, outbuf, inbuf, nblocks, encrypt); - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - return _gcry_aes_ppc8_ocb_crypt (c, outbuf, inbuf, nblocks, encrypt); - } -#endif /*USE_PPC_CRYPTO*/ - else if (encrypt) + if (encrypt) { union { unsigned char x1[16] ATTR_ALIGNED_16; u32 x32[4]; } l_tmp; rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; @@ -1644,66 +1407,30 @@ _gcry_aes_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) RIJNDAEL_context *ctx = (void *)&c->context.c; const unsigned char *abuf = abuf_arg; unsigned int burn_depth = 0; + union { unsigned char x1[16] ATTR_ALIGNED_16; u32 x32[4]; } l_tmp; + rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - return _gcry_aes_aesni_ocb_auth (c, abuf, nblocks); - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - return _gcry_aes_ssse3_ocb_auth (c, abuf, nblocks); - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - return _gcry_aes_armv8_ce_ocb_auth (c, abuf, nblocks); - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) - { - return _gcry_aes_ppc9le_ocb_auth (c, abuf, nblocks); - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - return _gcry_aes_ppc8_ocb_auth (c, abuf, nblocks); - } -#endif /*USE_PPC_CRYPTO*/ - else - { - union { unsigned char x1[16] ATTR_ALIGNED_16; u32 x32[4]; } l_tmp; - rijndael_cryptfn_t encrypt_fn = ctx->encrypt_fn; - - if (ctx->prefetch_enc_fn) - ctx->prefetch_enc_fn(); - - for ( ;nblocks; nblocks-- ) - { - u64 i = ++c->u_mode.ocb.aad_nblocks; - const unsigned char *l = ocb_get_l(c, i); + if (ctx->prefetch_enc_fn) + ctx->prefetch_enc_fn(); - /* Offset_i = Offset_{i-1} xor L_{ntz(i)} */ - cipher_block_xor_1 (c->u_mode.ocb.aad_offset, l, BLOCKSIZE); - /* Sum_i = Sum_{i-1} xor ENCIPHER(K, A_i xor Offset_i) */ - cipher_block_xor (l_tmp.x1, c->u_mode.ocb.aad_offset, abuf, - BLOCKSIZE); - burn_depth = encrypt_fn (ctx, l_tmp.x1, l_tmp.x1); - cipher_block_xor_1 (c->u_mode.ocb.aad_sum, l_tmp.x1, BLOCKSIZE); + for ( ;nblocks; nblocks-- ) + { + u64 i = ++c->u_mode.ocb.aad_nblocks; + const unsigned char *l = ocb_get_l(c, i); - abuf += BLOCKSIZE; - } + /* Offset_i = Offset_{i-1} xor L_{ntz(i)} */ + cipher_block_xor_1 (c->u_mode.ocb.aad_offset, l, BLOCKSIZE); + /* Sum_i = Sum_{i-1} xor ENCIPHER(K, A_i xor Offset_i) */ + cipher_block_xor (l_tmp.x1, c->u_mode.ocb.aad_offset, abuf, + BLOCKSIZE); + burn_depth = encrypt_fn (ctx, l_tmp.x1, l_tmp.x1); + cipher_block_xor_1 (c->u_mode.ocb.aad_sum, l_tmp.x1, BLOCKSIZE); - wipememory(&l_tmp, sizeof(l_tmp)); + abuf += BLOCKSIZE; } + wipememory(&l_tmp, sizeof(l_tmp)); + if (burn_depth) _gcry_burn_stack (burn_depth + 4 * sizeof(void *)); @@ -1724,89 +1451,56 @@ _gcry_aes_xts_crypt (void *context, unsigned char *tweak, rijndael_cryptfn_t crypt_fn; u64 tweak_lo, tweak_hi, tweak_next_lo, tweak_next_hi, tmp_lo, tmp_hi, carry; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_xts_crypt (ctx, tweak, outbuf, inbuf, nblocks, encrypt); - return; - } -#endif /*USE_AESNI*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_xts_crypt (ctx, tweak, outbuf, inbuf, nblocks, encrypt); - return; - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO_WITH_PPC9LE - else if (ctx->use_ppc9le_crypto) + if (encrypt) { - _gcry_aes_ppc9le_xts_crypt (ctx, tweak, outbuf, inbuf, nblocks, encrypt); - return; - } -#endif /*USE_PPC_CRYPTO_WITH_PPC9LE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - _gcry_aes_ppc8_xts_crypt (ctx, tweak, outbuf, inbuf, nblocks, encrypt); - return; + if (ctx->prefetch_enc_fn) + ctx->prefetch_enc_fn(); + + crypt_fn = ctx->encrypt_fn; } -#endif /*USE_PPC_CRYPTO*/ else { - if (encrypt) - { - if (ctx->prefetch_enc_fn) - ctx->prefetch_enc_fn(); - - crypt_fn = ctx->encrypt_fn; - } - else - { - check_decryption_preparation (ctx); - - if (ctx->prefetch_dec_fn) - ctx->prefetch_dec_fn(); + check_decryption_preparation (ctx); - crypt_fn = ctx->decrypt_fn; - } + if (ctx->prefetch_dec_fn) + ctx->prefetch_dec_fn(); - tweak_next_lo = buf_get_le64 (tweak + 0); - tweak_next_hi = buf_get_le64 (tweak + 8); + crypt_fn = ctx->decrypt_fn; + } - while (nblocks) - { - tweak_lo = tweak_next_lo; - tweak_hi = tweak_next_hi; + tweak_next_lo = buf_get_le64 (tweak + 0); + tweak_next_hi = buf_get_le64 (tweak + 8); - /* Xor-Encrypt/Decrypt-Xor block. */ - tmp_lo = buf_get_le64 (inbuf + 0) ^ tweak_lo; - tmp_hi = buf_get_le64 (inbuf + 8) ^ tweak_hi; + while (nblocks) + { + tweak_lo = tweak_next_lo; + tweak_hi = tweak_next_hi; - buf_put_le64 (outbuf + 0, tmp_lo); - buf_put_le64 (outbuf + 8, tmp_hi); + /* Xor-Encrypt/Decrypt-Xor block. */ + tmp_lo = buf_get_le64 (inbuf + 0) ^ tweak_lo; + tmp_hi = buf_get_le64 (inbuf + 8) ^ tweak_hi; - /* Generate next tweak. */ - carry = -(tweak_next_hi >> 63) & 0x87; - tweak_next_hi = (tweak_next_hi << 1) + (tweak_next_lo >> 63); - tweak_next_lo = (tweak_next_lo << 1) ^ carry; + buf_put_le64 (outbuf + 0, tmp_lo); + buf_put_le64 (outbuf + 8, tmp_hi); - burn_depth = crypt_fn (ctx, outbuf, outbuf); + /* Generate next tweak. */ + carry = -(tweak_next_hi >> 63) & 0x87; + tweak_next_hi = (tweak_next_hi << 1) + (tweak_next_lo >> 63); + tweak_next_lo = (tweak_next_lo << 1) ^ carry; - buf_put_le64 (outbuf + 0, buf_get_le64 (outbuf + 0) ^ tweak_lo); - buf_put_le64 (outbuf + 8, buf_get_le64 (outbuf + 8) ^ tweak_hi); + burn_depth = crypt_fn (ctx, outbuf, outbuf); - outbuf += GCRY_XTS_BLOCK_LEN; - inbuf += GCRY_XTS_BLOCK_LEN; - nblocks--; - } + buf_put_le64 (outbuf + 0, buf_get_le64 (outbuf + 0) ^ tweak_lo); + buf_put_le64 (outbuf + 8, buf_get_le64 (outbuf + 8) ^ tweak_hi); - buf_put_le64 (tweak + 0, tweak_next_lo); - buf_put_le64 (tweak + 8, tweak_next_hi); + outbuf += GCRY_XTS_BLOCK_LEN; + inbuf += GCRY_XTS_BLOCK_LEN; + nblocks--; } + buf_put_le64 (tweak + 0, tweak_next_lo); + buf_put_le64 (tweak + 8, tweak_next_hi); + if (burn_depth) _gcry_burn_stack (burn_depth + 5 * sizeof(void *)); } -- 2.25.1 From jussi.kivilinna at iki.fi Sun Sep 27 20:08:28 2020 From: jussi.kivilinna at iki.fi (Jussi Kivilinna) Date: Sun, 27 Sep 2020 21:08:28 +0300 Subject: [PATCH 4/4] rijndael: clean-up prepare_decryption function In-Reply-To: <20200927180828.744564-1-jussi.kivilinna@iki.fi> References: <20200927180828.744564-1-jussi.kivilinna@iki.fi> Message-ID: <20200927180828.744564-4-jussi.kivilinna@iki.fi> * cipher/rijndael-internal.h (rijndael_prepare_decfn_t): New. (RIJNDAEL_context_s): New member 'prepare_decryption'. * cipher/rijndael-padlock.c (_gcry_aes_padlock_prepare_decryption): New. * cipher/rijndael.c (_gcry_aes_padlock_prepare_decryption): New. (do_setkey): Setup 'ctx->prepare_decryption' for each acceleration type. (prepare_decryption): Remove calls to other prepare decryption functions. (check_decryption_preparation): Call 'ctx->prepare_decryption' instead of 'prepare_decryption'. -- Signed-off-by: Jussi Kivilinna --- cipher/rijndael-internal.h | 2 + cipher/rijndael-padlock.c | 6 ++ cipher/rijndael.c | 135 ++++++++++++++----------------------- 3 files changed, 59 insertions(+), 84 deletions(-) diff --git a/cipher/rijndael-internal.h b/cipher/rijndael-internal.h index bdd3bee1..92c47a9d 100644 --- a/cipher/rijndael-internal.h +++ b/cipher/rijndael-internal.h @@ -127,6 +127,7 @@ typedef unsigned int (*rijndael_cryptfn_t)(const struct RIJNDAEL_context_s *ctx, unsigned char *bx, const unsigned char *ax); typedef void (*rijndael_prefetchfn_t)(void); +typedef void (*rijndael_prepare_decfn_t)(struct RIJNDAEL_context_s *ctx); /* Our context object. */ typedef struct RIJNDAEL_context_s @@ -181,6 +182,7 @@ typedef struct RIJNDAEL_context_s rijndael_cryptfn_t decrypt_fn; rijndael_prefetchfn_t prefetch_enc_fn; rijndael_prefetchfn_t prefetch_dec_fn; + rijndael_prepare_decfn_t prepare_decryption; } RIJNDAEL_context ATTR_ALIGNED_16; /* Macros defining alias for the keyschedules. */ diff --git a/cipher/rijndael-padlock.c b/cipher/rijndael-padlock.c index 234751b9..3af214d7 100644 --- a/cipher/rijndael-padlock.c +++ b/cipher/rijndael-padlock.c @@ -101,4 +101,10 @@ _gcry_aes_padlock_decrypt (const RIJNDAEL_context *ctx, return do_padlock(ctx, bx, ax, 1); } +void +_gcry_aes_padlock_prepare_decryption (RIJNDAEL_context *ctx) +{ + /* Padlock does not need decryption subkeys. */ + (void)ctx; +} #endif /* USE_PADLOCK */ diff --git a/cipher/rijndael.c b/cipher/rijndael.c index bf9c0050..60d16c68 100644 --- a/cipher/rijndael.c +++ b/cipher/rijndael.c @@ -143,6 +143,7 @@ extern unsigned int _gcry_aes_padlock_encrypt (const RIJNDAEL_context *ctx, extern unsigned int _gcry_aes_padlock_decrypt (const RIJNDAEL_context *ctx, unsigned char *bx, const unsigned char *ax); +extern void _gcry_aes_padlock_prepare_decryption (RIJNDAEL_context *ctx); #endif #ifdef USE_ARM_ASM @@ -314,6 +315,7 @@ static void _gcry_aes_xts_crypt (void *context, unsigned char *tweak, /* Function prototypes. */ static const char *selftest(void); +static void prepare_decryption(RIJNDAEL_context *ctx); @@ -465,6 +467,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = _gcry_aes_aesni_decrypt; ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; + ctx->prepare_decryption = _gcry_aes_aesni_prepare_decryption; ctx->use_aesni = 1; ctx->use_avx = !!(hwfeatures & HWF_INTEL_AVX); ctx->use_avx2 = !!(hwfeatures & HWF_INTEL_AVX2); @@ -487,6 +490,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = _gcry_aes_padlock_decrypt; ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; + ctx->prepare_decryption = _gcry_aes_padlock_prepare_decryption; ctx->use_padlock = 1; memcpy (ctx->padlockkey, key, keylen); } @@ -499,6 +503,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = _gcry_aes_ssse3_decrypt; ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; + ctx->prepare_decryption = _gcry_aes_ssse3_prepare_decryption; ctx->use_ssse3 = 1; /* Setup SSSE3 bulk encryption routines. */ @@ -519,6 +524,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = _gcry_aes_armv8_ce_decrypt; ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; + ctx->prepare_decryption = _gcry_aes_armv8_ce_prepare_decryption; ctx->use_arm_ce = 1; /* Setup ARM-CE bulk encryption routines. */ @@ -540,6 +546,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = _gcry_aes_ppc9le_decrypt; ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; + ctx->prepare_decryption = _gcry_aes_ppc8_prepare_decryption; ctx->use_ppc_crypto = 1; /* same key-setup as USE_PPC_CRYPTO */ ctx->use_ppc9le_crypto = 1; @@ -562,6 +569,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = _gcry_aes_ppc8_decrypt; ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; + ctx->prepare_decryption = _gcry_aes_ppc8_prepare_decryption; ctx->use_ppc_crypto = 1; /* Setup PPC8 bulk encryption routines. */ @@ -581,6 +589,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->decrypt_fn = do_decrypt; ctx->prefetch_enc_fn = prefetch_enc; ctx->prefetch_dec_fn = prefetch_dec; + ctx->prepare_decryption = prepare_decryption; } /* NB: We don't yet support Padlock hardware key generation. */ @@ -705,94 +714,52 @@ rijndael_setkey (void *context, const byte *key, const unsigned keylen, static void prepare_decryption( RIJNDAEL_context *ctx ) { + const byte *sbox = ((const byte *)encT) + 1; int r; - if (0) - ; -#ifdef USE_AESNI - else if (ctx->use_aesni) - { - _gcry_aes_aesni_prepare_decryption (ctx); - } -#endif /*USE_AESNI*/ -#ifdef USE_SSSE3 - else if (ctx->use_ssse3) - { - _gcry_aes_ssse3_prepare_decryption (ctx); - } -#endif /*USE_SSSE3*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_prepare_decryption (ctx); - } -#endif /*USE_ARM_CE*/ -#ifdef USE_ARM_CE - else if (ctx->use_arm_ce) - { - _gcry_aes_armv8_ce_prepare_decryption (ctx); - } -#endif /*USE_ARM_CE*/ -#ifdef USE_PPC_CRYPTO - else if (ctx->use_ppc_crypto) - { - _gcry_aes_ppc8_prepare_decryption (ctx); - } -#endif -#ifdef USE_PADLOCK - else if (ctx->use_padlock) - { - /* Padlock does not need decryption subkeys. */ - } -#endif /*USE_PADLOCK*/ - else - { - const byte *sbox = ((const byte *)encT) + 1; + prefetch_enc(); + prefetch_dec(); - prefetch_enc(); - prefetch_dec(); - - ctx->keyschdec32[0][0] = ctx->keyschenc32[0][0]; - ctx->keyschdec32[0][1] = ctx->keyschenc32[0][1]; - ctx->keyschdec32[0][2] = ctx->keyschenc32[0][2]; - ctx->keyschdec32[0][3] = ctx->keyschenc32[0][3]; + ctx->keyschdec32[0][0] = ctx->keyschenc32[0][0]; + ctx->keyschdec32[0][1] = ctx->keyschenc32[0][1]; + ctx->keyschdec32[0][2] = ctx->keyschenc32[0][2]; + ctx->keyschdec32[0][3] = ctx->keyschenc32[0][3]; - for (r = 1; r < ctx->rounds; r++) - { - u32 *wi = ctx->keyschenc32[r]; - u32 *wo = ctx->keyschdec32[r]; - u32 wt; - - wt = wi[0]; - wo[0] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) - ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) - ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) - ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); - - wt = wi[1]; - wo[1] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) - ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) - ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) - ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); - - wt = wi[2]; - wo[2] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) - ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) - ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) - ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); - - wt = wi[3]; - wo[3] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) - ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) - ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) - ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); - } - - ctx->keyschdec32[r][0] = ctx->keyschenc32[r][0]; - ctx->keyschdec32[r][1] = ctx->keyschenc32[r][1]; - ctx->keyschdec32[r][2] = ctx->keyschenc32[r][2]; - ctx->keyschdec32[r][3] = ctx->keyschenc32[r][3]; + for (r = 1; r < ctx->rounds; r++) + { + u32 *wi = ctx->keyschenc32[r]; + u32 *wo = ctx->keyschdec32[r]; + u32 wt; + + wt = wi[0]; + wo[0] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) + ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) + ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) + ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); + + wt = wi[1]; + wo[1] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) + ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) + ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) + ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); + + wt = wi[2]; + wo[2] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) + ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) + ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) + ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); + + wt = wi[3]; + wo[3] = rol(decT[sbox[(byte)(wt >> 0) * 4]], 8 * 0) + ^ rol(decT[sbox[(byte)(wt >> 8) * 4]], 8 * 1) + ^ rol(decT[sbox[(byte)(wt >> 16) * 4]], 8 * 2) + ^ rol(decT[sbox[(byte)(wt >> 24) * 4]], 8 * 3); } + + ctx->keyschdec32[r][0] = ctx->keyschenc32[r][0]; + ctx->keyschdec32[r][1] = ctx->keyschenc32[r][1]; + ctx->keyschdec32[r][2] = ctx->keyschenc32[r][2]; + ctx->keyschdec32[r][3] = ctx->keyschenc32[r][3]; } @@ -1232,7 +1199,7 @@ check_decryption_preparation (RIJNDAEL_context *ctx) { if ( !ctx->decryption_prepared ) { - prepare_decryption ( ctx ); + ctx->prepare_decryption ( ctx ); ctx->decryption_prepared = 1; } } -- 2.25.1 From jussi.kivilinna at iki.fi Sun Sep 27 20:08:26 2020 From: jussi.kivilinna at iki.fi (Jussi Kivilinna) Date: Sun, 27 Sep 2020 21:08:26 +0300 Subject: [PATCH 2/4] cipher: setup bulk functions at each algorithms key setup In-Reply-To: <20200927180828.744564-1-jussi.kivilinna@iki.fi> References: <20200927180828.744564-1-jussi.kivilinna@iki.fi> Message-ID: <20200927180828.744564-2-jussi.kivilinna@iki.fi> * cipher/cipher-internal.h (cipher_mode_ops_t, cipher_bulk_ops_t): New. (gcry_cipher_handle): Define members 'mode_ops' and 'bulk' using new types. * cipher/cipher.c (_gcry_cipher_open_internal): Remove bulk function setup. (cipher_setkey): Pass context bulk function pointer to algorithm setkey function. * cipher/cipher-selftest.c (_gcry_selftest_helper_cbc) (_gcry_selftest_helper_cfb, _gcry_selftest_helper_ctr): Remove bulk function parameter; Use bulk function returned by setkey function. * cipher/cipher-selftest.h (_gcry_selftest_helper_cbc) (_gcry_selftest_helper_cfb, _gcry_selftest_helper_ctr): Remove bulk function parameter. * cipher/arcfour.c (arcfour_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/blowfish.c (bf_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_blowfish_ctr_enc, _gcry_blowfish_cbc_dec) (_gcry_blowfish_cfb_dec): Make static. (selftest_ctr, selftest_cbc, selftest_cfb): Do not pass bulk function to selftest helper. (selftest): Pass 'bulk_ops' to setkey function. * cipher/camellia.c (camellia_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_camellia_ctr_enc, _gcry_camellia_cbc_dec) (_gcry_camellia_cfb_dec, _gcry_camellia_ocb_crypt) (_gcry_camellia_ocb_auth): Make static. (selftest_ctr, selftest_cbc, selftest_cfb): Do not pass bulk function to selftest helper. (selftest): Pass 'bulk_ops' to setkey function. * cipher/cast5.c (cast_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_cast5_ctr_enc, _gcry_cast5_cbc_dec, _gcry_cast5_cfb_dec): Make static. (selftest_ctr, selftest_cbc, selftest_cfb): Do not pass bulk function to selftest helper. (selftest): Pass 'bulk_ops' to setkey function. * cipher/chacha20.c (chacha20_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/cast5.c (do_tripledes_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_3des_ctr_enc, _gcry_3des_cbc_dec, _gcry_3des_cfb_dec): Make static. (bulk_selftest_setkey): Change 'hd' parameter to 'bulk_ops'. (selftest_ctr, selftest_cbc, selftest_cfb): Do not pass bulk function to selftest helper. (do_des_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/gost28147.c (gost_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/idea.c (idea_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/rfc2268.c (do_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/rijndael.c (do_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (rijndael_setkey): Change 'hd' parameter to 'bulk_ops'. (_gcry_aes_cfb_enc, _gcry_aes_cfb_dec, _gcry_aes_cbc_enc) (_gcry_aes_cbc_dec, _gcry_aes_ctr_enc, _gcry_aes_ocb_crypt) (_gcry_aes_ocb_auth, _gcry_aes_xts_crypt): Make static. (selftest_basic_128, selftest_basic_192, selftest_basic_256): Pass 'bulk_ops' to setkey function. (selftest_ctr, selftest_cbc, selftest_cfb): Do not pass bulk function to selftest helper. * cipher/salsa20.c (salsa20_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/seed.c (seed_setkey): Change 'hd' parameter to 'bulk_ops'. * cipher/serpent.c (serpent_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_serpent_ctr_enc, _gcry_serpent_cbc_dec, _gcry_serpent_cfb_dec) (_gcry_serpent_ocb_crypt, _gcry_serpent_ocb_auth): Make static. (selftest_ctr_128, selftest_cbc_128, selftest_cfb_128): Do not pass bulk function to selftest helper. * cipher/sm4.c (sm4_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_sm4_ctr_enc, _gcry_sm4_cbc_dec, _gcry_sm4_cfb_dec) (_gcry_sm4_ocb_crypt, _gcry_sm4_ocb_auth): Make static. (selftest_ctr_128, selftest_cbc_128, selftest_cfb_128): Do not pass bulk function to selftest helper. * cipher/twofish.c (twofish_setkey): Change 'hd' parameter to 'bulk_ops'; Setup 'bulk_ops' with bulk acceleration functions. (_gcry_twofish_ctr_enc, _gcry_twofish_cbc_dec) (_gcry_twofish_cfb_dec, _gcry_twofish_ocb_crypt) (_gcry_twofish_ocb_auth): Make static. (selftest_ctr, selftest_cbc, selftest_cfb): Do not pass bulk function to selftest helper. (selftest, main): Pass 'bulk_ops' to setkey function. * src/cipher-proto.h: Forward declare 'cipher_bulk_ops_t'. (gcry_cipher_setkey_t): Replace 'hd' with 'bulk_ops'. * src/cipher.h: Remove bulk acceleration function prototypes for 'aes', 'blowfish', 'cast5', 'camellia', '3des', 'serpent', 'sm4' and 'twofish'. -- Signed-off-by: Jussi Kivilinna --- cipher/arcfour.c | 4 +- cipher/blowfish.c | 33 ++++--- cipher/camellia-glue.c | 53 +++++++---- cipher/cast5.c | 34 +++---- cipher/chacha20.c | 4 +- cipher/cipher-internal.h | 93 +++++++++---------- cipher/cipher-selftest.c | 29 +++--- cipher/cipher-selftest.h | 5 +- cipher/cipher.c | 88 +----------------- cipher/des.c | 36 ++++---- cipher/gost28147.c | 4 +- cipher/idea.c | 4 +- cipher/rfc2268.c | 4 +- cipher/rijndael.c | 188 ++++++++++++++++++++++----------------- cipher/salsa20.c | 4 +- cipher/seed.c | 4 +- cipher/serpent.c | 48 ++++++---- cipher/sm4.c | 45 +++++++--- cipher/twofish.c | 71 ++++++++++----- src/cipher-proto.h | 4 +- src/cipher.h | 127 -------------------------- 21 files changed, 392 insertions(+), 490 deletions(-) diff --git a/cipher/arcfour.c b/cipher/arcfour.c index 72decf08..9e71857c 100644 --- a/cipher/arcfour.c +++ b/cipher/arcfour.c @@ -171,11 +171,11 @@ do_arcfour_setkey (void *context, const byte *key, unsigned int keylen) static gcry_err_code_t arcfour_setkey ( void *context, const byte *key, unsigned int keylen, - gcry_cipher_hd_t hd ) + cipher_bulk_ops_t *bulk_ops ) { ARCFOUR_context *ctx = (ARCFOUR_context *) context; gcry_err_code_t rc = do_arcfour_setkey (ctx, key, keylen ); - (void)hd; + (void)bulk_ops; return rc; } diff --git a/cipher/blowfish.c b/cipher/blowfish.c index a1d81d31..7b001306 100644 --- a/cipher/blowfish.c +++ b/cipher/blowfish.c @@ -69,7 +69,7 @@ typedef struct { } BLOWFISH_context; static gcry_err_code_t bf_setkey (void *c, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd); + cipher_bulk_ops_t *bulk_ops); static unsigned int encrypt_block (void *bc, byte *outbuf, const byte *inbuf); static unsigned int decrypt_block (void *bc, byte *outbuf, const byte *inbuf); @@ -629,7 +629,7 @@ decrypt_block (void *context, byte *outbuf, const byte *inbuf) /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size BLOWFISH_BLOCKSIZE. */ -void +static void _gcry_blowfish_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -710,7 +710,7 @@ _gcry_blowfish_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_blowfish_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -787,7 +787,7 @@ _gcry_blowfish_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_blowfish_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -866,8 +866,7 @@ selftest_ctr (void) const int context_size = sizeof(BLOWFISH_context); return _gcry_selftest_helper_ctr("BLOWFISH", &bf_setkey, - &encrypt_block, &_gcry_blowfish_ctr_enc, nblocks, blocksize, - context_size); + &encrypt_block, nblocks, blocksize, context_size); } @@ -881,8 +880,7 @@ selftest_cbc (void) const int context_size = sizeof(BLOWFISH_context); return _gcry_selftest_helper_cbc("BLOWFISH", &bf_setkey, - &encrypt_block, &_gcry_blowfish_cbc_dec, nblocks, blocksize, - context_size); + &encrypt_block, nblocks, blocksize, context_size); } @@ -896,8 +894,7 @@ selftest_cfb (void) const int context_size = sizeof(BLOWFISH_context); return _gcry_selftest_helper_cfb("BLOWFISH", &bf_setkey, - &encrypt_block, &_gcry_blowfish_cfb_dec, nblocks, blocksize, - context_size); + &encrypt_block, nblocks, blocksize, context_size); } @@ -905,6 +902,7 @@ static const char* selftest(void) { BLOWFISH_context c; + cipher_bulk_ops_t bulk_ops; byte plain[] = "BLOWFISH"; byte buffer[8]; static const byte plain3[] = @@ -916,7 +914,8 @@ selftest(void) const char *r; bf_setkey( (void *) &c, - (const unsigned char*)"abcdefghijklmnopqrstuvwxyz", 26, NULL ); + (const unsigned char*)"abcdefghijklmnopqrstuvwxyz", 26, + &bulk_ops ); encrypt_block( (void *) &c, buffer, plain ); if( memcmp( buffer, "\x32\x4E\xD0\xFE\xF4\x13\xA2\x03", 8 ) ) return "Blowfish selftest failed (1)."; @@ -924,7 +923,7 @@ selftest(void) if( memcmp( buffer, plain, 8 ) ) return "Blowfish selftest failed (2)."; - bf_setkey( (void *) &c, key3, 8, NULL ); + bf_setkey( (void *) &c, key3, 8, &bulk_ops ); encrypt_block( (void *) &c, buffer, plain3 ); if( memcmp( buffer, cipher3, 8 ) ) return "Blowfish selftest failed (3)."; @@ -1119,11 +1118,17 @@ do_bf_setkey (BLOWFISH_context *c, const byte *key, unsigned keylen) static gcry_err_code_t bf_setkey (void *context, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { BLOWFISH_context *c = (BLOWFISH_context *) context; gcry_err_code_t rc = do_bf_setkey (c, key, keylen); - (void)hd; + + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cfb_dec = _gcry_blowfish_cfb_dec; + bulk_ops->cbc_dec = _gcry_blowfish_cbc_dec; + bulk_ops->ctr_enc = _gcry_blowfish_ctr_enc; + return rc; } diff --git a/cipher/camellia-glue.c b/cipher/camellia-glue.c index 4b0989ea..6577b651 100644 --- a/cipher/camellia-glue.c +++ b/cipher/camellia-glue.c @@ -203,9 +203,24 @@ extern void _gcry_camellia_aesni_avx2_ocb_auth(CAMELLIA_context *ctx, static const char *selftest(void); +static void _gcry_camellia_ctr_enc (void *context, unsigned char *ctr, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_camellia_cbc_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_camellia_cfb_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static size_t _gcry_camellia_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, + int encrypt); +static size_t _gcry_camellia_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, + size_t nblocks); + static gcry_err_code_t camellia_setkey(void *c, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { CAMELLIA_context *ctx=c; static int initialized=0; @@ -214,8 +229,6 @@ camellia_setkey(void *c, const byte *key, unsigned keylen, unsigned int hwf = _gcry_get_hw_features (); #endif - (void)hd; - if(keylen!=16 && keylen!=24 && keylen!=32) return GPG_ERR_INV_KEYLEN; @@ -239,6 +252,14 @@ camellia_setkey(void *c, const byte *key, unsigned keylen, ctx->keybitlength=keylen*8; + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cbc_dec = _gcry_camellia_cbc_dec; + bulk_ops->cfb_dec = _gcry_camellia_cfb_dec; + bulk_ops->ctr_enc = _gcry_camellia_ctr_enc; + bulk_ops->ocb_crypt = _gcry_camellia_ocb_crypt; + bulk_ops->ocb_auth = _gcry_camellia_ocb_auth; + if (0) { } #ifdef USE_AESNI_AVX @@ -353,7 +374,7 @@ camellia_decrypt(void *c, byte *outbuf, const byte *inbuf) /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size CAMELLIA_BLOCK_SIZE. */ -void +static void _gcry_camellia_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -442,7 +463,7 @@ _gcry_camellia_ctr_enc(void *context, unsigned char *ctr, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_camellia_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -529,7 +550,7 @@ _gcry_camellia_cbc_dec(void *context, unsigned char *iv, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_camellia_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -609,7 +630,7 @@ _gcry_camellia_cfb_dec(void *context, unsigned char *iv, } /* Bulk encryption/decryption of complete blocks in OCB mode. */ -size_t +static size_t _gcry_camellia_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int encrypt) { @@ -762,7 +783,7 @@ _gcry_camellia_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, } /* Bulk authentication of complete blocks in OCB mode. */ -size_t +static size_t _gcry_camellia_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) { @@ -912,8 +933,7 @@ selftest_ctr_128 (void) const int context_size = sizeof(CAMELLIA_context); return _gcry_selftest_helper_ctr("CAMELLIA", &camellia_setkey, - &camellia_encrypt, &_gcry_camellia_ctr_enc, nblocks, blocksize, - context_size); + &camellia_encrypt, nblocks, blocksize, context_size); } /* Run the self-tests for CAMELLIA-CBC-128, tests bulk CBC decryption. @@ -926,8 +946,7 @@ selftest_cbc_128 (void) const int context_size = sizeof(CAMELLIA_context); return _gcry_selftest_helper_cbc("CAMELLIA", &camellia_setkey, - &camellia_encrypt, &_gcry_camellia_cbc_dec, nblocks, blocksize, - context_size); + &camellia_encrypt, nblocks, blocksize, context_size); } /* Run the self-tests for CAMELLIA-CFB-128, tests bulk CFB decryption. @@ -940,8 +959,7 @@ selftest_cfb_128 (void) const int context_size = sizeof(CAMELLIA_context); return _gcry_selftest_helper_cfb("CAMELLIA", &camellia_setkey, - &camellia_encrypt, &_gcry_camellia_cfb_dec, nblocks, blocksize, - context_size); + &camellia_encrypt, nblocks, blocksize, context_size); } static const char * @@ -949,6 +967,7 @@ selftest(void) { CAMELLIA_context ctx; byte scratch[16]; + cipher_bulk_ops_t bulk_ops; const char *r; /* These test vectors are from RFC-3713 */ @@ -989,7 +1008,7 @@ selftest(void) 0x20,0xef,0x7c,0x91,0x9e,0x3a,0x75,0x09 }; - camellia_setkey(&ctx,key_128,sizeof(key_128),NULL); + camellia_setkey(&ctx,key_128,sizeof(key_128),&bulk_ops); camellia_encrypt(&ctx,scratch,plaintext); if(memcmp(scratch,ciphertext_128,sizeof(ciphertext_128))!=0) return "CAMELLIA-128 test encryption failed."; @@ -997,7 +1016,7 @@ selftest(void) if(memcmp(scratch,plaintext,sizeof(plaintext))!=0) return "CAMELLIA-128 test decryption failed."; - camellia_setkey(&ctx,key_192,sizeof(key_192),NULL); + camellia_setkey(&ctx,key_192,sizeof(key_192),&bulk_ops); camellia_encrypt(&ctx,scratch,plaintext); if(memcmp(scratch,ciphertext_192,sizeof(ciphertext_192))!=0) return "CAMELLIA-192 test encryption failed."; @@ -1005,7 +1024,7 @@ selftest(void) if(memcmp(scratch,plaintext,sizeof(plaintext))!=0) return "CAMELLIA-192 test decryption failed."; - camellia_setkey(&ctx,key_256,sizeof(key_256),NULL); + camellia_setkey(&ctx,key_256,sizeof(key_256),&bulk_ops); camellia_encrypt(&ctx,scratch,plaintext); if(memcmp(scratch,ciphertext_256,sizeof(ciphertext_256))!=0) return "CAMELLIA-256 test encryption failed."; diff --git a/cipher/cast5.c b/cipher/cast5.c index 7219e3ea..837ea0fe 100644 --- a/cipher/cast5.c +++ b/cipher/cast5.c @@ -74,7 +74,7 @@ typedef struct { } CAST5_context; static gcry_err_code_t cast_setkey (void *c, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd); + cipher_bulk_ops_t *bulk_ops); static unsigned int encrypt_block (void *c, byte *outbuf, const byte *inbuf); static unsigned int decrypt_block (void *c, byte *outbuf, const byte *inbuf); @@ -765,7 +765,7 @@ do_decrypt_block_3 (CAST5_context *c, byte *outbuf, const byte *inbuf ) /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size CAST5_BLOCKSIZE. */ -void +static void _gcry_cast5_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -846,7 +846,7 @@ _gcry_cast5_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_cast5_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -922,7 +922,7 @@ _gcry_cast5_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_cast5_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -1001,8 +1001,7 @@ selftest_ctr (void) const int context_size = sizeof(CAST5_context); return _gcry_selftest_helper_ctr("CAST5", &cast_setkey, - &encrypt_block, &_gcry_cast5_ctr_enc, nblocks, blocksize, - context_size); + &encrypt_block, nblocks, blocksize, context_size); } @@ -1016,8 +1015,7 @@ selftest_cbc (void) const int context_size = sizeof(CAST5_context); return _gcry_selftest_helper_cbc("CAST5", &cast_setkey, - &encrypt_block, &_gcry_cast5_cbc_dec, nblocks, blocksize, - context_size); + &encrypt_block, nblocks, blocksize, context_size); } @@ -1031,8 +1029,7 @@ selftest_cfb (void) const int context_size = sizeof(CAST5_context); return _gcry_selftest_helper_cfb("CAST5", &cast_setkey, - &encrypt_block, &_gcry_cast5_cfb_dec, nblocks, blocksize, - context_size); + &encrypt_block, nblocks, blocksize, context_size); } @@ -1040,6 +1037,7 @@ static const char* selftest(void) { CAST5_context c; + cipher_bulk_ops_t bulk_ops; static const byte key[16] = { 0x01, 0x23, 0x45, 0x67, 0x12, 0x34, 0x56, 0x78, 0x23, 0x45, 0x67, 0x89, 0x34, 0x56, 0x78, 0x9A }; @@ -1050,7 +1048,7 @@ selftest(void) byte buffer[8]; const char *r; - cast_setkey( &c, key, 16, NULL ); + cast_setkey( &c, key, 16, &bulk_ops ); encrypt_block( &c, buffer, plain ); if( memcmp( buffer, cipher, 8 ) ) return "1"; @@ -1071,10 +1069,10 @@ selftest(void) 0x80,0xAC,0x05,0xB8,0xE8,0x3D,0x69,0x6E }; for(i=0; i < 1000000; i++ ) { - cast_setkey( &c, b0, 16, NULL ); + cast_setkey( &c, b0, 16, &bulk_ops ); encrypt_block( &c, a0, a0 ); encrypt_block( &c, a0+8, a0+8 ); - cast_setkey( &c, a0, 16, NULL ); + cast_setkey( &c, a0, 16, &bulk_ops ); encrypt_block( &c, b0, b0 ); encrypt_block( &c, b0+8, b0+8 ); } @@ -1217,11 +1215,17 @@ do_cast_setkey( CAST5_context *c, const byte *key, unsigned keylen ) static gcry_err_code_t cast_setkey (void *context, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd ) + cipher_bulk_ops_t *bulk_ops) { CAST5_context *c = (CAST5_context *) context; gcry_err_code_t rc = do_cast_setkey (c, key, keylen); - (void)hd; + + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cfb_dec = _gcry_cast5_cfb_dec; + bulk_ops->cbc_dec = _gcry_cast5_cbc_dec; + bulk_ops->ctr_enc = _gcry_cast5_ctr_enc; + return rc; } diff --git a/cipher/chacha20.c b/cipher/chacha20.c index 9d95723b..c5967b6f 100644 --- a/cipher/chacha20.c +++ b/cipher/chacha20.c @@ -452,11 +452,11 @@ chacha20_do_setkey (CHACHA20_context_t *ctx, static gcry_err_code_t chacha20_setkey (void *context, const byte *key, unsigned int keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { CHACHA20_context_t *ctx = (CHACHA20_context_t *) context; gcry_err_code_t rc = chacha20_do_setkey (ctx, key, keylen); - (void)hd; + (void)bulk_ops; _gcry_burn_stack (4 + sizeof (void *) + 4 * sizeof (void *)); return rc; } diff --git a/cipher/cipher-internal.h b/cipher/cipher-internal.h index 262ca902..a2d0e5c7 100644 --- a/cipher/cipher-internal.h +++ b/cipher/cipher-internal.h @@ -100,6 +100,51 @@ typedef unsigned int (*ghash_fn_t) (gcry_cipher_hd_t c, byte *result, const byte *buf, size_t nblocks); +/* A structure with function pointers for mode operations. */ +typedef struct cipher_mode_ops +{ + gcry_err_code_t (*encrypt)(gcry_cipher_hd_t c, unsigned char *outbuf, + size_t outbuflen, const unsigned char *inbuf, + size_t inbuflen); + gcry_err_code_t (*decrypt)(gcry_cipher_hd_t c, unsigned char *outbuf, + size_t outbuflen, const unsigned char *inbuf, + size_t inbuflen); + gcry_err_code_t (*setiv)(gcry_cipher_hd_t c, const unsigned char *iv, + size_t ivlen); + + gcry_err_code_t (*authenticate)(gcry_cipher_hd_t c, + const unsigned char *abuf, size_t abuflen); + gcry_err_code_t (*get_tag)(gcry_cipher_hd_t c, unsigned char *outtag, + size_t taglen); + gcry_err_code_t (*check_tag)(gcry_cipher_hd_t c, const unsigned char *intag, + size_t taglen); +} cipher_mode_ops_t; + + +/* A structure with function pointers for bulk operations. The cipher + algorithm setkey function initializes them when bulk operations are + available and the actual encryption routines use them if they are + not NULL. */ +typedef struct cipher_bulk_ops +{ + void (*cfb_enc)(void *context, unsigned char *iv, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks); + void (*cfb_dec)(void *context, unsigned char *iv, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks); + void (*cbc_enc)(void *context, unsigned char *iv, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, int cbc_mac); + void (*cbc_dec)(void *context, unsigned char *iv, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks); + void (*ctr_enc)(void *context, unsigned char *iv, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks); + size_t (*ocb_crypt)(gcry_cipher_hd_t c, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, int encrypt); + size_t (*ocb_auth)(gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks); + void (*xts_crypt)(void *context, unsigned char *tweak, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, int encrypt); +} cipher_bulk_ops_t; + + /* A VIA processor with the Padlock engine as well as the Intel AES_NI instructions require an alignment of most data on a 16 byte boundary. Because we trick out the compiler while allocating the @@ -150,54 +195,12 @@ struct gcry_cipher_handle int algo; /* A structure with function pointers for mode operations. */ - struct { - gcry_err_code_t (*encrypt)(gcry_cipher_hd_t c, - unsigned char *outbuf, size_t outbuflen, - const unsigned char *inbuf, size_t inbuflen); - gcry_err_code_t (*decrypt)(gcry_cipher_hd_t c, - unsigned char *outbuf, size_t outbuflen, - const unsigned char *inbuf, size_t inbuflen); - gcry_err_code_t (*setiv)(gcry_cipher_hd_t c, const unsigned char *iv, - size_t ivlen); - - gcry_err_code_t (*authenticate)(gcry_cipher_hd_t c, - const unsigned char *abuf, size_t abuflen); - gcry_err_code_t (*get_tag)(gcry_cipher_hd_t c, unsigned char *outtag, - size_t taglen); - gcry_err_code_t (*check_tag)(gcry_cipher_hd_t c, const unsigned char *intag, - size_t taglen); - } mode_ops; + cipher_mode_ops_t mode_ops; /* A structure with function pointers for bulk operations. Due to limitations of the module system (we don't want to change the - API) we need to keep these function pointers here. The cipher - open function initializes them and the actual encryption routines - use them if they are not NULL. */ - struct { - void (*cfb_enc)(void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - void (*cfb_dec)(void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - void (*cbc_enc)(void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks, int cbc_mac); - void (*cbc_dec)(void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - void (*ctr_enc)(void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - size_t (*ocb_crypt)(gcry_cipher_hd_t c, void *outbuf_arg, - const void *inbuf_arg, size_t nblocks, int encrypt); - size_t (*ocb_auth)(gcry_cipher_hd_t c, const void *abuf_arg, - size_t nblocks); - void (*xts_crypt)(void *context, unsigned char *tweak, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks, int encrypt); - } bulk; - + API) we need to keep these function pointers here. */ + cipher_bulk_ops_t bulk; int mode; unsigned int flags; diff --git a/cipher/cipher-selftest.c b/cipher/cipher-selftest.c index eb3614ad..d7f38a42 100644 --- a/cipher/cipher-selftest.c +++ b/cipher/cipher-selftest.c @@ -1,5 +1,5 @@ /* cipher-selftest.c - Helper functions for bulk encryption selftests. - * Copyright (C) 2013 Jussi Kivilinna + * Copyright (C) 2013,2020 Jussi Kivilinna * * This file is part of Libgcrypt. * @@ -27,6 +27,7 @@ #include "cipher.h" #include "bufhelp.h" #include "cipher-selftest.h" +#include "cipher-internal.h" #ifdef HAVE_STDINT_H # include /* uintptr_t */ @@ -72,10 +73,10 @@ _gcry_cipher_selftest_alloc_ctx (const int context_size, unsigned char **r_mem) const char * _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey_func, gcry_cipher_encrypt_t encrypt_one, - gcry_cipher_bulk_cbc_dec_t bulk_cbc_dec, const int nblocks, const int blocksize, const int context_size) { + cipher_bulk_ops_t bulk_ops = { 0, }; int i, offs; unsigned char *ctx, *plaintext, *plaintext2, *ciphertext, *iv, *iv2, *mem; unsigned int ctx_aligned_size, memsize; @@ -105,7 +106,7 @@ _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey_func, ciphertext = plaintext2 + nblocks * blocksize; /* Initialize ctx */ - if (setkey_func (ctx, key, sizeof(key), NULL) != GPG_ERR_NO_ERROR) + if (setkey_func (ctx, key, sizeof(key), &bulk_ops) != GPG_ERR_NO_ERROR) { xfree(mem); return "setkey failed"; @@ -123,7 +124,7 @@ _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey_func, memcpy (iv, ciphertext, blocksize); /* CBC decrypt. */ - bulk_cbc_dec (ctx, iv2, plaintext2, ciphertext, 1); + bulk_ops.cbc_dec (ctx, iv2, plaintext2, ciphertext, 1); if (memcmp (plaintext2, plaintext, blocksize)) { xfree (mem); @@ -163,7 +164,7 @@ _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey_func, } /* Decrypt using bulk CBC and compare result. */ - bulk_cbc_dec (ctx, iv2, plaintext2, ciphertext, nblocks); + bulk_ops.cbc_dec (ctx, iv2, plaintext2, ciphertext, nblocks); if (memcmp (plaintext2, plaintext, nblocks * blocksize)) { @@ -195,10 +196,10 @@ _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey_func, const char * _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey_func, gcry_cipher_encrypt_t encrypt_one, - gcry_cipher_bulk_cfb_dec_t bulk_cfb_dec, const int nblocks, const int blocksize, const int context_size) { + cipher_bulk_ops_t bulk_ops = { 0, }; int i, offs; unsigned char *ctx, *plaintext, *plaintext2, *ciphertext, *iv, *iv2, *mem; unsigned int ctx_aligned_size, memsize; @@ -228,7 +229,7 @@ _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey_func, ciphertext = plaintext2 + nblocks * blocksize; /* Initialize ctx */ - if (setkey_func (ctx, key, sizeof(key), NULL) != GPG_ERR_NO_ERROR) + if (setkey_func (ctx, key, sizeof(key), &bulk_ops) != GPG_ERR_NO_ERROR) { xfree(mem); return "setkey failed"; @@ -245,7 +246,7 @@ _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey_func, buf_xor_2dst (iv, ciphertext, plaintext, blocksize); /* CFB decrypt. */ - bulk_cfb_dec (ctx, iv2, plaintext2, ciphertext, 1); + bulk_ops.cfb_dec (ctx, iv2, plaintext2, ciphertext, 1); if (memcmp(plaintext2, plaintext, blocksize)) { xfree(mem); @@ -284,7 +285,7 @@ _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey_func, } /* Decrypt using bulk CBC and compare result. */ - bulk_cfb_dec (ctx, iv2, plaintext2, ciphertext, nblocks); + bulk_ops.cfb_dec (ctx, iv2, plaintext2, ciphertext, nblocks); if (memcmp(plaintext2, plaintext, nblocks * blocksize)) { @@ -316,10 +317,10 @@ _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey_func, const char * _gcry_selftest_helper_ctr (const char *cipher, gcry_cipher_setkey_t setkey_func, gcry_cipher_encrypt_t encrypt_one, - gcry_cipher_bulk_ctr_enc_t bulk_ctr_enc, const int nblocks, const int blocksize, const int context_size) { + cipher_bulk_ops_t bulk_ops = { 0, }; int i, j, offs, diff; unsigned char *ctx, *plaintext, *plaintext2, *ciphertext, *ciphertext2, *iv, *iv2, *mem; @@ -351,7 +352,7 @@ _gcry_selftest_helper_ctr (const char *cipher, gcry_cipher_setkey_t setkey_func, ciphertext2 = ciphertext + nblocks * blocksize; /* Initialize ctx */ - if (setkey_func (ctx, key, sizeof(key), NULL) != GPG_ERR_NO_ERROR) + if (setkey_func (ctx, key, sizeof(key), &bulk_ops) != GPG_ERR_NO_ERROR) { xfree(mem); return "setkey failed"; @@ -374,7 +375,7 @@ _gcry_selftest_helper_ctr (const char *cipher, gcry_cipher_setkey_t setkey_func, } memset (iv2, 0xff, blocksize); - bulk_ctr_enc (ctx, iv2, plaintext2, ciphertext, 1); + bulk_ops.ctr_enc (ctx, iv2, plaintext2, ciphertext, 1); if (memcmp (plaintext2, plaintext, blocksize)) { @@ -429,7 +430,7 @@ _gcry_selftest_helper_ctr (const char *cipher, gcry_cipher_setkey_t setkey_func, } } - bulk_ctr_enc (ctx, iv2, ciphertext2, plaintext2, nblocks); + bulk_ops.ctr_enc (ctx, iv2, ciphertext2, plaintext2, nblocks); if (memcmp (ciphertext2, ciphertext, blocksize * nblocks)) { @@ -482,7 +483,7 @@ _gcry_selftest_helper_ctr (const char *cipher, gcry_cipher_setkey_t setkey_func, iv2[0] = iv2[1] = 0; iv2[2] = 0x07; - bulk_ctr_enc (ctx, iv2, plaintext2, ciphertext, nblocks); + bulk_ops.ctr_enc (ctx, iv2, plaintext2, ciphertext, nblocks); if (memcmp (plaintext2, plaintext, blocksize * nblocks)) { diff --git a/cipher/cipher-selftest.h b/cipher/cipher-selftest.h index a435080f..c3090ad1 100644 --- a/cipher/cipher-selftest.h +++ b/cipher/cipher-selftest.h @@ -1,5 +1,5 @@ /* cipher-selftest.h - Helper functions for bulk encryption selftests. - * Copyright (C) 2013 Jussi Kivilinna + * Copyright (C) 2013,2020 Jussi Kivilinna * * This file is part of Libgcrypt. * @@ -49,7 +49,6 @@ void *_gcry_cipher_selftest_alloc_ctx (const int context_size, const char * _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey, gcry_cipher_encrypt_t encrypt_one, - gcry_cipher_bulk_cbc_dec_t bulk_cbc_dec, const int nblocks, const int blocksize, const int context_size); @@ -57,7 +56,6 @@ _gcry_selftest_helper_cbc (const char *cipher, gcry_cipher_setkey_t setkey, const char * _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey, gcry_cipher_encrypt_t encrypt_one, - gcry_cipher_bulk_cfb_dec_t bulk_cfb_dec, const int nblocks, const int blocksize, const int context_size); @@ -65,7 +63,6 @@ _gcry_selftest_helper_cfb (const char *cipher, gcry_cipher_setkey_t setkey, const char * _gcry_selftest_helper_ctr (const char *cipher, gcry_cipher_setkey_t setkey, gcry_cipher_encrypt_t encrypt_one, - gcry_cipher_bulk_ctr_enc_t bulk_ctr_enc, const int nblocks, const int blocksize, const int context_size); diff --git a/cipher/cipher.c b/cipher/cipher.c index c77c9682..1039dff7 100644 --- a/cipher/cipher.c +++ b/cipher/cipher.c @@ -647,90 +647,6 @@ _gcry_cipher_open_internal (gcry_cipher_hd_t *handle, h->mode = mode; h->flags = flags; - /* Setup bulk encryption routines. */ - switch (algo) - { -#ifdef USE_AES - case GCRY_CIPHER_AES128: - case GCRY_CIPHER_AES192: - case GCRY_CIPHER_AES256: - h->bulk.cfb_enc = _gcry_aes_cfb_enc; - h->bulk.cfb_dec = _gcry_aes_cfb_dec; - h->bulk.cbc_enc = _gcry_aes_cbc_enc; - h->bulk.cbc_dec = _gcry_aes_cbc_dec; - h->bulk.ctr_enc = _gcry_aes_ctr_enc; - h->bulk.ocb_crypt = _gcry_aes_ocb_crypt; - h->bulk.ocb_auth = _gcry_aes_ocb_auth; - h->bulk.xts_crypt = _gcry_aes_xts_crypt; - break; -#endif /*USE_AES*/ -#ifdef USE_BLOWFISH - case GCRY_CIPHER_BLOWFISH: - h->bulk.cfb_dec = _gcry_blowfish_cfb_dec; - h->bulk.cbc_dec = _gcry_blowfish_cbc_dec; - h->bulk.ctr_enc = _gcry_blowfish_ctr_enc; - break; -#endif /*USE_BLOWFISH*/ -#ifdef USE_CAST5 - case GCRY_CIPHER_CAST5: - h->bulk.cfb_dec = _gcry_cast5_cfb_dec; - h->bulk.cbc_dec = _gcry_cast5_cbc_dec; - h->bulk.ctr_enc = _gcry_cast5_ctr_enc; - break; -#endif /*USE_CAMELLIA*/ -#ifdef USE_CAMELLIA - case GCRY_CIPHER_CAMELLIA128: - case GCRY_CIPHER_CAMELLIA192: - case GCRY_CIPHER_CAMELLIA256: - h->bulk.cbc_dec = _gcry_camellia_cbc_dec; - h->bulk.cfb_dec = _gcry_camellia_cfb_dec; - h->bulk.ctr_enc = _gcry_camellia_ctr_enc; - h->bulk.ocb_crypt = _gcry_camellia_ocb_crypt; - h->bulk.ocb_auth = _gcry_camellia_ocb_auth; - break; -#endif /*USE_CAMELLIA*/ -#ifdef USE_DES - case GCRY_CIPHER_3DES: - h->bulk.cbc_dec = _gcry_3des_cbc_dec; - h->bulk.cfb_dec = _gcry_3des_cfb_dec; - h->bulk.ctr_enc = _gcry_3des_ctr_enc; - break; -#endif /*USE_DES*/ -#ifdef USE_SERPENT - case GCRY_CIPHER_SERPENT128: - case GCRY_CIPHER_SERPENT192: - case GCRY_CIPHER_SERPENT256: - h->bulk.cbc_dec = _gcry_serpent_cbc_dec; - h->bulk.cfb_dec = _gcry_serpent_cfb_dec; - h->bulk.ctr_enc = _gcry_serpent_ctr_enc; - h->bulk.ocb_crypt = _gcry_serpent_ocb_crypt; - h->bulk.ocb_auth = _gcry_serpent_ocb_auth; - break; -#endif /*USE_SERPENT*/ -#ifdef USE_SM4 - case GCRY_CIPHER_SM4: - h->bulk.cbc_dec = _gcry_sm4_cbc_dec; - h->bulk.cfb_dec = _gcry_sm4_cfb_dec; - h->bulk.ctr_enc = _gcry_sm4_ctr_enc; - h->bulk.ocb_crypt = _gcry_sm4_ocb_crypt; - h->bulk.ocb_auth = _gcry_sm4_ocb_auth; - break; -#endif /*USE_SM4*/ -#ifdef USE_TWOFISH - case GCRY_CIPHER_TWOFISH: - case GCRY_CIPHER_TWOFISH128: - h->bulk.cbc_dec = _gcry_twofish_cbc_dec; - h->bulk.cfb_dec = _gcry_twofish_cfb_dec; - h->bulk.ctr_enc = _gcry_twofish_ctr_enc; - h->bulk.ocb_crypt = _gcry_twofish_ocb_crypt; - h->bulk.ocb_auth = _gcry_twofish_ocb_auth; - break; -#endif /*USE_TWOFISH*/ - - default: - break; - } - /* Setup mode routines. */ _gcry_cipher_setup_mode_ops(h, mode); @@ -816,7 +732,7 @@ cipher_setkey (gcry_cipher_hd_t c, byte *key, size_t keylen) } } - rc = c->spec->setkey (&c->context.c, key, keylen, c); + rc = c->spec->setkey (&c->context.c, key, keylen, &c->bulk); if (!rc || (c->marks.allow_weak_key && rc == GPG_ERR_WEAK_KEY)) { /* Duplicate initial context. */ @@ -850,7 +766,7 @@ cipher_setkey (gcry_cipher_hd_t c, byte *key, size_t keylen) case GCRY_CIPHER_MODE_XTS: /* Setup tweak cipher with second part of XTS key. */ rc = c->spec->setkey (c->u_mode.xts.tweak_context, key + keylen, - keylen, c); + keylen, &c->bulk); if (!rc || (c->marks.allow_weak_key && rc == GPG_ERR_WEAK_KEY)) { /* Duplicate initial tweak context. */ diff --git a/cipher/des.c b/cipher/des.c index e4d10caa..1580ea4e 100644 --- a/cipher/des.c +++ b/cipher/des.c @@ -199,7 +199,7 @@ static unsigned int do_tripledes_decrypt(void *context, byte *outbuf, const byte *inbuf ); static gcry_err_code_t do_tripledes_setkey(void *context, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd); + cipher_bulk_ops_t *bulk_ops); static int initialized; @@ -872,7 +872,7 @@ tripledes_ecb_crypt (struct _tripledes_ctx *ctx, const byte * from, /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size DES_BLOCKSIZE. */ -void +static void _gcry_3des_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -922,7 +922,7 @@ _gcry_3des_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_3des_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -971,7 +971,7 @@ _gcry_3des_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_3des_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -1050,7 +1050,7 @@ is_weak_key ( const byte *key ) /* Alternative setkey for selftests; need larger key than default. */ static gcry_err_code_t bulk_selftest_setkey (void *context, const byte *__key, unsigned __keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { static const unsigned char key[24] ATTR_ALIGNED_16 = { 0x66,0x9A,0x00,0x7F,0xC7,0x6A,0x45,0x9F, @@ -1058,11 +1058,10 @@ bulk_selftest_setkey (void *context, const byte *__key, unsigned __keylen, 0x18,0x2A,0x39,0x47,0x5E,0x6F,0x75,0x82 }; - (void)hd; (void)__key; (void)__keylen; - return do_tripledes_setkey(context, key, sizeof(key), NULL); + return do_tripledes_setkey(context, key, sizeof(key), bulk_ops); } @@ -1076,8 +1075,7 @@ selftest_ctr (void) const int context_size = sizeof(struct _tripledes_ctx); return _gcry_selftest_helper_ctr("3DES", &bulk_selftest_setkey, - &do_tripledes_encrypt, &_gcry_3des_ctr_enc, nblocks, blocksize, - context_size); + &do_tripledes_encrypt, nblocks, blocksize, context_size); } @@ -1091,8 +1089,7 @@ selftest_cbc (void) const int context_size = sizeof(struct _tripledes_ctx); return _gcry_selftest_helper_cbc("3DES", &bulk_selftest_setkey, - &do_tripledes_encrypt, &_gcry_3des_cbc_dec, nblocks, blocksize, - context_size); + &do_tripledes_encrypt, nblocks, blocksize, context_size); } @@ -1106,8 +1103,7 @@ selftest_cfb (void) const int context_size = sizeof(struct _tripledes_ctx); return _gcry_selftest_helper_cfb("3DES", &bulk_selftest_setkey, - &do_tripledes_encrypt, &_gcry_3des_cfb_dec, nblocks, blocksize, - context_size); + &do_tripledes_encrypt, nblocks, blocksize, context_size); } @@ -1315,15 +1311,19 @@ selftest (void) static gcry_err_code_t do_tripledes_setkey ( void *context, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd ) + cipher_bulk_ops_t *bulk_ops ) { struct _tripledes_ctx *ctx = (struct _tripledes_ctx *) context; - (void)hd; - if( keylen != 24 ) return GPG_ERR_INV_KEYLEN; + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cbc_dec = _gcry_3des_cbc_dec; + bulk_ops->cfb_dec = _gcry_3des_cfb_dec; + bulk_ops->ctr_enc = _gcry_3des_ctr_enc; + tripledes_set3keys ( ctx, key, key+8, key+16); if (ctx->flags.no_weak_key) @@ -1382,11 +1382,11 @@ do_tripledes_decrypt( void *context, byte *outbuf, const byte *inbuf ) static gcry_err_code_t do_des_setkey (void *context, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { struct _des_ctx *ctx = (struct _des_ctx *) context; - (void)hd; + (void)bulk_ops; if (keylen != 8) return GPG_ERR_INV_KEYLEN; diff --git a/cipher/gost28147.c b/cipher/gost28147.c index 24385915..df16c3c6 100644 --- a/cipher/gost28147.c +++ b/cipher/gost28147.c @@ -48,12 +48,12 @@ gost_do_set_sbox (GOST28147_context *ctx, unsigned int index) static gcry_err_code_t gost_setkey (void *c, const byte *key, unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { int i; GOST28147_context *ctx = c; - (void)hd; + (void)bulk_ops; if (keylen != 256 / 8) return GPG_ERR_INV_KEYLEN; diff --git a/cipher/idea.c b/cipher/idea.c index abfe6755..a503c08f 100644 --- a/cipher/idea.c +++ b/cipher/idea.c @@ -259,11 +259,11 @@ do_setkey( IDEA_context *c, const byte *key, unsigned int keylen ) static gcry_err_code_t idea_setkey (void *context, const byte *key, unsigned int keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { IDEA_context *ctx = context; int rc = do_setkey (ctx, key, keylen); - (void)hd; + (void)bulk_ops; _gcry_burn_stack (23+6*sizeof(void*)); return rc; } diff --git a/cipher/rfc2268.c b/cipher/rfc2268.c index 09149462..348c4d41 100644 --- a/cipher/rfc2268.c +++ b/cipher/rfc2268.c @@ -263,9 +263,9 @@ setkey_core (void *context, const unsigned char *key, unsigned int keylen, int w static gpg_err_code_t do_setkey (void *context, const unsigned char *key, unsigned int keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { - (void)hd; + (void)bulk_ops; return setkey_core (context, key, keylen, 1); } diff --git a/cipher/rijndael.c b/cipher/rijndael.c index d81ed809..ac70ba75 100644 --- a/cipher/rijndael.c +++ b/cipher/rijndael.c @@ -281,6 +281,29 @@ static unsigned int do_encrypt (const RIJNDAEL_context *ctx, unsigned char *bx, static unsigned int do_decrypt (const RIJNDAEL_context *ctx, unsigned char *bx, const unsigned char *ax); +static void _gcry_aes_cfb_enc (void *context, unsigned char *iv, + void *outbuf, const void *inbuf, + size_t nblocks); +static void _gcry_aes_cfb_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_aes_cbc_enc (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks, int cbc_mac); +static void _gcry_aes_cbc_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_aes_ctr_enc (void *context, unsigned char *ctr, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static size_t _gcry_aes_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, + int encrypt); +static size_t _gcry_aes_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, + size_t nblocks); +static void _gcry_aes_xts_crypt (void *context, unsigned char *tweak, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks, int encrypt); /* All the numbers. */ @@ -349,7 +372,7 @@ static void prefetch_dec(void) /* Perform the key setup. */ static gcry_err_code_t do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { static int initialized = 0; static const char *selftest_failed = 0; @@ -357,12 +380,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, int rounds; int i,j, r, t, rconpointer = 0; int KC; -#if defined(USE_AESNI) || defined(USE_PADLOCK) || defined(USE_SSSE3) \ - || defined(USE_ARM_CE) || defined(USE_PPC_CRYPTO) unsigned int hwfeatures; -#endif - - (void)hd; /* The on-the-fly self tests are only run in non-fips mode. In fips mode explicit self-tests are required. Actually the on-the-fly @@ -400,11 +418,7 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, return GPG_ERR_INV_KEYLEN; ctx->rounds = rounds; - -#if defined(USE_AESNI) || defined(USE_PADLOCK) || defined(USE_SSSE3) \ - || defined(USE_ARM_CE) || defined(USE_PPC_CRYPTO) hwfeatures = _gcry_get_hw_features (); -#endif ctx->decryption_prepared = 0; #ifdef USE_PADLOCK @@ -426,6 +440,19 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->use_ppc9le_crypto = 0; #endif + /* Setup default bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cfb_enc = _gcry_aes_cfb_enc; + bulk_ops->cfb_dec = _gcry_aes_cfb_dec; + bulk_ops->cbc_enc = _gcry_aes_cbc_enc; + bulk_ops->cbc_dec = _gcry_aes_cbc_dec; + bulk_ops->ctr_enc = _gcry_aes_ctr_enc; + bulk_ops->ocb_crypt = _gcry_aes_ocb_crypt; + bulk_ops->ocb_auth = _gcry_aes_ocb_auth; + bulk_ops->xts_crypt = _gcry_aes_xts_crypt; + + (void)hwfeatures; + if (0) { ; @@ -441,17 +468,16 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->use_aesni = 1; ctx->use_avx = !!(hwfeatures & HWF_INTEL_AVX); ctx->use_avx2 = !!(hwfeatures & HWF_INTEL_AVX2); - if (hd) - { - hd->bulk.cfb_enc = _gcry_aes_aesni_cfb_enc; - hd->bulk.cfb_dec = _gcry_aes_aesni_cfb_dec; - hd->bulk.cbc_enc = _gcry_aes_aesni_cbc_enc; - hd->bulk.cbc_dec = _gcry_aes_aesni_cbc_dec; - hd->bulk.ctr_enc = _gcry_aes_aesni_ctr_enc; - hd->bulk.ocb_crypt = _gcry_aes_aesni_ocb_crypt; - hd->bulk.ocb_auth = _gcry_aes_aesni_ocb_auth; - hd->bulk.xts_crypt = _gcry_aes_aesni_xts_crypt; - } + + /* Setup AES-NI bulk encryption routines. */ + bulk_ops->cfb_enc = _gcry_aes_aesni_cfb_enc; + bulk_ops->cfb_dec = _gcry_aes_aesni_cfb_dec; + bulk_ops->cbc_enc = _gcry_aes_aesni_cbc_enc; + bulk_ops->cbc_dec = _gcry_aes_aesni_cbc_dec; + bulk_ops->ctr_enc = _gcry_aes_aesni_ctr_enc; + bulk_ops->ocb_crypt = _gcry_aes_aesni_ocb_crypt; + bulk_ops->ocb_auth = _gcry_aes_aesni_ocb_auth; + bulk_ops->xts_crypt = _gcry_aes_aesni_xts_crypt; } #endif #ifdef USE_PADLOCK @@ -474,16 +500,15 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; ctx->use_ssse3 = 1; - if (hd) - { - hd->bulk.cfb_enc = _gcry_aes_ssse3_cfb_enc; - hd->bulk.cfb_dec = _gcry_aes_ssse3_cfb_dec; - hd->bulk.cbc_enc = _gcry_aes_ssse3_cbc_enc; - hd->bulk.cbc_dec = _gcry_aes_ssse3_cbc_dec; - hd->bulk.ctr_enc = _gcry_aes_ssse3_ctr_enc; - hd->bulk.ocb_crypt = _gcry_aes_ssse3_ocb_crypt; - hd->bulk.ocb_auth = _gcry_aes_ssse3_ocb_auth; - } + + /* Setup SSSE3 bulk encryption routines. */ + bulk_ops->cfb_enc = _gcry_aes_ssse3_cfb_enc; + bulk_ops->cfb_dec = _gcry_aes_ssse3_cfb_dec; + bulk_ops->cbc_enc = _gcry_aes_ssse3_cbc_enc; + bulk_ops->cbc_dec = _gcry_aes_ssse3_cbc_dec; + bulk_ops->ctr_enc = _gcry_aes_ssse3_ctr_enc; + bulk_ops->ocb_crypt = _gcry_aes_ssse3_ocb_crypt; + bulk_ops->ocb_auth = _gcry_aes_ssse3_ocb_auth; } #endif #ifdef USE_ARM_CE @@ -495,17 +520,16 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; ctx->use_arm_ce = 1; - if (hd) - { - hd->bulk.cfb_enc = _gcry_aes_armv8_ce_cfb_enc; - hd->bulk.cfb_dec = _gcry_aes_armv8_ce_cfb_dec; - hd->bulk.cbc_enc = _gcry_aes_armv8_ce_cbc_enc; - hd->bulk.cbc_dec = _gcry_aes_armv8_ce_cbc_dec; - hd->bulk.ctr_enc = _gcry_aes_armv8_ce_ctr_enc; - hd->bulk.ocb_crypt = _gcry_aes_armv8_ce_ocb_crypt; - hd->bulk.ocb_auth = _gcry_aes_armv8_ce_ocb_auth; - hd->bulk.xts_crypt = _gcry_aes_armv8_ce_xts_crypt; - } + + /* Setup ARM-CE bulk encryption routines. */ + bulk_ops->cfb_enc = _gcry_aes_armv8_ce_cfb_enc; + bulk_ops->cfb_dec = _gcry_aes_armv8_ce_cfb_dec; + bulk_ops->cbc_enc = _gcry_aes_armv8_ce_cbc_enc; + bulk_ops->cbc_dec = _gcry_aes_armv8_ce_cbc_dec; + bulk_ops->ctr_enc = _gcry_aes_armv8_ce_ctr_enc; + bulk_ops->ocb_crypt = _gcry_aes_armv8_ce_ocb_crypt; + bulk_ops->ocb_auth = _gcry_aes_armv8_ce_ocb_auth; + bulk_ops->xts_crypt = _gcry_aes_armv8_ce_xts_crypt; } #endif #ifdef USE_PPC_CRYPTO_WITH_PPC9LE @@ -518,17 +542,16 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->prefetch_dec_fn = NULL; ctx->use_ppc_crypto = 1; /* same key-setup as USE_PPC_CRYPTO */ ctx->use_ppc9le_crypto = 1; - if (hd) - { - hd->bulk.cfb_enc = _gcry_aes_ppc9le_cfb_enc; - hd->bulk.cfb_dec = _gcry_aes_ppc9le_cfb_dec; - hd->bulk.cbc_enc = _gcry_aes_ppc9le_cbc_enc; - hd->bulk.cbc_dec = _gcry_aes_ppc9le_cbc_dec; - hd->bulk.ctr_enc = _gcry_aes_ppc9le_ctr_enc; - hd->bulk.ocb_crypt = _gcry_aes_ppc9le_ocb_crypt; - hd->bulk.ocb_auth = _gcry_aes_ppc9le_ocb_auth; - hd->bulk.xts_crypt = _gcry_aes_ppc9le_xts_crypt; - } + + /* Setup PPC9LE bulk encryption routines. */ + bulk_ops->cfb_enc = _gcry_aes_ppc9le_cfb_enc; + bulk_ops->cfb_dec = _gcry_aes_ppc9le_cfb_dec; + bulk_ops->cbc_enc = _gcry_aes_ppc9le_cbc_enc; + bulk_ops->cbc_dec = _gcry_aes_ppc9le_cbc_dec; + bulk_ops->ctr_enc = _gcry_aes_ppc9le_ctr_enc; + bulk_ops->ocb_crypt = _gcry_aes_ppc9le_ocb_crypt; + bulk_ops->ocb_auth = _gcry_aes_ppc9le_ocb_auth; + bulk_ops->xts_crypt = _gcry_aes_ppc9le_xts_crypt; } #endif #ifdef USE_PPC_CRYPTO @@ -540,17 +563,16 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, ctx->prefetch_enc_fn = NULL; ctx->prefetch_dec_fn = NULL; ctx->use_ppc_crypto = 1; - if (hd) - { - hd->bulk.cfb_enc = _gcry_aes_ppc8_cfb_enc; - hd->bulk.cfb_dec = _gcry_aes_ppc8_cfb_dec; - hd->bulk.cbc_enc = _gcry_aes_ppc8_cbc_enc; - hd->bulk.cbc_dec = _gcry_aes_ppc8_cbc_dec; - hd->bulk.ctr_enc = _gcry_aes_ppc8_ctr_enc; - hd->bulk.ocb_crypt = _gcry_aes_ppc8_ocb_crypt; - hd->bulk.ocb_auth = _gcry_aes_ppc8_ocb_auth; - hd->bulk.xts_crypt = _gcry_aes_ppc8_xts_crypt; - } + + /* Setup PPC8 bulk encryption routines. */ + bulk_ops->cfb_enc = _gcry_aes_ppc8_cfb_enc; + bulk_ops->cfb_dec = _gcry_aes_ppc8_cfb_dec; + bulk_ops->cbc_enc = _gcry_aes_ppc8_cbc_enc; + bulk_ops->cbc_dec = _gcry_aes_ppc8_cbc_dec; + bulk_ops->ctr_enc = _gcry_aes_ppc8_ctr_enc; + bulk_ops->ocb_crypt = _gcry_aes_ppc8_ocb_crypt; + bulk_ops->ocb_auth = _gcry_aes_ppc8_ocb_auth; + bulk_ops->xts_crypt = _gcry_aes_ppc8_xts_crypt; } #endif else @@ -672,10 +694,10 @@ do_setkey (RIJNDAEL_context *ctx, const byte *key, const unsigned keylen, static gcry_err_code_t rijndael_setkey (void *context, const byte *key, const unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { RIJNDAEL_context *ctx = context; - return do_setkey (ctx, key, keylen, hd); + return do_setkey (ctx, key, keylen, bulk_ops); } @@ -943,7 +965,7 @@ rijndael_encrypt (void *context, byte *b, const byte *a) make sure that IV is aligned on an unsigned long boundary. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_aes_cfb_enc (void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1017,7 +1039,7 @@ _gcry_aes_cfb_enc (void *context, unsigned char *iv, make sure that IV is aligned on an unsigned long boundary. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_aes_cbc_enc (void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int cbc_mac) @@ -1100,7 +1122,7 @@ _gcry_aes_cbc_enc (void *context, unsigned char *iv, minimum alignment is for an u32. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size BLOCKSIZE. */ -void +static void _gcry_aes_ctr_enc (void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1357,7 +1379,7 @@ rijndael_decrypt (void *context, byte *b, const byte *a) make sure that IV is aligned on an unsigned long boundary. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_aes_cfb_dec (void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1429,7 +1451,7 @@ _gcry_aes_cfb_dec (void *context, unsigned char *iv, make sure that IV is aligned on an unsigned long boundary. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_aes_cbc_dec (void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1508,7 +1530,7 @@ _gcry_aes_cbc_dec (void *context, unsigned char *iv, /* Bulk encryption/decryption of complete blocks in OCB mode. */ -size_t +static size_t _gcry_aes_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int encrypt) { @@ -1616,7 +1638,7 @@ _gcry_aes_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, /* Bulk authentication of complete blocks in OCB mode. */ -size_t +static size_t _gcry_aes_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) { RIJNDAEL_context *ctx = (void *)&c->context.c; @@ -1690,7 +1712,7 @@ _gcry_aes_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) /* Bulk encryption/decryption of complete blocks in XTS mode. */ -void +static void _gcry_aes_xts_crypt (void *context, unsigned char *tweak, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int encrypt) @@ -1797,6 +1819,7 @@ selftest_basic_128 (void) RIJNDAEL_context *ctx; unsigned char *ctxmem; unsigned char scratch[16]; + cipher_bulk_ops_t bulk_ops; /* The test vectors are from the AES supplied ones; more or less randomly taken from ecb_tbl.txt (I=42,81,14) */ @@ -1844,7 +1867,7 @@ selftest_basic_128 (void) if (!ctx) return "failed to allocate memory"; - rijndael_setkey (ctx, key_128, sizeof (key_128), NULL); + rijndael_setkey (ctx, key_128, sizeof (key_128), &bulk_ops); rijndael_encrypt (ctx, scratch, plaintext_128); if (memcmp (scratch, ciphertext_128, sizeof (ciphertext_128))) { @@ -1866,6 +1889,7 @@ selftest_basic_192 (void) RIJNDAEL_context *ctx; unsigned char *ctxmem; unsigned char scratch[16]; + cipher_bulk_ops_t bulk_ops; static unsigned char plaintext_192[16] = { @@ -1887,7 +1911,7 @@ selftest_basic_192 (void) ctx = _gcry_cipher_selftest_alloc_ctx (sizeof *ctx, &ctxmem); if (!ctx) return "failed to allocate memory"; - rijndael_setkey (ctx, key_192, sizeof(key_192), NULL); + rijndael_setkey (ctx, key_192, sizeof(key_192), &bulk_ops); rijndael_encrypt (ctx, scratch, plaintext_192); if (memcmp (scratch, ciphertext_192, sizeof (ciphertext_192))) { @@ -1910,6 +1934,7 @@ selftest_basic_256 (void) RIJNDAEL_context *ctx; unsigned char *ctxmem; unsigned char scratch[16]; + cipher_bulk_ops_t bulk_ops; static unsigned char plaintext_256[16] = { @@ -1932,7 +1957,7 @@ selftest_basic_256 (void) ctx = _gcry_cipher_selftest_alloc_ctx (sizeof *ctx, &ctxmem); if (!ctx) return "failed to allocate memory"; - rijndael_setkey (ctx, key_256, sizeof(key_256), NULL); + rijndael_setkey (ctx, key_256, sizeof(key_256), &bulk_ops); rijndael_encrypt (ctx, scratch, plaintext_256); if (memcmp (scratch, ciphertext_256, sizeof (ciphertext_256))) { @@ -1958,8 +1983,7 @@ selftest_ctr_128 (void) const int context_size = sizeof(RIJNDAEL_context); return _gcry_selftest_helper_ctr("AES", &rijndael_setkey, - &rijndael_encrypt, &_gcry_aes_ctr_enc, nblocks, blocksize, - context_size); + &rijndael_encrypt, nblocks, blocksize, context_size); } @@ -1973,8 +1997,7 @@ selftest_cbc_128 (void) const int context_size = sizeof(RIJNDAEL_context); return _gcry_selftest_helper_cbc("AES", &rijndael_setkey, - &rijndael_encrypt, &_gcry_aes_cbc_dec, nblocks, blocksize, - context_size); + &rijndael_encrypt, nblocks, blocksize, context_size); } @@ -1988,8 +2011,7 @@ selftest_cfb_128 (void) const int context_size = sizeof(RIJNDAEL_context); return _gcry_selftest_helper_cfb("AES", &rijndael_setkey, - &rijndael_encrypt, &_gcry_aes_cfb_dec, nblocks, blocksize, - context_size); + &rijndael_encrypt, nblocks, blocksize, context_size); } diff --git a/cipher/salsa20.c b/cipher/salsa20.c index 5c5e2b54..58e2dc6f 100644 --- a/cipher/salsa20.c +++ b/cipher/salsa20.c @@ -367,11 +367,11 @@ salsa20_do_setkey (SALSA20_context_t *ctx, static gcry_err_code_t salsa20_setkey (void *context, const byte *key, unsigned int keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { SALSA20_context_t *ctx = (SALSA20_context_t *)context; gcry_err_code_t rc = salsa20_do_setkey (ctx, key, keylen); - (void)hd; + (void)bulk_ops; _gcry_burn_stack (4 + sizeof (void *) + 4 * sizeof (void *)); return rc; } diff --git a/cipher/seed.c b/cipher/seed.c index e36d3cf9..c8ae4a01 100644 --- a/cipher/seed.c +++ b/cipher/seed.c @@ -310,11 +310,11 @@ do_setkey (SEED_context *ctx, const byte *key, const unsigned keylen) static gcry_err_code_t seed_setkey (void *context, const byte *key, const unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { SEED_context *ctx = context; int rc = do_setkey (ctx, key, keylen); - (void)hd; + (void)bulk_ops; _gcry_burn_stack (4*6 + sizeof(void*)*2 + sizeof(int)*2); return rc; } diff --git a/cipher/serpent.c b/cipher/serpent.c index 71d843d0..3c5eed2c 100644 --- a/cipher/serpent.c +++ b/cipher/serpent.c @@ -222,9 +222,24 @@ extern void _gcry_serpent_neon_ocb_auth(serpent_context_t *ctx, #endif -/* A prototype. */ +/* Prototypes. */ static const char *serpent_test (void); +static void _gcry_serpent_ctr_enc (void *context, unsigned char *ctr, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_serpent_cbc_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_serpent_cfb_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static size_t _gcry_serpent_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, + int encrypt); +static size_t _gcry_serpent_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, + size_t nblocks); + /* * These are the S-Boxes of Serpent from following research paper. @@ -749,15 +764,13 @@ serpent_setkey_internal (serpent_context_t *context, static gcry_err_code_t serpent_setkey (void *ctx, const byte *key, unsigned int key_length, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { serpent_context_t *context = ctx; static const char *serpent_test_ret; static int serpent_init_done; gcry_err_code_t ret = GPG_ERR_NO_ERROR; - (void)hd; - if (! serpent_init_done) { /* Execute a self-test the first time, Serpent is used. */ @@ -767,6 +780,14 @@ serpent_setkey (void *ctx, log_error ("Serpent test failure: %s\n", serpent_test_ret); } + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cbc_dec = _gcry_serpent_cbc_dec; + bulk_ops->cfb_dec = _gcry_serpent_cfb_dec; + bulk_ops->ctr_enc = _gcry_serpent_ctr_enc; + bulk_ops->ocb_crypt = _gcry_serpent_ocb_crypt; + bulk_ops->ocb_auth = _gcry_serpent_ocb_auth; + if (serpent_test_ret) ret = GPG_ERR_SELFTEST_FAILED; else @@ -902,7 +923,7 @@ serpent_decrypt (void *ctx, byte *buffer_out, const byte *buffer_in) /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size sizeof(serpent_block_t). */ -void +static void _gcry_serpent_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1014,7 +1035,7 @@ _gcry_serpent_ctr_enc(void *context, unsigned char *ctr, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_serpent_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1123,7 +1144,7 @@ _gcry_serpent_cbc_dec(void *context, unsigned char *iv, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_serpent_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -1225,7 +1246,7 @@ _gcry_serpent_cfb_dec(void *context, unsigned char *iv, } /* Bulk encryption/decryption of complete blocks in OCB mode. */ -size_t +static size_t _gcry_serpent_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int encrypt) { @@ -1412,7 +1433,7 @@ _gcry_serpent_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, } /* Bulk authentication of complete blocks in OCB mode. */ -size_t +static size_t _gcry_serpent_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) { @@ -1592,8 +1613,7 @@ selftest_ctr_128 (void) const int context_size = sizeof(serpent_context_t); return _gcry_selftest_helper_ctr("SERPENT", &serpent_setkey, - &serpent_encrypt, &_gcry_serpent_ctr_enc, nblocks, blocksize, - context_size); + &serpent_encrypt, nblocks, blocksize, context_size); } @@ -1607,8 +1627,7 @@ selftest_cbc_128 (void) const int context_size = sizeof(serpent_context_t); return _gcry_selftest_helper_cbc("SERPENT", &serpent_setkey, - &serpent_encrypt, &_gcry_serpent_cbc_dec, nblocks, blocksize, - context_size); + &serpent_encrypt, nblocks, blocksize, context_size); } @@ -1622,8 +1641,7 @@ selftest_cfb_128 (void) const int context_size = sizeof(serpent_context_t); return _gcry_selftest_helper_cfb("SERPENT", &serpent_setkey, - &serpent_encrypt, &_gcry_serpent_cfb_dec, nblocks, blocksize, - context_size); + &serpent_encrypt, nblocks, blocksize, context_size); } diff --git a/cipher/sm4.c b/cipher/sm4.c index 75f2e412..c8dd0406 100644 --- a/cipher/sm4.c +++ b/cipher/sm4.c @@ -69,6 +69,21 @@ static const char *sm4_selftest (void); +static void _gcry_sm4_ctr_enc (void *context, unsigned char *ctr, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_sm4_cbc_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_sm4_cfb_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static size_t _gcry_sm4_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, + int encrypt); +static size_t _gcry_sm4_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, + size_t nblocks); + typedef struct { u32 rkey_enc[32]; @@ -329,14 +344,13 @@ sm4_expand_key (SM4_context *ctx, const byte *key) static gcry_err_code_t sm4_setkey (void *context, const byte *key, const unsigned keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { SM4_context *ctx = context; static int init = 0; static const char *selftest_failed = NULL; unsigned int hwf = _gcry_get_hw_features (); - (void)hd; (void)hwf; if (!init) @@ -359,6 +373,14 @@ sm4_setkey (void *context, const byte *key, const unsigned keylen, ctx->use_aesni_avx2 = (hwf & HWF_INTEL_AESNI) && (hwf & HWF_INTEL_AVX2); #endif + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cbc_dec = _gcry_sm4_cbc_dec; + bulk_ops->cfb_dec = _gcry_sm4_cfb_dec; + bulk_ops->ctr_enc = _gcry_sm4_ctr_enc; + bulk_ops->ocb_crypt = _gcry_sm4_ocb_crypt; + bulk_ops->ocb_auth = _gcry_sm4_ocb_auth; + sm4_expand_key (ctx, key); return 0; } @@ -491,7 +513,7 @@ sm4_crypt_blocks (const u32 *rk, byte *out, const byte *in, /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size 16. */ -void +static void _gcry_sm4_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -592,7 +614,7 @@ _gcry_sm4_ctr_enc(void *context, unsigned char *ctr, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_sm4_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -686,7 +708,7 @@ _gcry_sm4_cbc_dec(void *context, unsigned char *iv, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_sm4_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) @@ -783,7 +805,7 @@ _gcry_sm4_cfb_dec(void *context, unsigned char *iv, } /* Bulk encryption/decryption of complete blocks in OCB mode. */ -size_t +static size_t _gcry_sm4_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int encrypt) { @@ -955,7 +977,7 @@ _gcry_sm4_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, } /* Bulk authentication of complete blocks in OCB mode. */ -size_t +static size_t _gcry_sm4_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) { SM4_context *ctx = (void *)&c->context.c; @@ -1112,8 +1134,7 @@ selftest_ctr_128 (void) const int context_size = sizeof(SM4_context); return _gcry_selftest_helper_ctr("SM4", &sm4_setkey, - &sm4_encrypt, &_gcry_sm4_ctr_enc, nblocks, blocksize, - context_size); + &sm4_encrypt, nblocks, blocksize, context_size); } /* Run the self-tests for SM4-CBC, tests bulk CBC decryption. @@ -1126,8 +1147,7 @@ selftest_cbc_128 (void) const int context_size = sizeof(SM4_context); return _gcry_selftest_helper_cbc("SM4", &sm4_setkey, - &sm4_encrypt, &_gcry_sm4_cbc_dec, nblocks, blocksize, - context_size); + &sm4_encrypt, nblocks, blocksize, context_size); } /* Run the self-tests for SM4-CFB, tests bulk CFB decryption. @@ -1140,8 +1160,7 @@ selftest_cfb_128 (void) const int context_size = sizeof(SM4_context); return _gcry_selftest_helper_cfb("SM4", &sm4_setkey, - &sm4_encrypt, &_gcry_sm4_cfb_dec, nblocks, blocksize, - context_size); + &sm4_encrypt, nblocks, blocksize, context_size); } static const char * diff --git a/cipher/twofish.c b/cipher/twofish.c index 417d7378..d19e0790 100644 --- a/cipher/twofish.c +++ b/cipher/twofish.c @@ -85,6 +85,24 @@ /* Prototype for the self-test function. */ static const char *selftest(void); + +/* Prototypes for the bulk functions. */ +static void _gcry_twofish_ctr_enc (void *context, unsigned char *ctr, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_twofish_cbc_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static void _gcry_twofish_cfb_dec (void *context, unsigned char *iv, + void *outbuf_arg, const void *inbuf_arg, + size_t nblocks); +static size_t _gcry_twofish_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, + const void *inbuf_arg, size_t nblocks, + int encrypt); +static size_t _gcry_twofish_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, + size_t nblocks); + + /* Structure for an expanded Twofish key. s contains the key-dependent * S-boxes composed with the MDS matrix; w contains the eight "whitening" * subkeys, K[0] through K[7]. k holds the remaining, "round" subkeys. Note @@ -735,14 +753,12 @@ do_twofish_setkey (TWOFISH_context *ctx, const byte *key, const unsigned keylen) static gcry_err_code_t twofish_setkey (void *context, const byte *key, unsigned int keylen, - gcry_cipher_hd_t hd) + cipher_bulk_ops_t *bulk_ops) { TWOFISH_context *ctx = context; unsigned int hwfeatures = _gcry_get_hw_features (); int rc; - (void)hd; - rc = do_twofish_setkey (ctx, key, keylen); #ifdef USE_AVX2 @@ -753,6 +769,14 @@ twofish_setkey (void *context, const byte *key, unsigned int keylen, } #endif + /* Setup bulk encryption routines. */ + memset (bulk_ops, 0, sizeof(*bulk_ops)); + bulk_ops->cbc_dec = _gcry_twofish_cbc_dec; + bulk_ops->cfb_dec = _gcry_twofish_cfb_dec; + bulk_ops->ctr_enc = _gcry_twofish_ctr_enc; + bulk_ops->ocb_crypt = _gcry_twofish_ocb_crypt; + bulk_ops->ocb_auth = _gcry_twofish_ocb_auth; + (void)hwfeatures; _gcry_burn_stack (23+6*sizeof(void*)); @@ -1096,7 +1120,7 @@ twofish_decrypt (void *context, byte *out, const byte *in) /* Bulk encryption of complete blocks in CTR mode. This function is only intended for the bulk encryption feature of cipher.c. CTR is expected to be of size TWOFISH_BLOCKSIZE. */ -void +static void _gcry_twofish_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -1174,7 +1198,7 @@ _gcry_twofish_ctr_enc(void *context, unsigned char *ctr, void *outbuf_arg, /* Bulk decryption of complete blocks in CBC mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_twofish_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -1249,7 +1273,7 @@ _gcry_twofish_cbc_dec(void *context, unsigned char *iv, void *outbuf_arg, /* Bulk decryption of complete blocks in CFB mode. This function is only intended for the bulk encryption feature of cipher.c. */ -void +static void _gcry_twofish_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, const void *inbuf_arg, size_t nblocks) { @@ -1318,7 +1342,7 @@ _gcry_twofish_cfb_dec(void *context, unsigned char *iv, void *outbuf_arg, } /* Bulk encryption/decryption of complete blocks in OCB mode. */ -size_t +static size_t _gcry_twofish_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, const void *inbuf_arg, size_t nblocks, int encrypt) { @@ -1432,7 +1456,7 @@ _gcry_twofish_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, } /* Bulk authentication of complete blocks in OCB mode. */ -size_t +static size_t _gcry_twofish_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, size_t nblocks) { @@ -1546,8 +1570,7 @@ selftest_ctr (void) const int context_size = sizeof(TWOFISH_context); return _gcry_selftest_helper_ctr("TWOFISH", &twofish_setkey, - &twofish_encrypt, &_gcry_twofish_ctr_enc, nblocks, blocksize, - context_size); + &twofish_encrypt, nblocks, blocksize, context_size); } /* Run the self-tests for TWOFISH-CBC, tests bulk CBC decryption. @@ -1560,8 +1583,7 @@ selftest_cbc (void) const int context_size = sizeof(TWOFISH_context); return _gcry_selftest_helper_cbc("TWOFISH", &twofish_setkey, - &twofish_encrypt, &_gcry_twofish_cbc_dec, nblocks, blocksize, - context_size); + &twofish_encrypt, nblocks, blocksize, context_size); } /* Run the self-tests for TWOFISH-CFB, tests bulk CBC decryption. @@ -1574,8 +1596,7 @@ selftest_cfb (void) const int context_size = sizeof(TWOFISH_context); return _gcry_selftest_helper_cfb("TWOFISH", &twofish_setkey, - &twofish_encrypt, &_gcry_twofish_cfb_dec, nblocks, blocksize, - context_size); + &twofish_encrypt, nblocks, blocksize, context_size); } @@ -1585,7 +1606,8 @@ static const char* selftest (void) { TWOFISH_context ctx; /* Expanded key. */ - byte scratch[16]; /* Encryption/decryption result buffer. */ + byte scratch[16]; /* Encryption/decryption result buffer. */ + cipher_bulk_ops_t bulk_ops; const char *r; /* Test vectors for single encryption/decryption. Note that I am using @@ -1620,7 +1642,7 @@ selftest (void) 0x05, 0x93, 0x1C, 0xB6, 0xD4, 0x08, 0xE7, 0xFA }; - twofish_setkey (&ctx, key, sizeof(key), NULL); + twofish_setkey (&ctx, key, sizeof(key), &bulk_ops); twofish_encrypt (&ctx, scratch, plaintext); if (memcmp (scratch, ciphertext, sizeof (ciphertext))) return "Twofish-128 test encryption failed."; @@ -1628,7 +1650,7 @@ selftest (void) if (memcmp (scratch, plaintext, sizeof (plaintext))) return "Twofish-128 test decryption failed."; - twofish_setkey (&ctx, key_256, sizeof(key_256), NULL); + twofish_setkey (&ctx, key_256, sizeof(key_256), &bulk_ops); twofish_encrypt (&ctx, scratch, plaintext_256); if (memcmp (scratch, ciphertext_256, sizeof (ciphertext_256))) return "Twofish-256 test encryption failed."; @@ -1663,7 +1685,8 @@ int main() { TWOFISH_context ctx; /* Expanded key. */ - int i, j; /* Loop counters. */ + int i, j; /* Loop counters. */ + cipher_bulk_ops_t bulk_ops; const char *encrypt_msg; /* Message to print regarding encryption test; * the printf is done outside the loop to avoid @@ -1710,13 +1733,13 @@ main() /* Encryption test. */ for (i = 0; i < 125; i++) { - twofish_setkey (&ctx, buffer[0], sizeof (buffer[0]), NULL); + twofish_setkey (&ctx, buffer[0], sizeof (buffer[0]), &bulk_ops); for (j = 0; j < 1000; j++) twofish_encrypt (&ctx, buffer[2], buffer[2]); - twofish_setkey (&ctx, buffer[1], sizeof (buffer[1]), NULL); + twofish_setkey (&ctx, buffer[1], sizeof (buffer[1]), &bulk_ops); for (j = 0; j < 1000; j++) twofish_encrypt (&ctx, buffer[3], buffer[3]); - twofish_setkey (&ctx, buffer[2], sizeof (buffer[2])*2, NULL); + twofish_setkey (&ctx, buffer[2], sizeof (buffer[2])*2, &bulk_ops); for (j = 0; j < 1000; j++) { twofish_encrypt (&ctx, buffer[0], buffer[0]); twofish_encrypt (&ctx, buffer[1], buffer[1]); @@ -1728,15 +1751,15 @@ main() /* Decryption test. */ for (i = 0; i < 125; i++) { - twofish_setkey (&ctx, buffer[2], sizeof (buffer[2])*2, NULL); + twofish_setkey (&ctx, buffer[2], sizeof (buffer[2])*2, &bulk_ops); for (j = 0; j < 1000; j++) { twofish_decrypt (&ctx, buffer[0], buffer[0]); twofish_decrypt (&ctx, buffer[1], buffer[1]); } - twofish_setkey (&ctx, buffer[1], sizeof (buffer[1]), NULL); + twofish_setkey (&ctx, buffer[1], sizeof (buffer[1]), &bulk_ops); for (j = 0; j < 1000; j++) twofish_decrypt (&ctx, buffer[3], buffer[3]); - twofish_setkey (&ctx, buffer[0], sizeof (buffer[0]), NULL); + twofish_setkey (&ctx, buffer[0], sizeof (buffer[0]), &bulk_ops); for (j = 0; j < 1000; j++) twofish_decrypt (&ctx, buffer[2], buffer[2]); } diff --git a/src/cipher-proto.h b/src/cipher-proto.h index 97eb0d9a..d87559ed 100644 --- a/src/cipher-proto.h +++ b/src/cipher-proto.h @@ -129,11 +129,13 @@ typedef struct gcry_pk_spec * */ +typedef struct cipher_bulk_ops cipher_bulk_ops_t; + /* Type for the cipher_setkey function. */ typedef gcry_err_code_t (*gcry_cipher_setkey_t) (void *c, const unsigned char *key, unsigned keylen, - gcry_cipher_hd_t hd); + cipher_bulk_ops_t *bulk_ops); /* Type for the cipher_encrypt function. */ typedef unsigned int (*gcry_cipher_encrypt_t) (void *c, diff --git a/src/cipher.h b/src/cipher.h index f6b2151a..908e3b78 100644 --- a/src/cipher.h +++ b/src/cipher.h @@ -147,133 +147,6 @@ gcry_err_code_t _gcry_blake2_init_with_key(void *ctx, unsigned int flags, const unsigned char *key, size_t keylen, int algo); -/*-- rijndael.c --*/ -void _gcry_aes_cfb_enc (void *context, unsigned char *iv, - void *outbuf, const void *inbuf, - size_t nblocks); -void _gcry_aes_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_aes_cbc_enc (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks, int cbc_mac); -void _gcry_aes_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_aes_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -size_t _gcry_aes_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, - const void *inbuf_arg, size_t nblocks, int encrypt); -size_t _gcry_aes_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, - size_t nblocks); -void _gcry_aes_xts_crypt (void *context, unsigned char *tweak, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks, int encrypt); - -/*-- blowfish.c --*/ -void _gcry_blowfish_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -void _gcry_blowfish_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -void _gcry_blowfish_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -/*-- cast5.c --*/ -void _gcry_cast5_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -void _gcry_cast5_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -void _gcry_cast5_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -/*-- camellia-glue.c --*/ -void _gcry_camellia_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_camellia_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_camellia_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -size_t _gcry_camellia_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, - const void *inbuf_arg, size_t nblocks, - int encrypt); -size_t _gcry_camellia_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, - size_t nblocks); - -/*-- des.c --*/ -void _gcry_3des_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -void _gcry_3des_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -void _gcry_3des_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); - -/*-- serpent.c --*/ -void _gcry_serpent_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_serpent_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_serpent_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -size_t _gcry_serpent_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, - const void *inbuf_arg, size_t nblocks, - int encrypt); -size_t _gcry_serpent_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, - size_t nblocks); - -/*-- sm4.c --*/ -void _gcry_sm4_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_sm4_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_sm4_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -size_t _gcry_sm4_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, - const void *inbuf_arg, size_t nblocks, - int encrypt); -size_t _gcry_sm4_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, - size_t nblocks); - -/*-- twofish.c --*/ -void _gcry_twofish_ctr_enc (void *context, unsigned char *ctr, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_twofish_cbc_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -void _gcry_twofish_cfb_dec (void *context, unsigned char *iv, - void *outbuf_arg, const void *inbuf_arg, - size_t nblocks); -size_t _gcry_twofish_ocb_crypt (gcry_cipher_hd_t c, void *outbuf_arg, - const void *inbuf_arg, size_t nblocks, - int encrypt); -size_t _gcry_twofish_ocb_auth (gcry_cipher_hd_t c, const void *abuf_arg, - size_t nblocks); - /*-- dsa.c --*/ void _gcry_register_pk_dsa_progress (gcry_handler_progress_t cbc, void *cb_data); -- 2.25.1