[PATCH] Add script to run basic tests with all supported HWF combinations

[Jussi Kivilinna]

On 23.06.2017 11:00, Werner Koch wrote:
> Hi!
> 
> I indeed think that this test is very useful.
> 
> On Sun, 18 Jun 2017 10:23, jussi.kivilinna at iki.fi said:
> 
>> +++ b/tests/basic_all_hwfeature_combinations.sh
>> @@ -0,0 +1,73 @@
>> +#!/bin/bash
> 
> Unfortunately we can't depend on bash.  There are enough systems w/o
> bash installed.
> 
>> +#  For example for Windows executables: BINEXT=.exe
>> +if [ "x$BINEXT" != "x" ] && [ -e "tests/version$BINEXT" ]; then
>> +	binext="$BINEXT"
>> +else
>> +	binext=""
> 
> Hmmm, running the script on Windows would require the installation of a
> Unix shell.  That is too troublesome.  With the new gcry_get_config
> fucntion we can get the features form a C program and the we would only
> need to fork/exec - oh well, not on Windows :-(

Yes, that would be problematic if testing on Windows. One can instead
cross-compile win32 & win64 binaries on Linux and running tests
on command-line with Wine (+ binfmt plugin) which works quite
nicely. I've also found Qemu+binfmt to be quite useful when testing
code with different architectures, although there is some caveats
especially with missing unaligned memory access problems when
running program in Qemu on top of x86.

> 
> tests/random also forks but skips these checks on Windows.  Which it
> should not.  To avoid cluttering all code with a platform dependent
> implemtation of a spawing function I would suggest to wait until we have
> put such a function into libgpgrt (aka libgpg-error).  We plan this
> because we several such spawning implementations in the GnuPG stack and
> libgpgrt is a common dependency of all.> 
> If you want to push your change, I would suggest to do it as
> maintainer-only tool _for now_ so that it is not run by "make check".
> 

Dedicated program would be nicer, but I'm not sure if that would
be worth of the extra effort. Problem with this check is that
it can take quite a long time and I would not recommend to 
run it as part of "make check". For example, with current Intel
Skylake CPU, number of different HWF combinations is 4096 and
running all basic tests can take tens of minutes.

-Jussi