Exhaustive DTLS handshake test

[Nikos Mavrogiannopoulos]

On 02/14/2012 08:43 PM, Sean Buckheister wrote:

> I fear not, and I am terribly sorry. Due to an oversight, the programm
> dropped far less packets than it actually should. The patch to fix this
> is rather short: add "filter_current_idx++;" at the end of
> filter_run_next. 


So would filter_run_next contain filter_current_idx++ twice?

> As of current git head, there are almost 6000 variations that fail the
> test with 1000 child processes, but pass with only 100. This seems
> rather odd, running on a machine with four cores an 10ms between any two
> test run forks. Maybe i made a big mistake fixing my original mistake,
> but looking at the traces, it doesn't look much like it.


I noticed that too. However if I repeat the same tests in single mode
they succeed. I believe that is because the cases that fail require
quite some retransmissions and in 100 or 1000 process cases they might
take more time than the allowed timeout.

> Running the full suite, with 100 children, in nonblocking mode also
> fails a number of the tests. I am puzzled.


I have not thoroughly tested the non-blocking mode.

> (I'm not exactly sure whether I can legally transfer copyright. To my
> knowledge, I can grant a nonrevocable exclusive use license though,
> which should be pretty much the same thing.)


If it is not possible, you could release it under GPLv3 (e.g. with a
mail in gnutls-devel or so) and that would be fine with me since it is
an independent module.

>> Note that I've tweaked the code in order to compile with gnutls' cflags.
> I have tried to extract the gnutls cflags, they are "-std=gnu99 -g -O2"
> for me. The program compiled fine with those, but I must be missing
> something. Now it compiles with -std=c89, which should work for everyone.


We use more than that in development. Use ./configure
--enable-gcc-warnings to enable them.

> I've also made a number of modifications to avoid code duplication for
> filter_packet_* and filter_permute_*, more sensible error handling, more
> sensible child process handling (^C now kills the whole process tree,
> not just the master process). I would also add tests for certificate
> authentication of both client and server, four extra packets, which
> would make the test suite a bit more comprehensive. It would end up
> being more than 0.3 million test runs.

Feel free to send me any update.

> Also, there are a number of parameters missing:
> 
> * run tests with nonblocking DTLS
> * set debug output level
> * set child process limit
> * set retransmit timeouts?
> 
> I have noticed that low retransmit timeouts and large child process
> limits yield to a lot of false negatives in the batch run.


That could be because of the high number of retransmissions in case of a
low retransmit timeout.

regards,
Nikos