eunit: Add possibility to scale timeouts #7635
Conversation
CT Test Results 2 files 12 suites 5m 20s ⏱️ Results for commit 0d75a4b. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts// Erlang/OTP Github Action Bot |
269195e
to
e1543e3
Compare
|
Sorry for the delay in updating, for some reason, I failed to notice your comments until just recently. I (force-)pushed an updated version. |
What do you mean with above note? I think setting increased hardcoded timeout for all tests is NOT as good as setting a scaling factor (proposed in PR) which is more sensible. |
|
Exactly, I agree. When I first ran the tests on the slow host, I looked for a way to up the timeout without having to change tools along the way (rebar, makefiles etc). My first (ditched) attempt was actually to introduce an env variable and read that. Then I discovered the |
Add an option to eunit:test(..., Options):
{scale_timeouts, N}
to be able to increase the timeouts by some factor, or
decrease it if the factor is below 1.0.
This applies to both the the default 5 second timeout,
and to timeouts specified as {timeout, Seconds, Test}
in eunit test generators.
e1543e3
to
0d75a4b
Compare
Add an option to
eunit:test(..., Options):Motivation: I ran a bunch of tests on an s390 qemu guest on an x86_64 host to test some NIF code on big endian. This executed much slower, and some of the unit tests timed out. The default eunit timeout is 5 seconds. One can increase it, for example by using a test generator with a
{timeout, Seconds, TestOrTests}and change xyz_test/0 to xyz_test/1. But this becomes tedious when there are many tests. And to commit such a change to the test code just to accommodate for an unusually slow host doesn't seem like an attractive way to go. Being able to scale the timeouts seems like a good middle ground.Note that eunit can already read the
$EUNITenvironment var for one-off runs from the command line with increased timeouts.I made the PR to target the maint branch. Let me know if I should change it to the master branch instead.