[ESD-2010] Fix auto generated tests #955
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix autogenerated C tests
The tests as currently written are not really testing very much. Each test case is passed in a byte array from the spec which represents the encoded frame. This data is then passed in to
sbp_send_message
as the payload rather than just the payload being extracted and passed in. This means that the payload as encoded bysbp_send_message
is not the actual payload which relates to the message under test but a complete frame which contains the payload:If the test case for message X is passed a particular set of test data it should be encoded as
but it's actually encoded as
The encoded frame is then passed back in to libsbp and the payload extracted via a logging callback - the payload which is actually an entire frame. The "payload" is then compared against the encoded frame data defined in the test spec. The first part of each generated test case is pretty much just testing memcpy
The second part of the test case uses a series of expected fields and compares them against the encoded data. However it checks against the data which was passed in from the spec. This part is therefore checking that the test spec is correctly written and not checking anything in libsbp itself.
These things combined mean that libsbp is not being tested at all. This is demonstrated by a test case for MSG_VEL_NED_COV which has an incomplete set of expected fields and should be failing but isn't.
This PR reworks the auto generated C test so that the procedure for each case is:
1 - Create an instance of the message struct and fill out with the fields from the test spec
2 - Encode the test message using
sbp_send_message
3 - Check the encoded data from
sbp_send_message
against the expected encoded frame data from the test spec4 - Check that the SBP message callback was called properly
5 - Check that the SBP frame callback was called properly
6 - Cast the decoded payload back to the correct message struct and verify the fields.
By doing this it checks that
1 - The test spec is well written
2 - libsbp is behaving properly
Variable length arrays need special handling and complicated some of the macros in the test suite. The length of the encoded message is tracked while the test message is being built up since
sizeof
for messages containing variable length arrays is meaningless.Strings are similarly badly defined in SBP with some having no termination, single null termination, double null termination, or null delimination. These are now all handled with memcpy/memcmp rather than strcpy/strstr
For reviewers:
The only files in this PR which have been modified by hand are
generator/sbpg/targets/resources/sbp_c_test.c.j2
generator/sbp/targets/test_c.py
spec/tests/yaml/swiftnav/sbp/navigation/test_MsgVelNEDCOV.yaml
Everything else is auto generated