Interoperability concerns the ability of Defensics to communicate with the system under testing. This chapter discusses suggestions for debugging interoperability.
A protocol can define multiple use cases. Some of the more complex protocols are split into different sequences. Only one sequence can be tested by a single configuration. For testing multiple protocols features and sequences, it is necessary to create separate configurations.
Choose the tested sequence in Test Design > Sequence view. For example in HTTP Server Test suite you’ll find specific sequences for testing HTTP GET and HTTP POST.
There are two types of problems:
- Connectivity problems - Defensics and the target are not communicating at all.
- Configuration problems - Defensics is able to communicate with the target but interoperability is failing.
Debugging Interoperability
In order to debug interoperability issues, you should start with the log in Test Design. Run Test interoperability and after the run check the log. The log contains issues detected in the interoperability test.
Read the error messages to see what is wrong with the test.
When there are messages from the target, it means that connectivity is working and the failures are related to configuration issues. If you are not getting the expected responses, you most likely need to change the configuration in the target or the Test Suite.
There can also be legitimate reasons for valid case failures. These arise when the target does not support all the features of the protocol. In this case some valid cases will fail and some will pass. Defensics uses interoperability results to create test case groups for testing features which exist on the target.
Connectivity Problems
If the log results show that Defensics is not able to reach the target, check the following:
- Try to ping the target from the test machine
- Check that the target and the test machine have the addresses they should have
- Check if the target has a log file showing incoming mesaages from Defensics. If there is no log file, use tcpdump or Wireshark on the target to listen for incoming messages.
- Check if the target needs to whitelist your test machine's address
Configuration Problems
If you are communicating with the target but the results are not what you expected, you might have to change the configurations for Defensics, the test target or both.
In the following example:
Defensics and the target are clearly able to exchange HTTP messages, but the error is "HTTP Status does not match to 1xx 2xx 3xx". In this case, the test suite is examining the HTTP status code in the response. The response from the target is 404 Not Found. There are at least three ways to resolve this:
- Change Defensics' configuration (the URI in 1) Protocol and Network) to request an existing resource from the target.
- Change the configuration of the target so the requested resource can be returned.
- Change Defensics' configuration so that the 404 status code is considered a successful response. This is in 1) Protocol and Network > HTTP options > Filter for accepted HTTP status codes. You could simply add 404 to the field.