The test was implemented when version 0.3.1 of the XEP was the most current version. The Smack code that is being tested defines a namespace that was introduced in 0.4.0, making it plausible that this implementation matches that version of the XEP: 0.4.0 (which is _not_ the latest version of the XEP).
The test was implemented years after the most current version of the XEP was published, making it plausible that the implementation matches that version of the XEP: 1.9.
The test was implemented when version 1.2 of the XEP was the most current version. Later versions of the XEP do not significantly modify the specifications, making it plausible that this implementation matches the current version of the XEP: 1.3.1.
The test was implemented years after the most current version of the XEP was published, making it plausible that the implementation matches that version of the XEP: 1.3.0.
The test was implemented years after the most current version of the XEP was published, making it plausible that the implementation matches that version of the XEP: 2.1.
The test was implemented when version 1.5.1 of the XEP was the most current version. Later versions of the XEP do not significantly modify the specifications, making it plausible that this implementation matches the current version of the XEP: 1.6.0.
An optional configuration option for the Smack Integration Test framework has been added that allows one to bypass DNS when resolving a host for the XMPP domain that is the subject of the test.
The `host` option can be used with IP addresses (eg: `-Dsinttest.host=127.0.0.1`) and DNS names (eg: `-Dsinttest.host=example.org`).
This tests reliably fails, not only for me. I suspect that it is
related to the order of events checked by this tests, that can not be
reliably tested, even with sync listeners.
It is also is primarily a test for server behavior.
In order to be able to identify potential room leaks, use unique rooms
names for the two integration tests. Also destroy the room in
mucJoinSemiAnonymousRoomReceivedByNonModeratorTest().
Only declare the body of the participant listeners once. And increase
the try block, to account, for example, for
participantOneSeesTwoEnter.waitForResult() throwing.
MUCUser.getStatus() returns a set, checking if a particular MUC status
number is set should be done via a simple and efficient set operation
and not by resorting to using Java's stream API.
This method only had one call site and using such "helper" methods has
the drawback that it is not immediatly obvious what it does when
reading the integration test code. Therefore, we better inline it.
When comparing SINT-configuration to annotations, a bit of normalization occurs, to ensure that common variations in denoting a specification are detected to be equal to each-other.
The dash (`-`) character is commonly used when referencing a specification (eg: `XEP-0001`).
This commit ensures that usage of a dash (`-`) character is included in the normalization process, making `XEP 0001`, `XEP0001` and `XEP-0001` all to be identified as the same reference.
Some specifications are versioned. XEPs, for example, typically are. It is useful to annotate an implementation with the specific version of the specification that is being tested.
The default set of packages that is scanned for integration tests are referencing jivesoftware. This commit adds igniterealtime-oriented package names.
The implementation of the test that is being removed depends on a server
characteristic that will cause a loop of presence stanzas (which obviously is
bad). A RFC3921-compliant client can send an 'acknowledgement' after receiving
a presence 'subscribed' stanza, in the form of a presence 'subscribe' stanza.
See section 8.2 of RFC3921.
When a server implementation does not ignore this acknowledgement, the domain
of the recipient MUST (RFC6121 section 3.1.3) respond with a 'subscribed' on
behalf of the recipient (which is what the now removed test was verifying).
This can trigger the RFC3921-compliant sender to again receive 'subscribed',
that it again can acknowledge, which causes a loop.
To test RFC6121, the subscription state of the recipient must somehow be
modified to reflect a different state than that of the initiator. I'm not sure
if that is feasible with the SINT framework.
`AbstractSmackIntTest#assertResult()` takes an argument that is an assertion message. When the sync point times out, that message should be logged.
The following illustrates the change, as a result of this assertion failing:
```
assertResult(resultSyncPoint, "Expected " + conTwo.getUser() + " to receive message that was sent by " + conOne.getUser() + " in room " + mucAddress + " (but it did not).");
```
Prior to this change, this is logged:
```
SEVERE: MultiUserChatIntegrationTest.mucTest (Normal) failed: java.util.concurrent.TimeoutException: Timeout expired
java.util.concurrent.TimeoutException: Timeout expired
at org.igniterealtime.smack.inttest.util.ResultSyncPoint.waitForResult(ResultSyncPoint.java:49)
at org.igniterealtime.smack.inttest.AbstractSmackIntTest.assertResult(AbstractSmackIntTest.java:104)
at org.igniterealtime.smack.inttest.AbstractSmackIntTest.assertResult(AbstractSmackIntTest.java:99)
at org.jivesoftware.smackx.muc.MultiUserChatIntegrationTest.mucTest(MultiUserChatIntegrationTest.java:132)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
(snip)
```
With the change in this commit, that becomes:
```
SEVERE: MultiUserChatIntegrationTest.mucTest (Normal) failed: java.util.concurrent.TimeoutException: Expected smack-inttest-two-jskr4@example.org/two-jskr4 to receive message that was sent by smack-inttest-one-jskr4@example.org/one-jskr4 in room smack-inttest-message-jskr4-aud43i@conference.example.org (but it did not).
java.util.concurrent.TimeoutException: Expected smack-inttest-two-jskr4@example.org/two-jskr4 to receive message that was sent by smack-inttest-one-jskr4@example.org/one-jskr4 in room smack-inttest-message-jskr4-aud43i@conference.example.org (but it did not).
at org.igniterealtime.smack.inttest.util.ResultSyncPoint.waitForResult(ResultSyncPoint.java:53)
at org.igniterealtime.smack.inttest.AbstractSmackIntTest.assertResult(AbstractSmackIntTest.java:104)
at org.igniterealtime.smack.inttest.AbstractSmackIntTest.assertResult(AbstractSmackIntTest.java:99)
at org.jivesoftware.smackx.muc.MultiUserChatIntegrationTest.mucTest(MultiUserChatIntegrationTest.java:132)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
(snip)
```
This adds a new configuration option, `testRunResultProcessors`, that allows a user to customize the way the results of a test run is processed.
By default, the pre-exising printing-to-stderr is used.
When refactoring the original implementation, this annotation was expected to be present on methods. It later was changed to be a type-based annotation. This particular usage of the annotation was not properly modified to account for that change.
When testing for default role assignment based on affiliation, depending on the 'adminGranted' callback is dangerous, as this callback appears to be only invoked when the affected occupant has already joined the room.
The exact occupant count isn't something that these tests need to assert either.
This commit changes the test implementations to proceed when all expected occupants have been detected by the user performing the change.
These changes combined intend to make the tests more robust, with regards to the order in which several events are fired (which might be unexpeced, given threading implementation in server (clusters) and Smack stanza handling.
When using 'assertEquals', the first argument is to be the _expected_ value, the second the _actual_ value. When this is inverted, the functional test will still succeed, but any generated error message ("Expected X, got Y") will be wrong.
This commit fixes the order of arguments, mostly in the sinttest module.
This adds human-readable text to nearly all assertions in SINT. It is intended that these, together with an XMPP dump of the traffic that was exchanged during the test, allows an observer to have a chance to determine why a particular test failed, without analyzing the test code itself.
A new annotation is introduced (`SpecificationReference`) that can be used to annotate a SINT test class
The properties are available in the annotation:
- `document`: Identifier for a specification document, such as 'RFC 6120' or 'XEP-0485'
The pre-existing `SmackIntegrationTest` annotation has now received two new properties:
- `section`: Identifier for a section (or paragraph), such as '6.2.1'
- `quote`: A quotation of relevant text from the section
These are expected to be used in context of the `SpecificationReference` annotation.
The SINT execution framework is modified so that two new configuration options are available:
- `enabledSpecifications`
- `disabledSpecifications`
These operate on the value of the `document` property of the annotation. Their usage is comparable
to that of the pre-existing `enabledTests` and `disabledTest` configuration options.
Execution output now includes the document, section and quote that's on the annotated test, when
the test fails. This allows an end-user to easily correspond a test failure with a particular
specification.
Using 60 seconds makes it sometimes easy to miss that the some action
was 1 minute after the timeout, because only a single digit in the
timestamp changes. Using a prime number as timeout makes this more
obvious.
When performing IBR-based account registration, we do not need to
login nor are the admin credentials typically available.
Suggested-by: Guus der Kinderen <guus@goodbytes.nl>
Subclasses of AbstractSmackSpecificLowLevelIntegrationTest have test
methods with no parameters. This was, after the refactoring in
c5bb15c631 ("[sinttest] Add UnconnectedConnectionSource for
low-level tests") not handled properly.
Fixes: c5bb15c631 ("[sinttest] Add UnconnectedConnectionSource for low-level tests")
Previously low-level tests where run, potentially multiple times, with
the default connection descriptor.
Reported-by: Guus der Kinderen <guus@goodbytes.nl>
While markdown is easier to write, Smack's markdown documentation was
never tightly coupled with the source. For example, the markdown
documentation never provided links to the actual Java classes and
methods. This poses the risk that the documentation and the code
diverge over time. Furthermore, javadoc is constantly improving (for
example @snippet annotations) and I expect that one will be able to
write javadoc in markdown.
Fixes SMACK-928.
The UserTuneIntegrationTest, in rapid succession:
- add a listener for PEP-published usertune data
- publishes a usertune
- waits for a notification to arrive
Implicit to adding the listener is the publication of a change in
Pubsub notification filtering. This can involve a stanza handshake,
as CAPS is involved.
A race condition exists where the usertune data can be published
before the notification filter has been properly applied.
The changes in this commit add a synchronzation point that ensures
that the notification filter is in place, before the usertune data
is published.
Co-authored-by: Paul Schaub <vanitasvitae@fsfe.org>
Some roster-based tests depend on there not being any prior subscription state beteween entities. The utility method that
tries to guarantee that, acts on the state of the roster that's cached in memory, but acts on the one that's stored on the
server. This occasionally causes issues, as both representations might be different.
Stability is added in this commit by:
- refreshing the roster from the server prior to evaluating it
- ignoring an 'item-not-found' as returned by the server, when the code tries to remove that item.
If something goes wrong when destroying a MUC, we want to be notified
about it immediately, as further results of the integration test run
are unreliable anyway.
