It seems like most "Assert ... Results Contains" tests support retries. The HTTP connector tests do not. That feature should be available across all assertion tests.
Store passwords in a way that:
Ties the serialized value to the local Validator install
Allows tests to be imported with "clear text passwords" that are then stored securely.
Separates the password storage from the test definitions.
Perhaps reuse the "named password" features in IDM to implement the storage?
Put the Description of the test into the results tab. Looking at many very similar tests having the Description can make it so much easier.
And on a failed test, particularly one where values are compared. Show the value found. This is a standard feature of Junit where it shows both expected and actual values. Right now you show only expected value in the report.
Since many workflows are started automatically, it would be helpful In the UserApp connector if there was a way to get a list of workflow approvals pending for a particular user. This would allow us to know if the workflow was automatically started correctly.
Add a parameter to select a specific test in a suite file to run from command line.
I would like to create a suite with two tests, one basic for Swift verification and one for mor thorough tests.
Then I want to run these from command line.
It will be a maintenence problem to put these tests in diffrent suite files, since they will be based on the same set of templates.
We would like to put quite a lot of information in variables. It would simplify the overview if there was a built in namespace/grouping functionality. This would make it possible to introduce folding for each namespace/Group.
In the UI one could unfold only the section one is interested in.
To make validator tests more maintainable, it would be nice if one could reference other sets of variables/connectors or templates from a suite.
It will be easier to 'divide and conquer' testing if some common parts could be reused instead of having to copy/paste between suites.
When you copy a function to a complimentary fuinction, for example, Copy an Add Object and change it to a Delete Object, the corresponding fields should be maintained. The DN from the add object should become the DN of the delete object (right now it puts the class name in when you do that).
When your action makes a change to object attributes, the prior values should be preserved, so that a new action could "undo" the change. You could either stack the prior values, or allow the undo to reference a specific change.
Or it would be great to tie in new functionality into validator that would perform the Analyzer functionality. Compare values... more »
If test coverage for default driver config can be developed, consider a mode of storing the Validator config in an eDir object (DirXML-Resource class, with DirXML-ContentType=mime/Validator-test-suite) so that when you import drivers from Vault in Validator, you also get a default test suite. Would allow developers to ship drivers with test suites included.
I know you can run "Use XPath and Set Variable", then do an assertion on that variable. However, "Assert XPath Results Contains" and "Assert XPath Results Not Contains" would be very useful, by which you could specify an Xpath expression and do an assertion all in one step.
Use case: I make a SOAP... more »
Ideally the command line could spit out the test results in JSON or some useful format that could be used as input to another process.