This guide is intended to work as a useful reference for those who already have a basic understanding on how to use CallbackParams. It takes a little more than a basic-usage example to reach this understanding so instead of trying to present such an example here the reader is encouraged to turn to the tutorial articles.
The comparison to traditional paramterization and the maintenance examples presented in the part-one article "Patterns That Simplify Maintenance" is of particular importance for understanding the ways of CallbackParams!
This guide explains CallbackParams' functionality from a pretty low-level point of view ...
CallbackParams' primary mean for JUnit integration is CallbackParamsRunner, which extends JUnit's Runner class.
It enforces CallbackParams' functionality by making byte-code modifications and then sends the ~rebyted~ test-class to another suitable Runner implementation, which gets to take care of the actual test-execution while CallbackParamsRunner stays in the background to rig the tests with the right callback-records and parameter-values and also make sure the test-names are properly composed.
When a JUnit4-style test-class is annotated with ...
@RunWith(CallbackParamsRunner.class)
public class MyJUnit4ParameterizedTest {
...
... the CallbackParamsRunner instance will enforce the parameterization functionality through byte-code modifications and then use a callback-records factory to combine the callback-records.
Thereafter a suitable Runner class will be chosen to take care of the actual test-execution. For JUnit-4.5+ JUnit4 will be chosen. The chosen runner class gets to do its thing on the modified test-class once for each callback-record.
The chosen runner will of course know nothing about the parameterization, which has been enforced through byte-code modifications in this manner:
For more educational examples of this functionality - please consider the tutorial articles.
CallbackParamsRunner can also be used for a JUnit-3.x style test-class:
/* TestCase subclass ... */
@RunWith(CallbackParamsRunner.class)
public class MyJUnit3ParameterizedTest extends TestCase {
...
}
/* ... or suite-style tests: */
@RunWith(CallbackParamsRunner.class)
public class MyJUnit3SuiteParameterizedTest {
public static Test suite() {
...
}
...
}
When CallbackParamsRunner encounters a JUnit3-style test-class it acts pretty much as for the JUnit4-style test but with a few differences:
Please note that it is the test-class that will have its byte-code modified. - I.e. for the suite-style test-example above it is the class MyJUnit3SuiteParameterizedTest that will have its byte-code modified, regardless of the nature of whatever is returned by its suite()-method. This will not make much sense when using this pattern to create a simple test-suite. However, it makes good sense when using PowerMockSuite from the Powermock framework or similarly implemented 3rd-party frameworks.
By using the annotation @WrappedRunner it is possible to specify an arbitrary 3rd-party Runner, to which the modified test-class will be handed for test-execution:
@RunWith(CallbackParamsRunner.class)
@WrappedRunner(PowerMockRunner.class)
@PrepareForTest(ClassToBeTested.class)
public class MyParameterizedPowermockTest {
...
}
@Runwith(CallbackParamsRunner.class)
@WrappedRunner(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class MyParameterizedSpringContextTest {
...
}
Above are two examples of the power of @WrappedRunner. MyParameterizedPowermockTest works as a parameterized Powermock test and MyParameterizedSpringContextTest works as a parameterized test that also benenfits from all the Spring Framework features that SpringJUnit4ClassRunner can offer. - And that includes the annotation-based transaction support ...
For situations where the project for some reason is stuck under JDK-1.4, CallbackParams also offers two JUnit-3.x API alternatives that can be used under JDK-1.4 (i.e. when annotations cannot be used).
These alternatives work almost exactly as when a test-class using the JUnit-3.x API is run with CallbackParamsRunner in the way proxy-methods are introduced for the test-methods that take interface arguments. The main difference is that all non-static and non-final fields of interface-type will be initialized with callbacks, in order to make up for the fact that the @ParameterizedCallback-annotation cannot be used under JDK-1.4. - This JDK-1.4 field-initialization is one reason to why the fields are injected before constructor and initializers are executed:
public class MyTest extends CallbackTestCase {
interface Callback {
...
}
Callback foo;
Callback bar = null;
The fields foo and bar will both be initialized with callback-instances of Callback. However, since this initialization occurs before the initializers are executed, the field bar will be reset to null during the remaining object-initialization ...
Providing the JUnit-3.x style test-class with a suite-method that returns a CallbackTestSuite-instance of the test-class is the direct equivalent of annotating it with @RunWith(CallbackParams.class) ...
public class MyJUnit3ParameterizedTest extends TestCase {
public static Test suite() {
return new CallbackTestSuite(MyJUnit3ParameterizedTest.class);
}
...
}
... with the previously mentioned difference that all (non-static, non-final) fields of interface-type will be initialized with a callback, regardless of whether annotated with @ParameterizedCallback or not.
By having the test-class extend CallbackTestCase the test-class will make sure to create a byte-code modified version of itself as described above and then have the execution continue with instances of the modified class.
public class MyJUnit3ParameterizedTest extends CallbackTestCase {
...
Unlike CallbackTestSuite there are situations where CallbackTestCase provides an API that is desirable even if JUnit-4.x is used, since it does offer some special API that can be used to hamper with the combine machinary in a non-static context.
Having a test-class extend CallbackTestCase does not disqualify it from being annotated with @RunWith(CallbackParamsRunner.class):
@RunWith(CallbackParamsRunner.class)
public class MyJUnit3ParameterizedTest extends CallbackTestCase {
...
The only diffence from the non-annotated case is that only the @ParameterizedCallback-annotated fields will be initialized with callbacks.
One of the key features of CallbackParams is the automated combining of parameter values. For information on the purpose and benefits of this feature please turn to the tutorial article "Patterns That Simplify Maintenance"!
CallbackParams' combine API provides the possibility to add parameter-values without the need to manually make sure the new parameter-values are properly incorporated in decent callback-records. The mechanism for automated composing of callback-records can be controlled by the test-developer by specifying (and configuring) a particular combine strategy ...
Technically, a combine strategy is simply an implementation of the interface CombineStrategy. The resulting callback-records are returned by the method combine(...), which takes an argument of class ValuesCollection. A ValuesCollection instance is a list of object arrays. A typical array is the array that is retrieved by the static method values() (which is implicitly available on every enum-class).
Each one of the callback-records retrieved from combine(...) is expected (but not obliged) to contain exactly one element from each one of the arrays available in the argument ValuesCollection - but in reality a combine-strategy is free to produce callback-records any way it likes, possibly even ignoring whatever parameter-values are available in the argument ValuesCollection.
The recommended way to specify a non-default combine-strategy for a CallbackParams test-class is to annotate it with @CombineConfig:
@RunWith(CallbackParamsRunner.class) @CombineConfig(strategy=MyCombineStrategy.class)
The mandatory annotation property strategy is set to the CombineStrategy implementation class to use. The annotation also makes it possible to set two additional properties maxCount and randomSeed, which will be passed to their respective setters on the combine-strategy instance.
The annotation @CallbackRecords is to CallbackParams what @Parameterized.Parameters is to paremeterized tests in JUnit. I.e. it makes it possible to have a static method of the test-class compose callback-records for the testrun.
Please note that manual composing of callback-record is generally not recommended when using the CallbackParams framework. The main reason for offering this piece of API is to simplify for developers that wish to closely investigate the test execution for a certain callback-record.
@RunWith(CallbackParams.class)
// @CombineConfig(strategy=SomeCallbackStrategy.class)
public class MyTest {
@CallbackRecords
public static java.util.Collection tmpSingleRecordForDebugPurpose() {
return java.util.Arrays.asList(new Object[][] {{
MyEnum1.VALUE_A, MyEnum2.VALUE_B
}};
}
...
For the above test-class the developer has out-commented the @CombineConfig annotation in order to instead explicitly specify a certain callback-record by using a @CallbackRecords annotation.
The purpose is probably to debug the test execution for the specified callback-record, fix a specific problem and then remove the tmpSingleRecordForDebugPurpose() method and reactivate the @CombineConfig annotation.
When using JDK-1.4 it is not possible to benefit from any API that relies on annotations. I.e. the annotation @CombineConfig cannot be used for specifying the combine-strategy. Fortunatelly the JUnit-3.x API of CallbackParams offers other means for this.
The most flexible JDK-1.4 compatible alternative to the above annotations is to specify a CallbackRecordsFactory. When subclassing CallbackTestCase this is done by overriding the method getCallbackRecordsFactory ...
public class MyJUnit3ParameterizedTest extends CallbackTestCase {
protected CallbackRecordsFactory getCallbackRecordsFactory() {
return new CallbackRecordsFactory() {
/**
* This overridden method-implementation is the JDK-1.4 equivalent
* for annotating the test-class with
* @CombineConfig(strategy=CombineCompletely.class, maxCount=500)
*/
public CombineStrategy retrieveCombineStrategy(Class testClass) {
CombineStrategy strategy = new CombineCompletely();
strategy.setMaxCount(500);
return strategy;
}
};
}
...
}
... and when using CallbackTestSuite it is possible to leverage from a certain constructor that takes your choice of CallbackRecordsFactory as the second argument:
public class MyJUnit3ParameterizedTest extends TestCase {
public static Test suite() {
return new CallbackTestSuite(MyJUnit3ParameterizedTest.class,
new CallbackRecordsFactory() {
/**
* This overridden method-implementation is the JDK-1.4 equivalent
* for annotating the test-class with
* @CombineConfig(strategy=CombineCompletely.class, maxCount=500)
*/
public CombineStrategy retrieveCombineStrategy(Class testClass) {
CombineStrategy strategy = new CombineCompletely();
strategy.setMaxCount(500);
return strategy;
}
});
}
...
}
The CallbackParams framework currently offers two built-in combine-strategies that can be used out of the box ...
... where CombineAllPossible2Combinations is the default combine strategy that will be used when the test-class does not make use of the combine API. The javadoc offers more details on how these strategies work and there is also a section on this in one of the tutorial articles.
A callback-method invocation tries to find a callback for each callback-value (i.e. each element in the callback record). There are a few different mechanisms available to each callback-value - and they will be tried in this order ...
For number 4 there is not yet any stable API available - but it will come ...
This is when a callback-value itself implements the callback-interface. It constitutes the original idea of CallbackParams and is the mechanism that gets the exclusive attention in the tutorial article Patterns That Simplify Maintenance, where all the parameter values implement the callback-interface.
When a callback-value does not implement the callback-interface it can instead ~manufacture~ the callback. To do so it must implement CallbackFactory and have the implementation of getCallback return the ~manufactured~ callback. The framework will check whether the callback-interface is implemented by the returned object (i.e. the ~manufactured~ callback) and (if that is the case) invoke its implementation of the callback-method:
@RunWith(CallbackParamsRunner.class)
public class MyTest {
...
enum Foo implements CallbackFactory {
FOO_A('A'),
FOO_B('B');
Callback callback;
Supplier(char charValue) {
this.callback = new ComplexCallbackImpl("foo", charValue);
}
public Object getCallback() {
return this.callback;
}
}
enum Bar implements CallbackFactory {
BAR_1('1'),
BAR_2('2');
Callback callback;
Supplier(char charValue) {
this.callback = new ComplexCallbackImpl("bar", charValue);
}
public Object getCallback() {
return this.callback;
}
}
}
The above enums "Foo" and "Bar" both need to have Callback implemented in very similar ways and by using the CallbackFactory API it was possible to externalize the common logic to the separate class "ComplexCallbackImpl" and therewith achieve better compliance with the DRY-principle.
However, for enum-constants there is a mechanism that is much more convenient ...
If the class ComplexCallbackImpl extends WrappingSupport (or implements Wrapping) it is possible to have an annotation specify ComplexCallbackImpl as callback:
public class ComplexCallbackImpl extends WrappingSupport<Enum<?>>
implements Callback {
@WrappingCallbackImpls(ComplexCallbackImpl.class)
@Retention(RetentionPolicy.RUNTIME)
public @interface Args {
String name();
char charValue();
}
/**
* Annotation from wrapped enum-constant will be injected here by
* CallbackParams
*/
private Args argsFromEnumConstant;
...
}
The above declaration of annotation type @ComplexCallbackImpl.Args uses framework annotation @WrappingCallbackImpls to specify ComplexCallbackImpl.class as callback for any enum-constant that is annotated with @ComplexCallbackImpl.Args!
Therewith the enums Foo and Bar can use annotation @ComplexCallbackImpl.Args specify ComplexCallbackImpl as callback in a very slim manner:
@RunWith(CallbackParamsRunner.class)
public class MyTest {
...
enum Foo {
@ComplexCallbackImpl.Args(name="foo", charValue='A') FOO_A,
@ComplexCallbackImpl.Args(name="foo", charValue='B') FOO_B;
}
enum Bar {
@ComplexCallbackImpl.Args(name="bar", charValue='1') BAR_1,
@ComplexCallbackImpl.Args(name="bar", charValue='2') BAR_2;
}
}
The above case is very simple and obvious but enum constants and classes can have many annotations and these can specify different Callback implementations. What would happen in case the annotations provide more than one valid Callback implementation is mostly undefined but there are a few rules:
Note that it is possible for the developer to resolve any undefined scenario by using the annotation @WrappingCallbackImpls on the enum-constant directly but that workaround is no recommended, however. Instead it would perhaps be better for the test-developer to think about potential modifications of the parameter model in such cases.
Also note that a @WrappingCallbackImpls annotation can specify classes that implement CallbackFactory. For such a class it is irrelevant whether the class also implements the callback interface. - What matters is whether the callback-interface is implemented by the return-value from getCallback ...
The sandbox-class Property and its nested resources have been developed as an example on how annotation-specified callbacks could be used. In this case it is the nested annotation @Property.Value that specifies the sibling class Property.BasicImpl as callback implementation.
Please note how Property.BasicImpl is an example of how both the callback interface (Property.Callback) and CallbackFactory is implemented. Whether a callback can be arranged depends on whether the method getCallback() decides to return itself or not!
Though CallbackParams wish to offer pretty and generic patterns for all sorts of situations, there will always occur situations where the recommended patterns simply do not get the job done. Hopefully the test-developer will then come up with innovative ideas for new framework features but until those features are properly incorporated the developer will need to implement some sort of workaround and that is where low-level features could be handy.
An instance of the interface CallbackControlPanel is accessed by the testrun in the same manner as callback-interfaces are accessed but it is not a callback-interface! Instead it offers an API to the callback-record low-level details of the current testrun:
The method getCurrentCallbackRecord() offers immediate access to the current callback-record:
public class TestThatSortsTheValuesOfTheCallbackRecord {
/**
* The built-in combine-strategies do not sort the values of the
* callback-records in any particular order. That is here taken care of by
* having an initializer access the current callback-record through
* CallbackControlPanel and sort the values in the callback-record.
* Please notice how the use of an initializer can take care of this before
* the constructor and "@Before"-methods get going ...
*/
@ParameterizedCallback CallbackControlPanel panel;
{
Collections.sort(panel.getCurrentCallbackRecord(), new SomeComparator());
}
...
If the test-developer needs to sort the values of the callback-record then the above pattern can be used
To use CallbackControlPanel as a mean to modify the callback-record is generally not recommended, however. It is very likely to lead to a number of undefined situations when future features are implemented.
CallbackControlPanel is currently the only way to access the return values in case there are some non-void callback-methods:
@Test
public void testMethod(MyCallback callback, CallbackControlPanel panel) {
callback.isValidationFailureExpected();
if (panel.getLatestCallbackResults().values().contains(Boolean.TRUE)) {
testValidationFailure(callback);
} else {
testHappyPath(callback);
}
}
In the example above the boolean return-values of the callback-method "isValidationFailureExpected()" are investigated to determine whether any of them force the testrun to follow a validation-failure expecting path (instead of the happy path).
The above is a reasonable example on how to access return values of callback-methods but please note that the tuturial article Validation of Validation Failures shows how the framework offers much better patterns for testing validation failures.