Basic Usage

This guide is intended to work as a useful reference for those who already have a basic understanding on how to use CallbackParams. It takes a little more than a basic-usage example to reach this understanding so instead of trying to present such an example here the reader is encouraged to turn to the tutorial articles.

The comparison to traditional paramterization and the maintenance examples presented in the part-one article "Patterns That Simplify Maintenance" is of particular importance for understanding the ways of CallbackParams!

This guide explains CallbackParams' functionality from a pretty low-level point of view ...

JUnit Integration

CallbackParamsRunner

CallbackParams' primary mean for JUnit integration is CallbackParamsRunner, which extends JUnit's Runner class.

It enforces CallbackParams' functionality by making byte-code modifications and then sends the ~rebyted~ test-class to another suitable Runner implementation, which gets to take care of the actual test-execution while CallbackParamsRunner stays in the background to rig the tests with the right callback-records and parameter-values and also make sure the test-names are properly composed.

JUnit-4.x API

When a JUnit4-style test-class is annotated with ...

@RunWith(CallbackParamsRunner.class)
public class MyJUnit4ParameterizedTest {
  ...

... the CallbackParamsRunner instance will enforce the parameterization functionality through byte-code modifications and then use a callback-records factory to combine the callback-records.

Thereafter a suitable Runner class will be chosen to take care of the actual test-execution. For JUnit-4.5+ JUnit4 will be chosen. The chosen runner class gets to do its thing on the modified test-class once for each callback-record.

The chosen runner will of course know nothing about the parameterization, which has been enforced through byte-code modifications in this manner:

  • @Before, @Test and @After methods that accept interface arguments have been substituted by equivalently annotated proxy-methods, which implementations invoke the original methods after first picking the proper callback arguments.
  • Fields of interface-type that are annotated with @ParameterizedCallback will be initialized with the proper callbacks before(!) the test-class initializers and constructor are executed.
  • Fields that are annotated with @ParameterizedValue will be initialized with a combined parameter-value.
    The field-type itself specifies the available parameter-values with a static method values() that is expected to return an array with the available parameter-values. This is why an enum-class would be convenient because it will have an implicit method values() that returns all of its enum-values. There will be an error if a non-enum field-type does not have a static values() method to provide the parameter-values! The exceptions are fields of types boolean or Boolean, which obviously have true and false as their available parameter-values.

For more educational examples of this functionality - please consider the tutorial articles.

JUnit-3.x API

CallbackParamsRunner can also be used for a JUnit-3.x style test-class:

/* TestCase subclass ... */

@RunWith(CallbackParamsRunner.class)
public class MyJUnit3ParameterizedTest extends TestCase {
  ...
}

/* ... or suite-style tests: */

@RunWith(CallbackParamsRunner.class)
public class MyJUnit3SuiteParameterizedTest {

  public static Test suite() {
    ...
  }
  ...
}

When CallbackParamsRunner encounters a JUnit3-style test-class it acts pretty much as for the JUnit4-style test but with a few differences:

  • The modified test-class is handed to one of JUnit's JUnit-3.x runners, i.e. for JUnit-4.5+ JUnit38ClassRunner (for TestCase tests) or AllTests (for suite-style tests) will be used.
  • The methods to be proxied are the public and void methods that take interface arguments and has names that starts with "test". (This means that the equivalents of @Before and @After methods, i.e. setUp() and tearDown() cannot take callback-arguments. - Use @ParameterizedCallback-annotated fields instead!)

Please note that it is the test-class that will have its byte-code modified. - I.e. for the suite-style test-example above it is the class MyJUnit3SuiteParameterizedTest that will have its byte-code modified, regardless of the nature of whatever is returned by its suite()-method. This will not make much sense when using this pattern to create a simple test-suite. However, it makes good sense when using PowerMockSuite from the Powermock framework or similarly implemented 3rd-party frameworks.

@WrappedRunner

By using the annotation @WrappedRunner it is possible to specify an arbitrary 3rd-party Runner, to which the modified test-class will be handed for test-execution:

@RunWith(CallbackParamsRunner.class)
@WrappedRunner(PowerMockRunner.class)
@PrepareForTest(ClassToBeTested.class)
public class MyParameterizedPowermockTest {
  ...
}

@Runwith(CallbackParamsRunner.class)
@WrappedRunner(SpringJUnit4ClassRunner.class)
@ContextConfiguration
public class MyParameterizedSpringContextTest {
  ...
}

Above are two examples of the power of @WrappedRunner. MyParameterizedPowermockTest works as a parameterized Powermock test and MyParameterizedSpringContextTest works as a parameterized test that also benenfits from all the Spring Framework features that SpringJUnit4ClassRunner can offer. - And that includes the annotation-based transaction support ...

Stuck with JDK-1.4? - JUnit-3.x (without annotations)

For situations where the project for some reason is stuck under JDK-1.4, CallbackParams also offers two JUnit-3.x API alternatives that can be used under JDK-1.4 (i.e. when annotations cannot be used).

These alternatives work almost exactly as when a test-class using the JUnit-3.x API is run with CallbackParamsRunner in the way proxy-methods are introduced for the test-methods that take interface arguments. The main difference is that all non-static and non-final fields of interface-type will be initialized with callbacks, in order to make up for the fact that the @ParameterizedCallback-annotation cannot be used under JDK-1.4. - This JDK-1.4 field-initialization is one reason to why the fields are injected before constructor and initializers are executed:

public class MyTest extends CallbackTestCase {

  interface Callback {
    ...
  }

  Callback foo;
  Callback bar = null;

The fields foo and bar will both be initialized with callback-instances of Callback. However, since this initialization occurs before the initializers are executed, the field bar will be reset to null during the remaining object-initialization ...

CallbackTestSuite

Providing the JUnit-3.x style test-class with a suite-method that returns a CallbackTestSuite-instance of the test-class is the direct equivalent of annotating it with @RunWith(CallbackParams.class) ...

public class MyJUnit3ParameterizedTest extends TestCase {

  public static Test suite() {
    return new CallbackTestSuite(MyJUnit3ParameterizedTest.class);
  }
  ...
}

... with the previously mentioned difference that all (non-static, non-final) fields of interface-type will be initialized with a callback, regardless of whether annotated with @ParameterizedCallback or not.

CallbackTestCase

By having the test-class extend CallbackTestCase the test-class will make sure to create a byte-code modified version of itself as described above and then have the execution continue with instances of the modified class.

public class MyJUnit3ParameterizedTest extends CallbackTestCase {
  ...

Unlike CallbackTestSuite there are situations where CallbackTestCase provides an API that is desirable even if JUnit-4.x is used, since it does offer some special API that can be used to hamper with the combine machinary in a non-static context.

Having a test-class extend CallbackTestCase does not disqualify it from being annotated with @RunWith(CallbackParamsRunner.class):

@RunWith(CallbackParamsRunner.class)
public class MyJUnit3ParameterizedTest extends CallbackTestCase {
  ...

The only diffence from the non-annotated case is that only the @ParameterizedCallback-annotated fields will be initialized with callbacks.

Combining

One of the key features of CallbackParams is the automated combining of parameter values. For information on the purpose and benefits of this feature please turn to the tutorial article "Patterns That Simplify Maintenance"!

API

CallbackParams' combine API provides the possibility to add parameter-values without the need to manually make sure the new parameter-values are properly incorporated in decent callback-records. The mechanism for automated composing of callback-records can be controlled by the test-developer by specifying (and configuring) a particular combine strategy ...

Combine Strategy

Technically, a combine strategy is simply an implementation of the interface CombineStrategy. The resulting callback-records are returned by the method combine(...), which takes an argument of class ValuesCollection. A ValuesCollection instance is a list of object arrays. A typical array is the array that is retrieved by the static method values() (which is implicitly available on every enum-class).

Each one of the callback-records retrieved from combine(...) is expected (but not obliged) to contain exactly one element from each one of the arrays available in the argument ValuesCollection - but in reality a combine-strategy is free to produce callback-records any way it likes, possibly even ignoring whatever parameter-values are available in the argument ValuesCollection.

@CombineConfig

The recommended way to specify a non-default combine-strategy for a CallbackParams test-class is to annotate it with @CombineConfig:

@RunWith(CallbackParamsRunner.class)
@CombineConfig(strategy=MyCombineStrategy.class)

The mandatory annotation property strategy is set to the CombineStrategy implementation class to use. The annotation also makes it possible to set two additional properties maxCount and randomSeed, which will be passed to their respective setters on the combine-strategy instance.

@CallbackRecords

The annotation @CallbackRecords is to CallbackParams what @Parameterized.Parameters is to paremeterized tests in JUnit. I.e. it makes it possible to have a static method of the test-class compose callback-records for the testrun.

Please note that manual composing of callback-record is generally not recommended when using the CallbackParams framework. The main reason for offering this piece of API is to simplify for developers that wish to closely investigate the test execution for a certain callback-record.

@RunWith(CallbackParams.class)
// @CombineConfig(strategy=SomeCallbackStrategy.class)
public class MyTest {

  @CallbackRecords
  public static java.util.Collection tmpSingleRecordForDebugPurpose() {
    return java.util.Arrays.asList(new Object[][] {{
      MyEnum1.VALUE_A, MyEnum2.VALUE_B
    }};
  }
  ...

For the above test-class the developer has out-commented the @CombineConfig annotation in order to instead explicitly specify a certain callback-record by using a @CallbackRecords annotation.
The purpose is probably to debug the test execution for the specified callback-record, fix a specific problem and then remove the tmpSingleRecordForDebugPurpose() method and reactivate the @CombineConfig annotation.

JDK-1.4 (CallbackRecordsFactory)

When using JDK-1.4 it is not possible to benefit from any API that relies on annotations. I.e. the annotation @CombineConfig cannot be used for specifying the combine-strategy. Fortunatelly the JUnit-3.x API of CallbackParams offers other means for this.

The most flexible JDK-1.4 compatible alternative to the above annotations is to specify a CallbackRecordsFactory. When subclassing CallbackTestCase this is done by overriding the method getCallbackRecordsFactory ...

public class MyJUnit3ParameterizedTest extends CallbackTestCase {

  protected CallbackRecordsFactory getCallbackRecordsFactory() {

    return new CallbackRecordsFactory() {
  
      /**
       * This overridden method-implementation is the JDK-1.4 equivalent
       * for annotating the test-class with
       * @CombineConfig(strategy=CombineCompletely.class, maxCount=500)
       */
      public CombineStrategy retrieveCombineStrategy(Class testClass) {
        CombineStrategy strategy = new CombineCompletely();
        strategy.setMaxCount(500);
        return strategy;
      }
    };
  }
  ...
}

... and when using CallbackTestSuite it is possible to leverage from a certain constructor that takes your choice of CallbackRecordsFactory as the second argument:

public class MyJUnit3ParameterizedTest extends TestCase {

  public static Test suite() {
    return new CallbackTestSuite(MyJUnit3ParameterizedTest.class,
        new CallbackRecordsFactory() {
  
      /**
       * This overridden method-implementation is the JDK-1.4 equivalent
       * for annotating the test-class with
       * @CombineConfig(strategy=CombineCompletely.class, maxCount=500)
       */
      public CombineStrategy retrieveCombineStrategy(Class testClass) {
        CombineStrategy strategy = new CombineCompletely();
        strategy.setMaxCount(500);
        return strategy;
      }
    });
  }
  ...
}

Built-in Strategies

The CallbackParams framework currently offers two built-in combine-strategies that can be used out of the box ...

... where CombineAllPossible2Combinations is the default combine strategy that will be used when the test-class does not make use of the combine API. The javadoc offers more details on how these strategies work and there is also a section on this in one of the tutorial articles.

Mechanisms for Parameterized Callbacks

A callback-method invocation tries to find a callback for each callback-value (i.e. each element in the callback record). There are a few different mechanisms available to each callback-value - and they will be tried in this order ...

  1. Callback-Value Implements the Callback Interface
  2. Callback-Value Provides Callback by Implementing CallbackFactory
  3. Callback-Value Has Annotation which Specifies Callback Implementation
  4. The Callback-Interface Specifies Default Callback Implementations

For number 4 there is not yet any stable API available - but it will come ...

1) Callback-Value Implements the Callback Interface

This is when a callback-value itself implements the callback-interface. It constitutes the original idea of CallbackParams and is the mechanism that gets the exclusive attention in the tutorial article Patterns That Simplify Maintenance, where all the parameter values implement the callback-interface.

2) Callback-Value Provides Callback by Implementing CallbackFactory

When a callback-value does not implement the callback-interface it can instead ~manufacture~ the callback. To do so it must implement CallbackFactory and have the implementation of getCallback return the ~manufactured~ callback. The framework will check whether the callback-interface is implemented by the returned object (i.e. the ~manufactured~ callback) and (if that is the case) invoke its implementation of the callback-method:

@RunWith(CallbackParamsRunner.class)
public class MyTest {

  ...

  enum Foo implements CallbackFactory {
    FOO_A('A'),
    FOO_B('B');

    Callback callback;

    Supplier(char charValue) {
      this.callback = new ComplexCallbackImpl("foo", charValue);
    }

    public Object getCallback() {
      return this.callback;
    }
  }

  enum Bar implements CallbackFactory {
    BAR_1('1'),
    BAR_2('2');

    Callback callback;

    Supplier(char charValue) {
      this.callback = new ComplexCallbackImpl("bar", charValue);
    }

    public Object getCallback() {
      return this.callback;
    }
  }
}

The above enums "Foo" and "Bar" both need to have Callback implemented in very similar ways and by using the CallbackFactory API it was possible to externalize the common logic to the separate class "ComplexCallbackImpl" and therewith achieve better compliance with the DRY-principle.

However, for enum-constants there is a mechanism that is much more convenient ...

3) Callback-Value Has Annotation which Specifies Callback Implementation

If the class ComplexCallbackImpl extends WrappingSupport (or implements Wrapping) it is possible to have an annotation specify ComplexCallbackImpl as callback:

public class ComplexCallbackImpl extends WrappingSupport<Enum<?>>
implements Callback {

  @WrappingCallbackImpls(ComplexCallbackImpl.class)
  @Retention(RetentionPolicy.RUNTIME)
  public @interface Args {
    String name();
    char charValue();
  }

  /**
   * Annotation from wrapped enum-constant will be injected here by
   * CallbackParams
   */
  private Args argsFromEnumConstant;

  ...
}

The above declaration of annotation type @ComplexCallbackImpl.Args uses framework annotation @WrappingCallbackImpls to specify ComplexCallbackImpl.class as callback for any enum-constant that is annotated with @ComplexCallbackImpl.Args!

Therewith the enums Foo and Bar can use annotation @ComplexCallbackImpl.Args specify ComplexCallbackImpl as callback in a very slim manner:

@RunWith(CallbackParamsRunner.class)
public class MyTest {

  ...

  enum Foo {
    @ComplexCallbackImpl.Args(name="foo", charValue='A') FOO_A,
    @ComplexCallbackImpl.Args(name="foo", charValue='B') FOO_B;
  }

  enum Bar {
    @ComplexCallbackImpl.Args(name="bar", charValue='1') BAR_1,
    @ComplexCallbackImpl.Args(name="bar", charValue='2') BAR_2;
  }
}

The above case is very simple and obvious but enum constants and classes can have many annotations and these can specify different Callback implementations. What would happen in case the annotations provide more than one valid Callback implementation is mostly undefined but there are a few rules:

  1. If annotations from the enum-constant as well as the enum-class specify valid Callback implementations - then only the enum-constant annotations will be considered.
  2. If (after separating annotations on constant- and class-level) there are still more than one Callback implementation available and all Callback implementations are come from the same class hierarchy - then any implementation from a super-class of another will be excluded.

Note that it is possible for the developer to resolve any undefined scenario by using the annotation @WrappingCallbackImpls on the enum-constant directly but that workaround is no recommended, however. Instead it would perhaps be better for the test-developer to think about potential modifications of the parameter model in such cases.

Also note that a @WrappingCallbackImpls annotation can specify classes that implement CallbackFactory. For such a class it is irrelevant whether the class also implements the callback interface. - What matters is whether the callback-interface is implemented by the return-value from getCallback ...

Sandbox Example: Property

The sandbox-class Property and its nested resources have been developed as an example on how annotation-specified callbacks could be used. In this case it is the nested annotation @Property.Value that specifies the sibling class Property.BasicImpl as callback implementation.

Please note how Property.BasicImpl is an example of how both the callback interface (Property.Callback) and CallbackFactory is implemented. Whether a callback can be arranged depends on whether the method getCallback() decides to return itself or not!

4) The Callback-Interface Specifies Default Callback Implementations

This is not yet supported by a stable API. A modest estimate of the arrival of this API says September 2013 in the beta-8 release, which is planned to be the final beta before the first stable release - 1.0.0.

Parameterized Callback Return-Values

Invocation Has Only One Callback

When a parameterized callback contains only one callback then the return-value from this single callback will be returned from the callback-method invocation.

Return-Type and First Parameter-Type Are Same (Genericly) - Folding

When a callback-method's return-type is identical to the first parameter-type then the first argument in the callback-invocation will only be passed to the first callback. Thereafter the return-value from the previous callback will be passed as first argument to the next callback. - In the end the overall callback-invocation return the value returned by the last callback.
In the special situation when the overall callback-invocation cannot reach any callback then the first argument of the callback-invocation will be returned at once.

This mechanism encourages the callbacks to fold their data into a cumulative result. This is the only way for multiple callbacks to build a mutable cumulative return-value.

The strongest argument for CallbackParams to support folding on this kind of callback method signatures can be found in how the JUnit Rules API is designed ...

Parameterized Rules

The folding mechanism will apply to the apply-methods of the JUnit Rules interfaces. This opens a door for callback parameters to specify their own set-up and tear-down functionality without involving any @Before- or @After-method in the test-class. It is done by declaring a public callback-parameterized field of an interface-type for JUnit rules and annotate it with the JUnit @Rule annotation.

JUnit-rules interfaces come in two different flavors ...

TestRule (Since JUnit-4.9)

The rule-interface TestRule and its abstract implementation-class ExternalResource do probably offer the most interesting solution for decorating a parameter lifecycle with before-and-after functionality.
There is not one obvious way to do it, however. The code below present three parameter classes that incorporate ExternalResource with significantly different patterns ...

/**
 * The test-class uses this @Rule declaration
 * to properly fold all three TestRule callbacks.
 */
@Rule @ParameterizedCallback
public TestRule ruleCallbacks;

/**************************************************************************
 * Pattern I: Expose ExternalResource by implementing CallbackFactory
 *
 * The enum has a field beforeAndAfter for holding its ExternalResource
 * instance - and has the implementation of CallbackFactory#getCallback()
 * offer it as a callback.
 *
 * pro: Simple design that is easy to understand
 * con: Implementation of CallbackFactory#getCallback() is now occupied with this
 *      task and is not easily reused for exposing any other indirect callback.
 */
enum RuleFromCallbackFactory implements CallbackFactory {
  VALUE_1("value 1"),
  VALUE_2("value 2");

  String value;

  RuleFromCallbackFactory(String value) {this.value = value;}

  TestRule beforeAndAfter = new ExternalResource() {
    @Override protected void before() { /* Set-up code here */ }
    @Override protected void after() { /* Tear-down code here */ }
  };
  
  public Object getCallback() {
    return beforeAndAfter;
  }
}/* End of Pattern I ******************************************************/

/**************************************************************************
 * Pattern II: Apply ExternalResource by implementing TestRule directly
 *
 * This enum also has the beforeAndAfter field for holding its
 * ExternalResource instance but this time the implementation
 * of TestRule#apply(...) wraps the applying of its ExternalResource.
 *
 * pro: Does not occupy CallbackFactory#getCallback(), which is now freely
 *      available for other needs
 * con: Not as simple as "Pattern I" - developers, who are not necessarily
 *      familiar with the inner working of JUnit rules, are perhaps
 *      uncomfortable with this kind of code
 */
enum ImplementTestRule implements TestRule {
  VALUE_3("value 3"),
  VALUE_4("value 4");

  String value;

  ImplementTestRule(String value) {this.value = value;}

  TestRule beforeAndAfter = new ExternalResource() {
    @Override protected void before() { /* Set-up code here */ }
    @Override protected void after() { /* Tear-down code here */ }
  };

  public Statement apply(Statement base, Description description) {
    return beforeAndAfter.apply(base, description)
  }
}/* End of Pattern II *****************************************************/

/**************************************************************************
 * Pattern III: Abstract Wrapping extends ExternalResource
 * 
 * Fact is that the internals of CallbackParams do not actually know what
 * an enum is. What the internals actually do is to collect whichever
 * parameter values that are returned from a static values()-method, which
 * the Java-compiler implicitly adds to every enum-class!
 * The API interface Wrapping and WrappingSupport are designed to also
 * work as a powerful utility options for situations when the parameter
 * wraps primitive singleton data (in this case String-values).
 * By trusting the method WrappingSupport.html#wrap(...) to construct the
 * concrete Wrapping instances it is possible to have your parameter-type
 * extend another class (in this case ExternalResource) - whereas an enum-
 * class cannot extend any other class.
 *
 * pro: - by extending ExternalResource the before()- and after()-methods
 *        can be specified in the parameter-class directly - no need to wrap
 *        them inside any nested class
 *      - no constructor needed - the method WrappingSupport#wrap(...) makes
 *        sure the wrapped value is reached through the method wrappedValue()
 *      - less code
 * con: not an enum ...
 *      - the a static values()-method must be explicitly defined
 *      - a single parameter-value cannot easily override the default
 *        behaviour of a callback-method
 */
static abstract class WrappingExtendsExternalResource
extends ExternalResource implements Wrapping<String> {

  static WrappingExtendsExternalResource[] values() {
    return WrappingSupport.wrap(WrappingExtendsExternalResource.class,
        "value 5",
        "value 6");
  }

  String value = wrappedValue();

  @Override protected void before() { /* Set-up code here */ }
  @Override protected void after() { /* Tear-down code here */ }
}/* End of Pattern III ****************************************************/
MethodRule (Since JUnit-4.7)

This - a little older - JUnit Rules interface is no longer used by the core rules provided by JUnit. Its significant difference from TestRule is that JUnit sends the test instance to its apply-method - something that makes its API an interesting alternative for parameterized rules.

An interesting opportunity is to accomplish parameter-specific before-and-after functionality by replacing ExternalResource in the above code-examples with something similar to this:

public abstract class TargetAwareResource implements MethodRule {

  public Statement apply(final Statement base,
                         FrameworkMethod testMethodWhichIsIgnoredHere,
                         final Object target) {
    return new Statement() {
      @Override public void evaluate() throws Throwable {
        before(target);
        try {
          base.evaluate();
        } finally {
          after(target);
        }
      }
    };
  }

  /** Override to specify set-up code */
  protected void before(Object targetTest) throws Throwable {}

  /** Override to specify tear-down code */
  protected void after(Object targetTest) {}
}

The above code-example would need these modifications:

/**
 * The test-class uses this @Rule declaration
 * to properly fold all three TestRule MethodRule callbacks.
 */
@Rule @ParameterizedCallback
public TestRule MethodRule ruleCallbacks;

/**************************************************************************
 * Pattern I: Expose TargetAwareResource by implementing CallbackFactory
 * ...
 */
enum RuleFromCallbackFactory implements CallbackFactory {
  VALUE_1("value 1"),
  VALUE_2("value 2");

  String value;

  RuleFromCallbackFactory(String value) {this.value = value;}

  TestRule beforeAndAfter = new ExternalResource() {
  MethodRule beforeAndAfter = new TargetAwareResource() {
    @Override protected void before(Object targetTest) { /* Set-up code here */ }
    @Override protected void after(Object targetTest) { /* Tear-down code here */ }
  };

  public Object getCallback() {
    return beforeAndAfter;
  }
}/* End of Pattern I ******************************************************/

/**************************************************************************
 * Pattern II: Apply TargetAwareResource by implementing TestRule MethodRule directly
 * ...
 */
enum ImplementTestRule implements TestRule MethodRule {
  VALUE_3("value 3"),
  VALUE_4("value 4");

  String value;

  ImplementTestRule(String value) {this.value = value;}

  TestRule beforeAndAfter = new ExternalResource() {
  MethodRule beforeAndAfter = new TargetAwareResource() {
    @Override protected void before(Object targetTest) { /* Set-up code here */ }
    @Override protected void after(Object targetTest) { /* Tear-down code here */ }
  };

  public Statement apply(Statement base, Description description) {
  public Statement apply(Statement base, FrameworkMethod method, Object target) {
    return beforeAndAfter.apply(base, description method, target)
  }
}/* End of Pattern II *****************************************************/

/**************************************************************************
 * Pattern III: Abstract Wrapping extends TargetAwareResource
 * ...
 */
static abstract class WrappingExtendsExternalResource
extends ExternalResource TargetAwareResource implements Wrapping<String> {

  static WrappingExtendsExternalResource[] values() {
    return WrappingSupport.wrap(WrappingExtendsExternalResource.class,
        "value 5",
        "value 6");
  }

  String value = wrappedValue();

  @Override protected void before(Object targetTest) { /* Set-up code here */ }
  @Override protected void after(Object targetTest) { /* Tear-down code here */ }
}/* End of Pattern III ****************************************************/

Multiple Return-Values that Are Equal

When the parameterized callback reaches more than one callback and all return-values are equals, then the last of these return-values will be returned from the overall composite invocation. (Which return-value is the last depends on the order of the callback-values in the callback-record. This order is undefined unless the callback-record is sorted. See below ...)

Otherwise Return Default Value (null, false or 0)

Unless any of the above situations apply, the default value of the return-type will be returned. This means null for everything except the primitive return-types, which have false or 0 as their default values.

When you get this default return-value it is still possible to determine the individual return-values by using the method getLatestCallbackResults() on CallbackControlPanel

Low-Level API

Though CallbackParams wish to offer pretty and generic patterns for all sorts of situations, there will always occur situations where the recommended patterns simply do not get the job done. Hopefully the test-developer will then come up with innovative ideas for new framework features but until those features are properly incorporated the developer will need to implement some sort of workaround and that is where low-level features could be handy.

CallbackControlPanel

An instance of the interface CallbackControlPanel is accessed by the testrun in the same manner as callback-interfaces are accessed but it is not a callback-interface! Instead it offers an API to the callback-record low-level details of the current testrun:

getCurrentCallbackRecord()

The method getCurrentCallbackRecord() offers immediate access to the current callback-record:

public class TestThatSortsTheValuesOfTheCallbackRecord {

  /**
   * The built-in combine-strategies do not sort the values of the
   * callback-records in any particular order. That is here taken care of by
   * having an initializer access the current callback-record through
   * CallbackControlPanel and sort the values in the callback-record.
   * Please notice how the use of an initializer can take care of this before
   * the constructor and "@Before"-methods get going ...
   */
  @ParameterizedCallback CallbackControlPanel panel;
  {
    Collections.sort(panel.getCurrentCallbackRecord(), new SomeComparator());
  }
...

If the test-developer needs to sort the values of the callback-record then the above pattern can be used

To use CallbackControlPanel as a mean to modify the callback-record is generally not recommended, however. It is very likely to lead to a number of undefined situations when future features are implemented.

getLatestCallbackResults()

CallbackControlPanel is currently the only way to access the return values in case there are some non-void callback-methods:

  @Test
  public void testMethod(MyCallback callback, CallbackControlPanel panel) {
    callback.isValidationFailureExpected();
    if (panel.getLatestCallbackResults().values().contains(Boolean.TRUE)) {
      testValidationFailure(callback);
    } else {
      testHappyPath(callback);
    }
  }

In the example above the boolean return-values of the callback-method "isValidationFailureExpected()" are investigated to determine whether any of them force the testrun to follow a validation-failure expecting path (instead of the happy path).
The above is a reasonable example on how to access return values of callback-methods but please note that the tuturial article Validation of Validation Failures shows how the framework offers much better patterns for testing validation failures.