Unit-Testing Complex External Dependencies

The question of how to unit-test complex external dependencies arises pretty frequently. This is near-and-dear to our hearts because we have many of them. Mocking out complex responses is tedious and error-prone, so I’ll tell you what we do instead.

1: The setup

Stratosphere has an contract for obtaining instances from a cloud service provider, viz.,

public interface ListInstancesOperation extends ProviderOperation<List<Instances>> {
      @Override
      public List<Instance> perform();

      @Override
      public Provider getProvider(); 
     
      @Override
      public Secret() getSecret();
//...etc.

}

perform() will typically interact with the provider service endpoint. The body of perform is quite simple:

  public List<Instance> perform() {
    AmazonEC2 client = createClient();

    DescribeInstancesResult result = client.describeInstances();
    return toInstances(result.getReservations());
  }

This doesn’t look like a super-testable method, so what do we do?

2: Get the actual response and write it to a file

Before you write any code that interacts with an external dependency, you have to understand how that dependency behaves. I recommend keeping a set of IAM credentials in ~/.aws/credentials for testing. Once you do that, actually perform the request and see what you get back.

Now, we serialize the response using Java’s default serialization mechanism to a file that we check into source-control.

DescribeInstancesResult result = client.describeInstances();
Objects.write(result, relativeToRoot("src/test/resources/ec2/list-instances.obj")); // our utilities for writing an object using serialization.

3: Mock it real good

Recall that our method under test had 3 statements. One to create the client, one to make the request to the client, and one to map the results. The client’s method under test is public, so we can mock that. We gave createClient default visibility so that we could mock that while not exposing it as part of our Operation API, and then we put all our actual logic into a private method, whose behavior we want to test.

We can now set up our tests to mock out the external operation with a real result:

  private Secret secret;
  private AmazonEC2Client client;
  private EC2ListInstancesOperation operation;

  @BeforeEach
  void setUp() {
    secret = new Secret();
    operation = new EC2ListInstancesOperation(secret, "us-west-2", (AWS) AWS.getInstance());
    operation = spy(operation);

    client = mock(AmazonEC2Client.class);
    given(client.describeInstances())
        .willReturn(Objects.read("src/test/resources/ec2/list-instances.obj", true));
    given(operation.createClient()).willReturn(client);
  }

And test it as follows:

 @Test
  void ensureInstanceFirewallsAreCorrect() {
    List<Instance> perform = operation.perform();
    Instance instance = perform.get(0);
    assertThat(instance.getFirewalls().size(), is(1));
    Firewall firewall = instance.getFirewalls().iterator().next();
    assertThat(firewall.getName(), is("launch-wizard-2"));
    assertThat(firewall.getSecured().contains(instance), is(true));
  }

This approach works pretty well for external dependencies that don’t change frequently (like the public APIs of large cloud service providers), but less well for external dependencies that do. What are some approaches you use?