AWS S3 Client – Local Stack – Test Container

AWS S3 Client – Local Stack – Test Container

Test Container

Test container is a third party Java library which offers to test or validate anything that can run in a Docker container. For Example, we can create lightweight database instances of  MySQL, PostgreSQL or Oracle database to test our data access layer code (DAO , Repositories). Similarly we can test our AWS cloud services like server less apps without actually using the cloud. We can do this with the help of LocalStack, a fully functional local AWS cloud stack.

Local Stack

LocalStack is a cloud service emulator. We can able to run AWS applications like Lambda, S3 entirely on your laptop or machine without connecting to a remote cloud provider.

AWS S3 Client with Local Stack and Test Container

Why It is needed ?

We can able to connect to AWS services like S3 from Java using their SDK’s. But here the problem is, if we wants to validate our code then we are need to connect cloud sandboxes using AWS SAM or anything like that. And While we access the cloud services, It is billable as well. Secondly we can not write unit test or integration test for this sources in a efficient way which would be a another downside of it. So In this article, we are going to learn to write unit testing for AWS S3 services with the help of Local stack and Test containers. Let’s dive into that. Cheers.

Prerequisite

  • Java
  • Maven
  • Docker

Create a Spring initializer project

First, I am creating basic spring initializer project with Spring boot starter, Spring boot test and AWS SDK maven dependencies.

<dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk</artifactId>
            <version>1.11.163</version>
</dependency>

Creating AWS S3 Client

Second, now I am creating AWS S3 client with simple upload object functionality. Here I am validating whether the bucket is exist or not, followed by uploading an object into the S3 storage system.

package awsservicestutorial.client;

import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.ObjectMetadata;

import java.io.IOException;
import java.io.InputStream;
import java.net.URLConnection;
import java.nio.file.Files;
import java.nio.file.Path;

public class AwsS3Client {

    private final AmazonS3 amazonS3;

    public AwsS3Client(final AmazonS3 amazonS3) {
        this.amazonS3 = amazonS3;
    }

    public void uploadObject(final String bucketName, final Path path) throws IOException {

        isBucketExists(bucketName);

        final String filename = path.getFileName().toString();

        final ObjectMetadata metadata = new ObjectMetadata();
        metadata.setContentType(URLConnection.guessContentTypeFromName(filename));
        metadata.setSSEAlgorithm("AES256");
        metadata.setContentLength(Files.size(path));

        try(final InputStream stream = Files.newInputStream(path)){
            amazonS3.putObject(bucketName, filename, stream, metadata);
        }
    }

    private void isBucketExists(final String bucketName) {
        if(!amazonS3.doesBucketExist(bucketName)) {
            throw new IllegalStateException(String.format("Bucket %s does not exist", bucketName));
        }
    }

}

Local Stack and Test Container Dependencies

Add the following dependency to your pom.xml for using local stack and test container libraries.

        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>localstack</artifactId>
            <version>1.17.6</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>1.17.6</version>
            <scope>test</scope>
        </dependency>

Here I am using Junit-5 for unit testing the application. It is up to to decide whether Junit 5 or 4. We can do this with Junit 4 as well.

AWS S3 Client – Local Stack – Test Container – Create Unit Test

Creating unit test with local stack is really very simple. we just need to create local stack instance. This instance can act as AWS S3 services with the help of test container and docker. It will create a exact AWS mock in local or in our CI environment. This will reduce a lot money, since we don’t need to interact AWS cloud services during local development and testing.

package awsservicestutorial;

import awsservicestutorial.client.AwsS3Client;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import org.testcontainers.containers.localstack.LocalStackContainer;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import org.testcontainers.utility.DockerImageName;

import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Optional;

@Testcontainers(disabledWithoutDocker = true)
@ExtendWith(SpringExtension.class)
class AwsServicesTutorialApplicationTests {
    private static final String BUCKET_NAME = "test-bucket";
    @Container
    public final static LocalStackContainer LOCAL_STACK_CONTAINER =
            new LocalStackContainer(DockerImageName.parse("localstack/localstack:0.12.16"))
                    .withServices(LocalStackContainer.Service.S3).withEnv("DEFAULT_REGION", "us-east-1");
    private AmazonS3 amazonS3 = AmazonS3ClientBuilder.standard()
            .withEndpointConfiguration(LOCAL_STACK_CONTAINER.getEndpointConfiguration(LocalStackContainer.Service.S3))
            .withCredentials(LOCAL_STACK_CONTAINER.getDefaultCredentialsProvider()).build();

    private AwsS3Client awsS3Client;

    @BeforeEach
    public void beforeEach() throws IOException, InterruptedException {
        LOCAL_STACK_CONTAINER.execInContainer("awslocal", "s3", "mb", "s3://" + BUCKET_NAME);
        awsS3Client = new AwsS3Client(amazonS3);
    }

    @AfterEach
    public void afterEach() throws IOException, InterruptedException {
        LOCAL_STACK_CONTAINER.execInContainer("awslocal", "s3", "rb", "s3://" + BUCKET_NAME, "--force");
    }

    /**
     * First assert whether localstack is running!
     */
    @Test
    public void test_isLocalstackRunning() {
        Assertions.assertTrue(LOCAL_STACK_CONTAINER.isRunning());
    }

    @Test
    public void test_uploadObjectSuccess() throws URISyntaxException, IOException {

        awsS3Client.uploadObject(BUCKET_NAME, createPath());
        Assertions.assertTrue(amazonS3.doesObjectExist(BUCKET_NAME, "sample.csv"));
    }

    @Test
    public void tes_uploadObjectError() throws IOException, InterruptedException {

        LOCAL_STACK_CONTAINER.execInContainer("awslocal", "s3", "rb", "s3://" + BUCKET_NAME, "--force");
        Assertions.assertThrows(IllegalStateException.class, () -> awsS3Client.uploadObject(BUCKET_NAME, createPath()));
    }

    private Path createPath() throws URISyntaxException {
        return Optional.ofNullable(ClassLoader.getSystemResource("sample.csv").toURI())
                .map(Paths::get)
                .orElseThrow(IllegalArgumentException::new);
    }

}
  • First, I am validating whether the local or CI machine having docker environment.
  • Second, @Container creates the local stack container instance and mimic the AWS S3 as we are providing S3 as a service here. If you are using Junit-4, then you must use @ClassRule annotation here.
  • During beforeEach() method, we are just creating the bucket in the S3 where as afterEach() method deletes the bucket after each test execution.
  • You will need to create sample.csv file under test/resources directory for uploading into local stack S3.
  • During upload test, once the file upload has been done, we can assert that using amazonS3.doesObjectExist method, whether the file has been successfully uploaded or not.
  • And also we are validating after deleting the the bucket, whether it is throwing IllegalStateException exception or not.
  • Finally In this way, we tested our AWS S3 client class effectively without connecting into AWS services.

AWS S3 Client – Local Stack – Test Container – Code Sample

Leave a Reply

%d bloggers like this: