How Do We Deploy Temporary Acceptance Systems on AWS ?
We want to deliver our products as temporary acceptance systems to be tested by our developers, testers, and pre-sales engineers internally before their release. How do we have setup such system using CodeBuild, CloudFormation and ECS ?
Introduction : How GB&Smith delivers a global solution.
Our company, GBANDSMITH edits multiple software solutions (the 360Suite portfolio) to help our customers use efficiently SAP BusinessObjects (BOBJ in this article). But as our testing processing are improving, we are increasingly struggling with the installation and deployment of multiple products on test environments. It takes time and generates frustration for developers and pre-sales engineers.
We now have dedicated resources for testing itself, and we don’t want them to struggle with the installation/provisioning phases but to focus on testing.
The GB&SMITH Labs team has been asked to find a solution on this issue!
We have worked to deliver a global solution that answers the following needs:
- developers and pre-sales engineers can create themselves a solution
- the solution will contain our 2 main products (360WebPlatform — named 360wp in schema, and 360Eyes), and optionally a BOBJ engine
- it must be possible to select any git branch (work in progress) or released version (available as git tag) of the 2 products
- the solution must be easily available, like an accessible URL endpoint
- multiple solutions can be created at the same time and used by multiple persons.
The design and AWS.
From the beginning, we wanted to work on this project only with resources available on our main cloud environment, Amazon Web Services. The goal was to prove that the team is able to deliver a real cloud solution, and to trial-and-error on some new tools available on AWS. So the global design of the solution is intended to use instances only if needed.
We are using the following products of Amazon Web Service:
- CodeBuild is the Continous Integration / Continous Deployment tooling that triggers the solution creation. It is used directly by our teams.
- The solution is mainly deployed on Elastic Container Service (ECS) that is a container orchestration solution. Our products are deployed as docker containers.
- We rely on Elastic Container Repository (ECR) to store our private docker images.
- We rely on Amazon Relational Database Service (RDS) to host our PostgreSQL database instance, used by both 360Eyes and 360WebPlatform. The EC2 BOBJ instance is also able to use it.
- We have a single EC2 (Elastic Cloud Compute) instance to host our BOBJ system.
- The creation of the system (ECS + EC2 + RDS) is orchestrated by a CloudFormation script, that helps us bring up and down the full stack.
The Stack
Let’s dig a bit on our choice. Choosing the right tools is not obvious, so let me take a few minutes to explain why we have created such a stack.
Why use ECS ?
ECS is one of the solutions in AWS if you want to deploy docker containers. We think that it is easier to deal with docker containers and images to deploy our products. It is easier and faster than working with instance images:
- Build the code using our classic Maven tooling (
maven clean install) - Create a docker image (
docker build .) - Tag and push the image to ECR (
docker tag ...and thendocker push ...) - Run the ECS task based on ECR
We are dealing here with couple of Mb (around 500 Mb for a docker image) instead of multiple Gb for a instance image. There is no easy way to generate a instance image containing our product from a CI solution like CodeBuild (the best I know in this area is the very good Packer tool from HashiCorp).
Why use an EC2 instance ?
In an ideal scenario, I would avoid an EC2 instance. It relies on an EC2 image (Amazon Machine Image), and it is bulky to modify. But here we want to deliver a BI solution, and officially, this solution is not compatible with Docker containers. I know some users are trying to dockerize BOBJ (here for example), but we haven’t succeed in this task.
Why use RDS ?
RDS is the de-facto standard solution to deploy a database in AWS. Both our products relies on a database, so we need to ship one. We chose RDS.
Actually it is not the only solution, we can also imagine adding a database container in our ECS task, but this solution complexifies the connectivity between the BOBJ instance and the database.
Why use CloudFormation ?
CloudFormation is the Infrastructure as Code solution in AWS. There are also other tools available like Terraform from HashiCorp (again!).
We chose to use CloudFormation in this project because it allows us to create a full stack and also to tear-down the stack very easily from the AWS API when the user doesn’t need the solution anymore (it is a bit more difficult to teardown a terraform setup, not a single AWS API call !)
Why use CodeBuild ?
Our company mainly uses Jenkins as CI solution. Starting on CodeBuild is not obvious ! The goal of the project was to use a completly AWS solution without any external product.
CodeBuild is quite a mature product now, which allows to setup a project with conditional builds, and to consume multiple git repositories. It also allows to push docker images to ECR, and to trigger the CloudFormation stack creation. Here is an exemple of the buildspec provided in the project:
version: 0.2cache: paths: - '/root/.m2/**/*'phases: install: runtime-versions: java: openjdk8 pre_build: commands: - echo Logging in to Amazon ECR... - $(aws ecr get-login --no-include-email --region $AWS_REGION) build: commands: - | if test "$BUILD_EYES" = "1"; then echo "Build 360eyes docker image and push it to ECR"; cd $CODEBUILD_SRC_DIR/360ace/eyes; mvn clean install -Pdocker; docker tag 360eyes/360eyes.rest.service:latest xxx.dkr.ecr.$AWS_REGION.amazonaws.com/360ace/eyes:latest; docker push xxx.dkr.ecr.$AWS_REGION.amazonaws.com/360ace/eyes:latest; fi - echo Launching CloudFormation template - cd $CODEBUILD_SRC_DIR/360ace - aws cloudformation create-stack --template-body file://360ace.yaml --stack-name ace-$(date +%d%H%M)
This is just does the following:
- build the product and docker image (using this maven plugin)
- tag and push on ECR the docker image
- start the CloudFormation stack
A good point is that it allows to checkout a specific branch (like a developer working branch) or a specific tag (a release) of our product repositories.
Managing resources on AWS
Here we see that we have some temporary resources created when a solution is up, and some more stable resources that continue to live even when no solution is running:
- Temporary resources: EC2 instance, ECS service, RDS database
- Stable resources: ECR repositories, CodeBuild project
The temporary resources are directly managed by the CloudFormation stack: when the stack is created it fires up all the linked resources. When the stack is stopped, then the linked resources are deleted.
The stable resources are designed to live all the time. But instead of creating them directly in the AWS Console, we prefer to use Terraform (again again !) to manage that. It allows us to have a clean setup that we can reproduce on multiple AWS accounts. We are also able to review the infrastructure changes within a git repository and to take full advantage of Infrastructure as Code.
End result
As a result now our employees can start a temporary solution of our solutions just by triggering a CI job:
And all is presented in a quick and dirty React application (the home container):
Next steps
OK, the solution is here and available but there is still a lot of room for improvements.
How to trigger the solution creation ?
Because we are only on AWS, we imagined some additional improvements to our system: it is possible to setup a chatbot that is able to create and tear down a system just by speaking to it. It communicates only with the AWS API (CodeBuild and CloudFormation):
How to speed up the stack creation ?
Currently we are relying on a RDS snapshot to pop-up some existing data in the RDS database. It is easy to setup but it is dramatically long to fire, around 20 minutes…We are thinking on how we can speed up this (maybe by using a different system).
Add more flexibility
We can also imagine adding an option to select a specific BOBJ version to start (BI4.1, BI4.2, …), or select the RDS backend type (MySQL, PostgresSQL, Oracle…). All of that is possible and we are waiting for our internal customers to get some usage feedback !
To Conclude
Cloud is hard. We have just scratched the surface of the issues and the options offered by a cloud platform like AWS. We still have lots of methods, skills, and tools to learn. But we are very happy to be able to deliver a working tool to our co-workers, waiting for the next project !
