Getting off the Mule and start shipping containers

When we started building media systems integrations many years ago Enterprise Service Buses (ESBs) were all the rage and a good alternative to old point-to-point integration methods.

Things change though and on one of our projects it was time to get off that trusty old steed: Mule ESB.

WHY MIGRATE TO MICROSERVICES?

Improved testability

ESB code can be very hard to unit test fully, especially if it's in a proprietary language. We wanted to increase code quality and increasing the unit test coverage was one of the ways.

Control over technology choices

ESBs want to you to use its specific libraries, but we increasingly found that the ESB provided libraries didn't have the functionality we needed or contained bugs that were rare but affected our code.

Deployment alternatives

ESB locks you into a single deployment alternative and any upgrade to the ESB itself affects ALL applications. The ESB had become a single-point-of-failure.

Improved development environment

We’d had enough of proprietary languages and development environments that couldn’t keep up with the rest of the development community.

THE RESULT (1 MONTH & 6 APPS LATER)

All in all it took about a month to migrate 6 integration apps, including continuous code review and system integration testing.

Surprisingly, we found that the standalone Java applications had both lower CPU and memory usage. After running the old and new implementations in parallel for a few days the total memory footprint for the standalone Java applications was stable around 500 to 530 MB, while the ESB memory usage varied from 280 MB to 1.8GB!

Otherwise, the biggest advantage we found when moving away from an ESB was control over dependencies, thread management, and exception handling. With this increased control we could then change to a microservices architecture where each application could be packaged up and deployed in the cloud on AWS EC2 Container Service.

We plan to follow up in a later blog post with more details on how we use Packer and Terraform to create AMIs deployed on AWS EC2 Container Service.

WHAT NEEDED REPLACING (TECHNOLOGY DECISIONS)

Dependency injection

For dependency injection, Guice was chosen to completely remove the ability to configure DI externally from the code. It is also our subjective opinion that Guice has a better abstraction model than Spring.

Configuration management

Archaius v2 was chosen for configuration management, because of its simplicity of usage and the ability to dynamically load configuration from other sources in the background, for example from Consul.

REST Libraries

Embedded Jetty was chosen for the server because it is the easiest HTTP server implementation to work with, and because we want the all applications to be deployed as a single package per application, without external dependencies on an HTTP server. Resteasy was selected for JAX-RS because it requires far fewer lines of code to get up and running, and Feign was selected for the client because of its simplicity and small footprint.

JMS

The JMS communication we chose to write ourselves, simply because there are no maintained JMS client libraries available anymore. We also wanted to build a system where we can easily swap to using AWS SQS, or another alternative messaging solution, at a later stage.

Scheduled tasks

Guava was already a part of the libraries we use in the project, so that was an easy choice for service management. Guava already has support for recurring tasks, though only rate based, not cron expression based.


Image courtesy of Curtis Perry