Agile methodologies have database structures evolve on a weekly basis for most web applications. Database migrations make it easy for developers to manage these changes, both for local development but also for databases running in production with real data. The idea is to provide scripts that describe the queries to run on the database to make it reach its wanted new structure.
In the Java, Kotlin and other JVM based ecosystems, Liquibase is a popular solution which allows writing migrations in SQL, but also in XML, YAML or JSON which are then transformed into SQL queries understandable by most database flavors.
However, when wanting to change the database's structures, developers usually find themselves doing in two steps which are redundant:
In this article, we explore how we managed to automate the second and third step to increase our productivity and enforce coherency between the database structure and the ORM entities.
What we want to achieve is to have database migrations be generated by comparing the current state of the database with its wanted state by looking at the entities which are currently present in the code. Many libraries and frameworks include an out of the box system to make it possible. Here is some for them:
What we seek is thus a single one line command, where the developer simply has to provide a description label for the migration which will trigger
The following tutorial makes several assumptions due to stability issues I found using the few libraries required to make it work, namely:
We will assume we want to create a new entity which is to be persisted in the database. Here is the Hibernate entity, created in Kotlin
We thus want to create a new table with two columns in the database
We provided the relevant dependencies to include in the pom.xml. In our case, we are using spring-boot-starter-data-jpa, spring-boot-starter-jdbc and h2 for the ORM and the database connexion, and liquibase-core for handling migrations.
We also use the liquibase-maven-plugin in the pom.xml plugins section to provide the maven goal which will generate the migration file. Three things are notable:
Here is finally how we provide the ability to inject the diffChangeLog property in the pom.xml file
We here describe the directory structure we use for configuring Liquibase and holding the changelogs. Make sure all the db/changelog/changes directory is created before going further.
In the Spring application.properties file, we simply specify we use an H2 in file database and we make sure Hibernate will not use the entities to create or update the database on its own. Indeed, Liquibase will be performing that job.
In the liquibase.properties properties file which we specified in the liquibase maven plugin propertyFile field, we :
Finally, the master Liquibase changelog file should contain a single line and new migration files references will be appended to it with each generation (keep an empty line at the end of the file)
We want to define a single command to:
Makefile is a good tool for such a purpose
It's all ready! You can now test your new system using
make makeMigration MIGRATION_LABEL="user-creation"
We were able to quickly create a simple make command which generates migration on the fly from database entities. Make sure to always check the generated migrations as nothing beats an expert developer eye for:
Always use liquibase for running the generated migration up and down locally, before pushing it to production.
The whole exemple can be found in this Github repository