Automated code maintenance with OpenRewrite

Piet Schijven

Dieser Artikel ist auch auf Deutsch verfügbar

Imagine you work in a modern organisation and you are partly responsible for maintaining a large number of software projects that have been created over the years. This can be a daunting task, especially with the high release frequency of dependency updates and security patches. It usually involves manually updating each project’s build file, testing to make sure everything works, and finally releasing the new version to production.

Large framework upgrades can also be problematic, especially in software projects that are developed by more than one team. These migrations are usually carried out on a separate feature branch, which is then constantly out-of-date due to features being developed simultaneously by other project members. Performing the final merge is therefore error-prone and requires lot’s of coordination within the project.

OpenRewrite is a library that can help you automate most of these tasks. Its main feature is that it can perform automatic source code refactoring by applying “recipes” to your project. These recipes are fully defined as Java code and can be easily integrated into the build process by using the OpenRewrite Maven or Gradle plugin. It can not only refactor Java code, but also modify the Maven pom.xml, property files (.properties or .yml) and more. Because it can be integrated into the build process, there is no need to use feature branches to perform refactorings and framework upgrades - by using a separate CI-pipeline and/or build-profile, refactorings can be performed directly on the master branch.

OpenRewrite provides many available modules for code maintenance and framework upgrades, for example:

• Fixing issues reported by static analysis tools
• Automatically fixing checkstyle violations
• Migrating to Java 11 or Java 17
• Upgrading from JUnit 4 to JUnit 5
• Upgrading Spring Boot
• and many more…

An exhaustive list of all recipes can be found in the recipe catalog. Guides for the most popular recipes can be found here: Popular recipe guides.

How does OpenRewrite work

When applying recipes to a codebase, OpenRewrite constructs a tree representation of the code in question. This tree is essentially an advanced version of an Abstract Syntax Tree (AST). It not only provides the basic amount of information the compiler would need to compile the code, but also has the following structural properties:

• The tree stores information about whitespace before and after the tree-elements. This is used to be able to preserve the original formatting of the code when executing the recipes.
• It contains detailed type information for all elements, even if these types are not defined in the source file itself.
• The elements in the tree can also have markers associated to them. Markers are additional metadata that, for example, hold information about Checkstyle settings, the Java version of the source code or additional styling information. They can also be used to store custom information during a traversal of the tree.
• The tree is fully serializable, even if it contains cyclic elements. This allows the tree to be generated in advance in a JSON format and stored for future processing.

An AST with these additional properties is called a “Lossless Semantic Tree” or LST. To make this definition a little less abstract, consider the following simple example of a “Hello World” class containing a hello() method:

package com.yourorg;

public class HelloWorld {

    public String hello() {
        return "Hello!";
    }
}

When generating the LST for this class, you would obtain the following result:

#root
\---J.ClassDeclaration
    |---J.Modifier | "public"
    |---J.Identifier | "HelloWorld"
    \---J.Block
        \---#JRightPadded
            \---J.MethodDeclaration
                |---J.Modifier | "public"
                |---J.Identifier | "String"
                |---J.Identifier | "hello"
                |---#JContainer
                |   \---#JRightPadded
                |       \---J.Empty
                \---J.Block
                    \---#JRightPadded
                        \---J.Return | "return "Hello!""
                            \---J.Literal

As you can see, all elements in the LST are defined as internal classes (and implementations) of the interface J, which is the tree implementation for Java source files. To work with these LSTs, OpenRewrite uses the visitor pattern (implemented by the TreeVisitor class) to traverse through the tree and applies the required refactoring logic by using the appropriate callback methods for each LST-element. The relevant visitor methods in the general class JavaVisitor for the example above are as follows:

class JavaVisitor<P> extends TreeVisitor<J, P> {

    ...
    public J visitClassDeclaration(J.ClassDeclaration classDecl, P p) {...}
    public J visitIdentifier(J.Identifier ident, P p) {...}
    public J visitBlock(J.Block block, P p) {...}
    public J visitMethodDeclaration(J.MethodDeclaration method, P p) {...}
    public J visitReturn(J.Return retrn, P p) {...}
    ...

}

Inside these methods you can access all the metadata about the respective LST element and, most importantly, also modify them to create transformations of the source code. Note that not all LST elements have a corresponding visitor method. For example, the J.Modifier element can only be accessed from the corresponding parent element (in this case J.ClassDeclaration or J.MethodDeclaration).

By creating an implementation of this class, you can develop your own refactoring recipe. How this is done in practice will be discussed in a future blog post.

Using OpenRewrite in practice

OpenRewrite can be easily integrated into the build process by using the OpenRewrite Maven or Gradle plugin. In the configuration of the plugin, you should specify which recipes should be activated for the current project.

<plugin>
    <groupId>org.openrewrite.maven</groupId>
    <artifactId>rewrite-maven-plugin</artifactId>
    <version>4.41.0</version>
    <configuration>
        <activeRecipes>
            <!-- Refer to the fully qualified name of the recipe or to the name 
                 defined in rewrite.yml -->
            <recipe>...</recipe> 
            ...
        </activeRecipes>
    </configuration>
    <dependencies>
        <!-- Declare dependencies for recipes that are not bundled with 
             OpenRewrite -->
        ...
    </dependencies>
</plugin>

By executing mvn rewrite:run you can run OpenRewrite for your codebase. On completion you will have a set of changed files, which you can then review and commit if you are happy with the result. If you don’t want to actually change your source code, you can also use the mvn rewrite:dryrun command - this will only produce a set of diffs for all the changes.

If the recipe you want to run requires additional configuration parameters, you should define a rewrite.yml file and place it in the root of the project (or in META-INF/rewrite). This file allows you to specify any number of recipes or compositions of recipes that you might want to use. You then refer to the name of any of these recipes when specifying them in the configuration of the Maven/Gradle plugin. For example, suppose that you want to update the Apache POI dependency in your project from version 5.2.2 to 5.2.3. The rewrite.yml for this refactoring should then look like this:

---
type: specs.openrewrite.org/v1beta/recipe
name: com.yourorg.UpgradeDependencies
recipeList:
  - org.openrewrite.maven.UpgradeDependencyVersion:
      groupId: org.apache.poi
      artifactId: poi
      newVersion: 5.2.3

Note that we have named this recipe com.yourorg.UpgradeDependencies. When using it in the Maven plugin, the configuration looks as follows:

<plugin>
    <groupId>org.openrewrite.maven</groupId>
    <artifactId>rewrite-maven-plugin</artifactId>
    <version>4.41.0</version>
    <configuration>
        <activeRecipes>
            <recipe>com.yourorg.UpgradeDependencies</recipe>
        </activeRecipes>
    </configuration>
</plugin>

In the next section we will explore a more advanced example to illustrate the concepts and techniques discussed above.

Upgrading Spring Boot applications with OpenRewrite

Let us now create a small Spring Boot application to try out the Spring migration recipes and those for fixing common static analysis problems. The main changes when upgrading from Spring Boot 2.x to Spring Boot 3.x are the upgrade from Java 8/11 to Java 17 and the move from the javax to the jakarta namespace. Therefore, we want to build a Spring Boot 2.x application on Java 11 with an embedded Tomcat server and also write some badly written code that references classes from the javax namespace.

After building our application by using the Spring Initializr and selecting Java 11 and the spring-boot-starter-web dependency, we get a Maven project with the following pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" 
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
            https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.9</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>pietschijven</groupId>
    <artifactId>openrewrite-spring-demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>openrewrite-spring-demo</name>
    <description>openrewrite-spring-demo</description>
    <properties>
        <java.version>11</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Now let us write a simple Spring MVC controller with a /hello endpoint. The controller method should use the javax.servlet.ServletRequest to return a hello message containing the current request URL. We also deliberately write the code in a bad way to see how OpenRewrite will correct these coding errors. The controller should then look like this:

package pietschijven.openrewritespringdemo.web;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import javax.servlet.ServletRequest;
import javax.servlet.http.HttpServletRequest;

@RestController
public class HelloController {

    private final static String base_message = "Hello request";

    @GetMapping("/hello")
    public String hello(ServletRequest servletRequest){
        String semicolon = String.valueOf(": ");;
        if ((servletRequest != null) == true) {
            return getString((HttpServletRequest) servletRequest, semicolon, 
                base_message);
        }
        return "no request";
    }

    private static final String getString(HttpServletRequest servletRequest, 
            String semicolon, String Message) {

        return Message + semicolon + servletRequest.getRequestURI();
    }

}

In order to use OpenRewrite to fix our coding errors and to perform the Spring Boot upgrade, we need to configure it to use both the CommonStaticAnalysis and the UpgradeSpringBoot_3_0 recipes. Both are recipes made up of many smaller refactoring recipes. A list of all the recipes included in CommonStaticAnalysis can be found here. For the UpgradeSpringBoot_3_0 recipe, the list can be found here.

<plugin>
    <groupId>org.openrewrite.maven</groupId>
    <artifactId>rewrite-maven-plugin</artifactId>
    <version>4.41.0</version>
    <configuration>
        <activeRecipes>
            <activeRecipe>
                org.openrewrite.java.cleanup.CommonStaticAnalysis
            </activeRecipe>
            <activeRecipe>
                org.openrewrite.java.spring.boot3.UpgradeSpringBoot_3_0
            </activeRecipe>
        </activeRecipes>
    </configuration>
    <dependencies>
        <dependency>
            <groupId>org.openrewrite.recipe</groupId>
            <artifactId>rewrite-spring</artifactId>
            <version>4.33.2</version>
        </dependency>
    </dependencies>
</plugin>

When using the dry-run command (mvn rewrite:dryRun), OpenRewrite will generate the patch file target/rewrite/rewrite.patch which we can use to review the changes. For the Maven pom.xml it will generate the following unified diff:

diff --git a/pom.xml b/pom.xml
index 8ded2e4..f89560b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -5,7 +5,7 @@ org.openrewrite.java.migrate.javax.AddJaxbRuntime, /
    org.openrewrite.Recipe$AdHocRecipe, /
    org.openrewrite.maven.UpgradeDependencyVersion, /
    org.openrewrite.maven.AddDependency, /
    org.openrewrite.maven.UpgradeParentVersion, /
    org.openrewrite.java.migrate.UpgradeJavaVersion /

     <parent>
         <groupId>org.springframework.boot</groupId>
         <artifactId>spring-boot-starter-parent</artifactId>
-        <version>2.7.9</version>
+        <version>3.0.4</version>
         <relativePath/> <!-- lookup parent from repository -->
     </parent>
     <groupId>pietschijven</groupId>
@@ -14,14 +14,23 @@
     <name>openrewrite-spring-demo</name>
     <description>openrewrite-spring-demo</description>
     <properties>
-        <java.version>11</java.version>
+        <java.version>17</java.version>
     </properties>
     <dependencies>
         <dependency>
+            <groupId>jakarta.servlet</groupId>
+            <artifactId>jakarta.servlet-api</artifactId>
+        </dependency>
+        <dependency>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-web</artifactId>
         </dependency>
         <dependency>
+            <groupId>org.glassfish.jaxb</groupId>
+            <artifactId>jaxb-runtime</artifactId>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
             <groupId>org.springframework.boot</groupId>
             <artifactId>spring-boot-starter-test</artifactId>
             <scope>test</scope>

As you can see, OpenRewrite bumped up the Java version to 17 and changed the Spring Boot version to 3.0.4. In addition, as we expected, it has introduced the jakarta.servlet-api so that we can use the new jakarta namespace in our HelloController. In the diff file itself, OpenRewrite also creates comments for each recipe that was used to modify the file.

The diff generated by OpenRewrite for the HelloController looks as follows:

...
@@ -3,25 +3,25 @@ org.openrewrite.Recipe$AdHocRecipe, 
    org.openrewrite.java.cleanup.RenamePrivateFieldsToCamelCase, 
    org.openrewrite.java.cleanup.RenameLocalVariablesToCamelCase, 
    org.openrewrite.java.cleanup.SimplifyBooleanExpression, 
    org.openrewrite.java.cleanup.NoValueOfOnStringType, 
    org.openrewrite.java.ChangePackage, 
    org.openrewrite.java.cleanup.RemoveExtraSemicolons, 
    org.openrewrite.java.migrate.UpgradeJavaVersion, 
    org.openrewrite.java.cleanup.StaticMethodNotFinal

 import org.springframework.web.bind.annotation.GetMapping;
 import org.springframework.web.bind.annotation.RestController;

-import javax.servlet.ServletRequest;
-import javax.servlet.http.HttpServletRequest;
+import jakarta.servlet.ServletRequest;
+import jakarta.servlet.http.HttpServletRequest;

 @RestController
 public class HelloController {

-    private final String base_message = "Hello request";
+    private final String baseMessage = "Hello request";

     @GetMapping("/hello")
     public String hello(ServletRequest servletRequest){
-        String semicolon = String.valueOf(": ");;
-        if ((servletRequest != null) == true) {
-            return getString((HttpServletRequest) servletRequest, semicolon, 
-                base_message);
+        String semicolon = ": ";
+        if ((servletRequest != null)) {
+            return getString((HttpServletRequest) servletRequest, semicolon, 
+               baseMessage);
         }
         return "no request";
     }

-    private static final String getString(HttpServletRequest servletRequest, 
-       String semicolon, String Message) {
-        return Message + semicolon + servletRequest
+    private static String getString(HttpServletRequest servletRequest, 
+       String semicolon, String message) {
+        return message + semicolon + servletRequest
                 .getRequestURI();
     }

The following changes were made by OpenRewrite to the source file:

• ChangePackage changed the import statements for the javax dependencies to their jakarta counterparts.
• RenamePrivateFieldsToCamelCase renamed the field base_message to baseMessage since a common convention in Java code is to write all variables in camel case. It also renamed all references to this field in the class.
• RemoveExtraSemicolons removed the unnecessary semicolon after the declaration of the variable semicolon.
• NoValueOfOnStringType removed the unneeded String.valueOf(...) in the variable declaration of semicolon.
• SimplifyBooleanExpression simplified the if-statement.
• StaticMethodNotfinal removed the unneeded final modifier in the declaration of the static method getString.
• RenameLocalVariablesToCamelCase furthermore changed the name of the method parameter Message to message.

So we see that OpenRewrite solved a lot of coding issues in our codebase and made the upgrade to Spring Boot 3.x a breeze!

Final remarks

OpenRewrite is a very powerful tool that can really help you to systematically maintain many of your organisation’s software projects. It provides a large number of refactoring recipes to automate most maintenance tasks, such as updating dependencies and fixing problems reported by static code analysis tools. However, since OpenRewrite is still under active development, the recipes may not be as stable as you had hoped. My advice is to always try to use the latest versions of the recipes and OpenRewrite itself whenever possible!

As an alternative to manually creating rewrite.yml files for each project, you could consider using the SaaS solution provided by the creators of OpenWrite. This allows you to connect all your Github/Gitlab repositories to your account on the platform and perform mass refactorings on all your projects, (pre-)view the results and commit the code changes back to the repositories. For a curated set of open source repositories available on Github (e.g. Spring Boot), they provide free access on the platform.

In a future blog post, I will show you how to develop your own OpenRewrite recipes. This will also give you a much better understanding of how OpenRewrite works internally, and help you understand existing recipes when they don’t behave as you expect.

The sample Spring Boot project can be found here on my Github account.