Tech Blog from a Happy Coder: October 2018

Apache Maven is a software project management and comprehension tool. It helps the developers to manage java projects in a controlled manner. For the open source developers, maven central repository provides the free hosting service for sharing their open source libraries to the global users.

This step-by-step guide shows the whole process of publishing an open source project to maven central repository with the helps of Maven, Git, GitHub, GPG and OSSRH:

Maven, used for project management and build automation
Git, used for source code version control
GitHub, used for source code repository
GPG, used to sign the built artifact
OSSRH, used as artifact repository

The Publish Process

1. Prerequisite Steps

In this step, we need to configure and prepare various of tools and services required by the project publishing process.

Step 1.1 - install & configure various tools needed in your local system:

Step 1.2 - setup GitHub

Step 1.3 - setup GPG key pair (see my GPG survival guide)

verify GPG installation

$ gpg --version

generate your key pair, and record your passphrase for your GPG key pair

$ gpg --full-generate-key

distribute your key to GPG servers (it may take hours to propagate your key to the whole key server network)

$ gpg --keyserver hkp://pool.sks-keyservers.net --send-keys <key id>

verify your key from GPG servers

$ gpg --keyserver hkp://pool.sks-keyservers.net --receive-keys <key id>

Step 1.4 - setup OSSRH, i.e. Open Source Software Repository Hosting, which is an approved repository provided by Sonatype for any OSS Project that want to get their artifacts into Central Repository.

sign up an Sonatype Jira account for hosting service
create a new project hosting ticket using your Jira account (see my request ticket as a sample)

Step 1.5 (optional) - setup CI system with Travis CI

sign up Travis with GitHub
accept the authorization of Travis, you’ll be redirected to GitHub
click the green Activate button, and select the repositories you want to use with Travis
add a configure file called .travis.yml under your project root directory to tell Travis what to do with your project, see my sample file
add the file .travis.yml to git, commit and push, to trigger a Travis CI build
check the build status page to see if your build passes or fails

2. Maven Configuration Steps

In this step, we need to configure Maven so that the publishing process can be automatically performed with the Maven commands. It would be a simple task to release a new version later on, and can even be delegated to a CD (Continue Deployment) system.

Step 2.1 - configure pom.xml (see my sample pom file)

locate it under project root directory
add the project information

compulsory information required by OSSRH:

correct coordinates

project name, description and url

license information

developer information

SCM Information - connection details the read only connection, while developerConnection details read and write access connection details. The url contains the URL for a web front end to your SCM system

optional information if possible:

organization information

issue tracking information (detailed by issueManagement)

continuous integration system information (detailed by ciManagement)

sample code segment:

<project ...>
  ...
  
  <name>archetype-modern-starter</name>
  <description>a modern Java standalone application template</description>
  <url>https://github.com/coder168/archetype-modern-starter</url>

  <licenses>
    <license>
      <name>The Apache Software License, Version 2.0</name>
      <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
    </license>
  </licenses>

  <organization>
    <name>Coder168</name>
    <url>https://github.com/coder168</url>
  </organization>

  <developers>
    <developer>
      <name>Ming Li</name>
      <email>coder168@users.noreply.github.com</email>
      <organization>Coder168</organization>
      <organizationUrl>https://github.com/coder168</organizationUrl>
    </developer>
  </developers>

  <scm>
    <connection>scm:git:git://github.com/coder168/archetype-modern-starter.git</connection>
    <developerConnection>scm:git:ssh://github.com:coder168/archetype-modern-starter.git</developerConnection>
    <url>https://github.com/coder168/archetype-modern-starter</url>
    <tag>HEAD</tag>
  </scm>

  <issueManagement>
    <system>Github</system>
    <url>https://github.com/coder168/archetype-modern-starter/issues</url>
  </issueManagement>

  <ciManagement>
    <system>Travis</system>
    <url>https://travis-ci.org/coder168/archetype-modern-starter</url>
  </ciManagement>

  ...
</project>

add the plugin section for publishing to OSSRH

defined within a dedicated profile (ossrh-release) for clarity purpose
distributionManagement specifies the details of OSSRH repositories
properties specifies the plugins version details, easy for upgrade later on
maven-source-plugin creates a source jar for OSSRH requirement
maven-javadoc-plugin creates a javadoc jar for OSSRH requirement
maven-jar-plugin creates a fake javadoc jar contain README.md for OSSRH requirement. This is a workaround when maven javadoc plugin fails to generate javadoc jar, such as maven archetype project can not generate java doc. This one is not needed if maven javadoc plugin works properly
maven-gpg-plugin sign the artifacts for OSSRH requirement
nexus-staging-maven-plugin provides support to access staging functionality to remote Nexus server OSSRH. Note: in the following sample, the plugin is configured to immediately perform a release of staging repository (or repositories) after a successful close
maven-release-plugin provides support to create a release and to update the project's SCM accordingly. Note: in the following sample, git is configured to make local checkout and will not push changes to remote repository automatically during release deployment
sample code segment:

<project ...>
  ...

  <profiles>
    <profile>
      <id>ossrh-release</id>
          
      <distributionManagement>
        <snapshotRepository>
          <id>ossrh</id>
          <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        </snapshotRepository>
        <repository>
          <id>ossrh</id>
          <url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
        </repository>
      </distributionManagement>
          
      <properties>
        <source-plugin.version>3.0.1</source-plugin.version>
        <javadoc-plugin.version>3.0.1</javadoc-plugin.version>
        <jar-plugin.version>3.1.0</jar-plugin.version>
        <gpg-plugin.version>1.6</gpg-plugin.version>
        <nexus-staging-maven-plugin.version>1.6.8</nexus-staging-maven-plugin.version>
        <release-plugin.version>2.5.3</release-plugin.version>
      </properties>
          
      <build>
        <plugins>
          <!-- create a source jar for OSSRH requirement. -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-source-plugin</artifactId>
            <version>${source-plugin.version}</version>
            <!-- stop to add two maven files: pom.xml & pom.properties. 
            <configuration>
              <archive>
                <addMavenDescriptor>false</addMavenDescriptor>
              </archive>
            </configuration>
            -->
            <executions>
              <execution>
                <id>attach-sources</id>
                <goals>
                  <goal>jar-no-fork</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
          <!-- create a javadoc jar for OSSRH requirement. -->
          <!-- javadoc plugin does not work with maven archetype project. 
          <plugin>
              <groupId>org.apache.maven.plugins</groupId>
              <artifactId>maven-javadoc-plugin</artifactId>
              <version>${javadoc-plugin.version}</version>
              <configuration>
                <encoding>UTF-8</encoding>
                <archive>
                  <addMavenDescriptor>false</addMavenDescriptor>
                </archive>
              </configuration>
              <executions>
                  <execution>
                      <id>attach-javadocs</id>
                      <goals>
                          <goal>jar</goal>
                      </goals>
                  </execution>
              </executions>
          </plugin>
          -->
          <!-- create a fake javadoc jar contain README.md for OSSRH requirement. -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <version>${jar-plugin.version}</version>
            <executions>
              <execution>
                <id>fake-javadoc-jar</id>
                <phase>package</phase>
                <goals>
                  <goal>jar</goal>
                </goals>
                <configuration>
                  <classifier>javadoc</classifier>
                  <classesDirectory>${basedir}</classesDirectory>
                  <includes>
                    <include>README.md</include>
                  </includes>
                  <archive>
                    <addMavenDescriptor>false</addMavenDescriptor>
                  </archive>
                </configuration>
              </execution>
            </executions>
          </plugin>
          <!-- sign the components for OSSRH requirement. -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-gpg-plugin</artifactId>
            <version>${gpg-plugin.version}</version>
            <executions>
              <execution>
                <id>sign-artifacts</id>
                <phase>verify</phase>
                <goals>
                  <goal>sign</goal>
                </goals>
              </execution>
            </executions>
          </plugin>
          <!-- provide support to access staging functionality to remote Nexus server OSSRH. -->
          <plugin>
            <groupId>org.sonatype.plugins</groupId>
            <artifactId>nexus-staging-maven-plugin</artifactId>
            <version>${nexus-staging-maven-plugin.version}</version>
            <extensions>true</extensions>
            <configuration>
              <serverId>ossrh</serverId>
              <nexusUrl>https://oss.sonatype.org/</nexusUrl>
              <autoReleaseAfterClose>true</autoReleaseAfterClose>
            </configuration>
          </plugin>
          <!-- provide support to release deployment in two steps: prepare and perform. -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-release-plugin</artifactId>
            <version>${release-plugin.version}</version>
            <configuration>
              <autoVersionSubmodules>true</autoVersionSubmodules>
              <useReleaseProfile>false</useReleaseProfile>
              <!-- <releaseProfiles>ossrh-release</releaseProfiles>-->
              <goals>deploy</goals>
              <!-- git use a local checkout instead of checkout from the upstream repo -->
              <localCheckout>true</localCheckout>
              <!-- git will not push changes to the upstream repo -->
              <pushChanges>false</pushChanges>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>

  ...
</project>

Step 2.2 - configure settings.xml

locate it at ${user.home}/.m2/settings.xml, create it over there if unavailable
configure the username and password for accessing OSSRH using Sonatype Jira account's credential
(Optional step) configure the passphrase for GPG key pair. Alternative, the passphrase will be prompted for during the verify phrase of maven life cycle if it is not provided by settings.xml
sample code of settings.xml:

<settings>
  <servers>
    <!-- OSSRH credential -->
    <server>
      <id>ossrh</id>
      <username>sonatype jira username</username>
      <password>sonatype jira password</password>
    </server>
    <!-- GPG passphrase -->
    <server>
      <id>gpg.passphrase</id>
      <passphrase>clear or encrypted text</passphrase>
    </server>
  </servers>
</settings>

3. Publishing Steps

In this step, we release our project and publish the released artifact to OSSRH staging server.

Step 3.1 - prepare the release (see the official manual)

under maven project root directory:

$ mvn clean

$ mvn release:prepare -P ossrh-release

at this stage, what release:prepare goal do:

verify that there are no uncommitted changes in the workspace
prompt the user for the desired tag, release and development version names
modify and commit release information into the pom.xml file
tag the entire project source tree with the new tag name

if the prepare stage has done successfully, there is a file called release.properties to be created within the project root directory
if an error occurs, or the process is cancelled, then you can do either one of the follows:

run this command again, it will pick up from where the last one left off:

$ mvn release:prepare -P ossrh-release

start again from scratch:

$ mvn release:clean release:prepare -P ossrh-release

(Optional step) Dry Run - since the release prepare step performs a number of operations that change the project, it may be wise to do a dry run before a big release or on a new project:

to do this, commit all of your files as if you were about to run a full release and run the following command, which will ask all the same questions, run the same tests, and output a copy of how the POMs will look after transformation. You can check the output and review the POMs

$ mvn release:prepare -P ossrh-release -DdryRun=true

after review the POMs, then you will remove all of the files created above, and the project will be ready to execute the proper release:

$ mvn release:clean -P ossrh-release

Step 3.2 - perform the release (see the official manual)

this step is where publishing to maven central repository really happened!

under maven project root directory:

$ mvn release:perform -P ossrh-release

in this stage, what release:perform goal do:

extract file revisions versioned under the new tag name
execute the maven build life-cycle on the extracted instance of the project
deploy the versioned artifacts to appropriate local and remote repositories

according to our POM settings, the released artifacts are deployed to OSSRH staging server and perform a release of staging repository (or repositories) after a successful close, in other word, the released artifacts will sync to maven central repository automatically in a few hours
if an error occurs, or the process failed, figure out what cause the failure and fix the issues, and then run this command again if the file release.properties is still present from a previous release preparation:

$ mvn release:perform -P ossrh-release

after the release is done successfully, the release.properties and other release files will be removed from the checkout

4. Checking & Troubleshooting Steps

In this step, we verify the published artifacts and finish up the whole process.

Step 4.1 - verify the published artifacts

search Sonatype OSSRH staging repository for your released artifacts. It should be available immediately after an successful release of Step 3.2
search Maven Central Repository for your released artifacts. It may take up to 2 hours to show up your artifacts
if what your open source project provided is a library, a sample application can be created to use your released artifacts as a dependency. My open source archetype archetype-modern-starter can easily create a modern maven java application for testing your library:

create a sample maven project with archetype-modern-starter (see my quick guide):

mvn archetype:generate -Dfilter=archetype-modern-starter

add your library as a dependency into its pom.xml under the root directory of the created project
delete the local copy of your library artifacts from your local repository at ${user.home}/.m2/repository/, since Step 3.2 did deploy to both local and remote repositories
build and package the sample project to see if your library can be used without any issue under the real scenario, under its root directory:

$ mvn clean package

if what your open source project provided is a maven archetype, you also need to check if maven central repository's archetype catalog has been updated with your new released archetype, otherwise your archetype can not be found by archetype:generate goal

the archetype catalog of maven central repository is rebuilt once per week (Sunday night IIRC). You may search your archetype artifact directly with the remote catalog url
to check if your archetype released can be found by archetype:generate goal or not:

mvn archetype:generate -DarchetypeCatalog=remote -Dfilter=<your archetype artifactId>

Step 4.2 - (Optional step) troubleshooting if something are wrong:

roll back the local source changes made during the release if needed (by Git):

undo the release (you may have to do it a second time, depending upon when the error occurred):

git reset --hard HEAD~1

delete the tag of release:

git tag -d <tag name>

ask for Sonatype support by creating the Jira issue ticket with your Jira account:

OSSRH Issue Tracker for deployment related problems or request additional users to have deployment access to your project
MVNCENTRAL Issue Tracker for problems with components in maven central repository

Step 4.3 - finish the whole publishing process if everything is fine:

update your Sonatype Jira ticket to inform your release has been done
push the local source changes (tags & commits) made during the release to remote source repository, i.e., GitHub:

git push --tags

git push origin master

At this point, your open source project has been published to Maven Central Repository successfully. Next, you may consider to promote your great releases to the whole world, e.g., post it to your favorite social networks and etc.

Tech Blog from a Happy Coder

Saturday, 6 October 2018

How to Publish Your Open Source Library to Maven Central Repository

The Publish Process

1. Prerequisite Steps

2. Maven Configuration Steps

3. Publishing Steps

4. Checking & Troubleshooting Steps

References

Tools

Guides

Friday, 5 October 2018

Archetype-modern-starter Quick Guide

What is archetype-modern-starter?

Why use archetype-modern-starter?

How to use archetype-modern-starter?

Generate a modern java project

Work with the generated project

How to work with its latest version?

Reference