The Ceylon Gradle Plugin

Post date: Feb 08, 2016 5:20:12 AM

Ceylon is a well designed programming language created from scratch to be backend agnostic. This means that it can run as well on JavaScript engines (node.js and browsers) as on the JVM, currently its main backend.

    Efforts to get Ceylon running on the Dart VM are well under way.

However, my main interest in using Ceylon has been to run code on the JVM, and I noticed that to use Java libraries in Ceylon was not as straightforward as in Java itself using Maven or Gradle.

The problem is that, even though Ceylon has its own module dependency resolution engine which works great for Ceylon modules sourced from Herd or local repositories, and which theoretically supports also Maven dependencies, I found that in practice several different problems got on my way.

For example, one of the first libraries I wanted to use was log4j, given that advanced logging capabilities are certainly a requirement of any application, and that no equivalent library existed for Ceylon at the time (see ceylon.logging for a possible Ceylon alternative).

But unfortunately, after I created my module file which looked like the one shown below, I discovered that Ceylon requires me to actually declare not only direct Maven dependencies (as opposed to Ceylon dependencies), but also any transitive dependencies the library may have!

module.ceylon

native("jvm")

module com.athaydes.maven "1.0.0" {

  import java.base "8";

  import "org.apache.logging.log4j:log4j-core" "2.4.1";

}

The error didn't really help much:

source/com/athaydes/maven/run.ceylon:1: error: package not found in imported modules: 'org.apache.logging.log4j'

(add module import to module descriptor of 'com.athaydes.maven')

import org.apache.logging.log4j {

      ^

...

The problem is that the org.apache.logging.log4j package is located inside the log4j-api module, not log4j-core (log4j-api is a direct dependency of log4j-core). Of course, in this case the solution was simple: just add a direct dependency on log4j-api:

native("jvm")

module com.athaydes.maven "1.0.0" {

  import java.base "8";

  import "org.apache.logging.log4j:log4j-core" "2.4.1";

  import "org.apache.logging.log4j:log4j-api" "2.4.1";

}

Now, imagine that you wanted to try something more advanced and write a Spring Boot application in Ceylon. The number of transitive dependencies you'd have to declare (after hours trying to understand what's actually required and what's optional from several levels of pom declarations, or just going the trial-and-error approach) is quite large. In fact, here's the dependency tree starting from spring-boot-starter-web:

Obtained with the Ceylon Gradle plugin, via gradle dependencies

ceylonRuntime

\--- org.springframework.boot:spring-boot-starter-web:1.3.0.RELEASE

+--- org.springframework.boot:spring-boot-starter:1.3.0.RELEASE

|    +--- org.springframework.boot:spring-boot:1.3.0.RELEASE

|    |    +--- org.springframework:spring-core:4.2.3.RELEASE

|    |    |    \--- commons-logging:commons-logging:1.2

|    |    \--- org.springframework:spring-context:4.2.3.RELEASE

|    |         +--- org.springframework:spring-aop:4.2.3.RELEASE

|    |         |    +--- aopalliance:aopalliance:1.0

|    |         |    +--- org.springframework:spring-beans:4.2.3.RELEASE

|    |         |    |    \--- org.springframework:spring-core:4.2.3.RELEAS (*)

|    |         |    \--- org.springframework:spring-core:4.2.3.RELEASE (*)

|    |         +--- org.springframework:spring-beans:4.2.3.RELEASE (*)

|    |         +--- org.springframework:spring-core:4.2.3.RELEASE (*)

|    |         \--- org.springframework:spring-expression:4.2.3.RELEASE

|    |              \--- org.springframework:spring-core:4.2.3.RELEASE (*)

|    +--- org.springframework.boot:spring-boot-autoconfigure:1.3.0.RELEASE

|    |    \--- org.springframework.boot:spring-boot:1.3.0.RELEASE (*)

|    +--- org.springframework.boot:spring-boot-starter-logging:1.3.0.RELEASE

|    |    +--- ch.qos.logback:logback-classic:1.1.3

|    |    |    +--- ch.qos.logback:logback-core:1.1.3

|    |    |    \--- org.slf4j:slf4j-api:1.7.7 -> 1.7.13

|    |    +--- org.slf4j:jcl-over-slf4j:1.7.13

|    |    |    \--- org.slf4j:slf4j-api:1.7.13

|    |    +--- org.slf4j:jul-to-slf4j:1.7.13

|    |    |    \--- org.slf4j:slf4j-api:1.7.13

|    |    \--- org.slf4j:log4j-over-slf4j:1.7.13

|    |         \--- org.slf4j:slf4j-api:1.7.13

|    +--- org.springframework:spring-core:4.2.3.RELEASE (*)

|    \--- org.yaml:snakeyaml:1.16

+--- org.springframework.boot:spring-boot-starter-tomcat:1.3.0.RELEASE

|    +--- org.apache.tomcat.embed:tomcat-embed-core:8.0.28

|    +--- org.apache.tomcat.embed:tomcat-embed-el:8.0.28

|    +--- org.apache.tomcat.embed:tomcat-embed-logging-juli:8.0.28

|    \--- org.apache.tomcat.embed:tomcat-embed-websocket:8.0.28

|         \--- org.apache.tomcat.embed:tomcat-embed-core:8.0.28

+--- org.springframework.boot:spring-boot-starter-validation:1.3.0.RELEASE

|    +--- org.springframework.boot:spring-boot-starter:1.3.0.RELEASE (*)

|    +--- org.apache.tomcat.embed:tomcat-embed-el:8.0.28

|    \--- org.hibernate:hibernate-validator:5.2.2.Final

|         +--- javax.validation:validation-api:1.1.0.Final

|         +--- org.jboss.logging:jboss-logging:3.2.1.Final

|         \--- com.fasterxml:classmate:1.1.0

+--- com.fasterxml.jackson.core:jackson-databind:2.6.3

|    +--- com.fasterxml.jackson.core:jackson-annotations:2.6.0

|    \--- com.fasterxml.jackson.core:jackson-core:2.6.3

+--- org.springframework:spring-web:4.2.3.RELEASE

|    +--- org.springframework:spring-aop:4.2.3.RELEASE (*)

|    +--- org.springframework:spring-beans:4.2.3.RELEASE (*)

|    +--- org.springframework:spring-context:4.2.3.RELEASE (*)

|    \--- org.springframework:spring-core:4.2.3.RELEASE (*)

\--- org.springframework:spring-webmvc:4.2.3.RELEASE

+--- org.springframework:spring-beans:4.2.3.RELEASE (*)

+--- org.springframework:spring-context:4.2.3.RELEASE (*)

+--- org.springframework:spring-core:4.2.3.RELEASE (*)

+--- org.springframework:spring-expression:4.2.3.RELEASE (*)

\--- org.springframework:spring-web:4.2.3.RELEASE (*)

This is insane! With a Java/Maven project, you most likely don't even know how many dependencies you've just added to your project once you added that innocent-looking dependency to your project... now that you have around 55 libraries you rely on, and given Java's flat classpath (which means you must pray some other dependency you had in your project does not require a different version of a library you just imported transitively), chances are you'll face classpath hell, sooner or later.

Ceylon actually solves this problem with its module system (inherited from JBoss in the JVM) which isolates module's classpaths by default. However, if we needed to declare every single transitive dependency every time we wanted to use a new library in our project, this would become completely un-manageable.

Here's where the Ceylon Gradle Plugin can help!

To allow Ceylon developers to add any Java (in fact, any JVM library, including Groovy, Kotlin and Scala libraries) library they want to their project as easily as they've always done in Java, the Ceylon Gradle Plugin comes to the rescue, letting you use Ceylon in the basically same way you used Java... with a flat classpath, and letting Gradle resolve your JVM dependencies for you (notice that Ceylon dependencies are still resolved by Ceylon, as it can do a much better job there!). 

Even though you can still use the Ceylon module system if you want to, by default the Ceylon Gradle Plugin will use a flat classpath, like Java and most JVM languages, and it will resolve all your dependencies using Gradle's powerful dependency management. So all the dirty-tricks used by many Java libraries (scanning the classpath looking for annotated classes, instantiating classes that are not even visible to them etc) will work as you expect them to. And you can remain blissfully unaware of the tons of transitive dependencies your projects use if you choose to.

Going back to the Spring Boot example... here's how it works:

• First of all, declare a single dependency on Spring Boot in your Ceylon module file (plus Java and the Java interop module, as we'll be needing those):

module.ceylon

native("jvm")

module com.athaydes.springboot "1.0.0" {

   import java.base "8";

   import ceylon.interop.java "1.2.0";

   import "org.springframework.boot:spring-boot-starter-web" "1.3.0.RELEASE";

}

• Now, create a simple hello world application in Ceylon that uses Spring Boot annotations:

run.ceylon

import org.springframework.boot { ... }import org.springframework.boot.autoconfigure { ... }import org.springframework.stereotype { ... }import org.springframework.web.bind.annotation { ... }import java.lang { JString=String }import ceylon.interop.java { javaClass, javaString }controllerenableAutoConfigurationshared class SampleController() {      requestMapping({ "/" })     responseBody     shared JString home() {         return javaString("Hello World!");     }}"Run the module `com.athaydes.springboot`."shared void run() {     SpringApplication.run(javaClass<SampleController>());}

• Try to run it!

 gradle runCeylon

Caused by: java.lang.IllegalArgumentException: LoggerFactory is not a Logback LoggerContext but Logback is on the classpath. Either remove Logback or the competing implementation (class org.slf4j.impl.SimpleLoggerFactory loaded from file:/Users/renato/.sdkman/candidates/ceylon/1.2.0/repo/org/slf4j/simple/1.6.1/org.slf4j.simple-1.6.1.jar). If you are using WebLogic you will need to add 'org.slf4j' to prefer-application-packages in WEB-INF/weblogic.xml Object of class [org.slf4j.impl.SimpleLoggerFactory] must be an instance of class ch.qos.logback.classic.LoggerContext

• Ok, that didn't go so well... Spring Boot complains about the logging library. Unfortunately, the Ceylon runtime also needs a logging framework, and unluckily, it didn't choose Logback as Spring Boot did. But that's easy to solve. Just exclude logback from your dependencies using the Gradle build file:

build.gradle

dependencies {

   ceylonCompile "org.springframework.boot:spring-boot-starter-web:1.3.0.RELEASE", {

       exclude group: "ch.qos.logback"

   }

}

• Run again... now everything should work and you should see the Spring Boot banner!

  .   ____          _            __ _ _

 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \

( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \

 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )

  '  |____| .__|_| |_|_| |_\__, | / / / /

 =========|_|==============|___/=/_/_/_/

 :: Spring Boot ::        (v1.3.0.RELEASE)

To kill the server, see the README page of this sample project.

Pretty straightforward. Notice that, originally, we didn't even declare the dependency in the Gradle build file (and that would usually have worked). That's because the Ceylon Gradle Plugin will actually add it automatically to the Gradle project based on the module.ceylon file.

The reason for that is that the idea is to keep the Ceylon module file as the main, if not the only, location where you declare your direct dependencies. The build.gradle file should be seen as a complimentary file where you patch your dependencies if necessary. You can also add runtime dependencies to the classpath which will not be visible to the Ceylon code... this is very common in Java, and will only work if you do not turn off the flatClasspath option...

An example where using a runtime dependency is useful is when you decide to use an API library like slf4j, and the implementation of that library should be a deployment detail, so the actual library that implements the slf4j API is a runtime dependency.

To declare a runtime dependency, you use the Gradle build file as follows:

build.gradle

dependencies {

   ceylonRuntime "some.runtime:dependency:1.0"

}

One important thing to remember is that Gradle is used only to build a Maven and a Ceylon repository based on the dependencies of the Ceylon project, but that all commands to compile, run and test your Ceylon project are still just delegated to Ceylon. Therefore, you must have Ceylon installed in order to build your project with Gradle.

    Install Ceylon with SDKMAN, for example, before trying to use the Ceylon Gradle Plugin.

If you install Ceylon in a standard location, the Gradle Plugin will find it automatically and you don't need to explicitly declare that... if necessary, just let the Plugin know where your Ceylon installation is:

build.gradle

ceylon {

   module = "com.athaydes.maven"

   ceylonLocation = "/usr/shared/ceylon/ceylon-1.2.0"

}

The Gradle Ceylon Plugin tasks run in the following order, by default:

1. resolveCeylonDependencies

2. createDependenciesPoms

3. createMavenRepo

4. generatesOverridesFile

5. createModuleDescriptors

6. importJars

7. compileCeylon

You don't need to know what all these tasks do, but if you are curious read the documentation by running:

 gradle tasks --all

Once your project has been built with gradle importJars, you don't really need Gradle anymore! You'll get the following files and directories with everything that you need to compile and run your Ceylon project with Ceylon itself:

• modules/ - the Ceylon repository containing any dependencies your Ceylon module might have on other Gradle Ceylon modules. Ceylon dependencies that come from Herd will be downloaded by Ceylon.

• build/dependency-poms/ - the auto-generated poms of each of your project dependencies. This is good for documentation, but can be deleted if you prefer as it is not required by Ceylon.

• build/maven-repository/ - A Maven local repository containing all Maven dependencies of your project (so Ceylon won't even need to look at any remote repositories, ensuring dependency resolution of Maven dependencies is done by Gradle only).

• build/module-descriptors/ - properties files describing the Maven modules. These files may be used by the importJars task if you enable it.

• build/maven-settings.xml - a Maven settings file used to tell Ceylon where to look for Maven dependencies.

• build/overrides.xml - the Ceylon overrides.xml file, which allows the Ceylon runtime to know transitive dependencies of each module.

With all of these in place, you have now the choice to compile using Gradle:

 gradle compileCeylon

... or just get the actual Ceylon command necessary to do so:

 gradle -P get-ceylon-command compileCeylon

Which will print this (and you can run it later if you prefer):

/Users/renato/.sdkman/candidates/ceylon/current/bin/ceylon compile --overrides /Users/renato/programming/projects/ceylon-gradle-plugin/ceylon-gradle-plugin-tests/module-with-tests-sample/build/overrides.xml --rep=aether:/Users/renato/programming/projects/ceylon-gradle-plugin/ceylon-gradle-plugin-tests/module-with-tests-sample/build/maven-settings.xml --rep=/Users/renato/programming/projects/ceylon-gradle-plugin/ceylon-gradle-plugin-tests/module-with-tests-sample/modules --out=/Users/renato/programming/projects/ceylon-gradle-plugin/ceylon-gradle-plugin-tests/module-with-tests-sample/modules --source source --resource resource com.athaydes.simple 

    Gradle doesn't re-run tasks if it finds it unnecessary given the task inputs and outputs, so you may have to add the --rerun-tasks option to this command to force Gradle to run the  task and print the command even if re-compilation is unnecessary.

In either case, the car file created by Ceylon will be added to your Ceylon repository in the modules/ directory, ready to be run.

Again, you can run with Gradle just to make sure everything is working (make sure your module is runnable, which means having a run() function at the top-level package):

 gradle runCeylon

... or just get the actual Ceylon command:

 gradle -P get-ceylon-command runCeylon

Finally, make sure to run your test module:

 gradle testCeylon

To help you get started (actually, I wrote these samples to test the plugin!) there are several samples in the ceylon-gradle-plugin-tests directory on Github!

For example, the spring-boot-sample contains the code shown earlier in this blog post.

multi-modules-project has 2 Ceylon modules and a Java module, with one of the Ceylon modules using the other two. This sample actually uses the Ceylon module system instead of a flat classpath, so if that's what you need, have a look at the build.gradle file.

There's even the cross-language-modules project consisting of modules written in a different language each: one in Groovy, one in Kotlin, one in plain Java, all of which are used by a Ceylon module, proving that the JVM is now a happy family of cool languages you can mix-and-match as you see fit!

No other programming platform is as exciting to work with these days!

Comments/questions on Reddit!