< Sergii Kostenko's blog

Distributed caching with Wildfly/Infinispan and poor JCache support.

26 August 2020

Latest trends teach us to do development of stateless applications, and if you can - keep your design stateless. But, by some reason, you may need to cache and share state between nodes.

It would be nice to have JSR 107: JCACHE - Java Temporary Caching API support in Wildfly Application Server, but unfortunately, JCache still not a part of JakartaEE specification (i hope one day it will) and pity to realize that Wildfly does not support JCache by default.

From other point of view many well known vendors like Hazelcast, Infinispan, Ehcache etc, supports JCache API as well. In turn significant Infinispan part integrated into Wildfly Application Server and can be used as distributed cache provider over separate infinispan subsystem configuration.

So, let's design sample Jakarta EE application to see how distrubuted cache looks and works on practice.

First, we need for at least two node Wildfly cluster - please refer to my article about Wildfly domain mode cluster and load balancing from the box. And then we are ready to configure distributed cache for our application:

/profile=full-ha/subsystem=infinispan/cache-container=mycachecontainer:add
/profile=full-ha/subsystem=infinispan/cache-container=mycachecontainer/distributed-cache=mycache:add

After simply server configuration above, we are ready to create our sample application. And as usual with Jakarta EE - build.gradle looks pretty simple and clear :

apply plugin: 'war'
dependencies {
    providedCompile "jakarta.platform:jakarta.jakartaee-api:8.0.0"
    providedCompile "org.infinispan:infinispan-core:10.1.8.Final"
}

Now to use configured above mycache we need to register cache resource in the one from two ways :

@Startup
@Singleton
@LocalBean
public class MyCacheResource {
    @Resource(lookup = "java:jboss/infinispan/cache/mycachecontainer/mycache")
    private org.infinispan.Cache<String, Object> myCache;

OR provide resource reference in your WEB-INF/web.xml descriptor:

<web-app version="2.5"  xmlns="http://java.sun.com/xml/ns/javaee"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd">
    <display-name>JCache API example</display-name>
    <resource-env-ref>
        <resource-env-ref-name>mycache</resource-env-ref-name>
        <lookup-name>java:jboss/infinispan/cache/mycachecontainer/mycache</lookup-name>
    </resource-env-ref>    
</web-app>

I personally prefer second one because it allows move vendor specific code and dependencies from application source level to the descriptor which is designed for. Actually, i recommend to use standard API as much as possible and refer to custom vendor specific stuff very carefully.

Also to help Wildfly avoid casting exception like java.lang.IllegalArgumentException: Can not set org.infinispan.Cache field to org.jboss.as.clustering.infinispan.DefaultCache we need to configure module dependencies over MANIFEST.MF:

Manifest-Version: 1.0
Dependencies: org.infinispan export

OR over jboss-deployment-structure.xml :

<jboss-deployment-structure>
   <deployment>
      <dependencies>
         <module name="org.infinispan" export="TRUE" />
      </dependencies>
   </deployment>
</jboss-deployment-structure>

And again, I prefer second way as vendor specific descriptor is a right place for vendor specific stuff. Please refer to deployment module dependencies explanation for the details

Now when all preparation is complete, - let's implement simple service and JAX-RS resource to check how cache distribution works:

TestCacheService.java:

@Named
public class TestCacheService {

    @Resource(name = "mycache")
    org.infinispan.Cache cache;

    public void putIspnCache(String key, String value) {
        cache.put(key, String.format("%s (%s)", value, new Date()));
    }

    public Object getIspnCache(String key) {
        return cache.get(key);
    }
}

TestCacheEndpoint.java:

@Stateless
@ApplicationPath("/")
@Path("/jcache")
public class TestCacheEndpoint extends Application {

    @Inject
    TestCacheService service;

    @GET
    @Path("/ispn-put")
    public Response putIspn(@QueryParam("key") String key, @QueryParam("value") String value) {
        service.putIspnCache(key, value);
        return Response.ok("ok").build();
    }

    @GET
    @Path("/ispn-get")
    public Response getIspn(@QueryParam("key") String key) {
        return Response.ok(service.getIspnCache(key)).build();
    }
}    

Time to do deploy and test:

[domain@localhost:9990 /] deploy ~/work/kostenko/wildfly-infinispan-example/build/libs/jcache-examples.war --server-groups=backend-servers
curl -o - "http://localhost:8180/jcache-examples/jcache/ispn-put?key=KEY1&value=VALUE1"
ok
curl -o - "http://localhost:8280/jcache-examples/jcache/ispn-get?key=KEY1"
VALUE1 (Mon Aug 24 21:26:56 EEST 2020)
curl -o - "http://localhost:8280/jcache-examples/jcache/ispn-put?key=KEY2&value=VALUE2"
ok
curl -o - "http://localhost:8180/jcache-examples/jcache/ispn-get?key=KEY2"
VALUE2 (Mon Aug 24 21:27:52 EEST 2020)

As you can see from above, value we put on node1 available on node2 and vice versa. Even if we add new node to the cluster - cached values will be available on the fresh node as well:

[domain@localhost:9990 /] /host=master/server-config=backend3:add(group=backend-servers, socket-binding-port-offset=300)
[domain@localhost:9990 /] /host=master/server-config=backend3:start(blocking=true)
curl -o - "http://localhost:8380/jcache-examples/jcache/ispn-get?key=KEY2"
VALUE2 (Mon Aug 24 21:27:52 EEST 2020)

Great! So for now we able to share state between cluster members and, actually, this is enough for lot of typical use cases.
So, what about some standardization of our application ? As was noticed above JCache can be helpful here, but unfortunately enabling last one on Wildfly is not trivial at all.

To get JCache worked you can patch your Wildfly Application Server with Infinispan wildfly modules or just put missed libraries to the your application and exclude transitive ones to avoid conflicts with libraries that already present in the Wildfly.

build.gradle:

...
dependencies {
    providedCompile "jakarta.platform:jakarta.jakartaee-api:8.0.0"
    compile "javax.cache:cache-api:1.0.0"
    compile "org.infinispan:infinispan-jcache:10.1.8.Final"
    compile "org.infinispan:infinispan-cdi-embedded:10.1.8.Final"
}
configurations {
  runtime.exclude group: "org.infinispan", module: "infinispan-core"
  runtime.exclude group: "org.infinispan", module: "infinispan-commons"
  runtime.exclude group: "org.infinispan.protostream", module: "protostream"
}

jboss-deployment-structure.xml:

<?xml version="1.0" encoding="UTF-8"?>
<jboss-deployment-structure>
    <deployment>
        <dependencies>
           <module name="org.infinispan" export="TRUE" />
           <module name="org.infinispan.commons" export="TRUE" />
           <module name="org.infinispan.protostream" export="TRUE" />
        </dependencies>
    </deployment>
</jboss-deployment-structure>

After that you should be able to use JCache in the usual way:

TestCacheService.java:

...
@CacheResult(cacheName = "mycache")
public String getJCacheResult() {
    System.out.println("getJCacheResult");
    return new Date().toString();
}
...

beans.xml:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://xmlns.jcp.org/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee  http://xmlns.jcp.org/xml/ns/javaee/beans_1_1.xsd" bean-discovery-mode="all">
    <interceptors>
        <class>org.infinispan.jcache.annotation.CacheResultInterceptor</class>
    </interceptors>
</beans>

@CacheResult works and caching the result BUT it is not related to the configured on Wildfly mycache and ignores configured options like lifespans, distributions etc because Infinispan's JCache CachingProvider implementation created caches from an Infinispan native configuration file (based on the provided URI, interpreted as a file path) instead of WF configuration.

I did some digging about possibility to produce custom JCache CachingProvider but unfortunately did not find any workable solution for it. Also refer to my post about ispn distributed cache issues workaround.

As usual samle source code available on GitHub

Comments


Caucho Resin datasource configuration

16 August 2020

There is few possible ways to do datasource configuration for Jakarta EE application on Resin Application Server:

Way #1 - Application level. Place JDBC driver to the application classpath and edit WEB-INF/resin-web.xml

<web-app xmlns="http://caucho.com/ns/resin">
    <database jndi-name='jdbc/myds'>
        <driver type="com.microsoft.sqlserver.jdbc.SQLServerDriver">
            <url>jdbc:sqlserver://localhost:1433</url>
            <user>user</user>
            <password>password</password>
        </driver>
    </database>
</web-app>

Way #2 - Application Server level. Put JDBC driver to <resin_home>/lib directory and edit <resin_home>/conf/resin.xml

...
<cluster id="app">
  ...
  <database>
    <jndi-name>jdbc/myds</jndi-name>
    <driver type="com.microsoft.sqlserver.jdbc.SQLServerDriver">
      <url>jdbc:sqlserver://localhost:1433</url>
      <user>user</user>
      <password>password</password>
     </driver>
     <prepared-statement-cache-size>8</prepared-statement-cache-size>
     <max-connections>20</max-connections>
     <max-idle-time>30s</max-idle-time>
   </database>
</cluster>
...

Comments


Migration from Wildfly 18 to Wildfly 20

06 August 2020

Some time ago i wrote article about migration from Wildfly 10 to Wildfly 18 and application level migration issues. Migration from Wildfly 18 to Wildfly 20 does not provoke any application level issues and can be done in minutes:

git clone https://github.com/wildfly/wildfly-server-migration.git
cd ./wildfly-server-migration/
mvn clean install
cd ./dist/standalone/target/
unzip jboss-server-migration-1.10.0-SNAPSHOT.zip
cd ./jboss-server-migration

./jboss-server-migration.sh -s /opt/wildfly-18.0.0.Final -t /opt/wildfly-20.0.1.Final/

Why should i do migration to Wildfly 20 ?

Comments


Unzip without root but with java

05 August 2020

If you need to unzip file on the server, where is no root and no unzip installed then time to ask java about:

jar xvf wildfly-20.0.1.Final.zip

Comments


Rich web application on pure Java with Vaadin and Quarkus

29 April 2020

Recently I wrote about REST API with Eclipse Microprofile and Quarkus - and it is very useful for the microservices development, but from time to time every backend Java developer needs for the UI. With Vaadin web framework you can write UI 100% in Java without getting bogged down in JS, HTML, and CSS.

Quarkus provides Servlet and Websocket support as well, so there is no any blockers to run web application.
To bootstrap Quarkus from the scratch you can visit code.quarkus.io and select build tool you like and extensions you need. In our case we need for:

With Vaadin 8 dependencies my build.gradle looks pretty clear:

plugins {
    id 'java'
    id 'io.quarkus'
}
repositories {
     mavenLocal()
     mavenCentral()
}
dependencies {
    compile 'com.vaadin:vaadin-server:8.10.3'
    compile 'com.vaadin:vaadin-push:8.10.3'
    compile 'com.vaadin:vaadin-client-compiled:8.10.3'
    compile 'com.vaadin:vaadin-themes:8.10.3'
    implementation 'io.quarkus:quarkus-undertow-websockets'
    implementation 'io.quarkus:quarkus-undertow'
    implementation enforcedPlatform("${quarkusPlatformGroupId}:${quarkusPlatformArtifactId}:${quarkusPlatformVersion}")
}

group 'org.kostenko'
version '1.0.0-SNAPSHOT'

compileJava {
    options.encoding = 'UTF-8'
    options.compilerArgs << '-parameters'
}

Now we able to create com.vaadin.ui.UI

@Theme("dashboard")
public class MyUI extends UI {

    @Override
    protected void init(VaadinRequest vaadinRequest) {
      ...
    }

    /**
     * VaadinServlet configuration
     */
    @WebServlet(urlPatterns = "/*", name = "MyUIServlet", asyncSupported = true, initParams = {
        @WebInitParam(name = "org.atmosphere.websocket.suppressJSR356", value = "true")}
    )
    @VaadinServletConfiguration(ui = MyUI.class, productionMode = false)
    public static class MyUIServlet extends VaadinServlet {
    }
}

Put Vaadin static files to the /src/main/resources/META-INF/resources/VAADIN and run quarkus in dev mode as usual ./gradlew quarkusDev:

Listening for transport dt_socket at address: 5005
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2020-04-29 09:49:37,718 WARN  [io.qua.dep.QuarkusAugmentor] (main) Using Java versions older than 11 to build Quarkus applications is deprecated and will be disallowed in a future release!
2020-04-29 09:49:38,389 INFO  [io.und.servlet] (Quarkus Main Thread) Initializing AtmosphereFramework
2020-04-29 09:49:38,579 INFO  [io.quarkus] (Quarkus Main Thread) Quarkus 1.4.1.Final started in 0.995s. Listening on: http://0.0.0.0:8080
2020-04-29 09:49:38,579 INFO  [io.quarkus] (Quarkus Main Thread) Profile dev activated. Live Coding activated.
2020-04-29 09:49:38,579 INFO  [io.quarkus] (Quarkus Main Thread) Installed features: [cdi, servlet, undertow-websockets]
2020-04-29 09:49:46,423 WARNING [com.vaa.ser.DefaultDeploymentConfiguration] (executor-thread-1)                                                                                                                                                             
=================================================================                                                                                                                                                                                            
Vaadin is running in DEBUG MODE.
Add productionMode=true to web.xml to disable debug features.
To show debug window, add ?debug to your application URL.
=================================================================

Example application I did based on vaadin/dashboard-demo that uses nicely looking and responsive Valo theme

quarkus + vaadin

Current solution limitations and workaround:

Described example application source code available on GitHub

Comments