29 June 2019
Recently got unusual height CPU utilizaton on random wildly cluster instance. Thread dump shows the reason of a problem:
Found one Java-level deadlock:
=============================
"Thread-1 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@46fcd20a-1114701218)":
waiting to lock Monitor@0x00007f45f02e20f8 (Object@0x0000000603407950, a org/apache/activemq/artemis/core/server/cluster/impl/ClusterConnectionImpl$MessageFlowRecordImpl),
which is held by "Thread-29 (ActiveMQ-client-global-threads-2129186403)"
"Thread-29 (ActiveMQ-client-global-threads-2129186403)":
waiting to lock Monitor@0x00007f46203b5518 (Object@0x00000004cc79a7b8, a org/apache/activemq/artemis/core/server/impl/QueueImpl),
which is held by "Thread-1 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@46fcd20a-1114701218)"
Found a total of 1 deadlock.
Below is part of official documentation:
"If thread-pool-max-size is set to a positive integer greater than zero, the thread pool is bounded. If requests come in and there are no free threads available in the pool, requests will block until a thread becomes available. It is recommended that a bounded thread pool be used with caution since it can lead to deadlock situations if the upper bound is configured too low."
So, solution is:
/subsystem=messaging-activemq/server=default:write-attribute(name=thread-pool-max-size,value=-1)
28 June 2019
To load calsses in runtime java uses ClassLoader mechanism which is based on next core principles:
parent-first delegation, - child ClassLoader will be used if parent is not able to find or load class. This behavior can be changed to child-first by overwriting ClassLoader.loadClass(...);The main scenarios to use custom ClassLoader is:
So, let's see how using of custom ClassLoader looks from source code perspective:
List<File> jars = Arrays.asList(new File("/tmp/jars").listFiles());
URL[] urls = new URL[files.size()];
for (int i = 0; i < jars.size(); i++) {
try {
urls[i] = jars.get(i).toURI().toURL();
} catch (Exception e) {
e.printStackTrace();
}
}
URLClassLoader childClassLoader = new URLClassLoader(urls, ClassLoader.getSystemClassLoader());
Then load class with custom ClassLoader:
Class.forName("org.kostenko.examples.core.classloader.ClassLoaderTest", true , childClassLoader);
Note! If your loaded libraries uses some resources like properties or something else, you need to provide context class loader:
Thread.currentThread().setContextClassLoader(childClassLoader);
Also, you can use custom ClassLoaders to load services with Java Service Provider Interface(SPI)
ServiceLoader<MyProvider> serviceLoader = ServiceLoader.load(MyProvider.class, childClassLoader);
...
27 May 2019
Well designed API is very important in case you expect your API will use someone else instead of just you. To build readable and writable DSL(domain specific language) in Java usually uses Builder pattern with few simple rules:
So, lets see how it looks from source code perspective...
Design API entry point
public class MyAPI {
public static UserBuilder.Registration asUser() {
return new UserBuilder.User();
}
}
Design builder using interfaces to restrict available values depends on current step
public class UserBuilder {
public static class User implements Registration, Login, Password, Apply {
private String login;
private String password;
@Override
public Login doRegistration() {
return this;
}
@Override
public Password withLogin(String login) {
this.login = login;
return this;
}
@Override
public Apply withPassword(String password) {
this.password = password;
return this;
}
@Override
public void apply() {
// ...
}
}
public interface Registration {
Login doRegistration();
}
public interface Login {
Password withLogin(String login);
}
public interface Password {
Apply withPassword(String password);
}
public interface Apply {
void apply();
}
}
Then usage, with no chance to mistake, looks like
asUser().doRegistration()
.withLogin("login").withPassword("password")
.apply();
02 May 2019
Have published new YouTube video about Wildfly AS domain mode clustering
18 April 2019
Directly or indirectly every developer uses random number generation during application development by different reasons:
and many more...
In Java we have insecure: java.util.Random and secure: java.security.SecureRandom random generators. Main differences between is
As result of above, for described reasons using of SecureRandom much preferable. You may never get performance issues with it until you need for number of random numbers which can be generated per time unit. Especially on cloud environment because of poor entropy.
Simple ussage and speed comparison example:
public class RandomGenerationTest {
private static int count = 100_000_000;
public static void main(String... s) throws Exception {
System.out.println("Start...");
doGeneration(new Random());
doGeneration(new SecureRandom());
}
private static void doGeneration(Random random) {
long time = System.currentTimeMillis();
for (int i = 0; i < count; i++) {
random.nextInt();
}
time = System.currentTimeMillis() - time;
System.out.println(String.format("Generation of %s random numbers with %s time %s ms.", count, random.getClass().getName() ,time));
}
}
Generation of 100000000 random numbers with java.util.Random time 930 ms.
Generation of 100000000 random numbers with java.security.SecureRandom time 22036 ms.
By default SecureRandom will use random data from Linux kernel entropy pool /dev/random. So in case pool went empty - next generation can be delayed for several minutes. You also can switch to the pseudo random number generator /dev/urandom which is can be must faster (non blocking) but little bit less secure.
To make Java use /dev/urandom you need to change securerandom.source property in the file jre/lib/security/java.security
securerandom.source=file:/dev/urandom
The entropy gathering device can also be specified with the System property java.security.egd. For example:
java -Djava.security.egd=file:/dev/urandom MainClass
To check how much random data is currently available in the entropy pool, use next command:
cat /proc/sys/kernel/random/entropy_avail
Every number lower than 1.000 can be considered too low for normal operation; if you request more data than available the requesting process will block. To increase entropy of Linux environment you can use HArdware Volatile Entropy Gathering and Expansion with haveged open source implementation
apt-get install haveged
It will use additional hardware(CPU) statistic to increase entropy and as result speedup your application.