Archive for the ‘Java’ Category

CSV/Excel Utility improves performance

Friday, March 7th, 2014

I recently stumbled across a performance test of Java CSV libraries (can’t remember where). To my surprise, someone tested multiple Open Source products including my own CSV/Excel Utility Package. And even more surprising to me, mine was the worst. By far! It took 4 times as much than others spent in parsing a CSV file. Embarrasing! Why did I never spent effort in measuring performance?

Anyway. I took the time and wrote a comparable JUnit perfromance test for all major CSV libraries and gave them a 150MB file to read. Analyzing the results of my own library with JProfiler, I found a very stupid performance eater (simplified here):

1
2
3
4
5
6
String s = "";
for (char c : anotherString.toCharArray()) {
   ...
   s += c;
   ...
}

The “addition” of line 6 was called 150 million times – for each single character in the file. Replacing this by a StringBuilder construct, the performance rose near the other major CSV libraries:

1
2
3
4
5
6
StringBuilder s = new StringBuilder();
for (char c : anotherString.toCharArray()) {
   ...
   s.append(c);
   ...
}

So two things to learn here: (1) Do not under-estimate performance test before releasing something, (2) Take care when using the “addition” operator for strings. 🙂

PS: CSV/Excel Utility Package 1.7 will contain the fix.
PPS: StringBuilder is preferred against StringBuffer as it is not synchronized and therefore faster. Most use cases allow this simplifications.

RS Library 1.2.2 released

Wednesday, December 11th, 2013

A new version of the RS Library was released. Version 1.2.2 fixes many bugs found by unit tests. The previous version just had 12 unit tests, the new version got 190 unit tests to ensure quality of the code. A complete list of changes can be found here.

The Maven repositories shall be up-to-date by now. The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>baselib</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-file</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-hibernate</artifactid>
      <version>1.2.2</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>templating</artifactid>
      <version>1.2.2</version>
   </dependency>

The Class.forName() problem

Monday, November 4th, 2013

In earlier Java days, it was easy to get a Class object when you had its name only. You simply called:

1
2
3
4
5
   // Without parametrized
   Class c = Class.forName("java.lang.String");
 
   // With parameters
   Class<String> = (Class<String>)Class.forName("java.lang.String");

This worked as long as there have been only one class loader. Long time ago :). The world kept spinning and concepts of separated applications in a single Java engine appeared. And so did OSGI. OSGI specified its container in a way that you can have a class being loaded in multiple versions at the same time. Furthermore, an OSGI bundle (effectively a JAR) can describe what other bundles can see their classes and what not. OSGI containers implement these behaviour through different ClassLoader instances responsible for the respective bundles.

However, this introduces a problem on Class.forName(String) calls. The method uses the ClassLoader of the “caller”. That means that it will be restricted to the bundle’s definition. As long as you use this method within bundle A to find a class in bundle A, everything works fine. It will even find a class in bundle B when B exported correctly and A defined B as a pre-requisite.

However, modular code encapsulates common functions. That’s why it can happen that a Class.forName(String) call is implemented in bundle C and doesn’t know about bundle A at all. When bundle A now calls bundle C which in turn calls Class.forName(String) to load a class from bundle A, then it fails. The reason is that ClassLoader C is not allowed to access classes in bundle A.

Several solutions are possible. The OSGI solution for this problem is the friend. You declare bundle C as friend of bundle A and class loading will succeed. However, this works only as long as you don’t try to load a class from bundle B. Now B would be required to declare C as a friend. If B is a 3rd party bundle then this solution is out of reach.

Another solution is the iteration on all OSGI bundles and trying to find the class via their class loaders. However, this is error-prone. Unless you don’t know exactly what bundle contains your required class you might get in trouble. Several versions of the same class can be present (or just define a class with the same name). You would need some heuristics to identify the correct class.

There is a better and most-likely the best solution. Each thread in Java can have a Context Class Loader. This will allow us to resolve all the dependency problems in OSGI. We will define the method in bundle C as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
	/**
	 * Load a class by first checking the thread's class loader and then the caller's class loader.
	 * @param className name of class to be loaded
	 * @return the class found
	 * @throws ClassNotFoundException when the class cannot be found
	 */
	public static Class<?> forName(String className) throws ClassNotFoundException {
		return forName(className, null);
	}
 
	/**
	 * Load a class by directly specifying a class loader.
	 * @param className name of class to be loaded
	 * @param classLoader the class loader to be used - if null the thread's class loader will be  used first
	 * @return the class found
	 * @throws ClassNotFoundException when the class cannot be found
	 */
	public static Class<?> forName(String className, ClassLoader classLoader) throws ClassNotFoundException {
		if (classLoader == null) try {
			// Check the thread's class loader
			classLoader = Thread.currentThread().getContextClassLoader();
			if (classLoader != null) {
				return Class.forName(className, true, classLoader);
			}
		} catch (ClassNotFoundException e) {
			// not found, use the class' loader
			classLoader = null;
		}
		if (classLoader != null) {
			return Class.forName(className, true, classLoader);
		}
		return Class.forName(className);
	}

The method will (if no specific class loader was given) first test the Context Class Loader (lines 21-24). Only if this fails or is (for whatever reasons) not available, then the original Class.forName(String) method will be asked (line 30).

As the Context Class Loader is defined in the beginning of the executing thread, it is usually the class loader of Bundle A which has access to all other dependent bundles. Therefore we will be able to load classes from bundle A and B with code written in bundle C.

1
2
3
4
5
6
7
8
   // Code in bundle C
   public Class<?> getClass(String name) {
      return BundleC.forName(name);
   }
 
   // Code in bundle A
   Class<?> c1 = BundleC.forName("bundleA.className");
   Class<?> c2 = BundleC.forName("bundleB.className");

I added the helper methods in my RS Base Classes Library class LangUtils where you can use it directly from (V1.2.2 – available very soon) in your OSGI projects.

RS Library 1.2.1 released

Wednesday, October 23rd, 2013

A major milestone of RS Library version 1.2 is out. I already discovered a critical bug in 1.2.0. That’s why the latest version is 1.2.1 now. The most important changes are:

However, I already discovered a minor bug in V1.2.1: RSLIBS-37 describes an incorrect behaviour of SoftMapCache.containsValue(Object) and WeakMapCache.containsValue(Object) methods which will be fixed in next version. V1.2.1 does not use this methods anywhere, so you would need to work-around this function in case you require it.

I also added more documentation now so you will be able to understand and use the libraries. It can be found either through Javadocs or the appropriate module homepages:

The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>baselib</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-file</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-hibernate</artifactid>
      <version>1.2.1</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>templating</artifactid>
      <version>1.2.1</version>
   </dependency>

Eclipse/E4: Problem with Key Bindings

Sunday, October 13th, 2013

I recently updated one of my E4 applications from Juno to Kepler (V4.3) release. Beside a few things (e.g. adding the HandlerProcessingAddOn to e4xmi configuration) I suddenly lost my customized key bindings. Unfortunately, there was not even a slight hint anywhere in the net why this happened. Most of the web sites were just referring to the afore mentioned add-on. But this didn’t bring back the missing key bindings.

So the only solution was to debug the bootstrap of RCP/E4 and find out where the key bindings disappeared. I noticed very early that my Binding Table was read but didn’t contain the correct key bindings. Although, an earlier breakpoint revealed that they have been read correctly. However, somehow they were removed and replaced by other ones.

Debugging in Eclipse/E4 is quite exhausting escpecially when you try to trace a problem in the bootstrap mechanism. But finally I saw an extension point (a “processor” to be precise) as root cause. This extension just threw away my key bindings. It eventually turned out to be the org.eclipse.ui.workbench plugin which I required in my application. This plugin defines a BindingToModelProcessor that simply throws away all key bindings within the org.eclipse.ui.contexts.dialogAndWindow Binding Context except those tagged with type:user.

That’s the solution. Simple as that. I added the tag to all my key bindings and they were not removed and working again. So be aware when adding org.eclipse.ui.workbench as a dependency plugin to your E4 application. 😉

Hibernate 4 with JTA Transactions

Saturday, October 12th, 2013

The following problem occurred when I was migrating from Hibernate 3 to Hibernate 4.2.3. Scrollable result sets were suddenly throwing exceptions:

org.hibernate.exception.GenericJDBCException: could not advance using next()
	at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
	at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
	at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:121)
...
Caused by: java.sql.SQLException: Operation not allowed after ResultSet closed
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1078)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
	at com.mysql.jdbc.ResultSetImpl.checkClosed(ResultSetImpl.java:804)
	at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:6986)
	at org.hibernate.internal.ScrollableResultsImpl.next(ScrollableResultsImpl.java:116)

It turned out after 3 hours of debugging and research that Hibernate’s 4 JTA implementation relies on having a transaction started not via JTA User Transaction but through Hibernate’s own Session object. The correct configuration in such cases is simply to use CMTTransactionFactory as transaction factory class:

1
2
3
4
5
6
7
8
9
10
11
12
13
<hibernate-configuration>
   <session-factory>
      <!-- How to find the Transaction -->
      <property name="hibernate.transaction.factory_class">org.hibernate.engine.transaction.internal.jta.CMTTransactionFactory</property>
 
      <!-- How to produce transaction -->
      <property name="hibernate.transaction.jta.platform">org.hibernate.service.jta.platform.internal.JOTMJtaPlatform</property>
 
      <!-- Session context with JTA -->
      <property name="current_session_context_class">jta</property>
      ...
   </session-factory>
</hibernate-configuration>

As explanation: The error is primarily caused by a bug in JTA implementation within Hibernate (it does not synchronize Hibernate’s with JTA’s transaction object). It occurs mainly when you use the Entity Manager. In above’s exception, a second retrieval was executed within object iteration which caused Hibernate to think that there is no active transaction and hence, close any previous statements. That’s why the Scrollable Result loses its session.

The solution was published first by Koen Serneels. Many thanks!

Short notice

Friday, September 27th, 2013

I am sorry that the desperately awaited version of B4J is still not available. B4J is supposed to add XML-RPC session support which I couldn’t follow up for a long time. However, you can use the current Snapshot version available via Sonatype to benefit from the UTF-8 encoding fix, the Jira session support and integration with CSV/Excel Utility. The Maven coordinates are:

<dependency>
  <groupId>eu.ralph-schuster</groupId>
  <artifactId>b4j</artifactId>
  <version>1.5.0-SNAPSHOT</version>
</dependency>

CSV/Excel Utility Package V2.6.1 released

Monday, August 5th, 2013

The new version 2.6.1 adds some improvements as of character encoding and bean reading and writing. A complete change log is available through the Maven Site.

You can download the new version here or visit the Homepage of the utility where you will find some examples on how to use it.

The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>csv</artifactid>
      <version>2.6.1</version>
</dependency>

PS: Version 2.6.0 has been released just two days ago but was already replaced by V2.6.1 because the V2.6.0 TableReader interface had a superfluous method defined.

RS Library V1.1.0 released

Monday, August 5th, 2013

I had to do some rework for the RS Library. The most important changes are:

  • RSLIB-19 – Improved DAO registration in abstract DaoFactory implementation
  • RSLIB-20 – Upgrade to Hibernate 4
  • RSLIB-22 – Provide modules as OSGI packages
  • RSLIB-23 – Cache Control for DAOs

A complete documentation can be found either through Javadocs or the appropriate module homepages:

The Maven coordinates are:

   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>baselib</artifactid>
      <version>1.1.0</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data</artifactid>
      <version>1.1.0</version>
   </dependency>
 
   <dependency>
      <groupid>eu.ralph-schuster</groupid>
      <artifactid>data-hibernate</artifactid>
      <version>1.1.0</version>
   </dependency>

Apache ServiceMix, Camel and XSL Transformation

Tuesday, July 9th, 2013

This topic cost me a few hours. It is basically how to achieve XML transformation when your original XML file contains a DTD reference that cannot be resolved from your system.

My DTD reference looked like this:

<!DOCTYPE root-element SYSTEM "../dtd/core.dtd">

and the route was simply:

	<camelContext xmlns="http://camel.apache.org/schema/spring">
		<route>
			<from uri="file:data/inbox"/>
			<to uri="xslt://xslt/transformation.xsl"/>
			<to uri="file:data/outbox"/>
		</route>
	</camelContext>

The Camel transformation was always complaining about the missing DTD because it was looking for it in the filesystem instead of the bundle.

There exist many hints on the net but none of them really helped (e.g. using a CatalogResolver). So I decided to tamper my Camel route and adding a bean inbetween:

	<bean id="dtdTransformer" class="de.example.xslt.DtdTransformer"/>
 
	<camelContext xmlns="http://camel.apache.org/schema/spring">
		<route>
			<from uri="file:data/inbox"/>
			<bean ref="dtdTransformer"/>
			<to uri="xslt://xslt/bundesliga.xsl"/>
			<to uri="file:data/outbox"/>
		</route>
	</camelContext>

This DtdTransformer is a simple class having one method, transform():

	public static final String DOCTYPE_LINE = "<!DOCTYPE root PUBLIC \"-//PRIVATE//RS//EN\" \"###URI###\">";
 
	public String transform(String body) throws Exception {
		URL url = FileFinder.find(getClass(), "dtd/core.dtd");
		if (url != null) {
			body = body.replaceAll("<!DOCTYPE[^>]*>", DOCTYPE_LINE).replaceAll("###URI###", url.toURI().toString());
		}
		return body;
	}

The URI will be replaced by looking up the DTD file at runtime, here by using my FileFinder class. Then it continues with replacing the special marker ###URI### by the real URI from the file found.

Although the solution sounds so simple, it took me quite a while to find out that the DOCTYPE references needs to be a URI and not a file or URL reference.