Wednesday, 23 December 2015

JPA result lists and Java 8 streams

We recently changed a project to build using Java 8.
This decision was largely driven by the need to use the Date Time API available in Java 8 but several developers, including me, were itching to start using Functional Interfaces/programming and Lambdas.

Fortunately we have not gone down the road of "refactor" everything into Java 8 but new code is now being written using things like Streams and Lambdas. When the opportunity presents itself existing code is also being re-factored.

All was well until I started using Streams on result lists from JPA Entities (the same is valid for JPA query result lists). After changing code like this

into this
no results were being returned.

The result list is the returned by the zone.getPlaces() method whose type is List<Place>. No surprises there and in fact mocking this for a test would mean that the re-factored code would pass tests.
So what is causing the production code to behave differently?

Well the answer lies in the actual JPA implementation, in this case EclipseLink 2.5.2, and the IndirectList classes which are used to facilitate all the clever things which EclipseLink does under the hood. The actual explanation is too detailed to go into it here but there is a bug which does show what was wrong and, more importantly, that this has been fixed in EclipseLink versions 2.6 and beyond.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=433075

Worryingly tests which use mocks cannot not pick this problem up, tests which use the underlying JPA implementation would but do you want to write JUnit tests using the database?

Why is this important?

  • Because JUnit tests which mock database operations will provide false test results.
  • Because the JPA implementation we use for this project is provided by the application server we run the software on.
Newer versions of that application server which provide a later version of EclipseLink are available but that migration is a different story.

Monday, 16 November 2015

PrimeFaces Theme Development

I have been recently been involved in a book project in collaboration with Packt Publishing and co-authored by Sudheer Jonna, a PrimeFaces developer.

I am proud to be able to announce that the book as now been published: PrimeFaces Theme Development


About This Book

  • Build PrimeFaces themes that meet the industry standards
  • Customize your themes for mobile web applications with PrimeFaces Mobile
  • A precise, example-oriented guide to help you create interesting themes using the PrimeFaces tools

Who This Book Is For

If you are a Web Designer who wants to create and modify PrimeFaces themes then this book is for you. Basic knowledge of JSF, CSS, and HTML is assumed.

What You Will Learn

  • Use Maven to create JSF Projects
  • Create a custom theme using JQuery UI tools
  • Package your theme for distribution and use your PrimeFaces-based web applications as drop in libraries
  • Develop additional dynamic look and feel elements that themes do not already supply
  • Apply look and feel elements dynamically to views without causing page refreshes
  • Reduce code repetition by applying page templates to your JSF project

Thursday, 23 May 2013

Calling methods from Constructors, a definitive explanation by Dr Heinz M. Kabutz

I am a huge fan of Dr. Kabutz and have been following his Java Specialists newsletter for many years.

In the latest issue he explains and demonstrates the pitfalls of calling methods from within constructors.

Now I have always been a proponent of "don't put logic other than simple initialisation in constructors", in fact I have always introduced this as a "Golden Rule" when helping other developers.
I knew about most of the things that Dr. Kabutz explains here but never really found the time to really get into it and write things down. Now that Dr. Kabutz has done so I don't need to.

The majority of my work involves JSF/CDI/EJB/JPA. I still see many code examples which places significant code in the constructors of CDI Beans even from sources that claim to know better.
If you are in the habit of doing so you really need to read the newsletter and get into the habit of using @PostConstruct annotations in your code as well as adding lazy loading/evaluation to setter/getter methods.

Tuesday, 14 May 2013

JSF Beginners Tips #1: request parameter encoding and date/times

As a habitual visitor to the PrimeFaces Forums I and other expert members often have to answer questions related to problems related to request parameter encoding and date/times.

Without further rambling from me an example of the first problem.


Typically a web application will use UTF-8 encoding for pages, requests and responses. Unfortunately this can lead to unexpected problems as illustrated by the following observations.

The poster had discovered that a JSF page which uses PrimeFaces for a partial submit will submit request parameters as expected

However when the same page used a non-AJAX request (full form post submit) the same parameters become garbled.
I was able to reproduce this behaviour with a simple form like this

<?xml version='1.0' encoding='UTF-8' ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:h="http://java.sun.com/jsf/html"
      xmlns:tst="http://java.sun.com/jsf/composite/composites"
      xmlns:f="http://java.sun.com/jsf/core"
      xmlns:p="http://primefaces.org/ui">
    <f:view contentType="text/html" encoding="UTF-8">
        <h:head>
            <meta content="text/html; charset=UTF-8" http-equiv="Content-Type" />
            <title>Facelet Title</title>
        </h:head>
        <h:body>
            <h:form>
                <tst:buttonFacet labelText="Press me!">
                    <f:facet name="commandButton">
                        <p:commandButton ajax="false" value="Waiting to be pressed!"/>
                    </f:facet>
                </tst:buttonFacet>
                <p>
                    <h:panelGroup>
                        <p:inputText id="myInput" value="#{inputCapture.input}" required="true" requiredMessage="Required">
                            <p:ajax event="blur" listener="#{inputCapture.inputListener()}" update="myMsg"/>
                        </p:inputText>
                        <p:messages id="myMsg" globalOnly="false"/>
                    </h:panelGroup>
                </p>
            </h:form>
        </h:body>
    </f:view>
</html>

The composite component and backing bean don't actually play a role in this, I used an existing page to see if I could reproduce the posters findings and was successful. Starting with the failure case any "special" characters like Ö and ü were recieved in a garbled state and by the look of things they were being decoded as if they were ISO-8859-1 when they are actually UTF-8.
Ajaxifying the p:commandButton results in a partial submit with the values not being incorrectly decoded.
Using Glassfish I observed the following being logged.

WARNING: PWC4011: Unable to set request character encoding to utf-8 from context /XXX, because request parameters have already been read, or ServletRequest.getReader() has already been called
An old friend indeed.
Glassfish is expected to behave as the spec says and in this case it actually does. If a form POST does not specify an encoding it is assumed that the request is encoded using ISO-8859-1 and decodes request parameters accordingly.
To correct this for Glassfish simply add a glassfish-web.xml with the following contents

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE glassfish-web-app PUBLIC "-//GlassFish.org//DTD GlassFish Application Server 3.1 Servlet 3.0//EN" "http://glassfish.org/dtds/glassfish-web-app_3_0-1.dtd">
<glassfish-web-app error-url="">
  <class-loader delegate="true"/>
  <jsp-config>
    <property name="keepgenerated" value="true">
      <description>Keep a copy of the generated servlet class' java code.</description>
    </property>
  </jsp-config>
  <parameter-encoding default-charset="UTF-8"/>
</glassfish-web-app>

Other servers will have other ways of fixing this, Tomcat for instance.
For a more in-depth explanation of this try here courtesy of Omnifaces.org.

Problem #2


Dates causes chaos! Simply adding a calender type widget or even a simple h:outputText where the output is a Date will often result in the date being displayed with a different, unexpected, value where often the time is out by an hour or even the date being out by a day or more. The actual value is still correct however.
This infuriating behaviour has probably caused quite a lot of premature balding in devs over the years however a simple fix is available.
Add the following to your web.xml.
<context-param>
        <param-name>javax.faces.DATETIMECONVERTER_DEFAULT_TIMEZONE_IS_SYSTEM_TIMEZONE</param-name>
        <param-value>true</param-value>
    </context-param>

At least this means that all the other date problems you have are less likely to be caused by the JSF container.

For a more complete explanation of why this is so try here.

Wednesday, 24 April 2013

Reflecting on life, the Universe and what the hell is that BeanInfo class doing to my JSF!!!

As my career at my current employers comes to a close I have been revisiting all the Project sources that I have created for the company with an eye to making them suitable for handover to the guy taking them on when I leave.

In one of them I found a nice booby trap that causes an intermittent problem with a JSF based app.
Essentially a specific domain entity was being accused of not having a specific property by Glassfish but not all the time.

I checked, the property is there as it should be. But what the hell is this BeanInfo class doing here?

The BeanInfo class was also for the domain entity that was sometimes being accused of not having a property it actually did have. Armed with my trusty Sherlock Holmes Deerstalker and magnifying glass I had a look at it. Cause for the problem found! The PropertyDescriptor for the offending property was missing: the BeanInfo class was not in synch with the entity class that had since been changed.

But why would this cause JSF to sometimes complain?
Well with Introspection you can obtain information about a class with

    BeanInfo info = Introspector.getBeanInfo( SimpleBean.class );

where the bean info is used to obtain lists of fields, properties etc. What happens when you make the getBeanInfo call? Well the clever stuff under the hood checks in something called SimpleBeanBeanInfo.class exists in the same package as the class parameter. If one does not exist then one is automagically created for you and this is returned.

The inference here is that Mojarra, the Glassfish JSF Implementation, is doing similar things with Introspector as above but


  • Not all the time
    Or has a weird classloader thing going on so that the existing BeanInfo class is not found consistently
  • is picking up the out-of-synch BeanInfo class and causing the clever-under-the-hood stuff to appear broken.
Now before throwing the baby away with the bath water I also hunted around to see if deleting these things would have any inadvertant untoward effects on the rest of the system.

Sadly yes, another developer added a totally unneeded Replication framework for DTOs that was commited to the project repository and not the asylum where it belonged. Because this person is no longer with us it is my job to remove the junk.
Only good thing is no JUnit tests for this were added.... but that was also a bad thing.

Tuesday, 23 April 2013

A tale of detached Entities, identity and equals with JPA

I was asked to do update an old project of mine in order to accommodate some new requirements for a web application which manages manufacturing cycle times. Data about the resulting cycles is stored in a Database for analysis through another system.

The existing project was specifically designed with Cathode Dip Painting plants in mind and wasn't suitable for generic assembly line monitoring. In particular a requirement that the current cycle and production lot state be persisted through Web Application downtimes/restarts was new.

A majority of the new requirements were cosmetic and easily added without having to reach into the plumbing of the asynchronous progress state monitoring.

Yesterday however I added the cycle start persistence and suddenly everything broke. Granted I had things like telephone calls breaking my concentration but still I was flummoxed about why the Production Lot data was not being processed as expected.

Basically a Production Lot has a maxCycles and a currentCycle property.
When the currrent cycle completes, the currentCycle property is decremented. If this value is < 1, the current production lot is deleted and the next one is retrieved via JPA. The next cycle is then started.

So, being the optimist that I was and between phone calls I simply added a merge(ProductionLot entity).
This one line of code broke the app such that Production Lots were not being processed correctly.

After 2 hours bashing my head against the telephone I went home hoping that today would bring a fresh head and ideas to the problem.

This morning all sorts of light bulbs were lit up. In what way can an entity merge cause a problem like this... Consult the literature... lo and behold

The merge method of an EntityManager is used when you want to either update an Entity persistently OR when you want to attach the Entity to the current Persistent Context. It will also create a new Entity for the current Persistent Context if one does not exist (essentially equivalent to a persist method call). It also returns the Entity you merged. But, and this is what was causing me the problems, it returns the attached version of the entity and not the original.

Somewhere in my code I was checking for equality between the current Production Lot and one returned by the ProductionLot Stateless Session EJB. This worked before because the Production Lot only changed when the current one was exhausted and new one returned. In merging the current Production Lot I broke the semantics because the equals semantic was no longer strong enough.

So currentProductionLot != nextProductionLot && currentProductionLot.equals(nextProductionLot) need to be added.

This is something that is normally bread and butter to me but I still made the elemental mistake. Why?

Was it the telephone calls or the desire to leave before 19:00? No, actually it was a lack of preparation made when changing the actual semantics of the system.

On the face of things: never take for granted that a one line change will not profoundly change things. They will and the possibility should always be taken into consideration when making them.

Thursday, 7 February 2013

Bad (programming) habits instilled in me

What can happen when enforced terrible coding style practices become ingrained!


A company I used to work for had a "Style Sergeant" [ref: The Code Warriors] of the worst kind.
This person was allowed to enforce coding style rules on the rest of us as long as they could justify them.
In this case a stye rule for method local variable names was introduced on the premise that it shortened build times considerably.

The rule was simple, the names of the variables had to be as short as possible to remain unique with the accepted case being an abbreviation of what the variable was.

Now I am not going to get into the obvious fallacies here.
Build times went up because the code base was growing anyway.
Code readability went down the toilet. Mr Code Sergeant spent too much time refactoring existing code and committing it rendering existing code unreadable too.
Number of bugs introduced in new or changed code went up (it was expected that it would take time to get used to the new naming convention)

I was reviewing and refactoring some code I wrote a while back and was horrified that it took me so long to unravel what was going on. There was also a bug in the code that had the potential to cause a mountain of database orphans that was really hard to spot which was caused by the local names I was in the habit of using back then.

Here is the original code after refactoring for Java 7.

public PlantProperties replicate( PlantProperties toReplicate ) {
        Replicator<PlantProperties> rp = new Replicator<>();
        PlantProperties cpp = rp.replicate(toReplicate);
        PlantTypes pt = toReplicate.getTypeID();
        Replicator<PlantTypes> rpt = new Replicator<>();
        List<ProductionLot> lpl = toReplicate.getProductionLotList();
        if( lpl != null ) {
            try {
                List<ProductionLot> clpl = lpl.getClass().newInstance();
                Replicator<ProductionLot> rpl = new Replicator<>();
                for (ProductionLot productionLot : clpl) {
                    ProductionLot cpl = rpl.replicate( productionLot);
                    cpl.setPlantID(cpp);
                    clpl.add( cpl );
                }
            } catch (InstantiationException | IllegalAccessException ex) {
                Logger.getLogger(PlantPropertiesReplicator.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
        cpp.setProductionLotList(clpl);
        return cpp;
    }

Absolutely unreadable after 2 hours never mind 2 years.

This is the code after a refactor including the bug, the bug should become apparent almost immediately, at least it did to me.

public PlantProperties replicate( PlantProperties toReplicate ) {
        Replicator<PlantProperties> toReplicateReplicator = new Replicator<>();
        PlantProperties replicatedPlantProperties = toReplicateReplicator.replicate(toReplicate);
        List<ProductionLot> plantLotList = toReplicate.getProductionLotList();

        if( plantLotList != null ) {
            try {
                List<ProductionLot> copyOfPlantLotList = plantLotList.getClass().newInstance();
                Replicator<ProductionLot> plantLotReplicator = new Replicator<>();

                for (ProductionLot productionLot : copyOfPlantLotList) {
                    ProductionLot replicatedPlantLot = plantLotReplicator.replicate( productionLot);
                    replicatedPlantLot.setPlantID(replicatedPlantProperties);
                    copyOfPlantLotList.add( replicatedPlantLot );
                }

                replicatedPlantProperties.setProductionLotList(copyOfPlantLotList);
            } catch (InstantiationException | IllegalAccessException ex) {
                Logger.getLogger(PlantPropertiesReplicator.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
        return replicatedPlantProperties;
    }

In the for loop it turned out I was replicating the copy of the PlantLot list instead of the original PlantLot list.
By the way, the NetBeans <CTRL>+R refactoring of current selection makes this sort of thing trivial and was what led me to spotting the bug immediately.

The bug was that if the replicated version of the instance in question was used as a template for a potential Deletion Strategy then the PlantLot instances in the list would not be deleted as well.

Four things here


  1. If you are responsible for code style rules make sure they make sense. Good style means others like to look at what is created. Other reasons you can find here and here.
    You should have refused the responsibility anyway, this sort of thing is so totally bureaucratic and counter-productive
  2. Even though I have been free from the tyranny of the Style Sergeant, Nero and the others in my old dev team (yes, the blog I linked evoked some powerful memories) I STILL FIND MYSELF WRITING CODE USING THOSE UTTERLY AWFUL RULES!!!
  3. Protest vigorously if someone tried to oppress or repress you in this manner.
  4. This is not a rant against naming conventions which are very important but can also end up leading to a bureaucratic mess when overzealously applied. The Style Sergeant was applying the rule for coding style not as a naming convention. Not that it makes any difference, it is still utterly terrible.
The creepy thing is I only write like this when in the office. At home my code is (hopefully) self-documenting and more like the second example and I was compelled to do this refactoring while in the comfort and safety of my own living room!