• download PagedList

Lazy loading with the PagedList

Originally I made the PagedList because I needed paged data fetching using JMS queues for a JSF Rich datatable and datascroller; a common problem in JSF but less commonly solved, even less documented and less still solved in an elegant way. Most people are able to solve this problem by using an OpenSessionInView antipattern (keeping Hibernate-oid session open until the view requests the lazy loaded data) but since I’m using JMS queue, this was not an option. I decided to circumvent the entire JSF to JMS queue integration by providing my data in a paging lazy list. It works beautifully.

public class PagedList<Dto, QueryParameters> extends AbstractList<Dto> {

    private final QueryParameters queryParameters;

    private final PagedDataProvider<Dto, QueryParameters> pagedDataProvider;

    private final Map<Integer, List<Dto>> fetchedPages;

    private final int dataSize;

    private final int pageSize;

    public PagedList(PagedDataProvider<Dto, QueryParameters> pagedDataProvider, QueryParameters queryParameters) {
        this.pagedDataProvider = pagedDataProvider;
        this.queryParameters = queryParameters;
        fetchedPages = new HashMap<Integer, List<Dto>>();
        dataSize = pagedDataProvider.getDataSize();
        pageSize = pagedDataProvider.getPageSize();
    }

    public Dto get(int index) {
        int pageNr = (int) Math.floor(index / pageSize);
        if (fetchedPages.get(pageNr) == null) {
            fetchedPages.put(pageNr, pagedDataProvider.provide(pageNr, queryParameters));
        }
        return fetchedPages.get(pageNr).get(index % pageSize);
    }

    public int size() {
        return dataSize;
    }
}

It is different from Apache’s LazyList (and many other implementations) in that it relies on an external dataprovider to provide the dataset’s total size, pagesize and data provision itself. For example, using my lazy list version the size() method actually returns the total size of data potentially available, not just physically available in the list at the present. This way the list works completely transparent to its users. Also, there is no simple factory being passed in that creates dummy objects or something: actual remote data is being fecthed in pages by a data provider. Pages being fetched are independent of each other; when requesting page 5, page 1 through 4 are not need or fetched.

The PagedList works in combo with a PagedDataProvider interface that external dataproviders (some dao for example) could implement. So it has no dependencies whatsoever on some specific implementation. The paged data provider provides the total size of the dataset, the page size and pages of the dataset itself when needed.

Example implementation for JSF and JPA dao

Here’s an example implementation which ties a JSF Richfaces datatable to a JPA dao using a PagedList. I cut out the service layer, view handlers/controllers and whatnot for sake of simplicity.

<rich:dataTable id="searchResults" value="#{appleService.apples}" var="apple">
...
</rich:dataTable>

/**
 * JPA dao which acts as paged data provider for the paged list.
 *
 * Relies on two JPA named queries specified on the Entity being fetched.
 *
 * @author Benny Bottema
 * @see PagedDataProvider
 */
public class AppleJPADao implements AppleDao, PagedDataProvider<AppleDto, AppleQueryParameters> {

    // entitymanager stuff omitted for example (real world: injected by Spring)

    private static final int PAGESIZE = 50;

    /**
     * @see AppleDao#getAllApples(String, String)
     */
    public List<AppleDto> getAllApples(String type, String quality) {
        // optional query parameters: can be null
        AppleQueryParameters params = new AppleQueryParameters();
        params.setAppleType(type);
        params.setAppleQuality(quality);
        return new PagedList<AppleDto, AppleQueryParameters>(this, params);
    }

    /**
     * @see PagedDataProvider#provide(int, Object)
     */
    public List<AppleDto> provide(int page, AppleQueryParameters queryCriteria) {
        Query query = entityManager.createNamedQuery("Apple.findAll");
        query.setParameter("appleType", queryCriteria.getAppleType());
        query.setParameter("appleQuality", queryCriteria.getAppleQuality());
        // JPA's paging mechanism
        query.setFirstResult(PAGESIZE * page);
        query.setMaxResults(PAGESIZE);
        return (List<AppleDto>) query.getResultList();
    }

    /**
     * @see PagedDataProvider#getDataSize()
     */
    public int getDataSize() {
        Query countQuery = entityManager.createNamedQuery("Apple.count");
        return ((Long) countQuery.getSingleResult()).intValue();
    }

    /**
     * @see PagedDataProvider#provide()
     */
    public int getPageSize() {
        return PAGESIZE;
    }
}

Known issues

So far there is only one known issue, which is more of a problem with lazy loading in general: synchronizing the lazy loading list with the remote dataset to avoid becoming stale when changes happen to the dataset. Example: when the dataset changes, the size of the list is not being updated and the get() method may try to fetch wrong or even unavailable data throwing an exception. Since I’m using the paged list in a limited way, I’ve not yet encountered this problem.

One way would be to solve this problem for the Rich datable and datascroller specifically is to have a a4j poll component that listens for dataset changes on the server and refreshes the entire datatable as necessary. Just thinking out loud here…

trackback

[RE: JSF – Still pretty much a steaming pile of donkey turd] – I was replying to Wille Faler’s post about why JSF sucks, when the comment was getting too large, so I made it into a post on my own blog.

JSF is hard to learn yes, but in my opinion it doesn’t suck as bad as Wille says. All the points Wille mentions can be solved by certain libraries or write-once reusable solutions (I’ve included my ‘magical’ combination of frameworks on the bottom). JSF is a diamond in the rough, you just need to make it shine.

The devil is in the details

There are a couple of pitfalls you need to know about with JSF. It’s all really a matter of having read a good book about JSF that points them out, or having learned them the hard way. Luckily I’ve had some guidance by experienced colleagues of mine; I’m guessing that’s the missing factor in Wille’s situation.

Referring to the pitfalls you need to know about, I’m talking about the common beginner’s problems like using JSTL c:tags that cause havoc when combined with those of jsf and facelets. I’m talking about the Ajax4Jsf details as Wille mentioned: once you know what to use it’s easy. Those 48 attributes? most of them are inherited and never used and you can forget about them, and that’s the learning curve. Once you know the right ones, it’s using exactly those, over and over again. And things like how you can incorporate Spring security and stuff like that. Another pitfall that comes to mind is how action=”" work quirky with navigation rules when using the array notation (action=”controller['action']“). And ofcourse there’s the issue of parameterized JSF validators, which needs some understanding. All pitfalls that can baffle a beginner or intermediate.

The thing with JSF is there are a number of little things that are hard to solve, but once you’re past that it does make you much more productive. You just have to have a solid base to start from each project. The pitfalls learning curve is definitely a downside of JSF, but JSF 2.0 is in the making to solve all these issues.

Bad performance or beginner’s mistake?

Then there is this performance issue Wille mentions, where I think he missed the target: JSF does not have to be slow at all. It really depends on how you treat your backing beans. JSF has the weird tendency to call setters/getters multiple times, which seems utterly pointless (it’s related to resolving valuebindings in nested components). I’ve made the mistake before to directly retrieve values from a controller/service in my views instead of a static model (backing bean): this is what makes things slow. JSF in itself isn’t that slow for a webapp: yes there are many redundant calls but we’re not dealing with a realtime simulation application, webapps do just fine. On a sidenote, it also depends on the implementation… JSF is a reference specification, while there are various implementations of the framework (IBM’s, Sun’s etc.)

SEO friendly JSF?

Ahh, but for the JSF Search Engine Friendliness. That’s solved too, somewhat. There’s is a framework called PrettyFaces which allows you to dynamically generate/interpret pretty GET urls with bean values. It’s a great framework that solves a common problem, but there still exists the problem that existing jsf/facelets/richfaces components are not generating SEO HTML code. That’s really the fault of the component makers, not JSF, though JSF definitely isn’t innocent with its POST obsession: PrettyFaces fixes at least that.

Common sense: use frameworks to avoid most pitfalls

Once you’ve got those kind of things down and use a good framework combo, JSF is straightforward. You won’t really need to know about the various phases and you can be very productive if you’ve set up the jsf/facelets components right. I’ve worked with JSF intensively for about a year now in various projects with various implementations and I hardly ever need to use phaselisteners and that sort of stuff. The rare cases where you do need them, you can define these things ones and reuse them in other projects. With this approach I’ve created a template project with some standard libraries, phaselisteners, security set up, facelets components etc. and I’m up and running in no time.

I haven’t sanitized or otherwise cleaned up the template project so I won’t put that up for the time being, but at least I can tell you what frameworks are in it by default.

The magical combination that works for me and my colleagues:
• JSF (MVC)
• Facelets (Templating, use of XHTML)
• Richfaces (component suite)
• Richfaces A4J (Ajax support)
• Tomahawk (only for file uploads with the Extensions Filter)
• PrettyFaces (for indexable GET urls)

Now that I’ve mentioned these, you should check out JBoss Seam as well, which basically is supposed to do everything the above libraries do combined, plus a couple of things more like solving the conversational scope issue. I haven’t worked with this framework myself though so I can’t speak from experience.

Final note

Although I agree JSF has some annoying pitfalls, shortcomings and a high learning curve, it’s been out there for enough time to pass for supplement frameworks to pop up from under the ground to solve most of these problems.

With the right combination, most classic counter arguments have become obsolete. Maybe it’s time for critics to take JSF as is: incomplete but ultimately making you more productive if combined with the right frameworks.

trackback

Prologue

So I was looking into sending some emails from my little Java webapp, right. I thought: easy, java has built in mail support called JavaMail. Well, I was in for an unpleasant surprise when I found out I was merely handed the sand grains to build my castle. I can’t just say to Java’s mailing library Hey, here’s what I want to send, and here are the addresses I need it sent to, no. I can say however I want you to formulate a mixed multipart message with the following attributes and these addresses encoded into special objects and oh yeah, define some mimemessage bodyparts for these attachments and use the following headers and put everything in a very specific nested structure dot dot dot.

You actually need to research the RFC for mail structures to do it properly! In case you’re interested (which you’re not), check out this page of useful email rfc’s. The rfc’s 822, 2047 and 2045 are referenced in Java’s MimePart alone. That’s just stupid, I can’t think of anything else where the JDK is forcing developers to work on such a low level where you need to deal with rfc’s.

An example of the JavaMail madness can be found on a thread at velocityreviews.com, where someone asks for a simple tutorial to learn how to simply send emails using JavaMail. Here’s the answer of one of the members:

First, you need to read and understand the relevant RFCs governing Internet
email, if you don’t already understand this. I’d recommend starting with:

RFC 2821 – Simple Mail Transfer (SMTP)
RFC 2822 – Format of Internet Messages
RFC 1939 – Post Office Protocol (POP3)
RFC 2060 – Internet Mail Access Protocol (IMAP)

then moving on to MIME:

RFCs 2045 – 2049 – Multipurpose Internet Mail Extensions (MIME)

Once you understand this, you should be able to figure out JavaMail using
the tutorial at http://java.sun.com/developer/onlineTraining/JavaMail/contents.html,
(which is, incidentally, the first hit returned from a Google search on the
term “javamail tutorial”), along with the JavaDoc that is shipped along
with the JavaMail libraries. If you have any specific questions about how
JavaMail is to be used, please post them to the news group and you’ll be
pretty sure to get an answer.

Cheers!

With all his good intentions, this guy doesn’t realize he’s completely scaring the mail novice away; people looking for a way to send emails shouldn’t be required to be RFC educated email engineers. I just now realize that maybe JavaMail wasn’t meant to be used by Joe sixpack, but rather by third-party library vendors. Also, realize that JavaMail is much more than just smtp client API. Still, when I began writing Vesijama there weren’t a whole lot of third-party options, let alone one as simple to use as Vesijama.

Don’t get me wrong though: it’s not that the JavaMail API is so bad, it’s great actually. It allows for very finegrained configuration for pop3, smtp and everything mail related. It’s just that you don’t want regular Java developers to deal with API on such a low level: it takes a lot of time getting it right and the risk of a faulty implementation is very high, as the result will often work in some email (web) clients and subtle mistakes with significant consequences will go unnoticed.

Enter Vesijama, the Very Simple Java Mail library

To spare other developers the same fate, I decided to write a proper wrapper around the JavaMail smtp API so no Java developer needs to be bothered by this stuff and just keep pluggin’ on an abstract business logic level. You know, just send the damn mails.

It took me quite a while to get it right since email clients, especially web clients, are very sensitive to incorrect nested structures. I’m happy to say I’ve succesfully used this when I was working for a commercial company, which means it has been professionally tested with a broad range of email (web) clients.

Enter Vesijama. It´s been a while since I wrote this little framework, I´ve only recently had the time to properly prepare the code for an Open Source release. At the time of writing the library I didn’t know about Spring’s mailing facility or Apache’s Commons Mail libraries, but in hindsight I think there’s a place for Vesijama since in my completely biased opinion it is by far the easiest to use framework. Look it up over at Vesjiama at Google Code.

Here’s the one and only code example from the Vesijama project page:

final Email email = new Email();

email.setFromAddress("lollypop", "lolly.pop@somemail.com");
email.setSubject("hey");
email.addRecipient("C. Cane", "candycane@candyshop.org", RecipientType.TO);
email.addRecipient("C. Bo", "chocobo@candyshop.org", RecipientType.BCC);
email.setText("We should meet up! ;)");
email.setTextHTML("We should meet up!");

// embed images and include downloadable attachments
email.addEmbeddedImage("wink1", imageByteArray, "image/png");
email.addEmbeddedImage("wink2", imageDatesource);
email.addAttachment("invitation", pdfByteArray, "application/pdf");
email.addAttachment("dresscode", odfDatasource);

new Mailer("smtp.host.com", 25, "username", "password").sendMail(email);

Oh wow, is that it? Well, I just imagine masses of people gazing with open mouths thinking that, but that’ll never happen because nobody realizes how much effort went in the rfc research. And the very few lines that make up the mail structure don’t look very impressive either… the thing is it has to be just right; mail clients aren’t very forgiving. It often amazes me how many thousands of lines of rfc text results in miniscule implementations, but luckily in this case at least if you do it right, it works in all email clients that properly handle emails according the rfc’s.

Anyway, the library suited me and now it’s yours too. Enjoy.

trackback

This isn’t yet another blogpost about what DWR exactly is or why it is so cool; the objective of my post today is to simply set up a quick example of how to work with it. A quick tutorial if you will. I’m the kind of guy that likes many examples and diagrams and whatnot to take apart and learn from, so here’s my contribution to the ‘code by example’ paradigm. If you want to know more about DWR before actually trying to use it, I suggest you check out the DWR homepage.

Setting the stage

One of the cool things (two sentences and I already contradict myself :), is that it is so easy to setup, as you will soon see. To get DWR to expose the classes to javascript, you first need to add the dwr library and define the specific classes. Basically, this is a 3-step dance:

1. Drop the dwr.jar you’ve downloaded from getahead in the lib folder of your webapplication (ie. \lib\dwr\dwr.jar).
2. Add DWR to your web.xml as a servlet:

   <servlet>
   	<servlet-name>dwr-invoker</servlet-name>
   	<display-name>DWR Servlet</display-name>
   	<servlet-class>uk.ltd.getahead.dwr.DWRServlet</servlet-class>
   	<init-param>
   		<param-name>debug</param-name>
   		<param-value>true</param-value>
   	</init-param>
   	<load-on-startup>1</load-on-startup>
   </servlet>
   
   <servlet-mapping>
   	<servlet-name>dwr-invoker</servlet-name>
   	<url-pattern>/dwr/*</url-pattern>
   </servlet-mapping>

3. Create a dwr.xml, which informs DWR about the classes you need exposed:

   <dwr>
   	<allow>
   		<create creator="new" javascript="MyService">
   			<param name="class" value="org.foo.MyServiceClass"/>
   		</create>
   	</allow>
   </dwr>

Testing DWR

To check if you’ve done everything right, you can open up the DWR testpage http://localhost/dwr/ on which all exposed classes and methods should be visible. Remember, this works because you configured a DWR servlet that triggers on /dwr/*. If you click on any of the classes that are exposed, you’ll get a list of methods that are available to you javascript and are ready for testing right away. Note that this is all javascript already!

Specifying nothing but /dwr/ results in this testpage. But If you want to use a specific java class in your own page to call methods on, you’d call /dwr/TheService. What happens is that the DWR library dynamically generates all the required javascript to communicate with the server through this servlet, runtime. Pretty cool huh?

Using DWR on the client side

To use the methods of the classes that are exposed through dwr.xml, you need to include a virtual javascript file with the name of such an exposed class. Since the classes are identified with the javascript attribute in dwr.xml, you’ll use that identifier when importing the javascript file. In addition to the scriptfile for your javaclass, you need at least engine.js as well.

<script src="/dwr/interface/MyService.js" mce_src="/dwr/interface/MyService.js"></script>
<script src="/dwr/engine.js" mce_src="/dwr/engine.js"></script>

There are some other useful scripts you can include from DWR, such as util.js, which has a couple of nifty functions like $() (a shorthand version for document.getElementById()).

About DWR callbacks

To actually make use of the Java serviceclass, you simply call a method on the Myservice.js file. The catch here is that you need to provide a callback function as well. This is because of the fact that javascript runs asynchronously in the client and doesn’t wait for Java to come back with an answer. Instead, DWR relies on the callback method you defined that will handle the result in due time. Note that the function you call on the serviceclass.js has all the parameters as defined in the Java class, safe for the callback parameter and peripheral parameters such as HttpSession or HttpRequest.

The callback function can only have one parameter, data, which bares the result of the DWR servicecall.

Take a look:

// call function on DWR generated service.js
MyService.getInfo(arg1, arg2, parseResult);

// callback function that parses whatever result comes back
function parseResult(data) {
  // do stuff with result data
}

On a side note, there are a couple of varying ways of calling a function on a serviceclass. For example, you can have the callback function as first or last argument, or not at all. You can also specify a time-out and some other options. For more details on check out the DWR page on this usage subject.

Matching arguments types

DWR already provides most common conversions such as Strings, Dates, Arrays (nested as well) and a host of other types, including stuff like Vector. If you’d like to use your own type for a parameter, you will need to create a javascript version with the same fields. Here’s an example:

// Java version:
public Person {
	private String name;
	private int age;
	private Date[] appointments;

	// getters and setters ...
}

// javascript version:
var person = {
	name:"Fred Bloggs",
	age:42,
	appointments:[ new Date(), new Date("1 Jan 2008") ]
};

MyService.setPerson(person);

To make these conversions possible, you need to inform DWR about these ‘custom’ classes. These so called ‘beans’ allow DWR converters to marshal these classes from/to javascript.

More on DWR converters here:

• http://getahead.ltd.uk/dwr/server/dwrxml/converters
• http://getahead.ltd.uk/dwr/server/dwrxml/converters/bean
• http://getahead.ltd.uk/dwr/server/dwrxml/converters/collection

 Slashdot It!   /   trackback