Tuesday, April 15, 2014

NIO 2 in Apache Tomcat 8

There is a new NIO 2 based connector in Apache Tomcat 8 that is nearing reasonably useful status, being now labeled as beta. Besides NIO 2 aligning well with the asynchronous IO from Servlet 3.1, it is not its only benefit.


First, a quick speed test. Raw speed is measured using a Servlet writing 1KB of data, using ab -k -c 100 (keepalive enabled over 100 concurrent connections) so that it only measures the ability of doing a blocking write and a keepalive between two requests. Obviously a horrible real world benchmark, but the idea is only to see if NIO 2 is fast enough, since it does look kinda slow when you look at its high level API. This could have eliminated NIO 2 as a useful solution since a stable NIO connector already exists in Tomcat, while at the other end of the spectrum APR is available for raw speed. I am happy to report that NIO 2 is significantly faster than NIO for this pure blocking/polling stress test, up to about +50%, and is comparable to APR for that task.

After taking this most critical issue out of the equation, we have a connector that is more elegant than the current connectors, as poller management for NIO and APR, blocking IO for NIO, and native code for APR have been proven to be a seemingly endless source of complications / deadlocks / crashes / platform specific issues.

However, it is not known yet how good the real world scalability and resource consumption is, although some initial weak points can already be identified with JSSE and static file serving (see below). With thread and poller management being nearly completely abstracted away, the JVM has everything in its hands to provide an optimized behavior, eventually.

A simple API

Or is it ? Actually, only blocking IO is very simple with NIO 2. A read or write returns immediately like with NIO, but unlike NIO the operation does not have to be complete, it could still be in progress asynchronously. To represent that, the most basic read/write API uses a Future object that can be polled (bad idea) or blocked upon. So, simple blocking, with per operation timeout, looks great.

"Non blocking" as is called in Servlet 3.1, requires using the more complex API that uses completion handlers to notify that the operation is now done. That also still sounds simple, but the special cases need to be handled while the NIO 2 API does not provide everything to do that easily. A call can complete inline (or not, obviously), synchronization is not intuitive (there's no code block to sync on while an operation is pending, but evidently the state of some important objects like the buffers is undefined; the risk of deadlock is also present), incomplete operations are possible, etc.

The API does allow some more significant IO optimization, with scatter and gather. I tried taking advantage of the latter in Tomcat, with more work on that possible in the future.

How NIO 2 could be better

NIO 2 looks simple, fast and intuitive, but many things in it could still use some improvement.

Sendfile support

NIO transferTo API is not supported with NIO 2 asynchronous channels, and I don't see a good reason for that. As a result, although the NIO 2 connector raw speed is good and it can be fast enough in most cases, it is not the most efficient file server. Not critical, but since it is such a low cost thing to implement, it is unfortunate.

JSSE integration

It is the same as with NIO and uses the SSL engine API, which allows good control and non blocking. But everyone will do mostly the same asynchronous channel wrapper code. This JSSE channel code could have been included with NIO 2.

JSSE (non) speed

JSSE is still as slow as it used to compared with OpenSSL. Immunity to that bleed thing only gets you so far though, JSSE as it is now looks like a waste of server resources. However, this component of the JVM is pluggable, so we'll see if this can be improved in the future.

Better state handling

There is no way to do basic things like query the operation state when using the completion handlers, while it is included when using a Future. The pending flag should be available somewhere and should actually be an integrated semaphore shared with the future (to be able to wait for any pending operation to complete). In the end, although it does look intuitive and nothing is insurmountable, it ends up being more complex that it needs to.

So there is room for some nice additions in NIO 2.next, if it happens !

Monday, November 25, 2013

Websockets !

Some interesting developments in JBoss EAP 6.x, with the introduction of an experimental implementation of the Websockets 1.0 JSR, a specification introduced in Java EE7. Developers will thus be able to enable websockets support in their existing EE 6 application, a key capability of modern web applications, without having to immediately migrate to brand new application server implementations and having to do without support. Since it uses the Websockets JSR standard, the web application will then be able to run on EE 7 application servers when the time comes for the big migration.

In addition to that, the implementation is based on the websockets implementation introduced in Apache Tomcat 7 and has compatibility with its Servlet 3.1 packaged renamed async IO.

Enabling websockets is not going to be automatic though, as Websockets 1.0 is not part of the previous JavaEE standard.

Using it involves the following checklist:
  • Use a build of the JBoss Web container from the 7.4 branch with JBoss EAP 6.2; the resultant jbossweb JAR can be placed in the org.jboss.as.web module of EAP in place of the existing web JAR, either by replacing it or after editing the module descriptor to have it use the updated JAR. The JAR contains the Websockets 1.0 API classes as well as the implementation as it is meant to be used in a Java EE6 environment, so no other JAR is necessary.
  • Required runtime is Java 7 for the Websockets 1.0 implementation
  • Use either the APR/native connector or the NIO2 connector (the latter should be used by default in the Web 7.4 builds)
  • Enable Websockets in the webapp, by loading the Servlet container initializer of the websockets implementation
The last step can be done easily using a dedicated "descriptor" JAR containing the services definition. One appropriate can be found in the web container svn repository: http://anonsvn.jboss.org/repos/jbossweb/branches/7.4.x/res/enable-websockets.jar

Friday, April 09, 2010

JBoss Web 3.0.0 Beta 2 Released

Building on JBoss Web Beta1, this new release focuses on integration refactorings with JBoss AS, as well as trimming down the configuration profile used to focus on the standalone Servlet and JSP functionality. As a result, the download sizes and startup times are significantly reduced over Beta1, while allowing easily scaling up to JBoss AS when needed.

Looking forward to future releases, integration of EE 6 naming as well as annotation processing optimizations are planned.

As with Beta1, this release is Servlet 3.0 and JSP 2.2 compliant.

JBoss Web downloads page: http://www.jboss.org/jbossweb/downloads/jboss-web/

Wednesday, February 17, 2010

JBoss AS 6.0.0 M2 Released

This release is significant for web as it moves off the AS 5 development branch and now supports Servlet 3.0. As I have listed in the previous entries on JBoss Web 3, there is more than simply this new API that has been added.

One of the main refactorings has been around the integration of the JSP container. Although not the most trendy EE technology, JSP is still used in many applications, and its integration was a big issue. The main difficulty is that tag library descriptors (TLD) processing is serious business (possibly the most convoluted item in the entire EE, actually), where Jasper (the JSP container in Tomcat and JBoss Web) would end up doing its own parsing. Very hackish and fragile code too ... As the Servlet container also had to be aware of TLDs, this meant duplicating parsing. So a lot of work went into factoring this out of Jasper, and moving off all TLD processing to JBoss Metadata and AS deployers, and replacing Jasper's code with usage of metadata objects representing the TLDs. The end result is that, in addition to a massive speed improvement over AS 5 (for the first JSP compilation in a webapp), TLDs are now fully configurable using AS deployers. Pretty powerful.

Another work area has been around connectors, with the addition of async support in Servlet 3. Rather than adding a new code path to support it, I chose to simply take advantage of the HttpEvent API that was introduced in JBoss Web 2.1. Of course, there was a problem for the connectors which did not support events. So I separated the event functionality in two parts: the part that required polling and non blocking IO (which was not needed to implement Servlet 3's async), and the rest (suspend, resume and timeout, mostly). All connectors now have support for the later (with APR HTTP also supporting the former). Pretty painless overall. Except the dispatcher logic, but having it look like the other include and forward dispatchers seemed like a maintainable solution.

Last, lots of time went into supporting web.xml modularity, Servlet container initializers, and all the annotation scanning (injections and the new Servlet 3 features). That means lots of scanning. Although this is convenient for developers and library packagers, this also causes a war deployment time impact. To address that, the war deployers support scanning exclusions. Large legacy webapps can be tagged with a jboss-scanning.xml and/or started on demand. We did that for our own admin-console. This can allow keeping war deployment time in line with the web container in AS 5, although as users will need to do some homework (given the default is to be EE compliant), some will see apparent regressions here.

A lot of work remains for AS M3 and beyond, including extracting a JBoss Web standalone packaging which makes sense in terms of size and feature set.

For the general AS 6.0.0 M2 release announcement, see Brian's blog entry.

Wednesday, January 06, 2010

JBoss Web passes standalone Servlet 3.0 TCK

The JBoss Web team is happy to report that the standalone TCKs for Servlet 3.0, JSP 2.2 and EL 2.2 now pass on JBoss Web 3.0.0 Beta 1.

JBoss Web downloads page: http://www.jboss.org/jbossweb/downloads/jboss-web/

Friday, December 18, 2009

JBoss Web 3.0.0 Beta 1 Released

The first beta release leading to JBoss Web 3.0 is now out. This release provides support for Servlet 3.0, JSP 2.2 and EL 1.2. Departing from the standalone packaging used in JBoss Web 2.x, JBoss Web 3.0 is based on JBoss AS, and also serves as a preview of the web capabilities that will be included in JBoss AS 6.0.

Major changes include:
- Jasper now integrates with JBoss VDF, for much faster JSP initialization
- Servlet 3.0 support: async, pluggability, annotations, servlet container initializer, new APIs, overlays
- JSP 2.2 support
- Expression Lanaguage 1.2 support
- Implement Servlet 3.0 async functionality on top of JBoss Web event mode
- Add IO-less event mode to all connectors
- Rebased commons fileupload
- Lots of other fixes and improvements

Work will continue in future builds to optimize the packaging, focusing releases on Servlet & JSP, and produce corresponding documentation.

Downloads page: http://www.jboss.org/jbossweb/downloads/jboss-web/

Tuesday, February 17, 2009

JBoss Web 2.1.2 GA Released

A new minor release of JBoss Web is now available, and will be integrated in the upcoming JBoss AS 5.0.1. In addition to many bugfixes, this release extends functionality of the URL rewrite valve.

Downloads page: http://www.jboss.org/jbossweb/downloads/jboss-web/