Tuesday, July 12, 2011

[benchmark] myfaces codi scopes - owb vs weld part 2

the last few days we heard a lot about how great jboss as 7 (jboss application server v7) should be. i already tested the pre-release version concerning the compatibility with codi. it worked pretty well. that was awesome. in this test i'm using the final version which starts in a bit more than 2 sec. (on my machine). but now it's time to measure the runtime performance. therefore i used the latest releases of all containers. (just in case of glassfish i used the nightly of glassfish v3.2 to be able to use the latest version.) in case of jboss as 7 and glassfish v3 i took the weld versions they provide out of the box and on tomcat 7 and jetty 7 i used the current openwebbeans (= owb) release.

the result - jboss as 7 starts quite fast but the runtime performance is still better with a servlet container like tomcat or jetty + owb. however, i'm glad to see that there is a step forward from weld v1.1.1 to v1.1.2 but there is still a lot of room for improvements.

the tested scopes are the scopes provided by myfaces codi (with the exception of the request scope - it's just included as an orientation). the most significant result in the first chart is that the slowest scope (which is still very fast) is faster with owb than the fastest scope with weld. (if you keep the tolerance in mind we could say it's maybe not faster but as least equivalent.)

as you know the results depend on a lot of parameters. anyway, the following charts should give you a basic impression about the difference. on my system i can reproduce the results (tolerance: <0,25ms). the benchmark invokes a method of an injected bean 1000 times. the benchmark was started 3 times and the chart shows the rounded average.

as you see there are more or less the same results in tomcat and jetty. the results depend on the cdi version and not the container. the differences are within the tolerance. the difference between glassfish and jboss as 7 is caused by the newer version of weld in jboss as 7. so independent of how fast jboss as 7 is (e.g. during startup), the runtime-performance depends on parts like the used cdi implementation. this benchmark shows what you get out-of-the-box. for sure with servlet containers like tomcat, jetty,... you have the choice. if performance is important for your project, go with owb (it's already used by several big real world projects, it's fast, very solid and provides a powerful spi to customize the behaviour if you need to).

so it might be interesting to compare the currently released versions directly:

owb has a special proxy for request scoped beans - that's the reason why there is such a huge difference.