[benchmark] myfaces codi scopes - weld vs owb part 3

in the previous benchmark i basically compared the scope performance of owb and weld with direct calls. this time i'm using jmeter to measure the performance under heavy load of a whole page. in the previous benchmarks the results were pretty stable. that's different this time. the results fluctuate way more and with some servers i (sometimes) got an OutOfMemoryError (depending on the settings). however, the conditions for the benchmark were the same for all servers:
- starting the server
- opening the page (which will be used for the jmeter tests)
- specifying how many el-expressions should be tested
- starting the jmeter benchmark

pages with 100, 1.000, 5.000 and 10.000 el-expressions were used for the benchmark.

all application servers were started without changes in the configuration. in tomcat mojarra 2.1.3 was used instead of myfaces-core to avoid an impact on the result e.g. due to a differing performance of renderers.

every value-binding in the page queried the value of a conversation scoped bean (codi conversations). this bean delegates the call to an injected request scoped bean. and the request scoped bean delegates to an application scoped bean which caches the list of objects which should be displayed.

as you know the results depend on a lot of parameters. anyway, the following charts should give you a basic impression about the difference. on my system i can reproduce the overall results.

the result:
esp. tests of pages with few el-expressions didn't show a huge difference between the servers, but the results fluctuated more compared to pages with a lot of el-expressions. as you see, tomcat 7 with owb is a bit better than tomcat 7 with weld and jboss as 7. in the end the difference is not big compared to glassfish v3.1.1. the following chart should give you a basic idea about a page which uses 100 value-bindings. that might sound a lot, but a page which contains a table with 4 columns and 15 rows already leads to 60 el-calls during the rendering phase only to display the values (in case of one displayed value per cell). and if you think about the rendered attribute or expressions to switch the formatting within cells dynamically,... you will get over 100 el-calls in the rendering phase very easily.

(the higher - the better)

as mentioned before i benchmarked the page with different amounts of el-expressions. the following chart shows the average of the throughput in percent. the fastest setup (in this case tomcat 7 + owb 1.1.0) is used as measuring staff.

(the higher - the better)

it might be interesting to see the results of the benchmarks above in a different context. the following charts shows the overhead in percentage. a static html page with the same content served by jsf on tomcat 7 is the basis for the comparison and would have 0% (that means: no overhead).

(the lower - the better)

measurements beyond hello word:
there are applications with even larger pages. the following chart shows the numbers for 30.000 value bindings per page. compared to the value-bindings of the benchmark above, it uses more complex bindings like
#{bean1.bean2.bean3.property} instead of directly delegating to a property of the first bean which resolves the final property. as you see owb gets even better with more complex scenarios.

(the higher - the better)

as mentioned before i used the settings which are shipped by the vendors out-of-the-box. since they are different i run the same constellation described above with 1.000 (instead of 30.000) value bindings per page. the following chart shows the result with -Xms128m -Xmx1500m -XX:MaxPermSize=256m (these values are the highest values used at least by one of the servers).

(the higher - the better)

codi meets scala

werner wrote nice blog entries about using scala in combination with jsf. in the second part he shows the basic approach how to implement the type-safe view-config the scala way. let's hope we will see a follow-up about nice view-config features like multiple inheritance of configs (like a SecuredPages interface).

[benchmark] myfaces codi scopes - owb vs weld part 2

the last few days we heard a lot about how great jboss as 7 (jboss application server v7) should be. i already tested the pre-release version concerning the compatibility with codi. it worked pretty well. that was awesome. in this test i'm using the final version which starts in a bit more than 2 sec. (on my machine). but now it's time to measure the runtime performance. therefore i used the latest releases of all containers. (just in case of glassfish i used the nightly of glassfish v3.2 to be able to use the latest version.) in case of jboss as 7 and glassfish v3 i took the weld versions they provide out of the box and on tomcat 7 and jetty 7 i used the current openwebbeans (= owb) release.

the result - jboss as 7 starts quite fast but the runtime performance is still better with a servlet container like tomcat or jetty + owb. however, i'm glad to see that there is a step forward from weld v1.1.1 to v1.1.2 but there is still a lot of room for improvements.

the tested scopes are the scopes provided by myfaces codi (with the exception of the request scope - it's just included as an orientation). the most significant result in the first chart is that the slowest scope (which is still very fast) is faster with owb than the fastest scope with weld. (if you keep the tolerance in mind we could say it's maybe not faster but as least equivalent.)

as you know the results depend on a lot of parameters. anyway, the following charts should give you a basic impression about the difference. on my system i can reproduce the results (tolerance: <0,25ms). the benchmark invokes a method of an injected bean 1000 times. the benchmark was started 3 times and the chart shows the rounded average.

as you see there are more or less the same results in tomcat and jetty. the results depend on the cdi version and not the container. the differences are within the tolerance. the difference between glassfish and jboss as 7 is caused by the newer version of weld in jboss as 7. so independent of how fast jboss as 7 is (e.g. during startup), the runtime-performance depends on parts like the used cdi implementation. this benchmark shows what you get out-of-the-box. for sure with servlet containers like tomcat, jetty,... you have the choice. if performance is important for your project, go with owb (it's already used by several big real world projects, it's fast, very solid and provides a powerful spi to customize the behaviour if you need to).


so it might be interesting to compare the currently released versions directly:

note:
owb has a special proxy for request scoped beans - that's the reason why there is such a huge difference.

codi for all - all for codi

it's really great. people notify us about successful tests on different environments.

now codi is tested on:
- Tomcat
- TomEE
- Jetty
- JEE 6 Application-Servers (JBossAS, WebSphere, Glassfish)
- JEE 5 Application-Servers (WLS, WebSphere,...)
- dmServer

an up to date list and further details are available here.

7th release of myfaces-extensions-cdi -> v1

The Apache MyFaces team is pleased to announce the 7th release of Apache MyFaces Extensions CDI (aka CODI). It's v1.0.0!