Sunday, July 17, 2011

[benchmark] myfaces codi scopes - weld vs owb part 3

in the previous benchmark i basically compared the scope performance of owb and weld with direct calls. this time i'm using jmeter to measure the performance under heavy load of a whole page. in the previous benchmarks the results were pretty stable. that's different this time. the results fluctuate way more and with some servers i (sometimes) got an OutOfMemoryError (depending on the settings). however, the conditions for the benchmark were the same for all servers:
- starting the server
- opening the page (which will be used for the jmeter tests)
- specifying how many el-expressions should be tested
- starting the jmeter benchmark

pages with 100, 1.000, 5.000 and 10.000 el-expressions were used for the benchmark.

all application servers were started without changes in the configuration. in tomcat mojarra 2.1.3 was used instead of myfaces-core to avoid an impact on the result e.g. due to a differing performance of renderers.

every value-binding in the page queried the value of a conversation scoped bean (codi conversations). this bean delegates the call to an injected request scoped bean. and the request scoped bean delegates to an application scoped bean which caches the list of objects which should be displayed.

as you know the results depend on a lot of parameters. anyway, the following charts should give you a basic impression about the difference. on my system i can reproduce the overall results.

the result:
esp. tests of pages with few el-expressions didn't show a huge difference between the servers, but the results fluctuated more compared to pages with a lot of el-expressions. as you see, tomcat 7 with owb is a bit better than tomcat 7 with weld and jboss as 7. in the end the difference is not big compared to glassfish v3.1.1. the following chart should give you a basic idea about a page which uses 100 value-bindings. that might sound a lot, but a page which contains a table with 4 columns and 15 rows already leads to 60 el-calls during the rendering phase only to display the values (in case of one displayed value per cell). and if you think about the rendered attribute or expressions to switch the formatting within cells dynamically,... you will get over 100 el-calls in the rendering phase very easily.

(the higher - the better)

as mentioned before i benchmarked the page with different amounts of el-expressions. the following chart shows the average of the throughput in percent. the fastest setup (in this case tomcat 7 + owb 1.1.0) is used as measuring staff.

(the higher - the better)

it might be interesting to see the results of the benchmarks above in a different context. the following charts shows the overhead in percentage. a static html page with the same content served by jsf on tomcat 7 is the basis for the comparison and would have 0% (that means: no overhead).

(the lower - the better)

measurements beyond hello word:
there are applications with even larger pages. the following chart shows the numbers for 30.000 value bindings per page. compared to the value-bindings of the benchmark above, it uses more complex bindings like
#{} instead of directly delegating to a property of the first bean which resolves the final property. as you see owb gets even better with more complex scenarios.

(the higher - the better)

as mentioned before i used the settings which are shipped by the vendors out-of-the-box. since they are different i run the same constellation described above with 1.000 (instead of 30.000) value bindings per page. the following chart shows the result with -Xms128m -Xmx1500m -XX:MaxPermSize=256m (these values are the highest values used at least by one of the servers).

(the higher - the better)