Among his Cocoon ramblings, Steven points to a hidden gem on my harddisk: ab, the Apache HTTP server benchmarking tool. Very useful to quickly test a server's performance.

But does the Cocoon flow add soo much processing time as Steven mentions? 70 milliseconds is huge on a system which seems otherwise able to serve 100 requests per second from a java program.

Let's see - I've uploaded my (very simple) benchmark to the Cocoon Wiki if you want more details.

Let's start with a small (127 bytes) XML document.

The document paths below explain what's going on: raw XML generation + serialization, then a simple XSLT identity transform, then four times the same XSLT in the pipeline, and finally the same requests but started with a minimal Flowscript function which just calls cocoon.sendPage. All this runs on a non-caching pipeline, looking for worst-case times.

Document Path: /benchmarks/noflow/noXsl/smallfile.xml Requests per second: 77.76 [#/sec] (mean) Time per request: 12.86 [ms] (mean)

Document Path: /benchmarks/noflow/oneXsl/smallfile.xml Requests per second: 33.66 [#/sec] (mean) Time per request: 29.70 [ms] (mean)

Document Path: /benchmarks/noflow/fourXsl/smallfile.xml Requests per second: 9.13 [#/sec] (mean) Time per request: 109.47 [ms] (mean)

Document Path: /benchmarks/flow/noXsl/smallfile.xml Requests per second: 56.61 [#/sec] (mean) Time per request: 17.67 [ms] (mean)

Document Path: /benchmarks/flow/oneXsl/smallfile.xml Requests per second: 26.95 [#/sec] (mean) Time per request: 37.11 [ms] (mean)

Document Path: /benchmarks/flow/fourXsl/smallfile.xml Document Length: 127 bytes Requests per second: 9.81 [#/sec] (mean) Time per request: 101.90 [ms] (mean)

So even though the Flow call adds a few milliseconds, it's not that bad as what Steven sees - but my Flow is doing nothing interesting of course, it's just

function flowToPage() { page = cocoon.parameters["page"]; cocoon.sendPage(page); }

To find out about the overhead of calling Flow.

The last test is actually faster with Flow than without, but it might just be that my system was busy at times during these tests, I won't get deep in the analysis with so few samples. Anyway, here the minimal Flow call does not seem to make a big difference.

On the other hand, the XSLT transform is much more costly, and as Steven mentions, chaining transforms is the big time eater in the above test.

Let's see what happens with a much larger (108 kBytes) XML document:

Document Path: /benchmarks/noflow/noXsl/bigfile.xml Requests per second: 20.58 [#/sec] (mean) Time per request: 48.60 [ms] (mean)

Document Path: /benchmarks/noflow/oneXsl/bigfile.xml Requests per second: 8.60 [#/sec] (mean) Time per request: 116.23 [ms] (mean)

Document Path: /benchmarks/noflow/fourXsl/bigfile.xml Requests per second: 2.86 [#/sec] (mean) Time per request: 350.06 [ms] (mean)

Document Path: /benchmarks/flow/noXsl/bigfile.xml Requests per second: 15.39 [#/sec] (mean) Time per request: 64.96 [ms] (mean)

Document Path: /benchmarks/flow/oneXsl/bigfile.xml Requests per second: 8.06 [#/sec] (mean) Time per request: 124.04 [ms] (mean)

Document Path: /benchmarks/flow/fourXsl/bigfile.xml Requests per second: 2.46 [#/sec] (mean) Time per request: 406.42 [ms] (mean)

The results are similar, except for the last test where Flow seems to add 50 msec to the request. But rerunning the test a few times, I got down to 360 msec, which is only 10 milliseconds more than the noFlow version.

Obviously this would require much more samples to be really meaningful: if you want to find out more, grab the code from the wiki and let me know what you find!

Steven is certainly right about chained XSLT processing having a big impact on performance, but I'm not sure what's happening with his Flow code: my tests show that the overhead of calling Flow is not that bad, so it must be something in his Flow code that makes the big 70 msec difference. Question is, does any Flow instruction add X msec, or is something wrong with a particular construct or component?

All this can be hidden by clever caching, which Cocoon is very capable of, so this shouldn't block you from writing this killer web app with Cocoon. But it's always good to know thy enemy: as Mom said, don't forget to test, profile, benchmark, tune the cache, and brush your teeth!

Update: Steven indicates having compared the same simple code written in Flow and in Java, so it looks that the Flow interpreter might well be the problem (actually I think this was clear in his original writings, but I was in an investigating mood this morning ;-)

Another reason to keep your Flow short and sweet! And another indication that the javascript Flow might not be the final solution...