Comparing Component speed on different CFML engines
Posted At : November 27, 2006 10:54 AM
| Posted By : Mark Drew
Related Categories:
coldfusion
After my previous couple of posts of comparing different speeds for instantiating structures vs components, I thought I would measure the same process on different Coldfusion engines.
I set up a small test that basically logs speeds to a database (for easy charting mainly) and I measure just the time of the inner loop (the creation of X number of objects). Now, this is not a scientific test and you should take the stats with a pinch of salt, they were run on my MacBook Pro (not exactly a server machine is it) and I tried to make everything as equal as possible (turning on the Trusted Cache for both Adobe ColdFusion and New Atlanta's BlueDragon, I wasn't able to spot something similar in Railo) and run the cycle a few times before setteling on the figure I was going to use (so the code could 'bed in').
The code that I am running on these tests is basically the same as before, but I am posting it here for comparison:
<cfloop from="0" to="2000" index="x" step="50">
.<cfflush>
<cfset tStartStruct = GetTickCount()>
<cfloop from="1" to="#x#" index="i">
<cfset stItem = StructNew()>
<cfset stItem.name = "Bob" & i>
<cfset stItem.age = 20>
</cfloop>
<cfset tTotalStruct = GetTickCount() - tStartStruct>
<cfset tStartComponent= GetTickCount()>
<cfloop from="1" to="#x#" index="i">
<cfset oItem = CreateObject('component', 'Person')>
<cfset oItem.setname("Bob" & i )>
<cfset oItem.setage(20)>
</cfloop>
<cfset tTotalComponent = GetTickCount() - tStartComponent>
<cfquery name="logThis" datasource="reporting">
UPDATE reports
SET bdStruct = #tTotalStruct#,
bdComponent = #tTotalComponent#
WHERE Instances = #x#
</cfquery>
|<cfflush>
</cfloop>
done!
I can post the precise results if people want but the charts kind of speak for themselves. Anything I missed?
EDIT: The versions of the engines are:
Adobe ColdFusion 7.02
New Atlanta Bluedragon Server JX 6.2.1.9
Railo (Live) 1.0.0.027
All these are the versions that run on OS X by the way
In theory they *SHOULD* be linear, since I am testing how long it takes to create 50,100,150 etc objects. so it should take roughly twice the time to create 2000 objects than it takes to create 1000 objects . Would you expect exponential curves or something else?
Totally missed the fact that you were doing this on a MAC. However the reason I ask is because I would like to see if .Net handles this better then Java. It would be interesting.
I have to mention, that even "/" is a mapping. It is defined in the server administrator and points to the web's root. But in order to make the "/" mapping trusted just define it in your local web accordingly by pointing it to your root directory.
*sitting on the fence*
- Disable Scope cascading (Admin/Settings)
- Convert the mapping into a railo archive (Admin/Mappings)
- enable trusted cache per mappings ((Admin/Mappings))
- upgrade to Railo 1.1
The whole point of this post is to show people that randomly creating objects (that for example, could be held in a recordset) simply because Java does it isnt the best idea in the world.
Besides, as I have also mentioned I am doing this test on a laptop, a MAC laptop, which would never be used as a server.
(Also was using java version 1.4.2 by default if that matters)
BD standalone
and Railo-live (the download on their site, just run from the command line)
If you really want to dig into this, the first thing you need to do is run the test with "empty" CFLOOP tags; it could be that the performance difference is due to running tens of thousands of CFLOOP iterations (which would be unusual in a "normal" CFML page). Then you'll want to run with only the CFSETs that contain the StructNew and CreateObject function calls; it could be that the other CFSETs are actually taking longer than these, espcially the one that contains the string concatenation.
However, if you really want to do a CFML engine comparison that has "real world" applicability, the thing to do is create a CFML page that contains a "normal" number of CreateObject calls (maybe 20 or so? maybe one of the Mach II sample applications?) and then run a load tester on it and compare requests/second, overall response times, and CPU and memory usage during the tests. Only then will you get meaningful data that reasonably translates to the results you'd expect in the real world.
in fact you can even download the WAR of Railo in order to deploy it. Just as a normal JEE application place it into the webapps folder and restart the server. It should install automatically.
Here it is (it addition to the comments I made here previously):
http://blog.newatlanta.com/index.cfm?mode=entry&am...