String Concatenation performance test

August 19, 2008

I've always tried to avoid using the underlying Java if I can do the same thing with CFML. This is because (as far as I know) it is undocumented so you may have issues if you try to run your code on one of the other CFML engines.

Anyway, I recently needed to do some string concatenation with large strings, which is notoriously slow in ColdFusion, so I thought I run some tests to compare performance of various ways to do it. Here is my test code.


<cfset iterations = 1000 />
<cfset sampletext = RepeatString("Lorem ipsum dolor sit amet, consectetuer adipiscing elit.", 10) />

<!--- concatenate with coldfusion --->
<cfset startTime = getTickCount() />
<cfset cfstring = "" />
<cfloop from="1" to="#iterations#" index="i">
  <cfset cfstring = cfstring & sampletext />
</cfloop>
<cfset endTime = getTickCount() />
<cfoutput>
  concatenate with coldfusion : #endTime-startTime#ms<br />
</cfoutput>

<!--- listappend coldfusion --->
<cfset startTime = getTickCount() />
<cfset cflist = "" />
<cfloop from="1" to="#iterations#" index="i">
  <cfset cflist = ListAppend( cflist, sampletext, Chr(1) ) />
</cfloop>
<cfset cfstring = ListChangeDelims( cflist, "", Chr(1) ) />
<cfset endTime = getTickCount() />
<cfoutput>
  listappend with coldfusion : #endTime-startTime#ms<br />
</cfoutput>

<!--- arrayappend coldfusion --->
<cfset startTime = getTickCount() />
<cfset cfarray = ArrayNew(1) />
<cfloop from="1" to="#iterations#" index="i">
  <cfset ArrayAppend( cfarray, sampletext ) />
</cfloop>
<cfset cfstring = ArrayToList( cfarray, "" ) />
<cfset endTime = getTickCount() />
<cfoutput>
  arrayappend with coldfusion : #endTime-startTime#ms<br />
</cfoutput>

<!--- concatenate with cfsavecontent --->
<cfset startTime = getTickCount() />
<cfsavecontent variable="cfstring"><cfoutput><cfloop from="1" to="#iterations#" index="i">#sampletext#</cfloop></cfoutput></cfsavecontent>
<cfset endTime = getTickCount() />
<cfoutput>
  coldfusion with cfsavecontent: #endTime-startTime#ms<br />
</cfoutput>

<!--- concatenate with java stringbuffer --->
<cfset startTime = getTickCount() />
<cfset oStringBuffer = createObject( "java", "java.lang.StringBuffer" ).init() />
<cfloop from="1" to="#iterations#" index="i">
  <cfset oStringBuffer.append( sampletext ) />
</cfloop>
<cfset cfstring = oStringBuffer.toString() />
<cfset endTime = getTickCount() />
<cfoutput>
  concatenate with java stringbuffer : #endTime-startTime#ms<br />
</cfoutput>

<!--- concatenate with java StringBuilder --->
<cfset startTime = getTickCount() />
<cfset oBuilder = createObject( "java", "java.lang.StringBuilder" ).init() />
<cfloop from="1" to="#iterations#" index="i">
  <cfset oBuilder.append( sampletext ) />
</cfloop>
<cfset cfstring = oBuilder.toString() />
<cfset endTime = getTickCount() />
<cfoutput>
  concatenate with java stringbuilder : #endTime-startTime#ms<br />
</cfoutput>

I ran this on ColdFusion 8.01 Developer Edition (which is the equivalent of ColdFusion 8.01 Enterprise). The results are really interesting.

concatenate with coldfusion : 6813ms
listappend with coldfusion : 21609ms
arrayappend with coldfusion : 47ms
coldfusion with save content: 47ms
concatenate with java stringbuffer : 63ms
concatenate with java StringBuilder : 62ms

I'm really surprised by this as ColdFusion is built of Java but as you can see, the fastest way to concatenate strings in ColdFusion is to use ArrayAppend and not Java Strings. It also shows that you should avoid using ListAppend.

24 comments

  1. Just as a comment, I think you should initialize the Java Objects before you start to count, as they normally take some time to initialize, and this might be affecting the results.

    ColdFusion already has all the objects initialized with it, so when you call a list Append for example, you're just invoking the method, but the object is already there.

    Try to invoke the objects first outside of the counter and then just invoke the methods inside of the counter. That should really change things.

    Cheers

    Comment by Marcos Placona – August 19, 2008
  2. Hi Marcos, thanks for the comment.

    That could explain the difference in performance. However, I think it is fair to include the time it takes to initialize the Java String object as you would have to do this in your code if you wanted to do it that way.

    Comment by John Whish – August 19, 2008
  3. Out of interest, I just tried running the test again but didn't include the CreateObject part in my timings. Here are the results:

    concatenate with coldfusion : 6875ms
    listappend with coldfusion : 24328ms
    arrayappend with coldfusion : 47ms
    concatenate with java stringbuffer : 109ms
    concatenate with java StringBuilder : 63ms

    I then ran it again and also excluded the .toString() method which made the biggest difference. The results are:

    concatenate with coldfusion : 7375ms
    listappend with coldfusion : 24360ms
    arrayappend with coldfusion : 47ms
    concatenate with java stringbuffer : 46ms
    concatenate with java StringBuilder : 47ms

    ArrayAppend still holds up really well.

    Comment by John Whish – August 19, 2008
  4. Cool mate.. sounds more reasonable now ;-)

    Comment by Marcos Placona – August 19, 2008
  5. here are my results with the provided code:

    concatenate with coldfusion : 421ms
    listappend with coldfusion : 1579ms
    arrayappend with coldfusion : 0ms
    concatenate with java stringbuffer : 0ms
    concatenate with java StringBuilder : 15ms

    and here are the results for same code but 10,000 iterations:

    concatenate with coldfusion : 87250ms
    listappend with coldfusion : 453312ms
    arrayappend with coldfusion : 78ms
    concatenate with java stringbuffer : 125ms
    concatenate with java StringBuilder : 110ms

    Comment by Ed – August 19, 2008
  6. Hi Ed, Thanks for posting your results. Are you running 8.01 Developer as well? Your machine is faster than mine - I tried 10,000 iterations and it died!

    I think I'll try on CF8.01 Standard to see if there is a difference.

    Comment by John Whish – August 19, 2008
  7. my tests was done on a CF8 Enterprise box and for the 10K iterations i've set a large timeout.

    Comment by Ed – August 19, 2008
  8. I tried your test and got generally similar results with the string concatenation, and using the list and array functions. However I also added <cfsavecontent> as one of the test conditions. You'll be surprised at the results

    coldfusion with save content: 16ms

    <cfset startTime = getTickCount() />
    <cfsaveContent variable="sampleText">
    <cfoutput>
    <cfloop from="1" to="#iterations#" index="i">
    #sampletext#
    </cfloop>
    </cfoutput>
    </cfsaveContent>
    <cfset endTime = getTickCount() />
    <cfoutput>
    coldfusion with save content: #endTime-startTime#ms<br />
    </cfoutput>

    This is with CFMX 7.02 on an WinXP (sp2) box with 2 gig of ram.

    Generally string buffer and string builder are very similar, although string builder is optimized for performance. However there is a risk in using it since it is not thread safe.

    Here's a discussion of of these issues recently on CF-Talk. www.houseoffusion.com/groups/cf-talk/thread.cfm/threadid:56583 />
    regards,
    larry

    Comment by larry c. lyons – August 19, 2008
  9. Hi Larry,

    Thanks for the comment. Would you mind posting the results for all the tests to see if there is any difference between CF8.01 and CF7.02.

    I've seen people use cfsavecontent but heard that it can cause Java Heap errors, so didn't include that one, but you're right I should add it. I've updated my post (I removed the carriage returns from your code so that it returns the same string as the other tests.)

    For subscribers here are my results:

    concatenate with coldfusion : 6797ms
    listappend with coldfusion : 21344ms
    arrayappend with coldfusion : 47ms
    coldfusion with save content: 47ms
    concatenate with java stringbuffer : 62ms
    concatenate with java StringBuilder : 63ms

    Comment by     John Whish – August 20, 2008
  10. Out of curiosity I changed the concatenate with ColdFusion test from:
    <cfset cfstring = cfstring & sampletext />
    to the new CF syntax of:
    <cfset cfstring &= sampletext />

    The results are:

    concatenate with coldfusion : 6813ms
    concatenate with coldfusion &= : 6265ms

    Comment by John Whish – August 20, 2008
  11. I'm running CF 8 Microsoft Windows Server 2003, Service Pack 2

    Pentium 4 3.00GHz
    1GB Ram

    concatenate with coldfusion : 641ms
    listappend with coldfusion : 2047ms
    arrayappend with coldfusion : 15ms
    coldfusion with cfsavecontent: 0ms
    concatenate with java stringbuffer : 32ms
    concatenate with java stringbuilder : 31ms

    Comment by Tim Rubel – September 04, 2008
  12. @Tim, 0ms for cfsavecontent - now that's fast :)
    Would you mind posting the results for 10,000 iterations? My tests were done on my dev box which is Windows 2000 SP4, CF8.01, 1GB RAM, 2 x 1Ghz Pentium III.

    Comment by John Whish – September 04, 2008
  13. results for 10,000 iterations is taking for ever!! Not sure what is going on.

    Comment by Tim Rubel – September 04, 2008
  14. Hi Tim, hope your server has recovered. You may have used up all the available space in your JVM heap and the garbage collection is kicking in. If that is your production server you might want to think about doubling the RAM and max JVM heap size.

    Comment by John Whish – September 05, 2008
  15. Do you know how I could go about dumping the garbage collection and the JVM heap without rebooting or shutting down the CF services?

    Comment by Tim Rubel – September 05, 2008
  16. I've not tried it myself but I believe this does it.

    <cfset runtime = CreateObject("java", "java.lang.Runtime").getRuntime() />
    <cfset runtime.gc() />

    I should point out that (as I understand it) it is the garbage collection that makes the server run slowly.

    Comment by John Whish – September 05, 2008
  17. The server didn't seem to slow down, that page took forever and then finally timed out. I even set the timeout to 500 seconds.

    Comment by Tim Rubel – September 05, 2008
  18. @Tim, if you've got ColdFusion 8 (developer or enterprise) then fire up the server monitor before you run the script. That should give you a pretty good idea of what is going on. You could also add a cfflush to the script to see where it stops.

    Hope that helps.

    Comment by John Whish – September 05, 2008
  19. I wish I could say you helped, but this is Coldfusion 8 standard. Perhaps adding a cfflush after each concatenate instance would help clean things up?

    Comment by Tim Rubel – September 05, 2008
  20. I ran the code twice and got different results. What might cause this? I cleaned the JVM gc before running both times.

    concatenate with coldfusion : 74640ms
    listappend with coldfusion : 349782ms
    arrayappend with coldfusion : 94ms
    coldfusion with cfsavecontent: 422ms
    concatenate with java stringbuffer : 515ms
    concatenate with java stringbuilder : 578ms

    concatenate with coldfusion : 302687ms
    listappend with coldfusion : 626172ms
    arrayappend with coldfusion : 141ms
    coldfusion with cfsavecontent: 109ms
    concatenate with java stringbuffer : 172ms
    concatenate with java stringbuilder : 203ms

    Comment by Tim Rubel – September 05, 2008
  21. @Tim - sorry, but I can't really help you with that as it is virtual impossible to say what might cause that. I suggest you install Seefusion or FusionReactor (both have trial versions) which both work on ColdFusion 8 Standard and go from there.

    Comment by John Whish – September 08, 2008
  22. I ran a slightly altered version of the script on my machine with the following results:
    (changed the string size / repitition)

    All values are in ms.
    All test were repeated 500 times in random order to calculate the averages.
    ColdFusion service was restarted before each test.

    My machine:
    (2x xeon 3ghz 5160, 4gb ecc, cf8.01 developer, sata raid0, vista business)

    concatenate below is using &= method.

    ------------------------------
    Test 1 - small strings, large repitition:
    Added 6 characters to a string 10,000 times.

    -- average:
    concatenate : 351.83
    listAppend : 346.90
    arrayAppend : 5.56
    cfsavecontent: 2.54
    java.lang.StringBuffer : 31.84
    java.lang.StringBuilder : 27.12

    -- average (less slowest and fastest 15%):
    concatenate : 344.91
    listAppend : 344.72
    arrayAppend : 5.32
    cfsavecontent: 2.45
    java.lang.StringBuffer : 27.02
    java.lang.StringBuilder : 26.92

    -- slowest:
    concatenate : 779
    listAppend : 410
    arrayAppend : 13
    cfsavecontent: 11
    java.lang.StringBuffer : 396
    java.lang.StringBuilder : 34

    -- fastest:
    concatenate : 334
    listAppend : 333
    arrayAppend : 5
    cfsavecontent: 2
    java.lang.StringBuffer : 26
    java.lang.StringBuilder : 25

    ------------------------------
    Test 2 - medium strings, medium repitition:
    Added 12 characters to a string 5,000 times.

    -- average:
    concatenate : 174.27
    listAppend : 174.44
    arrayAppend : 2.70
    cfsavecontent: 1.59
    java.lang.StringBuffer : 14.20
    java.lang.StringBuilder : 14.02

    -- average (less slowest and fastest 15%):
    concatenate : 173.23
    listAppend : 173.18
    arrayAppend : 2.67
    cfsavecontent: 1.47
    java.lang.StringBuffer : 13.97
    java.lang.StringBuilder : 13.88

    -- slowest:
    concatenate : 224
    listAppend : 209
    arrayAppend : 9
    cfsavecontent: 8
    java.lang.StringBuffer : 21
    java.lang.StringBuilder : 21

    -- fastest:
    concatenate : 166
    listAppend : 168
    arrayAppend : 2
    cfsavecontent: 1
    java.lang.StringBuffer : 13
    java.lang.StringBuilder : 13

    ------------------------------

    Test 3 - large strings, small repitition:
    Added 24 characters to a string 2,500 times.

    -- average:
    concatenate : 88.21
    listAppend : 88.55
    arrayAppend : 1.47
    cfsavecontent: 1.04
    java.lang.StringBuffer : 7.32
    java.lang.StringBuilder : 7.30

    -- average (less slowest and fastest 15%):
    concatenate : 87.22
    listAppend : 87.32
    arrayAppend : 1.46
    cfsavecontent: 1.02
    java.lang.StringBuffer : 7.17
    java.lang.StringBuilder : 7.05

    -- slowest:
    concatenate : 118
    listAppend : 147
    arrayAppend : 2
    cfsavecontent: 2
    java.lang.StringBuffer : 15
    java.lang.StringBuilder : 22

    -- fastest:
    concatenate : 82
    listAppend : 83
    arrayAppend : 1
    cfsavecontent: 0
    java.lang.StringBuffer : 6
    java.lang.StringBuilder : 6

    It seems cfsavecontent is the winner!

    Comment by Mike Causer – January 09, 2009
  23. Interesting discussion. I thought I'd add a possibility of the java.util.Vector class, using the AddAll function to concatenate the string together. It wasn't worth the effort in the end, as you have to do more to get the string out of it that you want, but it still came in at 2nd place in my testing, with ArrayAppend and CFSaveContent coming in at equal first.

    Here's the code I used:
    <!--- Vector append --->
    <cfset startTime = getTickCount() />
    <CFSet SampleArray = [SampleText]><!--- The AddAll function requires an array (a collection actually). --->
    <cfset objVector = createObject( "java", "java.util.Vector" ) /><!--- Include object creation time... negligible. --->
    <cfloop from="1" to="#iterations#" index="i">
    <CFSet objVector.AddAll(SampleArray)>
    </cfloop>
    <cfset cfstring = ArrayToList(objVector.toArray(), "") /><!--- If you use toString() then you end up with extra characters in the resulting string. This way works (can be improved on I'd guess). --->
    <cfset endTime = getTickCount() />
    <cfoutput>
    Vector Append : #endTime-startTime#ms (#Len(cfstring)#) <br />
    </cfoutput>



    In order to rank them I scrapped the idea of measuring the time to do each of the tests because the ListAppend was so slow and memory hogging that I couldn't get the iterations high enough to create a significant difference between the faster methods. i.e. When the ArrayAppend takes 141ms and the CFSaveContent takes 109ms I don't consider that significant enough as the server could have been doing something else at the time. To solve that you can run the tests multiple times, OR you can increase the iterations significantly. I ended up with 900,000 iterations to get the times up high enough to notice a significant difference.

    In order to go that high I had to disable the slower tests, because they simply would never finish!

    So, I found that the ArrayAppend and the CFSaveContent were the fastest, with the CFSaveContent being generally faster. They did 900,000 iterations in about 4000ms each, creating a string of 513 million characters each. However, from execution to execution they would actually switch position as to who was faster (it's one of my production boxes so I'm sure there is some other stuff happening). Either way, it's good enough for me to continue using ArrayAppend over CFSaveContent because of the possible JVM heap concerns about CFSaveContent (which gave a heap error at 1 million iterations, and ArrayAppend kept going until 1.6 million iterations and survived 850 million characters), and besides, it isn't significantly faster anyway - at least, it isn't significantly faster with under a billion characters.

    BTW this CF8 instance has 6GB allocated to it.
    Steven

    Comment by Steven Van Gemert – May 06, 2009
  24. Hi Steven, Always good to see an alternative approach, so thanks for posting. You make a good point about cfsavecontent throwing a heap exception. I've not seen that happen myself (my dev server died long before it had done a million iterations!), but it's worth knowing.

    Comment by John Whish – May 06, 2009

Leave a comment

If you found this post useful, interesting or just plain wrong, let me know - I like feedback :)

Please note: If you haven't commented before, then your comments will be moderated before they are displayed.