There's an oft-cited technote indicating that running with multiple instances somehow performs much better than running with a single instance, but no testing methodology is cited on the technote.
In my experience load testing, in the general case, the opposite is true-- you get better performance with fewer instances. In the specific case where you're doing a very large number of very small requests, the single-threaded nature of the datasource connection pool becomes a bottleneck, but it's easier (and less resource intensive) to work around that specific issue by using a modest number of identically-configured datasources, and choosing a datasource randomly at the start of the request.
If you're actually memory constrained, then you're better off using a single large 64-bit instance vs. a bunch of small 32-bit instances.
(I proved via load testing to one customer that their app had 10-15% better throughput when three of their four instances were disabled, and the techie that had pushed for Enterprise + multiple instances said, and I quote, "I don't believe you." Believe? A result that changes consistently and predictably when one variable changes somehow involves "belief"? What's happened to /science/ in this country?)
Posts
Glad to you see some great posts from you Daryl. Thanks.
ReplyDeleteWhat do you use nowadays for load testing?
Thanks for the heads-up, Daryl.
ReplyDeleteCan you clarify something for me? The "identically-configured datasources" - they all point to the same one dataBASE instance, correct? They do not point to some sort of a clustered database environment, right?
Also (and this may be a trivial question), I have a 64-bit machine on which I was just about to install a second CF instance (to be clustered). I believed this was needed to allocate more than 2Gb of memory to the instance, am I wrong? Is there no jrun/cf memory cap in 64-bit land?
Lastly, do you have any sort of data/results from your tests that we can all see?
Cheers!
Mike
TalkingTree,
ReplyDeleteI was using the customer's LoadRunner installation for those tests, and their scripts. The test results were consistent from test to (identical) test so I was confident of their usefullness. (Conversely, if the same test gives results that vary significantly from run to run, you really can't use that test for measuring the effects of changes.)
@Mike,
If you're running ColdFusion 64-bit, then you have a memory cap only barely theoretically reachable. (You need the 64-bit installer and ColdFusion 8.0.1 Enterprise to run 64-bit, though.)
Aside: It once seemed only theoretically reachable to use the whole of 2^32 bytes of RAM, because it was 65,536 times larger than the address space of a 16bit computer. Took only a few years to get 65,000x larger address spaces... but going from 2^32 to 2^64 is not "double that" by any means... we'd have to start needing four billion times more memory than present for 2^64 to become "crowded". The move from 64-bit address spaces to 128-bit address spaces will take a LOT longer than 16-bit to 32-bit did.
Thanks again, Daryl.
ReplyDeleteI'm still hoping for a couple more answers...
1) The "identically-configured datasources" - they all point to the same one database instance, correct? They do not point to some sort of a clustered database environment, right?
2) Do you have any sort of data/results from your tests that we can all see?
You rock,
Mike
@Mike,
ReplyDelete1) They're all regular datasources that point to a regular database.
2) I didn't keep copies of the results of those tests-- and even if I had, it would have been a nightmare to get publishing permission from the Fortune 500 company involved. I guess that makes me something of a hypocrite, since I complained in my original post about no testing methodology being shown, but I'll find a way to live with that pain. ;)
Thanks!!
ReplyDelete