CFLOCK Further Explained!
In a comment on my recent post explaining CFLOCK (http://www.horwith.com/index.cfm/2008/4/28/cflock-explained), Duncan asked an excellent question: "why has the (cflock) Scope attribute not been deprecated?". He also pointed out that I made the statement in that blog entry that "we won't discuss scope locks since their use has not been recommended since ColdFusion 5". Addressing Duncan's question isn't as straight forward as I'd like because I have to make some assumptions and draw on personal experience, and even then the answer can be somewhat confusing... so in order to do so I felt I had to post my response in the form of a blog entry.
First, to address the question as to why the "scope" attribute hasn't been deprecated (Duncan pointed out that in CF8 they even added a scope - you can now specify "request" as a valid value). The truth is, I'm not sure why it hasn't been deprecated because, if you read the available CF 8 documentation on locking http://livedocs.adobe.com/coldfusion/8/sharedVars_01.html#1159066, it appears to be absolutely no different to a named lock - you're just using the name of a scope and specifying it via a different attribute. One possible difference is that in the case of session scope locking, it's reasonable to assume that the tag is "smart" enough to limit the lock to THAT USER'S session scope (and not effect other users' sessions). You could still achieve the same result by doing something like name="session#session.sessionid#", though. One of the things I dislike most about ColdFusion is the fact that, being a commercial product owned and controlled by a company, not enough of the details of its inner workings are explained. If someone on the CF Server team would like to clarify the difference between how CFLOCK works internally depending on whether it's named or has a scope specified, it'd be greatly appreciated.
I promised a confusing response, and that didn't seem too confusing, did it? If you took the advice I just gave and read everything in the documentation about locking variables however, then you are probably extremely confused at this point. The reason being: the documentation is in direct conflict with my statement that you don't need to lock scopes and that it's not a recommended practice. So what's the truth?
It is true that ColdFusion is thread safe - this means threads will not try to modify the same variable simultaneously. However, Adobe does not guarantee the order in which the server, which is a multi-threaded engine handling a lot of simultaneous requests (among other things), will execute those threads. In other words, while trying to read and write to the same variable with 5 simultaneous threads won't crash the server, there's no guarantee in what order those threads will execute. This can cause inconsistent/inaccurate data. Basically, take my prior discussion about race conditions and apply the same principals to server threads in an environment where extreme load is causing VERY frequent simultaneous writes/reads to the same variable. Another environment where this may be an issue is in the case of a very slow server where processing isn't measured in milliseconds. The only way Adobe will guarantee data integrity is if you lock. Now, I have some thoughts on this.
First, I'll state for the record that I've never seensuch extreme load on CFMX servers that unlocked scope access caused incorrect results... this includes one environment in which the load was so great that CF had to be all-together abandoned (yes, that's A LOT of load). I'll also say that, in an application that follows what I'd call "ColdFusion Software Architecture Best Practices"), there is a very unlikely chance of data becoming corrupt in any of the ways described in the documentation. So maybe if you write really poor code you should play it safe and use a lot of CFLOCKing (remember though, CFLOCK will slow down your app). I also think it's worth noting that in the two environments I describe where the problem is more likely to arise (that of a server under massive load and that of an extremely slow server) there is a better answer. A server under THAT much load probably means (hopefully means is more like it) that you can afford to buy more (or better) servers. In the case of a server that is taking a really long time to process threads/requests - I'd say you've got bigger problems to worry about. Either way, based on my past experience, I wouldn't use scope locking.
Hopefully, that helps to clear things up. Again, I would love for an engineer from the CF Server Team to chime in on this one.
First, to address the question as to why the "scope" attribute hasn't been deprecated (Duncan pointed out that in CF8 they even added a scope - you can now specify "request" as a valid value). The truth is, I'm not sure why it hasn't been deprecated because, if you read the available CF 8 documentation on locking http://livedocs.adobe.com/coldfusion/8/sharedVars_01.html#1159066, it appears to be absolutely no different to a named lock - you're just using the name of a scope and specifying it via a different attribute. One possible difference is that in the case of session scope locking, it's reasonable to assume that the tag is "smart" enough to limit the lock to THAT USER'S session scope (and not effect other users' sessions). You could still achieve the same result by doing something like name="session#session.sessionid#", though. One of the things I dislike most about ColdFusion is the fact that, being a commercial product owned and controlled by a company, not enough of the details of its inner workings are explained. If someone on the CF Server team would like to clarify the difference between how CFLOCK works internally depending on whether it's named or has a scope specified, it'd be greatly appreciated.
I promised a confusing response, and that didn't seem too confusing, did it? If you took the advice I just gave and read everything in the documentation about locking variables however, then you are probably extremely confused at this point. The reason being: the documentation is in direct conflict with my statement that you don't need to lock scopes and that it's not a recommended practice. So what's the truth?
It is true that ColdFusion is thread safe - this means threads will not try to modify the same variable simultaneously. However, Adobe does not guarantee the order in which the server, which is a multi-threaded engine handling a lot of simultaneous requests (among other things), will execute those threads. In other words, while trying to read and write to the same variable with 5 simultaneous threads won't crash the server, there's no guarantee in what order those threads will execute. This can cause inconsistent/inaccurate data. Basically, take my prior discussion about race conditions and apply the same principals to server threads in an environment where extreme load is causing VERY frequent simultaneous writes/reads to the same variable. Another environment where this may be an issue is in the case of a very slow server where processing isn't measured in milliseconds. The only way Adobe will guarantee data integrity is if you lock. Now, I have some thoughts on this.
First, I'll state for the record that I've never seensuch extreme load on CFMX servers that unlocked scope access caused incorrect results... this includes one environment in which the load was so great that CF had to be all-together abandoned (yes, that's A LOT of load). I'll also say that, in an application that follows what I'd call "ColdFusion Software Architecture Best Practices"), there is a very unlikely chance of data becoming corrupt in any of the ways described in the documentation. So maybe if you write really poor code you should play it safe and use a lot of CFLOCKing (remember though, CFLOCK will slow down your app). I also think it's worth noting that in the two environments I describe where the problem is more likely to arise (that of a server under massive load and that of an extremely slow server) there is a better answer. A server under THAT much load probably means (hopefully means is more like it) that you can afford to buy more (or better) servers. In the case of a server that is taking a really long time to process threads/requests - I'd say you've got bigger problems to worry about. Either way, based on my past experience, I wouldn't use scope locking.
Hopefully, that helps to clear things up. Again, I would love for an engineer from the CF Server Team to chime in on this one.
I always thought that an exclusive scope lock suspended access that that entire scope for ALL RUNNING THREADS.
My understanding of an exclusive named lock is that the code inside the lock would only be executed by one thread at a time.
I could see both of those being useful for different reasons. That being said, I never use scope locks. Usually I use a named lock which has the name of the variable I am accessing. (like “application.cached_struct.key”) That way, I am essentially only locking access to the certain part of the application or session scope that I am fiddling with.
Of course, I rarely need locks anyway. ColdFusion already makes sure that two processes won't try to write to the same variable and corrupt memory. It's really only useful when a series of commands need to happen in a row on a resource without another process interfering in the mean time. For instance, if this application variable is undefined, run this process to populate it. Of course, even that can be more cleanly handled now days by onApplicationStart().
I know about a system using ColdFusion MX 7 (with all the patches applied), ~400 users, Windows and Oracle, clustered at two servers, that even with CFLOCK in the session scope for all session variables creating/updating calls, is suffering from session issues. They even tried to shut one server and let the other alone take care of all the load, but still they have sessions disappearing, and please note: it's not a session that is gone and replaced, it's *the same session being cleaned up*. They revised the code, everything and can't find a way to solve it - I didn't had the chance to try, but I would like to point it to you, as I ever advised the use of CFLOCK and even with it CF seems to be having problems with the session variables.
Tks for the post anyway :) I hope some Adobe engineer joins the discussion!
Abs,
Fernando
At my current job, we rarley persist information in shared scopes. Our app is very DB driven and for the most part all of our concurency is dealt with by the database because hey-- that's what DB's are good at!
The only times we've really gotten bit is when we use with(nolock) in the wrong places but that's a different story...
Yes, that's true in the case of sessions and I'll update the blog entry to reflect it... but do bear in mind that you can simply make the value of the "name" in named locks unique to the session and achieve the same result (something like name="session#session.sessionid#").
- you think it's curious to have the SCOPE attribute when we can just use the application.name for the app scope and session.sessionid for the session?
- you find it curious that the Adobe technote asserts that one could have a race condition, thus justifying bothering with CFLOCK ever at all? It seems you think it to be more of a theoretical problem than a real one?
I'll grant that locks are seriously misunderstood (and even misused) by most, but I'm kind of surprised by the assertions above. I'm confident in trusting that there can be race conditions that justify their continued use (though far more often they're used where they no longer need to be). Maybe someone else will chime in with some nice examples where race conditions warranted still doing locking.
And as for the named scopes, I'd assert that they're there simply as another element of CF's simplicity. Why force a developer to think to use the application name or session id, when the language can instead be clear to all as to what was intended? It just kind of seems superfluous to debate if the attribute should be deprecated. I can't see it doing anything wrong, and without it, some developers seeing the name attribute would perhaps be left wondering what to choose for their specific situation.
Honestly, I think there are a lot bigger problems with misuse of CFLOCK that people could use help with. Heck, I still feel that most don't honeslty understand the tag and its use at all. Your previous entry was a step in that direction. Not saying you shouldn't have bothered with this. Just proposing more fodder if you get the itch to write again. :-)
As for race conditions, today I applied my first ever cflock since using CF5.
We have a PayPal processing template that some nasty customer managed to trigger twice, instantaneously. The logic already included a check to see if it had been processed then immediately actions the outcome of that. Anyway, whatever caused the race condition (double clicking the return from PayPal link we think - but we couldn't replicate it), locking it with a named lock based on the paymentID allows the code block to run simultaneously for many users but only once for any one payment. Problem solved - we think... it has only happened once in thousands of transactions and we have not been able to replicate it. Time will tell.