[FoRK] Programming languages, operating systems, despair and anger

J. Andrew Rogers andrew at ceruleansystems.com
Thu Nov 12 20:26:15 PST 2009

On Nov 12, 2009, at 6:27 PM, Jeff Bone wrote:
> Jim Backus called it in his 1977 Turing Award Lecture re:  the dilemma we now face doing multi-core.

In the last year or so I have been working with a relatively rich and diverse set of massively parallel architectures, including some that are pretty exotic.  One of the hard-won lessons learned from a software design stand point is that the model of multi-threading that almost every programmer is learning today is, to be blunt, dead wrong.  The experience has been enlightening.

In conventional multi-threading models we are given the concept of synchronization primitives. We are correctly lead to believe that synchronization is expensive and most code is written for this model, but that assumption does not scale. 

When designing software for massively parallel systems, the correct model is one of two assumptions: that synchronization is free or that synchronization is non-existent. Both assumptions will vex the conventional programmer. For the former assumption, just about everything you traditionally learn in computer science is naive and under-exploits the power of this model. These models also tend to be exotic; while they scale, not that many people have experience with them.  For the latter assumption, which most real-world "synchronization is expensive" models collapse to at scale, we have a different problem where synchronization -- such as it is -- is not a simple matter of defining a sync variable. We really do not prepare people for this scenario either.

The problem with multi-core is not so much the idea of multi-core as it is an implementation that is attempting to scale the idea that synchronization is expensive. Just about all of our programming languages parrot this assumption. If they want the concept to scale, they will have to make it either free or non-existent at the architecture and language level.  A few languages do an adequate job of "synchronization is non-existent", and none (that I can think of) support the notion that synchronization is free (it is typically implemented as a set of extensions to standard languages in lieu of traditional synchronization primitives).

More information about the FoRK mailing list