[FoRK] Programming lang etc. (details for Stephen, comment for JAR)

J. Andrew Rogers andrew at ceruleansystems.com
Fri Nov 13 10:48:31 PST 2009

On Nov 13, 2009, at 6:42 AM, Jeff Bone wrote:
>> synchronization is free
> I have generally been agreeing 100% with everything JAR's had to say for quite a long time, but it's worth throwing this slight objection out there on this one...
> I'm guessing I have a pretty good idea of what environments you're working on which make this a practical assumption and it's worth pointing out that this is a special case;  in general / in the limit, synchronization cannot be free, in fact it is *impossible.*  Cf. previous thread re dispersed vs. distributed;  parallel is a special case of distributed in strictly the same sense that distributed is a special case of dispersed.  With each specialization comes various constraints and, if your operating environment meets those constraints, you can make some assumptions as you note.  Otherwise, special relativity will get you every time;  and note that this also has hard implications for any general theory of intelligence or computability, much less specific models of computing.
> I.e. "shared-nothing" isn't just a good idea --- it's the *law.* ;-) :-)

Yeah I know, the idea that synchronization is "free" is a practical programming model rather than an objective model of reality.  While there are scale limits in theory, this model scales up to some moderately large systems in practice. You can fit a lot of processing power within the implied light-cone  Codes that efficiently exploit these systems violate a lot of the code design heuristics that are second nature to veteran programmers, so they are interesting in that you have to think differently about software design to use them effectively; many programmers have a hard time adapting.

This model has a strong bias toward massive numbers of simple sub-gigahertz processors packed together in order to minimize relativistic effects. It won't work well with a conventional CPU architecture, but with appropriate hardware mods you can implement a very effective approximation of "free" synchronization that scales to an extremely large number of threads within a light-cone of tens of meters.  The low clock speed also does good things for power consumption. You'll just have to redesign all of your code to use it.

My point was really that programming models are bifurcating over the long-term and that popular programming languages are poorly suited to both models.  We can effectively scale massively threaded systems within a given light cone by increasing the number of processors/features that we can usefully fit within that volume of space.  Shared nothing models scale geographically, but they have other limitations and difficulties from a practical programming standpoint.  

More information about the FoRK mailing list