[FoRK] the fix is in? <rant> lang, os, etc. continued </rant>

Jeff Bone jbone at place.org
Sat Dec 12 05:47:30 PST 2009

Sean writes:

> It was also round this time that I was hired by a local company to  
> help port Unix programs to QNX. QNX is an underrated operating  
> system (kernel is only 8K on an 80486 class machine, Pentium kernel  
> probably isn't much larger) but you really need a network of  
> machines to see the full potential of it: it's entirely network  
> transparent. Sure, at the command line it looks like a Unix system,  
> but that's just a superficial layer over the operating system. Each  
> machine on the network (and really, it's a network segment as the  
> underlying protocol is Ethernet only) has an id and you could do  
> stuff like: //5 ls | //3 grep foobar | //7/tr -d '\r' | iconv -f  
> utf-8 -t iso8859-1 >//8/tmp/output

Good stuff, Maynard!  Like the Forth thing, never thought about it in  
the shell context though (beyond e.g. using dc....)  Wrote a bunch of  
Forth back in the day (on mucks, and before that on the PC;  my second  
"serious" program I ever wrote was in a language called Graforth that  
ran on the original PCs.  The program was a 3d animated star map fed  
from real astronomical data (meticulously hand-typed in by me from  
some paper map that showed xyz in c-years with Sol at the origin.)   
Graforth was extremely cool for graphics programming;  it provided 3- 
ary cartesian coordinates and 3d point and wireframe animation at ~10  
FPS on a first-gen Compaq luggable!  That coupled with an early  
fixation on RPN sealed my fate for me for a while.  That said, I can't  
really stomach it enough anymore to take things like Factor  

Familiar w/ QNX;  had it running on a few machines at the house a few  
years back when they made it more freely available, and had read the  
papers etc. long before that.

One problem here, with this:  that style of network transparency isn't  
really network transparent at the level of the individual composition  
of commands.  For one thing, network transparency is a bit of a canard  
anyway, cf. Waldo et. al.  Anytime you've got a network of any  
complexity involved, you've got all kinds of failure modes that you  
don't have in the purely local case.  W/o making those explicit, your  
code tends to break in weird ways.  Second thing:  explicit *node*  
addressing is a bit troublesome, as it requires certain things of the  
namespace;  either a global name resolver or some significant amount  
of shared context between nodes.  This induces quite a bit of coupling  
in the code.  Various languages / systems of course tackle this in  
different ways;  in the "delay-tolerant networking" limit, it implies  
async store-and-forward and routing coupled with some clever naming  
tricks.  In the more usual sense, Erlang's process addressing (rather  
than e.g. channels) provides a bit looser coupling but still doesn't  
entirely suffice;  though at least failure is handled explicitly, even  
if remote PIDs embed the node name.

Still, yeah, even simply adding QNX-style node addressing into the  
shell leads to a lot of power, particularly in the more homogenous /  
well-networked cluster case.  The next big problem, dealt with  
reasonably in Plan 9's approach via e.g. its 'cpu' command, is that  
each element in that pipeline runs in a completely unique  
environment.  Plan 9 and Inferno get around this by arranging to have  
the invoking shell's entire environment exported to the remote node,  
so that it sees the same filesystem / resource namespace as the  
invoking process.  (Which is pretty weird, but cool, if you think  
about it.)  If necessary the subprocess can then customize that  
namespace, mounting in local resources as appropriate.

No conclusions here, just musing.  Summing up, composition and  
coordination in the distributed case are interesting problems.   
Several approaches:

   - channels (very tight coupling)
   - explicit node-addressed synchronous composition / dispatch /  
coordination (tight coupling)
   - global / local namespace mapping and synchronous RESTful  
interaction (fairly tight coupling)
   - pid-addressed async messaging (semi-tight coupling)
   - async store-and-forward w/ some clever abstract naming and  
routing (moderate coupling)
   - completely content-based dispatch (i.e. Linda and generalizations  
thereof;  very loose coupling)

I'm sure that's far from a complete taxonomy, but it's early am and  
the caffeine hasn't kicked in yet. ;-)


More information about the FoRK mailing list