Coordination / Orchestration: Polyphony vs. Temporal Logic
Gregory Alan Bolcer
gbolcer at endeavors.com
Thu Oct 30 12:06:34 PST 2003
Sorry, I was just overly being a smartass about the "wrong" part.
I am very interested in this subject in particular and didn't
have time to write up a proper reply. I grew up academically
in Osterweil's "Software Processes are software too", Boehm's
"It's all about the cost metrics and repeatability", Humprey's
"the world is both capable and mature" model, Ted Smith's
"We can engineer anything with enough money", Wasserman's
"software can be (re)presented through pictures" and Taylor's
(not related to Frederick Taylorism) "architectural styles
are software too" worlds.
A couple of notes. EDI has mostly focused on the declartive
data and fuctions and not procedural. To boot, they've also
taken a pass on the runtime requirements (read IT surrounding
requirements left up to implementer) to actually prove out
the system. The equivalent in the programming world is either a
virtual machine or a lowest-common-demoninator, stateless interface
that doesn't bind the semantics to the runtime like so many
languages require. I believe this was one of the items that helped
the Web succeed.
In the software process field where they can lockstep the
developers and en/actors into a forced compliance mode, about
20% of workflows will be kicked out as exceptions which represents
more than 99% of the time spent on the workflow. You can
eliminated most of these kickouts through more rigorous declarative
and data checking, but in the end it comes down to human factors.
Your paper example would be a kickout. If you are HP processing
tens of thousands of human resources and payroll workflows, one
per each employee every month and quarter, then you need the
dataspace completeness assurances. Part of the value of
production workflow is completeness.
My experience is that, when you need completeness, it's theoretically
impossible to specify all the exceptions, and the ones that don't
get caught will include an inordinate amount of resources to fix.
Declarative will only get you around 50-60% and at most 90% with
very, very strict controls, but the cost increases exponentially
with the barriers created by the checks and controls.
The equivalent to the second is mostly in the ad hoc workflow community
versus the naziware. Ignoring the representability problems with
graphical workflow presentations mapping into concurrent languages
(most people like to design and maintain workflows graphically using
boxes and arrows and there's limitations on how concurrent structures
get represented and displayed) the best thing is to completely
decentralize the workflow definition and/or allow the definition to
move with the data. Give the people access to events and simple
exception handling that doesn't require programming. There's some
declarative email rules in Outlook, but if you've ever tried to
do something complex with them, you quickly get bogged down as
there's too many similar rules that do slightly different things.
The parallel would be able to capture IM messages, email
attachment arrivals, context representative text and attach a
(apple, tcl, perl, python, java) script to it. This of
online documents being able to go everywhere where offline
documents could go in both a push and pull manner.
Think of a workflow triggering that is simple enough
that users can set it up themselves. Here's the best part too, think
of a desktop or laptop computer environment similar to automated
testing tools, where you use VCR like controls to attach a repeatable
and generic enough set of actions as a pre- or post-condition on
any event and all the while having a whole simple PKI, ad hoc
VPN, extranet access management, and centralized reporting and auditing
where you can push, pull, start, stop, resume, monitor processes,
data, and workflow across any boundaries.
You have a personal Web server, I have a personal Web server. I
send it to you for approval. It's sitting on your server, I have
automated permissions to see if you've opened it, looked at it,
changed it, signed it, etc. You can regrab, I can repush.
Our manager can find all copies on the network instantly and
what state of the process it's in. We did a very similar
demonstration on stage with some of the Intel executives a long
time ago. We had some wireless ipaqs. The Intel exec needed
to sign an NDA, so it got sent to the first ipaq where our
exec signed the word doc NDA with the pen-based interface who
then pushed it on to the Intel exec and notified him that
it was ready for signing, both signed copies got sent off to
each other and then off to the different legal departments, and
then confirmed paper copies were automatically faxed all while
being able to watch the whole progression centrally without having
to put the stuff up on yet another portal and clogging
the exchange server up with version after version of attachments.
The real workflow would be automating all the revisions for the
NDA before it got sent out for signing without having to burden
those two poor non-technical execs with figuring out how to
to interact with a check-in and check-out document portal.
I'm telling you, ad hoc with metadata visibility is really the way
to go. How that maps onto the lower abstraction of concurrent
programming in some sense doesn't really matter unless you are
trying to prove otu livelock, choke points, or deadlock in a modeled
system. I've found out that you can typically eyeball these things
in a company with the loosest of mappings without all the formal
rigamorale days or weeks faster with a little bit of decentralized
The bottom line is, there's been a lot of tries, but they are
just at different levels of abstraction for good reasons.
Meltsner, Kenneth wrote:
> Greg, I didn't make myself clear enough. Note the (technical) in my
> comment: I think the problems are similar technically (thanks for the
> pointer to Patrick Young). And I believe we agree that the complexities
> of people and business make workflow/orchestration the complicated mess
> that it is today.
> Look at EDI for a good example of a simple technical problem complicated
> by people/business issues. It's complex because the business problems
> and processes are hard to represent in a way that is unambiguous (even
> though most purchasing processes are filled with opportunities for
> people to screw things up) and unlikely to end up with the various
> partners in court.
> The modelling part of the EDI specs is relatively simple, and is
> straightforward to translate to a more popular syntax (i.e. XML).
> Unfortunately, the EDI world had a lot of infrastructure (transaction
> exchange services, consultants, hooks into ERP systems, etc.) and the
> need for that infrastructure didn't go away simply because the transport
> problem was "solved" by the Internet.
> Back to workflow and concurrent programming: they're the same, except
> where they're different. Here's an example of two workflow
> implementation approachs for a sequential, serial technical paper review
> * a workflow that completely specifies all of the exceptions (e.g.
> paper reviewer doesn't respond in XX days, paper is shredded by legal
> reviewer) and how they have to be escalated. This process is fully
> specified and documented, (rah for ISO 9000!) but also completely
> * a really simple workflow, plus a decent reporting system so managers
> and other staff can see where items are stuck waiting for review,
> approval, etc. and can grab the request or resend a document to fix the
> problem. Managers have to do some work to keep an eye on the papers,
> but have the flexibility to intervene in an appropriate fashion as and
> when needed, e.g. before a more rigorously defined process would hit a
> timeout waiting for a step to be completed.
> The former process might look a lot like the latter in a typical
> workflow graphical editor. There will be more entities in the former
> case (to handle the exceptions explicitly) unless a rules-bases system
> is used to add them in a declarative fashion. It's a somewhat
> interesting technical problem as to the best way to represent these (and
> more complicated) workflows, how to validate them, etc.
> If you're in a high-"quality" environment -- certified laboratory,
> government procurement -- the former might be the only way to avoid
> getting sued. Personally, I'd hate it if I ever have to implement a
> "naziware" solution. The rigorous process is also "de-skilling" in that
> it removes the need for experienced staff and managers, and reduces the
> problem to a McDonald's level of experience. Consistency is often a
> synonym for mediocrity.
> In contrast, I think there are opportunities to produce *good* work, not
> just adequate work, within the latter sort of workflow. The automation
> features should (in a perfect world) provide support and handle mundane
> detail, not eliminate the need for skills and/or creativity.
> So, is there a parallel to the second example of workflow in concurrent
> programming? Is there a role for intervention (automatic or human) when
> things get snafu'd? A formal approach to mucking with tasks in-process?
>  Recently read an article on radiology interpretation off-shore, and
> it was near an article on the inflexibility of interpretation workflow
> systems -- it's hard to send an X-ray to a colleague for a casual
> opinion, or to send back comments to the radiologist if there's a
> question about the image, or to add a query about something unrelated to
> the original purpose for the X-ray.
More information about the FoRK