[FoRK] Software's Dirty Little Secret ZapFlash

Mark Day mark_s_day at yahoo.com
Tue Sep 7 08:43:41 PDT 2004


There is a less-fortunate corollary to the programmability and resulting
flexibility of hardware: a programmable machine, in itself, does nothing
very useful.  It requires a considerable investment of programming effort
before anything interesting happens.

The basic trend right now is away from exposed programmable "computers" and
toward embedded intelligence in special-purpose devices.  This seems roughly
analogous to the way that most of us acquire motors, heaters, magnetrons,
and other electrical components. Programmability is cool for those of us who
like it, but most people would rather get on with their lives.

I don't think many people in the Real World(tm) would be interested in a
software system that offers them the same kind of programmability/utility
tradeoff as a bare machine, which appears to be roughly what the author is
advocating.

--Mark

-----Original Message-----
From: fork-bounces at xent.com [mailto:fork-bounces at xent.com] On Behalf Of
Stephen D. Williams
Sent: Tuesday, September 07, 2004 9:56 AM
To: fork at xent.com
Subject: [FoRK] Software's Dirty Little Secret ZapFlash

This sentiment is the culmination of a long history of experience showing
that it is usually better to build very flexible meta-solutions:

"Here is the critical point: virtually all software created up to this point
is fundamentally of low quality, since it's all built to meet pre-defined
requirements, and those pre-defined requirements are rarely the true
requirements for the software. The only way software quality will ever
improve is for software to become high tech. The key insight here is that
today's common understanding of what software quality means is lacking,
because those definitions apply to purpose-built software. 
We must take a step back, so that we can judge software quality based upon
its flexibility and agility, rather than how few defects it has, or how
well-documented the process of creating it might be."

http://www.zapthink.com/report.html?id=ZAPFLASH-09012004
Software's Dirty Little Secret ZapFlash

By Jason Bloomberg
Document ID: ZAPFLASH-09012004 | Document Type: ZapFlash

Since the days of Bletchley Park's Colossus, the world's first programmable
computer, the creation of software has been separate from the production of
the machines that run that software. For the first time, a machine was
programmable instead of purpose-built. In other words, people did not
determine the specific requirements for computers before they built them.
Instead, programmers wrote software later to meet those requirements. No
longer did hardware have purpose-built requirements; instead, people had the
meta-requirement of programmability -- the requirement that they be flexible
enough to meet as-yet undefined requirements at some point in the future.

Today, we use the term high technology to refer in part to programmable
machines -- whether they be computers or any number of other devices with
the computer chips that enable them to meet the meta-requirement of
programmability. While computers are high tech, the surprising fact is that
software itself is almost always low tech, because software is generally
built for specific purposes. Programmers rarely, if ever, build software
applications for the sort of general purpose uses that a computer is
suitable for. We have done an extraordinary job of building computer chips
that can run everything from weather forecasters to video games -- but the
software itself is only good for one kind of activity or the other.

Software has been low-tech for so long that we're all used to it that way.
After all, software production is still a craft-based enterprise, with
experienced practitioners turning out custom software in small teams. Just
as when cobblers produced shoes and seamstresses fashioned clothing, today's
software is either entirely custom or appallingly ill-fitting. Custom
software is enormously expensive, often requiring teams of developers, and
only meets the particular needs of the moment. 
Meanwhile, commercial off-the-shelf (COTS) software is rarely flexible
enough to meet all the needs of its users, requiring users to work within
the constraints of the software, rather than the other way around. In any
case, regardless of whether it's custom or COTS, today's software is of
appallingly low quality -- and that's software's dirty little secret.

The challenge facing us today is whether new, Service-oriented (SO)
approaches to software will help to resolve this quality issue. Clearly,
there is no guarantee that implementing a Service-Oriented Architecture
(SOA) will improve quality -- people can still implement SOAs poorly, after
all. The real questions are first, whether SOA done right will raise the
level of software quality overall, and second, what will it take to do SOA
right?

Revisiting Software Quality
Bring up the word "quality" in a business context, and people think of one
of two things -- either lowering the defect rate as approaches like Six
Sigma and ISO 9001 advocate, or formalizing the manufacturing and
development processes, as Total Quality Management espouses. Both approaches
can apply to the production of software, as well as a wide range of
manufactured goods. Developers and testers are the most familiar with the
bug hunt and fix approach to software quality, and engineers and IT
management within the enterprise often look to the Capability Maturity Model
as the established approach to formalizing the processes of creating and
managing software.

Unfortunately, both perspectives on software quality fall within the
low-tech, craft approach to software production we follow today. Neither
perspective addresses the more important and fundamental definition of
software quality, which is how well software meets the requirements set out
for it. The unfortunate fact is that when people build software to meet
specific, pre-defined requirements, it almost always falls short of meeting
the true requirements for that software, primarily because the actual
requirements for software are typically in a state of constant flux. In
other words, our true requirements for software is for it to be high tech --
that is, for software to be flexible enough to meet as yet undefined
requirements at some point in the future, just as high-tech hardware does
today.

Here is the critical point: virtually all software created up to this point
is fundamentally of low quality, since it's all built to meet pre-defined
requirements, and those pre-defined requirements are rarely the true
requirements for the software. The only way software quality will ever
improve is for software to become high tech. The key insight here is that
today's common understanding of what software quality means is lacking,
because those definitions apply to purpose-built software. 
We must take a step back, so that we can judge software quality based upon
its flexibility and agility, rather than how few defects it has, or how
well-documented the process of creating it might be.

Will Service Orientation Lead to Higher Software Quality?
Business agility is the most fundamental benefit of SOA, where business
agility means being able to meet as-yet undefined, changing business
requirements, and leverage such change for competitive advantage. 
Business agility is thus the linchpin of high-tech software. Businesses
require software that responds quickly and efficiently to a changing
business environment, and SOAs promise this level of agility.

However, it is crucial that we temper this long-term vision with a
short-term reality check. It will take many years to achieve the high-tech
vision of agile software, and there will be many roadblocks along the way.
It won't be easy, by any means; such a transition will require substantial
change, both in technology and human behavior. 
Nevertheless, today's SO approaches can provide a good measure of business
agility.

What is it about SO software approaches, then, that provide such business
agility? What technology advances are we making today that are precursors of
the vision of high-tech software? The vision of high-tech software paints a
long-term goal, and the following achievements illustrate the progress we're
already making:

    * Metadata-driven functionality. An increasing number of products are
moving business logic from the code into the metadata, as metadata
management becomes increasingly important to building and running agile
SOAs.

    * Declarative "programming." Instead of true programming, business users
are increasingly able to determine the functionality of the software they
use through configuration, rather than compilation. Such declarative
approaches to determining software functionality move the power over
business logic into the hands of business users.

    * The maturation of the discipline of software architecture. Today,
architects learn most of their craft on the job. As the discipline of
architecture matures, what it means to be an architect will stabilize,
enabling companies to plan, design, and produce high-tech software in an
industry standard manner. Enterprise architects will increasingly be able to
architect their enterprises with business agility in mind.

    * The rise of standards. We are in the early chapters of what will
likely be the long story of IT standards, but we're on our way nevertheless.
Standards are one important element to moving beyond craft production of
software. Just as standardization in manufacturing led to the
industrialization of production, standardization in software will lead to
software that is built to be agile.

    * The Service Orientation abstraction layer. Software grows by the
addition of abstraction layers that simplify complexity. SO abstracts
application functionality, giving businesses unprecedented control and
flexibility over the software they use. Such an abstraction layer is thus a
necessary first step to building high-tech software.

The ZapThink Take
ZapThink believes that Service Orientation is one of the key mechanisms for
leaving the world of low-tech, low-quality software behind, and achieving
the goals of high-tech software. Clearly, much work remains to be done, and
SO alone does not guarantee quality. However, the trend toward high-tech
software is apparent in the progress vendors and IT end-users are making
today. ZapThink predicts that we will see a gradual decline in custom
programming in the enterprise, as users become able to configure business
applications to meet changing requirements. At the same time, companies will
need fewer IT developers, as the software they purchase becomes more agile.
After all, software vendors should be the companies that produce the
software, while enterprises in other lines of business should concentrate on
what they do best instead.

Finally, ZapThink predicts the end of craft software, as standards mature
and vendors learn how to build high-tech software. Software production will
achieve benefits analogous to those that mass production achieved --
consistent quality at substantially lower cost, leading to a new era of
software quality. As the industry matures and becomes high-tech, software
will no longer be judged on how well it can meet today's requirements, but
rather how flexibly it can meet tomorrow's challenges. Will Service
Orientation solve the software industry's "dirty little secret" of appalling
quality? Only time will tell, of course. There's no question, however, that
if vendors become able to deliver software that can truly meet as-yet
undefined business requirements, then enterprises will demand such software.

sdw

--
swilliams at hpti.com http://www.hpti.com Per: sdw at lig.net http://sdw.st
Stephen D. Williams 703-724-0118W 703-995-0407Fax 20147-4622 AIM: sdw

_______________________________________________
FoRK mailing list
http://xent.com/mailman/listinfo/fork




More information about the FoRK mailing list