Gosling gets it --- but a little late.
jbone at deepfile.com
Fri Jun 13 23:10:52 PDT 2003
Gosling's a great bellwether. He's *almost* never among the very first
to clue into things --- mobile code in NeWS being a possible exception,
translated slowly into Java, though they missed on the ANDF thing that
C# / .Net got right, despite Sun's centrality in participation in ANDF
in the early 90s --- and he's usually on the trailing edge of the
bleeding edge in applicability. But he's almost always among the very
first to *deeply* grok various things.
Gosling's recently grokked the fact that *programs are graphs* and
*graphs can be manipulated algorithmically.* Of course Koza (and
others) grokked this years ago. But IMHO, this more than any other
thing is one of the very few key technologies / insights that will make
automated general reasoning (aka Strong AI) possible. (Moreso than the
naive connectionist / neural net efforts by themselves, and more than
the too-low-level GA approaches, Genetic Programming by mutating parse
trees offers lots of opportunity.)
A Conversation with James Gosling, Part I
by Bill Venners
Jun 9, 2003
James Gosling talks with Bill Venners about his current research
project, code-named Jackpot, which builds annotated parse trees for
programs and can help you analyze, visualize, and refactor your program.
For the past several years, Java's creator James Gosling has been
working at Sun Labs, researching ways to analyze and manipulate
programs represented as annotated parse trees, a project called
Jackpot. Compilers have long built parse trees when they translate
source code into binary. But traditionally, programmers have worked
with source code primarily by manipulating text with editors. The goal
of the Jackpot project is to investigate the value of treating the
parse tree as the program at development time, not just at compile time.
In this interview, which will be published in multiple installments,
James Gosling talks about many aspects of programming. In this first
installment, Gosling describes the ways in which Jackpot can help
programmers analyze, visualize, and refactor their programs.
Treating Programs as Algebraic Structures
Bill Venners: What's the state of Jackpot, your current research
James Gosling: Jackpot has been really cool lately. It's what I'm
spending most of my time on, and it's been a lot of fun. I was really
hoping to have something I could hand out at JavaOne this year, but
I've been finding too many entertaining things to do.
It's a very different world when a program is an algebraic structure
rather than a bag of characters, when you can actually do algebra on
programs rather than just swizzling characters around. A lot of things
Bill Venners: Like what?
James Gosling: If you look at any of the refactoring books, most of
those refactoring actions become much more straightforward, in ways
that are fairly deep.
Moving a method isn't just cutting and pasting text. It's a lot more
than renaming the parameters and swizzling them around, because you
really want to be able to do things like construct forwarding methods.
When you construct forwarding methods, they're different from the
You can't just replace all uses of the forwarding method by uses of the
moved method, because they actually behave slightly differently. The
difference is usually around what happens when the pivot parameter is
null. That can lead you into a deep morass of essentially theorem
proving about properties of the code fragments that you're moving, to
understand how they behave with respect to null. And you can treat all
kinds of code manipulation that way.
So Jackpot has a baby theorem prover, or algebraic simplifier, that
knows an awful lot about data flow and the implications of values. And
it really does treat your program as a piece of algebra to be
simplified and transformed. It can do an awful lot of interesting
analysis that pays off when you want to make fairly significant
pervasive changes to very large programs. That analysis pays off, for
example, when you want to replace one API with another API that is
almost the same. Often "almost the same" is actually harder than
"radically different." I spent most of the last four months working on
this baby theorem prover, and that's been a lot of fun.
Creating Visual Representations of Programs
Bill Venners: I read that Jackpot can create interesting graphical
representations of a program. What is that about?
James Gosling: Jackpot can take this underlying algebraic structure—
it's really the annotated parse tree—and generate a visual
representation from that. Our internal notion of the truth is not text.
But once it's not text, all of a sudden you can display it in really
We've got an underlying rule engine that's able to do structural
pattern matching very efficiently. We can go from the structural
patterns it sees in your code to visual representations. So you can
write what is kind of like a reverse grammar, where you associate
structural patterns with what you can think of almost as TeX
descriptions of how to represent the patterns graphically. What you see
on the screen has been generated from this pattern matching. So we can,
on a user chosen basis, turn various program structures into all kinds
of visual representations.
You can, for example, turn the square root function into the obvious
mathematical notation. You can turn the identifier theta into the Greek
letter theta. You can turn division into the horizontal bar with
numbers stacked. And we've done experiments with wackier things, such
as trying to generate real time flow charts. That's kind of goofy, but
entertaining. Other things like hiding block contents, doing
interesting typography, doing typography on comments, all actually work
out reasonably well.
Bill Venners: At previous JavaOnes, I have seen some visualization
tools that I thought were useful. One of them analyzed your code and
drew diagrams that showed the coupling between packages. I felt those
diagrams could help you realize that you've got a lot of coupling going
from one package to another, which you may not realize by looking at
individual source files. Visualization tools like that can help, I
think, but a lot of tools that draw graphical representations from
source don't seem to help much. Let's say you analyze a big program and
generate inheritance charts, with thousands of boxes and lines going
all over the place. That often looks as confusing as the source code
James Gosling: Yes, doing that kind of visualization is a real
Coupling Analysis with Action
Bill Venners: What kind of analysis does Jackpot do?
James Gosling: We've got a bunch of hooks in Jackpot for plugging in
analysis modules. We want not only to be able to do analysis, but to be
able to act on that analysis. We have some pieces that do pretty
For example, often it's considered bad form to have public instance
variables. One piece of analysis, therefore, is to find all the public
instance variables. But we can find them and also make them private,
add all the setters and getters, and account for what it means to
actually access the variables via setters and getters. We can also do
things like find all methods whose natural home is not the class they
are actually in.
Bill Venners: How do you detect that?
James Gosling: If you look at the analysis books, there are various
ways of detecting that. For instance, if you've got a static method
that takes an object as a parameter, and it modifies that object, then
somebody probably just slapped that method in there because it was
easy. They were editing that file, so they put the method there. But
they really should have put it someplace else. We've got something that
will find those methods and actually move them, then change all the
uses of that method to do the right thing. So we're trying to couple
analysis with action.
Visualizing with JavaDoc
Bill Venners: What I mostly use for visualization is JavaDoc, because
it's an abstract view of the public interface. I generate JavaDoc a lot
as I'm designing and developing. I look at the HTML pages generated by
JavaDoc and think, well, this looks kind of confusing. And I go back
and make some changes to the code. I may not be able to see that it is
confusing just by looking at the code. So I think having different ways
to visualize code and designs as it is being developed can help guide
James Gosling: Jackpot's editor component tries essentially to do what
amounts to real time JavaDoc. JavaDoc is a funny thing. When I did the
original JavaDoc in the original compiler, even the people close around
me pretty soundly criticized it. And it was interesting, because the
usual criticism was: a good tech writer could do a lot better job than
the JavaDoc does. And the answer is, well, yeah, but how many APIs are
actually documented by good tech writers? And how many of them actually
update their documentation often enough to be useful?
Bill Venners: For me JavaDoc doesn't just serve as a way to document
the design, it serves as a way to visualize the design. If I see huge
classes with 300 public methods, or dozens of packages with only few
classes in each, I know there's a problem. It's more obvious when
you're looking at the JavaDoc than at the source.
James Gosling: Right. JavaDoc has been enormously successful and
enormously powerful. It's really been quite wonderful. And it's also
been interesting to see the way that professional tech writers have
taken to JavaDoc. A lot of the early criticism from them were things
like formatting. That's largely been solved by the sophisticated
doclets that people have been able to write. But the tech writers seem
to now spend a lot more time just documenting the semantics of what's
going on, and a lot less time fussing with structure and formatting.
And it actually feels like tech writers end up being more productive.
Bill Venners: So the tech writers are in there changing the code also,
and checking in their changes?
James Gosling: Yeah. That's certainly what happens around here. The
tech writers are intimately involved in the engineering. And actually
I've always found that to be a really good thing to do. One of my
general design principles is that it's really helpful to have a good
tech writer on the engineering team early on. If you're building
something and you have a tech writer trying to document it, and the
tech writer walks into your office and says, "I don't know how to
describe this," it means one of two things. Either you've got a really
stupid tech writer who you should fire. Or much more likely, you've got
a bad piece of design and you ought to rethink it. You have to rethink,
because an API that isn't comprehensible isn't usable.
Bill Venners: So the tech writer is giving you feedback on your design.
One of the values of design reviews is that programmers give you
feedback, and that's useful if it's an API because the users of APIs
James Gosling: The problem with programmers as reviewers, and
especially programmers that have been involved in the program for a
while, is that they are kind of oblivious to the complexity. And lots
of engineers are complexity junkies. Complexity is in many ways just
evil. Complexity makes things harder to understand, harder to build,
harder to debug, harder to evolve, harder to just about everything. And
yet complexity is often much easier than simplicity. There's that
really famous Blaise Pascal letter, where he starts, "I apologize for
this long letter. I didn't have the time to make it any shorter." And
that's really true.
Bill Venners: I always think the designer's job is not only to create
something that will work correctly and efficiently, but something that
is also easy for the client to understand and use.
James Gosling: Yeah, you've always got a customer on the other side,
whether that's some high end engineer, or some chemist in the lab who's
writing a piece of code. Often you've got people whose real job is
something other than software, and software is their tool. They don't
necessarily get off on all of the complexity. Making things simple can
be a real challenge.
Come back Monday, June 16 for Part II of a conversation with Elliotte
Rusty Harold. I am now staggering the publication of several interviews
at once, to give the reader variety. The next installment of this
interview with James Gosling will appear in the near future. If you'd
like to receive a brief weekly email announcing new articles at
Artima.com, please subscribe to the Artima Newsletter.
Have an opinion about refactoring tools, program visualization, or
JavaDoc? Discuss this article in the News & Ideas Forum topic, Analyze
James Gosling's Home Page:
Bill Venners has interviewed James Gosling each year for the past five
years. The four previous interviews are:
James Gosling discusses semantic models, mobile behavior, abstraction
versus vagueness, testing, and repetitive stress injury. (February
James Gosling speaks on inheritance and composition, JSPs and Servlets,
community design processes, and more. (May 2001):
James Gosling discusses developer tools, the realtime JVM, mobile
objects, strict interfaces, and more. (May 2000):
James Gosling speaks on interfaces and protocols, servers and services,
Jini, Java in the enterprise, and more. (May 1999):
More information about the FoRK