Gosling gets it --- but a little late.

[Jeff Bone]

Gosling's a great bellwether.  He's *almost* never among the very first 
to clue into things --- mobile code in NeWS being a possible exception, 
translated slowly into Java, though they missed on the ANDF thing that 
C# / .Net got right, despite Sun's centrality in participation in ANDF 
in the early 90s --- and he's usually on the trailing edge of the 
bleeding edge in applicability.  But he's almost always among the very 
first to *deeply* grok various things.

Gosling's recently grokked the fact that *programs are graphs* and 
*graphs can be manipulated algorithmically.*  Of course Koza (and 
others) grokked this years ago.  But IMHO, this more than any other 
thing is one of the very few key technologies / insights that will make 
automated general reasoning (aka Strong AI) possible.  (Moreso than the 
naive connectionist / neural net efforts by themselves, and more than 
the too-low-level GA approaches, Genetic Programming by mutating parse 
trees offers lots of opportunity.)

----

	http://www.artima.com/intv/jackpotP.html

Analyze this!
A Conversation with James Gosling, Part I
by Bill Venners
Jun 9, 2003
Summary
James Gosling talks with Bill Venners about his current research 
project, code-named Jackpot, which builds annotated parse trees for 
programs and can help you analyze, visualize, and refactor your program.

For the past several years, Java's creator James Gosling has been 
working at Sun Labs, researching ways to analyze and manipulate 
programs represented as annotated parse trees, a project called 
Jackpot. Compilers have long built parse trees when they translate 
source code into binary. But traditionally, programmers have worked 
with source code primarily by manipulating text with editors. The goal 
of the Jackpot project is to investigate the value of treating the 
parse tree as the program at development time, not just at compile time.

In this interview, which will be published in multiple installments, 
James Gosling talks about many aspects of programming. In this first 
installment, Gosling describes the ways in which Jackpot can help 
programmers analyze, visualize, and refactor their programs.
Treating Programs as Algebraic Structures

Bill Venners: What's the state of Jackpot, your current research 
project?

James Gosling: Jackpot has been really cool lately. It's what I'm 
spending most of my time on, and it's been a lot of fun. I was really 
hoping to have something I could hand out at JavaOne this year, but 
I've been finding too many entertaining things to do.

It's a very different world when a program is an algebraic structure 
rather than a bag of characters, when you can actually do algebra on 
programs rather than just swizzling characters around. A lot of things 
become possible.

Bill Venners: Like what?

James Gosling: If you look at any of the refactoring books, most of 
those refactoring actions become much more straightforward, in ways 
that are fairly deep.

Moving a method isn't just cutting and pasting text. It's a lot more 
than renaming the parameters and swizzling them around, because you 
really want to be able to do things like construct forwarding methods. 
When you construct forwarding methods, they're different from the 
original methods.

You can't just replace all uses of the forwarding method by uses of the 
moved method, because they actually behave slightly differently. The 
difference is usually around what happens when the pivot parameter is 
null. That can lead you into a deep morass of essentially theorem 
proving about properties of the code fragments that you're moving, to 
understand how they behave with respect to null. And you can treat all 
kinds of code manipulation that way.

So Jackpot has a baby theorem prover, or algebraic simplifier, that 
knows an awful lot about data flow and the implications of values. And 
it really does treat your program as a piece of algebra to be 
simplified and transformed. It can do an awful lot of interesting 
analysis that pays off when you want to make fairly significant 
pervasive changes to very large programs. That analysis pays off, for 
example, when you want to replace one API with another API that is 
almost the same. Often "almost the same" is actually harder than 
"radically different." I spent most of the last four months working on 
this baby theorem prover, and that's been a lot of fun.
Creating Visual Representations of Programs

Bill Venners: I read that Jackpot can create interesting graphical 
representations of a program. What is that about?

James Gosling: Jackpot can take this underlying algebraic structure— 
it's really the annotated parse tree—and generate a visual 
representation from that. Our internal notion of the truth is not text. 
But once it's not text, all of a sudden you can display it in really 
interesting ways.

We've got an underlying rule engine that's able to do structural 
pattern matching very efficiently. We can go from the structural 
patterns it sees in your code to visual representations. So you can 
write what is kind of like a reverse grammar, where you associate 
structural patterns with what you can think of almost as TeX 
descriptions of how to represent the patterns graphically. What you see 
on the screen has been generated from this pattern matching. So we can, 
on a user chosen basis, turn various program structures into all kinds 
of visual representations.

You can, for example, turn the square root function into the obvious 
mathematical notation. You can turn the identifier theta into the Greek 
letter theta. You can turn division into the horizontal bar with 
numbers stacked. And we've done experiments with wackier things, such 
as trying to generate real time flow charts. That's kind of goofy, but 
entertaining. Other things like hiding block contents, doing 
interesting typography, doing typography on comments, all actually work 
out reasonably well.

Bill Venners: At previous JavaOnes, I have seen some visualization 
tools that I thought were useful. One of them analyzed your code and 
drew diagrams that showed the coupling between packages. I felt those 
diagrams could help you realize that you've got a lot of coupling going 
from one package to another, which you may not realize by looking at 
individual source files. Visualization tools like that can help, I 
think, but a lot of tools that draw graphical representations from 
source don't seem to help much. Let's say you analyze a big program and 
generate inheritance charts, with thousands of boxes and lines going 
all over the place. That often looks as confusing as the source code 
does.

James Gosling: Yes, doing that kind of visualization is a real 
challenge.
Coupling Analysis with Action
Bill Venners: What kind of analysis does Jackpot do?

James Gosling: We've got a bunch of hooks in Jackpot for plugging in 
analysis modules. We want not only to be able to do analysis, but to be 
able to act on that analysis. We have some pieces that do pretty 
interesting things.

For example, often it's considered bad form to have public instance 
variables. One piece of analysis, therefore, is to find all the public 
instance variables. But we can find them and also make them private, 
add all the setters and getters, and account for what it means to 
actually access the variables via setters and getters. We can also do 
things like find all methods whose natural home is not the class they 
are actually in.

Bill Venners: How do you detect that?

James Gosling: If you look at the analysis books, there are various 
ways of detecting that. For instance, if you've got a static method 
that takes an object as a parameter, and it modifies that object, then 
somebody probably just slapped that method in there because it was 
easy. They were editing that file, so they put the method there. But 
they really should have put it someplace else. We've got something that 
will find those methods and actually move them, then change all the 
uses of that method to do the right thing. So we're trying to couple 
analysis with action.
Visualizing with JavaDoc

Bill Venners: What I mostly use for visualization is JavaDoc, because 
it's an abstract view of the public interface. I generate JavaDoc a lot 
as I'm designing and developing. I look at the HTML pages generated by 
JavaDoc and think, well, this looks kind of confusing. And I go back 
and make some changes to the code. I may not be able to see that it is 
confusing just by looking at the code. So I think having different ways 
to visualize code and designs as it is being developed can help guide 
the design.

James Gosling: Jackpot's editor component tries essentially to do what 
amounts to real time JavaDoc. JavaDoc is a funny thing. When I did the 
original JavaDoc in the original compiler, even the people close around 
me pretty soundly criticized it. And it was interesting, because the 
usual criticism was: a good tech writer could do a lot better job than 
the JavaDoc does. And the answer is, well, yeah, but how many APIs are 
actually documented by good tech writers? And how many of them actually 
update their documentation often enough to be useful?

Bill Venners: For me JavaDoc doesn't just serve as a way to document 
the design, it serves as a way to visualize the design. If I see huge 
classes with 300 public methods, or dozens of packages with only few 
classes in each, I know there's a problem. It's more obvious when 
you're looking at the JavaDoc than at the source.

James Gosling: Right. JavaDoc has been enormously successful and 
enormously powerful. It's really been quite wonderful. And it's also 
been interesting to see the way that professional tech writers have 
taken to JavaDoc. A lot of the early criticism from them were things 
like formatting. That's largely been solved by the sophisticated 
doclets that people have been able to write. But the tech writers seem 
to now spend a lot more time just documenting the semantics of what's 
going on, and a lot less time fussing with structure and formatting. 
And it actually feels like tech writers end up being more productive.

Bill Venners: So the tech writers are in there changing the code also, 
and checking in their changes?

James Gosling: Yeah. That's certainly what happens around here. The 
tech writers are intimately involved in the engineering. And actually 
I've always found that to be a really good thing to do. One of my 
general design principles is that it's really helpful to have a good 
tech writer on the engineering team early on. If you're building 
something and you have a tech writer trying to document it, and the 
tech writer walks into your office and says, "I don't know how to 
describe this," it means one of two things. Either you've got a really 
stupid tech writer who you should fire. Or much more likely, you've got 
a bad piece of design and you ought to rethink it. You have to rethink, 
because an API that isn't comprehensible isn't usable.

Bill Venners: So the tech writer is giving you feedback on your design. 
One of the values of design reviews is that programmers give you 
feedback, and that's useful if it's an API because the users of APIs 
are programmers.

James Gosling: The problem with programmers as reviewers, and 
especially programmers that have been involved in the program for a 
while, is that they are kind of oblivious to the complexity. And lots 
of engineers are complexity junkies. Complexity is in many ways just 
evil. Complexity makes things harder to understand, harder to build, 
harder to debug, harder to evolve, harder to just about everything. And 
yet complexity is often much easier than simplicity. There's that 
really famous Blaise Pascal letter, where he starts, "I apologize for 
this long letter. I didn't have the time to make it any shorter." And 
that's really true.

Bill Venners: I always think the designer's job is not only to create 
something that will work correctly and efficiently, but something that 
is also easy for the client to understand and use.

James Gosling: Yeah, you've always got a customer on the other side, 
whether that's some high end engineer, or some chemist in the lab who's 
writing a piece of code. Often you've got people whose real job is 
something other than software, and software is their tool. They don't 
necessarily get off on all of the complexity. Making things simple can 
be a real challenge.
Next Week

Come back Monday, June 16 for Part II of a conversation with Elliotte 
Rusty Harold. I am now staggering the publication of several interviews 
at once, to give the reader variety. The next installment of this 
interview with James Gosling will appear in the near future. If you'd 
like to receive a brief weekly email announcing new articles at 
Artima.com, please subscribe to the Artima Newsletter.
Talk Back!

Have an opinion about refactoring tools, program visualization, or 
JavaDoc? Discuss this article in the News & Ideas Forum topic, Analyze 
this!.
Resources

James Gosling's Home Page:
http://java.sun.com/people/jag/

Bill Venners has interviewed James Gosling each year for the past five 
years. The four previous interviews are:

James Gosling discusses semantic models, mobile behavior, abstraction 
versus vagueness, testing, and repetitive stress injury. (February 
2002):
http://www.artima.com/intv/goslingD.html

James Gosling speaks on inheritance and composition, JSPs and Servlets, 
community design processes, and more. (May 2001):
http://www.artima.com/intv/gosling3.html

James Gosling discusses developer tools, the realtime JVM, mobile 
objects, strict interfaces, and more. (May 2000):
http://www.artima.com/intv/gosling2.html

James Gosling speaks on interfaces and protocols, servers and services, 
Jini, Java in the enterprise, and more. (May 1999):
http://www.artima.com/intv/gosling1.html