jbone at place.org
Thu Oct 22 18:47:22 PDT 2009
re: ConceptNet etc...
Stephen quotes me and says:
>> Neural networks and other similar systems are something else
>> though, and while there's a mapping here it's a bit elusive.
>> Spreading activation in semantic networks with fuzzy, defeasible
>> semantics seems like a pretty rich topic at present.
> I'm glad you now see a mapping / equivalence.
Slow your roll, there, Stephen. I did *not* say there was an
equivalence and a mapping is not an equivalence. Neural networks (the
perceptron / single-layer / ANN kind) do one thing and one thing
only: they statistically learn a discrimination surface in n-feature
space. That's all they do and it's all they *can* do, and there are
hard computational limits on what that enables. These limits were
conclusively demonstrated by Minsky and Papert et. al. in the late 60s
(cf. their book Perceptrons) though they both overstated the
implications of these limits *and* were largely the genesis, through
other people misinterpreting their work, for the almost-complete
disappearance of ANN and connectionist-type models from AI and
computer science research for a decade and a half. (Which is a damn
shame, of course.)
More complex neural network wiring schemes have some different
properties than e.g. the pure perceptron, but it's still the case that
what they do is build classification or discrimination surfaces or
predict values according to either a learned linear relationship or
(in the case of recurrent networks) some essentially tail-recursive
What's going on w/ a neural network is *not* semantics; it's formal,
it's math, and it's not even particularly topological in itself.
Hence the "something else entirely, though." The "while there's a
mapping here it's a bit elusive" comment regards the stuff that's
being done on the frontier of connectionist research, *not* the
traditional ANN but rather its extrapolations in things like
hierarchical temporal memories and Geoffrey Hinton's work. I'm *not*
stipulating any change of position regarding any earlier debate we
had; your insistence that any of these things have any particular
*equivalence* is about as useful as saying that hash tables and lists
are "equivalent" because they are both examples of data structures. ;-)
Just to clarify.
More information about the FoRK