[FoRK] Science without explicit theory

Jeff Bone jbone at place.org
Sun Jul 6 16:29:04 PDT 2008

On Jul 6, 2008, at 5:12 PM, Stephen D. Williams wrote:

> They have explicit variables

Think about this:  a Bayesian network cannot have state, external to  
the temporary state maintained by the analytical algorithm.  An ANN  
with cycles --- can and does.  And the state so maintained is not  
explicit nor a function of the inputs, it's a function of the way the  
inputs coevolve over time.

The total number of such implicit stateful variables in a complex ANN  
with cycles is unknown, and possibly strictly unknowable.  Finding all  
the possible routes through a sufficiently cyclic and large topology  
is NP-hard, and a general method of finding the space of all variable  
relationships (input values, functions on input values, and algorithms  
over input values and functions on input values) that can map onto  
such a set is (so I conjecture) either impossible or AI-hard.

>> variables, and for ANNs with cycles you have significant  
>> nonlinearity,
>> which is particularly defeating of
> This is exactly where ANNs needed to go. I've fallen behind, digging
> into other things for a while. What is the best literature and
> commercial and open source software for the state of the art?

Not particularly.  The folks that are pushing the wavefront of  
understanding haven't been publishing, and those that are --- are  
looking at toy problems.

> What are you doing with it? (Forecasting financial markets or similar,
> right?)

Roughly speaking.

> Modern Bayesian / HMM probabilistic reasoning has cycles too. Not
> necessarily the same thing of course.

Right, those are probabilistic causal cycles, chains of entailment.   
But that's not too good at capturing chaotic systems.

> Then it sounds like an interesting area to investigate.

I agree...


More information about the FoRK mailing list