Distributed file systems, using http

Gregory Alan Bolcer gbolcer@endeavors.com
Sat, 15 Mar 2003 08:02:12 -0800

Vinod Kulkarni wrote:
> With DAV now going mainstream, we should expect Proxy DAV services
> that connect multiple, geographically distributed DAV services.
> Such services should help in:

Just for the record.  Magi is our DAV product that does
a pretty good job of being a secure DAV file system. 
Our model is listed below.  I feel like Rodney Dangerfield


>   - Centralized namespace management - i.e. a lot of freedom
>     in how different people would like to access same resource.

There's a lightweight central server component that
optimizes dynamic DNS and does name mapping.  There's
sub-namespaces that represent individual users and
any number of machines or devices can be registered into
that sub-namespace by the user.  This solves the problem of
having too much central control and allows self-management
of your own namespace.  

We generate our own x.509 certs so that every single node
is an HTTP client and server, DAV client and server, SSL 
client and server, and contains it's own sPKI/SDSI and PKCS#12 

These nodes can be mounted as a Web drive on any
Windows file system, works with Web Folders, can create
Web shortcuts to any resource across our ad hoc DAV 
SSL tunnel, can create an OLE link to any resource
or sub-DAV-resource across our ad hoc DAV SSL 
tunnel, and even better, can trick any non-DAV or
DAV application into thinking it's using a local resource. 

>   - Of course, decentralized administration

Individual users can add and remove buddies which causes
automated access controls.  Add and remove groups and
add and remove buddies to those groups.  These are
all things that IT has to do now for most users by
adding and removing passwords.  They are happy to be
freed from the task, and having strong credentials
activated through a local desktop proxy prevents you
from having to have a name and password for every other
server on the network without watering down the
authetication and access controls.  

>   - Local attributes attached to remote DAV resources

We have this concept of a DAV virtual folder.  You can
dynamically assemble within a group or project all of the
metadata across all the machines of all the buddies in that
particular group folder on each and create a coherent,
complete file view.  We synchronize the metadata, unlike
Groove who syncrhonizes everything.  We also add online and
offline caching, local locking of remote resources, change
notification, disambiguation, caching, and a whole lot
of other useful davfs features.  

>   - Good control over versioning

We've found that the easies way to do versioning is
to do automated write-back to a DAV content server such as
livelink, stellent, sharepoint, vignette, panagon, notes or any
other half dozen automated-versioning servers.   That way the
user gets the benefits of seemingly local performance and files
and the project manager or IT get all the benefits of 
versioning and content control at the desktop.

The best feature of this is that you can now have a 
local project view of all the project files that are
stored across companies in a secure manner across
dozens upon dozens of different content servers 
with different access and authentication all rolled
up into one single project folder without having
to replicate all the content from all those multiple

>   and so on. They will probably be integrated with firewalls.

We have a way to allow two desktops behind two different
firewalls to exchange keys in a secure way so that they
can create and end-to-end encrypted, mutually authenticated
SSL connection, two-ways across both firewalls without
any changes to either firewalls.  

> Such proxy DAVs also move the filesystems from being OS specific to
> being user specific. So irrespective of where I login from, I will see
> exact same structure. Different parts of the directory tree will
> probably imported from other users/agencies, and/or exported to other users.
> We also require good control on offline data. The user-centric
> filesystem should help in this: It can integrate offline and online data
> in a seamless manner. What that means is: The filesystem under my
> administration is partly online and partly offline (in CDROMs, tapes
> etc.). Any Writable CD or harddisk newly added to such system will be
> formatted such that it has permanent ID allocated. And the "space" is
> added to your pool of available space. And so on ...
> -Vinod