[FoRK] Amazon S3 storage service

J. Andrew Rogers < andrew at ceruleansystems.com > on > Tue Mar 14 22:07:22 PST 2006

On Mar 14, 2006, at 6:29 PM, Stephen D. Williams wrote:
> I firmly believe that both full filesystem semantics and ACID  
> integrity constraints are red herrings and have seen a number  
> projects reach the same conclusion.  If you relax those semantics  
> and go for things like immutable distributed RAIN with  
> opportunistic caching and manage metadata separately, you can be  
> more efficient and far simpler to implement.

ACID is an ideal, not something we can actually implement.  Some  
systems get closer than others, and some are so close that we pretend  
the ideal has been achieved (e.g. proper database systems).  Relaxing  
semantics is fine, but smart design can minimize the probability of  
being impacted by relaxed semantics while gaining most of the  
benefits, and it is this particular type of optimization that appears  
to be under-developed.  There is no reason one could not run a  
database on a distributed file system if one was willing to accept  
the downsides (and some people do), but the assumptions behind most  
distributed file systems are sufficiently incorrect with state-of-the- 
art network topologies that they come nowhere near their potential.

For example, one of the things that interests me is optimizing  
distributed file systems for feature-rich Ethernet switch fabrics,  
which already exist on transcontinental and transoceanic scales and  
which have many capabilities which are eminently exploitable for  
distributed file system designs if one assumes this type of backing  

I am not arguing for kitchen sink feature sets, but more that a  
decent feature set be implemented sufficiently well that it really is  
useful for most people and many enterprises.  Centralizing metadata  
management would be a perfectly acceptable tradeoff the vast majority  
of the time, and allow a pretty solid implementation of ACID  
semantics without performance going down the tubes.

> With database-based applications, even when you have ACID  
> capabilities, there are a number of reasons to avoid updates, avoid  
> "accumulators", and otherwise avoid many of the situations where  
> you needed transactions to begin with.

Of course and the app designer should take this into consideration,  
though the impact of such features varies greatly with choices in  
locking and concurrency control.  Most conventional file systems do  
not assume high concurrency, never mind distributed ones.  Which is  
why there is no real reason full conventional semantics cannot be  
achieved, particularly if there is an assumption of centralized  

And it could be damn fast too.  Modern large-scale network fabrics  
frequently have far more bandwidth than the computers connected to  
them can drive, and would allow certain types of guarantees before  
data ever hits a disk.

J. Andrew Rogers

More information about the FoRK mailing list