[Noisebridge-discuss] Haschool

[Jeremy Fitzhardinge]

Jason Dusek wrote:
>> your proposed project is quite similar to one I've been toying
>> with (ie, thought about but not written a line of code), which
>> is a not-completely-braindead fuse<->s3 filesystem using the
>> Haskell fuse and curl bindings.
>>     
>
>   How do you think you'll handle the underlying eventual
>   consistency? This is an interesting problem with S3, in any
>   language.
>   

The existing fuse/s3 filesystems seem to basically ignore the problem.  
If you 1) assume that there's nothing else concurrently changing the 
files and therefore 2) the s3 filesystem always has up-to-date copies of 
recently written data, then it just needs to maintain a local cache 
until s3 is fully consistent with writes (ie, don't read back until the 
data is consistent).

Coordinated multiple writers might be fairly easy to build on top of 
that (where all writers talk to each other and maintain some kind of 
cache-coherency protocol where they can always fetch the most recent 
version of a given file).  Uncoordinated writers is going to be more of 
a challenge...

I haven't looked closely enough to see what tools s3 itself has for 
dealing with this (like, is there some kind of per-file version 
identifier/timestamp?).

A somewhat related project would be a Haskell implementation of Tahoe, 
which has all kinds of crypto goodness as well.

    J