quantadev 4 days ago

I've been wondering for about 20 years why File Systems basically died and stopped innovating. For example we have lots of hierarchical data structures in the world, and no one seems to have figured out how to let a folder be the storage, instead of always just databases.

For example, if we simply had the ability to have "ordered" files inside folders, that would instantly make it practical for a folder structure to represent "Documents". After all, documents are nothing but a list of paragraphs and images, so if we simply had ordering in file systems we could have document editors which are using individual files for each paragraph of text or image. It would be amazing.

Also think about use cases like Jupyter Notebooks. We could stop using the XML file format, and just make it a folder structure instead. Each cell (node) being in a file. All social media messages and chatbot conversations could be easily saved as folders structures.

I've heard many file copy tools ignore XATTR so I've never tried to use it for this purpose, so maybe we've had the capability all along and just nobody thought to use it in a big way that became popular yet. Maybe I should consider XATTR and take it seriously.

1
wfn 3 days ago

I agree! I'm sort of exploring "programmable filesystem" concept (using FuseFS) (for some notes, see [1]).

Re: ordered files: depends on FS. e.g. filesystems which use B+ trees will tend to have files (in directories) in lexical order. So in some cases you may not need a new FS:

    echo 'for f in *.txt; do cat "$f"; done' > doc.sh; chmod +x doc.sh
=> `doc.sh` in dir produces 'documents' (add newlines / breaks as needed, or add piping through Markdown processor); symlink to some standardized filename 'Process', etc...

That said... wouldn't it be nice to have ridiculous easily pluggable features like

    echo "finish this poem: roses are red," > /auto-llm/poem.txt; cat ..
:)

[1]: chaotic notes: https://kfs.mkj.lt/#welcome (see bullet point list below)

quantadev 3 days ago

Interesting stuff. Thanks for posting that.