I want to represent my directory structure in some format (currently I'm just using JSON.)
This is how a sample JSON might look. For those curious it was generated using unix tree command:
tree /path/to/folder -J --noreport -h.
So this is just a representation of the entire folder structure of some path as JSON.
I can have many such separate JSON files, each representing a directory tree. There's no co-relation / links between these files.
On running the tree command on standard Windows "C:\" partition, I get a JSON file of ~30 MB in size. So I think we can assume that the max file size would be ~100 MB.
Once the files are stored, these are the operations I plan to make on a file:
- Get the entire file.
- Given a path, get it's immediate children (akin to doing
ls on that path.)
- Given a path, get the complete sub tree of the path.
- Modify metadata of some item, say change its name or add a new
note with it.
2 & 3 are the operations that I expect to happen the most.
Here are the ways of storing this data that I've come up with:
NODB: Store the file as-is on disk
- Operation no 1 is fast & easy, send the entire file.
- but the others can get slow because they all involve parsing the entire JSON first and then iterating on it.
- I've never used nosql databases before but read a few posts about them
- I think op 1 (entire file read) would be similarly fast
- but no idea if they'll be
- I've used relational DBs before but don't think my data has anything to do with tables etc.
- I did google around though and found that postgres has an
ltree type to store hierarchical data, but I'm not sure If that is what I need.
- If it is, HOW will I get the data in?
- Again, no prior experience with these, just shooting in the dark
- At the end of the day, a directory is just a tree
Instead of creating a vanilla JSON, maybe I could generate a format that one of the Graph DBs can read-in
Once I have some graph DB object, maybe all the operations become fast enough.
My question is this: For my use case, what is the best way to store the data?