I came across Seaweed FS, which I think is perfectly architectured:
- Acknowledgement of existence of other technologies, so master metadata can be stored in Redis(Cluster), Cassandra, Etcd
- Any use case I could think of covered — FUSE mount, HDFS, S3 API gateway, WebDAV, Async Backup into the cloud
- Tiered Storage.
- Cross-Datacentre replication (I don’t need it at home, but the number of databases/storages supporting cross DC replication out of the box can be counted on one hand)
Quickstart with a single master on one RPI4 server and volume server on another took 10 minutes. Why did I spend so much time going through edge fs, Ceph and Gluster FS? Now I need to figure out how to migrate GlusterFS to it, currently, I have 3 RPI4 nodes each with 128 GB USB stick and 1 TB external drives holding:
- Docker volume — 3 replicas over 3 microSD cards
- largerpivolumes — JBOD configuration in GlusterFS over all disks (12 TB)
- Fastvolume — JBOD over usb sticks 337G
- slow volume — replicate over 2 disks with one arbiter
Now I need to plan for 30 minutes to have the same volumes setup and more importantly figure out a way to move data from Gluster into seaweed fs.
The challenge: I don’t have a single large enough disk to hold all data. Any suggestions?
Written on March 7, 2021 by Alex Mikhalev.
Originally published on Medium