Over the years I have accumulated a sizable music library (mostly flacs, adding up to a bit less than 1TB) that I now want to reorganize (ie. gradually process with Musicbrainz Picard).
Since the music lives in my NAS, flacs are relatively big and my network speed is 1GB, I insalled on my computer a hdd I had laying around and replicated the whole library there; the idea being to work on local files and the sync them to the NAS.
I setup Syncthing for replication and… everything works, in theory.
In practice, Syncthing loves to rescan the whole library (given how long it takes, it must be reading all the data and computing checksums rather than just scanning the filesystem metadata - why on earth?) and that means my under-powered NAS (Celeron N3150) does nothing but rescanning the same files over and over.
Syncthing by default rescans directories every hour (again, why on earth?), but it still seem to rescan a whole lot even after I have set rescanIntervalS to 90 days (maybe it rescans once regardless when restarted?).
Anyway, I am looking into alternatives.
Are there any you would recommend? (FOSS please)
Notes:
- I know I could just schedule a periodic rsync from my PC to the NAS, but I would prefer a bidirectional solution if possible (rsync is gonna be the last resort)
- I read about unison, but I also read that it’s not great with big jobs and that it too scans a lot
- The disks on my NAS go to sleep after 10 minutes idle time and if possible I would prefer not waking them up all the time (which would most probably happen if I scheduled a periodic rsync job - the NAS has RAM to spare, but there’s no guarantee it’ll keep in cache all the data rsync needs)
Change the full-rescan interval to monthly, or yearly even, problem solved.
Syncthing should have inotify support which allows it to watch for changes rather than polling. Does that help?
Yep, this is how I do it on my NAS, which is some RockPro64 board attached to WD Red spin drives. I have music, movies, game saves, documents, pics, etc. that equal around 1.5TB and I don’t seem to get excess scanning when “watch files” is turned on.
Yes, Syncthing does watch for file changes… that’s why I am so puzzled that it also does full rescans :)
Maybe they do that to catch changes that may have been made while syncthing was not running… it may make sense on mobies, where the OS like to kill processes willy-nilly, but IMHO not on a “real” computer
why I am so puzzled that it also does full rescans
Because they’re not so foolish as to believe that inotify is infallible and complete.
You can set it to do full scans however often you like, even monthly.
Is it worth raising an issue with the project? Also enable logging to see if there are any clues as to why a rescan is being done?
Why do you need the files in your local?
Is your network that slow?I’ve heard of multiple content creators which have their video files in their NAS to share between their editors, and they work directly from the NAS.
Could you do the same? You’ll be working with music, so the network traffic will be lower than with video.If you do this you just need a way to mount the external directory, either with rclone or with sshfs.
The disks on my NAS go to sleep after 10 minutes idle time and if possible I would prefer not waking them up all the time
I think this is a good strategy to not put additional stress in your drives (as a non-expert of NAS), but I’ve read the actual wear and tear of the drives is mostly during this process of spinning up and down. That’s why NAS drives should be kept spinning all the time.
And drives specifically built for NAS setups are designed with this in mind.
