Unless the vm does all its writes synchronously, it's probably better to use zfs on the vm instead of snapshotting the zvol the vm is sitting on.

Could you explain this? I thought snapshots are part of the whole vector chain in zfs? Or do you mean using ZFS in the Guest VM?

So you have zfs on the VM host, that is, where you run your VMs...

When the VM goes to do a write, the VM thinks the data is on disk, but it could still be sitting in the ZIL/SLOG on the host. If you run into issues with the SLOG, you could potentially lose writes and scramble the zvol.

Of course you can get around this somewhat by using sync=always on the zvol, at the cost of write performance. While I've never had issues from incomplete writes, it seems bad to risk leaving the host in charge of when the data actually gets written.

Not to mention, if you're using a filesystem on the VM that writes all over the zvol, you wind up snapshotting more blocks than you need to if you leave it up to the host.

On my setup, I run with sync=always/logbias=throughput/primarycache=none/secondarycache=none for the zvol, and use zfs in the VM... as well as make the snapshots inside the VM.

Interesting, thank you. Did not know this. Maybe my knowledge of ZFS needs some improvement :-)

I won't claim to be a zfs expert, and most of this is based on my interpretation of using zfs for the past ~5 years. If I'm wrong, hopefully someone will correct me. ;-)

cool, and has to be pretty fast, but it needs attention.

we use proxmox pbs, and just ordinary backups, which are so fast we never even check snapshots

The problem with all the proxmox cron / backup stuff is, that these solutions do not consider snapshots are fast, reliable, sendable and do not take up much disk space, as long as the fs has not changed too much.

So zfs-auto-snapshot is performing a snapshot every 5 minutes for an hour, every hour for a day, etc. And it auto-removes these snapshots after one "round" of snapshotting, so it should not take too much space in the long term. This is absolutely amazing, because data loss is pretty unlikely to happen and you can always free up space by removing snapshots manually. I still have backups on an external disk though :-)

I've got the point, thank you so much.

I'll dig on snapshots which I always ignored