There's no perfect answer, since different approaches to this will introduce more complexity and inconvenience at the same time they block some of these threats. You need to consider which kinds of loss/disaster you are trying to mitigate. An overly complex solution introduces new kinds of failure you didn't have before.
As others mention, backup needs more than replication. You recover from a ransomware attack or other data-destruction event by using point-in-time recovery to restore good data that was backed up prior to the event. You need a sufficient retention period for older backups depending on how long it might take you to recognize a data loss event and perform recovery. A mere replica is useless since it does not retain those older copies. With retention, your worry is how to prevent the compromised machines from damaging the older time points in the backup archive.
The traditional method was offline tape backups, so the earlier time points are physically secure. They can only be destroyed if someone goes to the storage and tampers with the tapes. There is no way for the compromised system to automatically access earlier backups. You cannot automate this because that likely makes it an online archive again. A similar technique in a personal setting might be backing up to removable flash drives and physically rotating these to have offline drives. But, the inconvenience means you lose protection if you forget to perform the periodic physical rituals.
With the sort of rsync over ssh mechanism you are describing, one way to reduce the risk a little bit is to make a highly trusted and secured server and _pull_ backups from specific machines instead of _pushing_. This is under the assumption that your desktops and whatnot are more likely to be hacked and subverted. Have a keypair on the server that is authorized to connect and pull data from the more vulnerable machines. The various machines do not get a key authorized to connect to the server and manipulate storage. However, this depends on a belief that the rsync+ssh protocol is secure against a compromised peer. I'm not sure if this is really true over the long term.
A modern approach is to try to use an object store like S3 with careful setup of data retention policies and/or access policies. If you can trust the operating model, you can give an automated backup tool the permission to write new snapshots without being allowed to delete or modify older snapshots. The restic tool mentioned elsewhere has been designed with this in mind. It effectively builds a content-addressable store of file content (for deduplication) and snapshots as a description of how to compose the contents into a full backup. Building a new snapshot is adding new content objects and snapshot objects to the archive. This process does not need permission to delete or replace existing objects in the archive. Other management tools would need higher privilege to do cleanup maintenance of the archive, e.g. to delete older snapshots or garbage collect when some of the archived content is no longer used by any of the snapshots.
The new risk with these approaches like restic on s3 or some ZFS snapshot archive with deduplicative storage is that the tooling itself could fail and prevent you from reconstructing your snapshot during recovery. It is significantly more complex than a traditional file system or tape archive. But, it provides a much more convenient abstraction if you can trust it. A very risk-averse and resource rich operator might use redundant backup methods with different architectures, so that there is a backup for when their backup system fails!
As others mention, backup needs more than replication. You recover from a ransomware attack or other data-destruction event by using point-in-time recovery to restore good data that was backed up prior to the event. You need a sufficient retention period for older backups depending on how long it might take you to recognize a data loss event and perform recovery. A mere replica is useless since it does not retain those older copies. With retention, your worry is how to prevent the compromised machines from damaging the older time points in the backup archive.
The traditional method was offline tape backups, so the earlier time points are physically secure. They can only be destroyed if someone goes to the storage and tampers with the tapes. There is no way for the compromised system to automatically access earlier backups. You cannot automate this because that likely makes it an online archive again. A similar technique in a personal setting might be backing up to removable flash drives and physically rotating these to have offline drives. But, the inconvenience means you lose protection if you forget to perform the periodic physical rituals.
With the sort of rsync over ssh mechanism you are describing, one way to reduce the risk a little bit is to make a highly trusted and secured server and _pull_ backups from specific machines instead of _pushing_. This is under the assumption that your desktops and whatnot are more likely to be hacked and subverted. Have a keypair on the server that is authorized to connect and pull data from the more vulnerable machines. The various machines do not get a key authorized to connect to the server and manipulate storage. However, this depends on a belief that the rsync+ssh protocol is secure against a compromised peer. I'm not sure if this is really true over the long term.
A modern approach is to try to use an object store like S3 with careful setup of data retention policies and/or access policies. If you can trust the operating model, you can give an automated backup tool the permission to write new snapshots without being allowed to delete or modify older snapshots. The restic tool mentioned elsewhere has been designed with this in mind. It effectively builds a content-addressable store of file content (for deduplication) and snapshots as a description of how to compose the contents into a full backup. Building a new snapshot is adding new content objects and snapshot objects to the archive. This process does not need permission to delete or replace existing objects in the archive. Other management tools would need higher privilege to do cleanup maintenance of the archive, e.g. to delete older snapshots or garbage collect when some of the archived content is no longer used by any of the snapshots.
The new risk with these approaches like restic on s3 or some ZFS snapshot archive with deduplicative storage is that the tooling itself could fail and prevent you from reconstructing your snapshot during recovery. It is significantly more complex than a traditional file system or tape archive. But, it provides a much more convenient abstraction if you can trust it. A very risk-averse and resource rich operator might use redundant backup methods with different architectures, so that there is a backup for when their backup system fails!