When we need to remove existing backup for whatever reason (e.g. because
we've just successfully saved the file), we should do that regardless of
whether backups are enabled or not, since a backup may exist regardless
(it could have been created before the `backup` option got disabled).