Rdiff backup для windows

github-actions released this Apr 10, 2021

The sole purpose of this release is to fix the issue with the 64 bits build for Windows. Everything else is the same as v2.1.0a0.

This is an alpha release so that many people can test new features. All changes are documented in the changelog, but the highlights are the new command line interface, the —help option, the support of long names under Windows v1607 and higher, and the fixing of a few more long standing bugs.

Please use this release to test and report issues, especially regarding:

• usage in client/server in combination with version v2.0.5 of rdiff-backup
• do you like the new interface? Where is it not so intuitive?
• usage under Windows (and MacOS/X)

It’s an alpha version, everything might change, disappear and/or fail, do NOT get in love with a specific feature and do NOT use this version for production.

The installation procedure hasn’t changed much and is available from the readme (alpha packages might or might not be offered by the maintainers, in doubt install via pip install rdiff-backup==2.1.0a1 under Linux and use the attached binaries under Windows). You might want to use the side-by-side installation to safeguard your productive installation.

github-actions released this Apr 8, 2021

This is an alpha release so that many people can test new features. All changes are documented in the changelog, but the highlights are the new command line interface, the —help option, the support of long names under Windows v1607 and higher, and the fixing of a few more long standing bugs.

Please use this release to test and report issues, especially regarding:

• usage in client/server in combination with version v2.0.5 of rdiff-backup
• do you like the new interface? Where is it not so intuitive?
• usage under Windows (and MacOS/X)

It’s an alpha version, everything might change, disappear and/or fail, do NOT get in love with a specific feature and do NOT use this version for production.

The installation procedure hasn’t changed much and is available from the readme (alpha packages might or might not be offered by the maintainers, in doubt install via pip install rdiff-backup==2.1.0a0 under Linux and use the attached binaries under Windows). You might want to use the side-by-side installation to safeguard your productive installation.

rdiff-backup-admin released this Jul 25, 2020

Main changes

In this version we’ve fixed 10 issues reported by users (thanks!), some of them quite long standing. We’ve also improved our development and testing infrastructure, adding test coverage and profiling instructions.
Detailed release notes are at the top of the changelog.

Install

The installation, described in details in the read me, has the quick version:

• Linux, trust on your distro to provide the latest version of rdiff-backup «soon-ish» or:
  • On Fedora/CentOS/RHEL, you may use Frank’s COPR repo
  • On Ubuntu, you may use Otto’s PPA repo
• Windows, download the asset rdiff-backup-2.0.5.win32exe.zip from the tagged release and unpack it somewhere in your PATH .
• Any supported Platform, and a few unsupported like BSD and MacOSX, use pip install rdiff-backup (or download and install the proper asset from the tagged release).

Notes

It’s probably the last bug-fix release of this 2.0 branch because we now need to focus on improving/modernizing/simplifying the code, and this will take a little while. This doesn’t mean that there aren’t known bugs but they require the code to be cleaner before they can be properly addressed. This will surely require some months so be patient or help us to be faster.

This is also most probably the last version of rdiff-backup officially supporting Python 3.5, which is due end-of-life in September 2020, and also because we want to support the upcoming Python 3.9, due one month later, and can’t support more than 4 versions of Python in parallel.
In other words, if you’re still using Python 3.5, it’s time to think about upgrading; your security will thank you anyway.

If you encounter issues, contact us on the rdiff-backup-users mailing list and/or report an issue.

We could also use some help, so don’t hesitate to raise your hand on the mailing list if you’d like to help your favourite backup tool!

Rdiff backup для windows

rdiff-backup is a simple backup tool which can be used locally and remotely, on Linux and Windows, and even cross-platform between both. Users have reported using it successfully on FreeBSD and MacOS X.

Beside its ease of use, one of the main advantages of rdiff-backup is that it does use the same efficient protocol as rsync to transfer and store data. Because rdiff-backup only stores the differences from the previous backup to the next one (a so called reverse incremental backup), the latest backup is always a full backup, making it easiest and fastest to restore the most recent backups, combining the space advantages of incremental backups while keeping the speed advantages of full backups (at least for recent ones).

If the optional dependencies pylibacl and pyxattr are installed, rdiff-backup will support Access Control Lists and Extended Attributes provided the file system(s) also support these features.

In older Linux distributions the rdiff-backup versions are of the 1.x series, which is not recommended for new installs anymore. Follow the instructions below to install the latest 2.x release of rdiff-backup.

CAUTION: rdiff-backup 1.x and 2.x aren’t compatible and can’t be mixed in server/client mode! See how to migrate side-by-side.

Ubuntu Focal or Debian Bullseye or newer (has 2.0)

Ubuntu backports for older versions (needs a backported 2.0)

CentOS and RHEL 7 (From COPR)

NOTE: the last line is optional to get ACLs and EAs support.

CentOS and RHEL 8 (From COPR)

NOTE: you can add the option —setopt=install_weak_deps=False to the last line if you don’t need ACLs and EAs support. You can install python3-pylibacl and python3-pyxattr also separately. Under RHEL, the repo to enable is codeready-builder-for-rhel-8-x86_64-rpms in order to get access to pyxattr, instead of PowerTools.

NOTE: for earlier versions, see the COPR instructions below.

Debian and derivatives, Raspbian, etc. (from PyPi)

NOTE: If your platform is not i386 or amd64, e.g. ARM/MIPS/etc, you may need the build dependencies build-essentials , librsync-dev .

CentOS and RHEL 6 (from PyPi)

Fedora and derivatives (from PyPI)

Other Linux and UN*X-oid systems, e.g. BSD (From PyPi)

You need to make sure that the following requirements are met:

• Python 3.6 or higher
• pip3 e.g. installed with curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; sudo python3 get-pip.py .
• librsync 1.0.0 or higher
• pylibacl (optional, to support ACLs)
• pyxattr (optional, to support extended attributes) — the xattr library (without py) isn’t regularly tested but should work and you will be helped
• if Python’s version is 3.7.x or below, importlib-metadata 1.x (or alternatively setuptools)
• SSH for remote operations

Then you should only need to call the following before you can use rdiff-backup:

NOTE: especially if your platform is not i386 or amd64, e.g. ARM/MIPS/PowerPC/etc, but also if the pip3 installation fails with include [. ].h files missing, you may need the build dependencies named like python3-devel or librsync-dev .

Just download and unpack the file rdiff-backup-VERSION.win32exe.zip available as asset attached to one of the releases available in the releases section and drop the binary rdiff-backup.exe somewhere in your PATH and it should work, as it comes with all dependencies included.

For remote operations, you will need to have an SSH package installed. We recommand using OpenSSH from http://www.mls-software.com/opensshd.html

Creating your first backup is as easy as calling rdiff-backup (possibly as root), e.g. rdiff-backup -v5 /home/myuser /run/media/myuser/MYUSBDRIVE/homebackup would save your whole home directory (under Linux) to a USB drive (which you should have formatted with a POSIX file system, e.g. ext4 or xfs). Without the -v5 (v for verbosity), rdiff-backup isn’t very talkative, hence the recommendation.

Subsequent backups can simply be done by calling exactly the same command, again and again. Only the differences will be saved to the backup directory.

If you need to restore the latest version of a file you lost, it can be as simple as copying it back using normal operating system means (cp or copy, or even pointing your file browser at the backup directory). E.g. taking the above example cp -i /run/media/myuser/MYUSBDRIVE/homebackup/mydir/myfile /home/myuser/mydir/myfile and the lost file is back!

There are many more ways to use and tweak rdiff-backup, they’re documented in the man pages, in the documentation directory, or on our website.

If you have everything installed properly, and it still doesn’t work, see the enclosed FAQ, the rdiff-backup web page and/or the rdiff-backup-users mailing list.

We’re also happy to help if you create an issue to our GitHub repo. The most important is probably to explain what happened with which version of rdiff-backup, with which command parameters on which operating system version, and attach the output of rdiff-backup run with the very verbose option -v9 .

The FAQ in particular is an important reference, especially if you are using smbfs/CIFS, Windows, or have compiled by hand on Mac OS X.

Rdiff-backup is an open source software developed by many people over a long period of time. There is no particular company backing the development of rdiff-backup, so we rely very much on individual contributors who «scratch their itch». All contributions are welcome!

There are many ways to contribute:

• Testing, troubleshooting and writing good bug reports that are easy for other developers to read and act upon
• Reviewing and triaging existing bug reports and issues, helping other developers focus their efforts
• Writing documentation (e.g. the man page), or updating the webpage rdiff-backup.net
• Packaging and shipping rdiff-backup in your own favorite Linux distribution or operating system
• Running tests on your favorite platforms and fixing failing tests
• Writing new tests to get test coverage up
• Fixing bug in existing features or adding new features

If you don’t have anything particular in your mind but want to help out, just browse the list of issues. Both coding and non-coding tasks have been filed as issues.

For source code related documentation see docs/DEVELOP.md

Installing latest development release

To provide meaningful bug reports and help with testing, please use the latest development release.

Ubuntu and Debian development releases

Бэкап большого количества мелких файлов

Рано или поздно настройкой резервного копирования рабочих файлов озадачивается любой уважающий себя современный IT специалист. После ряда опечаток/ошибок программистов нашлось время для этого и у меня.
Специфика веб-приложения такова, что рабочий каталог занимает более 50ГБ на жестких дисках, включая в себя около 900 тысяч мелких файлов (картинки, превьюшки, . ). Поэтому в лоб решить задачу с помощью tar и аналогов не вышло. Да и хотелось бы иметь некоторую вариативность хранимых данных, а в случае с полным бэкапом реализация требовала больших затрат на хранение по сути одинаковых данных с небольшими изменениями. Плюс неплохо было бы дублировать копии на удаленном сервере бэкапов для снижения риска потери критической информации в результате краха железа. После скрупулезного анализа поисковых выдач и отбрасывания заведомо неподходящих мне методов, остановился на паре вариантов, навязываемых чаще всего в комментариях к самописным shell-велосипедам энтузиастов.

rdiff-backup

rdiff-backup показался с виду более подходящим и удобным. Написанный на python, он хранит данные инкрементально, позволяя получить состояние файла или директории в любой момент времени (в обозримом прошлом, учитывая время хранения снимков). Гибкое управление из консоли дает полную свободу действий и предоставляет полный контроль за ситуацией. Автоматическое создание бэкапов требует добавления в планировщик пары команд (вторая для подчистки старых снимков, не несущих какой-либо ценности в силу давности изменений).
Но тестирование показало, что утилита весьма прожорлива и справляется с моей задачей крайне неохотно. Дело в том, что ежедневно изменяется небольшой объем данных (300МБ), но изменения затрагивают около 30 тысяч файлов. На выявление измененных файлов, судя по всему, тратится бОльшая часть времени работы программы. После часа наблюдений на увеличившийся до неприличных 20% iowait во время очередного запуска скрипта было принято решение попробовать другой софт и сравнить их между собой.

rsnapshot

rsnapshot, написанный на Perl, базируется на rsync. В рабочей директории программы (назовём так место, куда складываются бэкапы), создается ряд папок с индексом, который при каждом следующем запуске программы увеличивается до указанного в конфигурации значения. Затем устаревшая копия удаляется. Если пройти в любую из созданных папок, то внутри можно найти полную копию резервируемых данных. На это указывает и общий размер папки (при просмотре стандартными средствами Midnight Commander, например) — он равен сумме всех папок. На самом деле это не так. Программа создает жесткие ссылки между одинаковыми данными в пределах рабочего каталога. Таким образом, последняя актуальная копия является самой «тяжелой», а размер всех остальных составляет разницу в измененных данных.

Тестирование

Так, как оба варианта используют приблизительно одинаковый размер для хранения, пришло время проверить скорость выполнения заданий по созданию резервных копий.

Для тестов была взята случайная папка проекта размером 11ГБ, содержащая 593 подкаталога разной степени вложенности и 230911 файлов. Размер файлов плавает от 4 до 800КБ, как указано выше, это в графический материал. Обе утилиты тестировались поочередно, внешние факторы практически полностью отсутствовали (нет других пользователей, нагрузки и тяжелых процессов). С помощью утилиты time было подсчитано время выполнения каждого из тестовых заданий, а также для сравнения копирование всего каталога средствами cp: