Sync linux and windows

Синхронизация данных между ПК с Windows и Linux через интернет

Проблема такова: есть два компьютера, достаточно далеко друг от друга, на одном Windows, на другом Kubuntu. Необходимо забрать с ПК с Windows около 100 гигабайт файлов. Чем это сделать? Пробовал Bittorrent Sync, но он платный, linux-версия консольная непригодна к использованию юзером левела «юзер», версия с GUI устарела и ссылки с windows-версии не жрёт. Выдрал код из ссылки, вставил в GUI-версию — не качает.

Больше ничего подоходящего не нашёл. Есть ещё варианты?

P.S. Однако же, долго меня тут небыло 😉 Как чо, что нового? Скоро вендекапец?

собственное облако, ftp, например

rsync под Cygwin

Собственное или любое облако — решение хорошее, но платное, например Dropbox’у надо заплатить 10 баксов за 1Тб. Это самый дешёвый вариант из платных. Но неужто нет свободных решений?

А связать их как, у обоих ПК серые IP же.

У меня в закладках syncthing, но я его не пробовал и только что увидел, что оно на go.

OwnCloud — свободное бесплатное решение.

FTP, SFTP. Последний поднять особо просто, но для вантуза понадобится клиента поставить.

Поднимать вебсервер с овнклаудом только ради разового переброса?
А не треснет спопывалка?

Источник

Как лучше реализовать непрерывную синхронизацию файлов и прав доступа к ним между двумя серверами на Linux и Windows?

Имеются два удалённых друг от друга сервера на Ubuntu 16.04 и Win2012r2.
Оба втянуты в один домен с AD, на обоих имеются файловые шары с множеством различных прав доступа через AD.
В шарах порядка 3млн. файлов в 400тыс. директорий. Всего чуть более 3ТБ данных.
Всё это дело надо непрерывно синхронизировать с минимальными задержками между серверами хотя бы в одну сторону (с линукс на винду) и, желательно, бесплатными решениями.

Рассмотренные варианты:
Resilio Sync — не умеет синхронизировать расширенные атрибуты (xattr, ACLs), есть Connect, но там платно уже,
Syncthing — тоже не умеет синхронизировать расширенные права доступа,
rsync — всё отлично синхронизирует, но долго строит дерево каталогов,
lsyncd — нельзя синхронизировать с виндой, пробовал монтировать виндовую шару через CIFS, но там нет inotify и ничего не получилось,
osync — тоже не получилось синхронизировать с примонтированной виндовой CIFS директорией, выдаёт
Unison — не умеет синхронизировать расширенные права. Хотя двусторонняя синхронизация на тесте с параметрами -repeat watch -fastcheck true быстрее всех остальных, но на больших объёмах данных я так и не дождался окончания начального построения,
robocopy — встроенная утилита в винде, аналогична rsync в линуксе, на нём и остановился пока, т.к. там есть встроенные параметры мониторинга изменений в исходной шаре.

Запускаю в несколько процессов команды на каждую шару (для ускорения):

/MON:1 — мониторит изменения источника за 1 минуту после окончания синхронизации и перезапускает процесс.
Проблема та же, что и у rsync — долгое построение дерева, отключить это, как в lsyncd или Unison, по видимому никак нельзя. Если кто знает, как можно отключить первоначальное построение дерева и сразу передавать изменения, подскажите, пожалуйста.
То что без начальной синхронизации менее надёжно знаю, планирую раз в сутки полностью прогонять rsync или robocopy.

Есть ли что-то ещё, что умеет синхронизировать так же и расширенные атрибуты, без первичного построения дерева?
Robocopy умеет как-то мониторить изменения файловой системы в линуксовой шаре, под линукс ничего, что так же работало бы с шарами, я не нашёл, только lsyncd и его производные, следящие за inotify.

В отдалённой перспективе снести винду и сделать везде lsyncd или zfs репликацию, но это дело будущего, сейчас надо решение в текущей конфигурации.

Источник

4 Free Software to Sync Files Between Windows and Linux Computers

I have a Windows laptop and an Ubuntu netbook, and it is a constant struggle to keep data in sync between both of them.

So, I started researching on the free software that can keep files and folders in sync between Windows and Linux computers. Most of the results that came use some sort of shell scripting, which isn’t something I prefer. I was looking for simple GUI tools to sync Linux and Windows computer.

After some time, I was able to find some good options. All these options are completely free, and simple to setup.

1) Dropbox

I am sure if anyone has heard about Dropbox, they would have immediately guessed that an easy solution to my problem is Dropbox. Dropbox installs on both Windows as well as Linux computer. After that, you create a Dropbox folder on both the computers. Dropbox will monitor any changes that happen to that folder in real time, and will sync it with Dropbox folder on other computer. Dropbox supports Mac as well, so you can actually keep data in sync between Linux, Windows, and Mac.

Dropbox doesn’t requires both the computers to be online at the same time. Dropbox first backs data to its own server, and from there it propagates changes to other computers. So, if you work on a Windows computer in office during day, Dropbox will keep backing up your data to its own server. When you go home and start your Linux computer, Dropbox will fetch the changes from the server, and will sync the data on your Linux computer.

Dropbox runs in the background, and you don’t have to do anything once you have installed it. It will automatically take care of syncing data between Windows and Linux computer for you.

The only limitation with Dropbox is that free version supports only 2GB of data. You can get more free space by referring your friends (250 MB per friend). (If you use the link above to sign up with Dropbox, you will automatically get 250MB additional free, and I will get the same).

2) SpiderOak

SpiderOak is a lot similar to Dropbox (we call it one of the dropbox alternatives). This can also sync data between Windows and Linux computers, and also stores a copy of the data online. Both the computers do not need to be online at the same time.

SpiderOak syncs data in a completely secure manner. The free version of SpiderOak comes with 2GB of free space. However, SpiderOak measures space after compressing data, so you might potentially be able to sync more data with SpiderOak. On the other hand, SpiderOak does not gives option of getting extra free space by referring friends.

As is with Dropbox, SpiderOak also supports Mac too.

Both Dropbox and SpiderOak provide cloud based file syncing, so that your Windows and Linux computer do not need to be online at same time. The tools that I will mention below need both the computers to be online at same time (or, using a server to provide temporary storage.)

3) Unison File Synchronizer

Unison File Synchronizer is an intelligent file synchronization software that can keep data in sync between Windows and Linux computer. Unison File Synchronizer can update changes done on Windows computer to your Linux computer, and vice versa.

Unison makes use of intelligent algorithm to find the changes that needs to be synced. If a file is present on one computer, and not present on other, it is copied to other computer. In case there is a conflict that Unison cannot resolve, it asks you for resolution.

Unison needs both the computers to be online at the same time. Or, you can use an external drive to act as an intermediary. So, you can first sync your Windows computer to your external hard drive or USB stick, and then sync your Linux computer with that.

Unison has a GUI based interface, but still it might not be very intuitive to setup Unison on multiple computers.

Here are two good resources that explain how to setup Unison:

4) JFileSync

JFileSync stands for Java File Synchronization. JFileSync is a file synchronization software made in Java to sync files between multiple computers. JFileSync is truly cross platform, and can sync data across both Windows and Linux computer.

JFileSync comes with a nice GUI that makes it easier to setup. The library is around just 200 KB, and it does not needs any other additional software to work.

To sync data between Windows and Linux computer using JFileSync, you define the folders of both the computers as Sync targets. JFileSync will then do a comparison between them, and start the sync process.

JFileSync also requires both the computers to be online at same time, or you can use an external drive as an intermediary.

I went through JFileSync documentation, and it looks quite simple to setup. However, I did not get a chance to test it myself, so I am listing it at fourth position. If you have tried JFileSync, do share your feedback in comments section.

If you just want to sync data between multiple computers running Windows, you can try easier options like Live Mesh, and Microsoft Sync.

Источник

FreeFileSync Open Source File Synchronization

Synchronize Files and Folders

Latest News

Download the new FreeFileSync version 11.14 from the official project homepage:
https://freefilesync.org

Latest changes:

• Authenticate (S)FTP connections using OpenSSL 3.0
• Fixed E_NOINTERFACE error after synchronization
• Preempt crashes due to Nahimic Sonic Studio 3
• Hide main window when minimizing progress window (macOS)
• Avoid second dock icon when minimizing progress window (macOS)

With FreeFileSync version 10.20 donors to the project can now get their synchronization logs sent to them as a nicely-formatted email .
This is implemented by integrating a professional transactional email service (Mailgun) which means that no complicated configuration is needed, e.g. for an SMTP server login. Just enter your email address in the synchronization settings and be done.

The conditions for the FreeFileSync Donation Edition have changed – for the better. Each user may now install the Donation Edition on up to three of his devices instead of just one.
Previously it wasn’t clear how to «count users» without requiring some inconvenient form of authentication. FreeFileSync solves this problem by acknowledging that it’s normal to own multiple devices and counting them instead. Three devices is probably enough for most users, but not for all:

For donors who are maintaining a larger number of devices, the update intervals have been doubled. And as an added bonus, installing FreeFileSync on a virtual machine does not increase the overall device count anymore.

The FreeFileSync Donation Edition with support for email notifications can be obtained via the official website: https://freefilesync.org

FreeFileSync version 10.8 adds support for Google Drive as a new option to synchronize cloud storage besides SFTP and FTP.

FreeFileSync directly accesses the files online, without the need to install additional synchronization software such as Google Backup & Sync.
This saves the hard drive space that is usually needed for a full local copy of all Google Drive files, and enables further performance improvements with FreeFileSync’s ability to copy multiple files in parallel.

The result of directory listings is buffered by FreeFileSync, which allows for future accesses to complete instantaneously. Changes that happen outside of FreeFileSync are synchronized automatically by only transferring the small delta of differences.

Google generously has granted high quotas for FreeFileSync’s users, so even the most-demanding sync scenarios are covered.

FreeFileSync with Google Drive support can be downloaded from the official website: https://freefilesync.org

FreeFileSync version 10 has added support for copying multiple files in parallel. It is now possible to specify the number of parallel file operations for each device inside the settings. During synchronization FreeFileSync will then spawn several tasks accordingly instead of processing only one file after another.

This new design offers huge performance improvements for all scenarios that are dominated by latency, like synchronization against network shares or cloud devices including SFTP and FTP(S). The idle time after issuing each file I/O request while waiting for the network response is not the limiting factor anymore: The number of parallel operations can be set as high as needed until the bandwidth is saturated and the network card is operating at full speed.

The new parallel file operations architecture is not limited to the synchronization step, but used throughout the application. Folder comparison will issue multiple requests at a time even when traversing only a single base folder, by dynamically managing the workload while recursively reading the folder tree. Binary-comparison also processes all files in parallel.

The FreeFileSync installer is now ad-free! Before version 10, FreeFileSync had shown a single advertisement during installation to help fund the project. Beginning with version 10, the project is fully relying on donations from users to finance its ongoing expenses in software development and support. A huge thanks to everyone who is contributing to make this possible!

– Zenju [project lead] –

FreeFileSync.org has released video tutorials that help you get up to speed quickly with the most important synchronization scenarios:

https://freefilesync.org/tutorials.php

• Mirror Synchronization
• Two Way Synchronization
• Folder Comparison
• External Applications
• Batch Jobs
• Task Scheduling
• RealTimeSync
• Tips and Tricks

Cross-Platform File and Folder Synchronization Software adds Support for FTP and FTPS

FreeFileSync (https://freefilesync.org) has released version 9.1 of its free open source file synchronization utility for Windows, Linux, and macOS.

The new version supports synchronization via FTP (File Transfer Protocol) and the encrypted variant FTPS (SSL/TLS). This further enhances FreeFileSync’s capabilities to not only work with local drives and network shares but also synchronize mobile and cloud storage.

FreeFileSync eliminates the tedious and error-prone task of manually identifying files that have changed on the source folder and copying them to the target folder, no matter what protocol is used to access files. Users can set up mirror, two-way, or custom synchronization rules, and then create batch jobs that run the sync without requiring manual interaction. Batch jobs can be started with a simple mouse double-click or triggered automatically by a task scheduler.

FreeFileSync detects moved files and reproduces the move on the target drive. This avoids time consuming file copy and delete operations and instead completes the task with a single hard drive access. This even works in mirror synchronization scenarios where the target file system does not support file IDs (e.g. FTP/SFTP) or has unstable file IDs (e.g. FAT).

FreeFileSync includes features that are normally found only in the premium versions of commercial applications. It can copy locked files using the Volume Shadow Copy Service, compare file content in binary mode, handle symbolic links and run real two-way synchronizations including detecting conflicts and propagating deletions. FreeFileSync fully supports long file paths with more than 260 characters, copies NTFS extended attributes, security permissions and alternate data streams. The application takes advantage of 64-bit and multiple CPU cores if available and is fully Unicode compatible.

FreeFileSync supports MTP (Media Transfer Protocol), making it easy for users to synchronize files and folders between their PCs and their iPad, iPhone, Android smartphone, tablet, digital camera, digital audio players, or other MTP devices. The tool is fully integrated with Windows Explorer and allows users to jump to their MTP files in Explorer, open them directly with their default application, or to drag and drop MTP folders from Explorer into FreeFileSync. FreeFileSync supports FTP, FTPS, SFTP, enabling web developers to publish changes to their local website copy by running a FreeFileSync batch job — with only a double-click of the mouse.

The software has been optimized for usability and performance. It can scan a hard drive with hundreds of thousands of files in a few seconds, while scanning multiple folders in parallel. The synchronization steps are laid out in the optimal order that minimizes peaks in disk space consumption and FreeFileSync’s fail-safe file copy algorithm prevents data corruption when the synchronization process is terminated unexpectedly, e.g. due to power loss. Graphs and detailed messages give relevant feedback and provide structure on multiple visual and textual levels: getting an overview of a hard drive’s space distribution can be quickly accomplished by navigating a directory tree showing file count and sizes below each node.

Pricing and Availability

FreeFileSync is available without cost. FreeFileSync runs on Windows XP/Vista/7/8/8.1/10, Mac OS X 10.8 and newer, and all major distributions of Linux.

Источник