Linux syncthing server selected unsupported protocol version 303

Syncthing

Syncthing is an open-source file synchronization client/server application, written in Go, implementing its own, equally free Block Exchange Protocol. All transit communications between syncthing nodes are encrypted using TLS, and all nodes are uniquely identified with cryptographic certificates.

Contents

Installation

Syncthing provides a #Web-GUI for control and monitoring. GUI wrappers like #Syncthing-GTK and #Syncthing Tray (provided in separate packages) also exist.

Running Syncthing

Starting Syncthing

Run the syncthing binary manually from a terminal. The multiple optional parameters are described in syncthing(1) .

Autostarting Syncthing

Syncthing can either be installed as a systemd system-wide service or as a systemd user service to run automatically at startup.

System service

Running Syncthing as a system service ensures that it is running at startup even if the user has no active session, it is intended to be used on a server. Enable and start the syncthing@myusername.service where myusername is the actual name of the Syncthing user.

User service

Running Syncthing as a systemd user service ensures that Syncthing only starts after the user has logged into the system (e.g., via the graphical login screen, or ssh). Thus, the user service is intended to be used on a (multiuser) desktop computer. To use the user service, start/enable the user unit syncthing.service (i.e. with the —user flag).

Syncthing-GTK

syncthing-gtk AUR provides a GTK graphical user interface, desktop notifications and integration with the file managers Nautilus, Nemo and Caja. Syncthing can be launched by Syncthing-GTK: use the interface settings to run syncthing-gtk at startup, and to state whether to launch the syncthing daemon.

Web-GUI

Syncthing provides a web interface accessible by default on http://localhost:8384.

Syncthing Tray

syncthingtray AUR complements the Web-GUI by providing a Qt-based system tray icon and desktop notifications. There exists a desktop environment neutral version and a plasmoid for Plasma 5. It also provides integration with systemd and the Dolphin file manager.

The packages also comes with the syncthingctl utility which allows to interact with Syncthing from the command line.

Configuration

After installation, Syncthing already has a proper start-up configuration. New servers and/or folders can be added by visiting the web interface. For detailed instructions on how to setup a simple network, read Syncthing’s getting started.

After a successful first start, a default repository at

/Sync is created. You can see this in the web admin interface. On the right is the list of nodes you have added. On the left is the list of repositories, which are folders you can choose to share with other nodes.

To add another node, click «Add Node» underneath the list of nodes. You will be prompted for their Node ID (which can be found on the other machine by clicking Edit > Show ID ) as well as a short name and the address. If you specify «dynamic» for the address, the syncthing announce server will be used to automatically exchange addresses between nodes. If you want to know more about Node IDs, including the cryptographic implications, you can read the appropriate Syncthing documentation page.

After saving the configuration, the syncthing server restarts automatically. Next, you can either change the configuration of the default node (click its name and then Edit ), or create a new one to share data with. Simply tick the node you wish to share the data with, and they will have permission to access it.

Local network setup

In the typical case several machines share a LAN (Local Area Network) behind a NAT (Network Address Translation) router, it is advised for a versatile configuration to:

• Activate both local and global discovery on each node. This will allow discovery in all situations, including if some of the nodes are mobile devices like laptops or Android phones, and leave the LAN and connect to the internet from the outside. This way they will still be found with global discovery.

• Use a different listen address port for each machine, like tcp://:22010 , tcp://:22011 , tcp://:22012 and so forth. This will differentiate the nodes on the global discovery servers and avoid the «Connected to myself — should not happen» message on the other local devices whenever they leave the LAN.

• If running multiple instances for different users on the same machine, set a different port for each user’s localAnnouncePort (IPv4 broadcasts) as to avoid Syncthing complaints and choose the same localAnnounceMCAddr (IPv6 multicasts) as to find other devices on the LAN without global discovery (see Options Element).

• If two instances on the same machine should find each other without global discovery, add tcp://127.0.0.1:xxxxx as device’s second address, e.g., tcp://127.0.0.1:22001 and tcp://127.0.0.1:22002 (see Device Element).

• Enable if possible UPnP port forwarding or manually forward each port to the right machine on the LAN. When a new node is discovered, Syncthing tries to use its configured listening port, 22000 by default. If this port happens to be closed, it will seek another port locally: whenever NAT traversal is enabled in Syncthing, it will attempt to use UPnP to map a random external port to the internal listening port chosen, for example 22000. If UPnP is not supported or if this is not desirable, each port should be manually forwarded to the right machine on the LAN. Eventually, if no open port can be found on both sides, relaying will be used.

Using inotify

inotify (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. Syncthing supports inotify and the functionality can be enabled in the configuration menu for individual folders.

Participate in the infrastructure

One can participate in the Syncthing infrastructure by running a global discovery server or a relay server.

Running a relay

Syncthing has the ability to connect two devices via a relay when it is not possible to establish a direct connection between them. Relayed connections are end-to-end encrypted in the usual manner, so the relay has no insight into the connection other than the knowledge of the IP addresses and device IDs.

Anyone can run a relay server and it will automatically join the Syncthing relay pool and be available to all Syncthing’s users. To run your own relay, install syncthing-relaysrv and Start/Enable syncthing-relaysrv.service . Rate limiting and other options can be configured via the command line. These options can be set in the ExecStart directive of the service drop-in file as follows:

Running a discovery server

Global discovery is used by Syncthing to find peers on the internet. Any device announces itself at startup to the discovery server which stores the device ID, IP address, port and current time. Then on request, for a given device ID, it returns the information stored in JSON format, for instance.

A list of public of global discovery server is provided. In addition, anyone can run a discovery server, to run your own, install the syncthing-discosrv AUR package.

The discovery server requires certificates to run, which should ideally be placed in /var/discosrv . The user/group syncthing needs permissions to be able to read the certificate files. You need to edit the systemd unit file to correctly point to the certificates (and to undertake any other configuration change you may want, see list).

To point the client to your discovery server, change the Global Discovery Servers variable under Settings to https://yourserver:8443/ (default port) or whatever port you have reconfigured to. The variable takes a comma-separated list of discovery servers. It is possible to include multiple ones, including the default one.

If you are using self-signed certificates, the client refuses to connect unless you append the discovery server ID to its domain. The ID is printed to stdout upon launching the discovery server. Amend the Global Discovery Servers entry to add the ID: https://yourserver.com:8443/?id=AAAAAAA-BBBBBBB-CCCCCCC-DDDDDDD-EEEEEEE-FFFFFFF-GGGGGGG-HHHHHHH .

Tips and tricks

Stop journal spam

Syncthing can be quite noisy even while it is not doing anything. The service ExecStart can be overridden to filter output directly without an extra script (adjust «grep» as needed):

Run in VirtualBox

It is possible to have Syncthing connect both locally and globally within a VirtualBox virtual machine (VM) while keeping its network adapter in the standard NAT mode (as opposed to bridged networking attached to the host computer’s adapter).

To enable this mode, Syncthing should listen to a port in the VM different from the listening port already used by the host. For example, if the default 22000 port is used by the host, one could use 22001 in the VM. The listening port in the VM can be changed through Syncthing’s Sync Protocol Listen Addresses to tcp://:22001 in the GUI Settings.

The 22001/TCP port of the host must be forwarded to the guest in this configuration. This can be done with the following command:

In this setup, relaying should not be necessary: local devices can connect to the VM on port 22001 while global devices are accessible as long as they have themselves an open port.

Running through a proxy

Syncthing can be run through a proxy to enable use behind a corporate firewall or tunneling via SSH. According to the using proxies documentation it is necessary to set the all_proxy environment variable, and it must indicate a socks5 proxy type.

• If the service is run from a script or from the command line, you must set the variables beforehand as follows:

• If it is run as a service, you must define the variables in the service configuration as follows:

You must then reload systemd daemons configurations:

and restart the syncthing@myusername.service .

This file can be edited using systemd facility systemctl edit —full syncthing@myusername.service according to the systemd#Editing provided units section.

Troubleshooting

Database issue

One may encounter database issue at some stage. To force a rescan of files and resync of database use the following command:

read-only file system error on /etc although run as root

In case Syncthing complains it is a read-only file system although the user (e.g. root on /etc ) has write permissions, check the template unit’s definition:

Within the [Service] part, there is a Hardening part and below that, there is a ProtectSystem directive which is set to full by default. See systemd.exec(5) § SANDBOXING for more information on this directive.

Create a drop-in file to override the value to something that suits your needs. If you are trying to sync a sub-folder of /etc , ProtectSystem=true should do the trick.

Источник

Настраиваем Syncthing. Синяя изолента в мелком бизнесе и дома

У меня тут внезапно завалялся гайд по настройке syncthing. Я уверен, что многим это может пригодиться.

Про революцию в дистрибуции файлов, которую совершил torrent-протокол рассказывать излишне. Одна проблема была у классических версии технологии — статичность. Если данные в каталоге изменялись, приходилось заново создавать и передавать всем новый torrent-файл. И вдруг, внезапно появляется BitTorrentSync. Наступает всеобщее счастье, теперь можно синхронизировать целые массивы изменяющихся данных. Однако, спустя время проприетарность софта выливается в сильное урезание функциональности, привязку к своим серверам и довольно агрессивную монетизацию. Вот тут и выходит на сцену идеологически кошерный свободный преемник — Syncthing.

Syncthing, как и обычный torrent может пролезть практически через любые барьеры в виде файерволлов, NAT и тому подобное. В случае серых IP-адресов полагается на централизованные релеи и сервера-анонсеры. При этом можно спокойно поднять их самостоятельно, что дает полную независимость от инфрастуруктуры разработчиков. Все узлы в кластере равноправны. Можно задать двустороннюю и одностороннюю синхронизацию, когда изменения на удаленных копиях не затронут оригинал. Умеет версионирование. Красота, одним словом.

В моем случае решалась задача ежедневного создания резервных копий баз данных. 3 независимых отделения в разных концах города. Интернет через Yota-модем, воткнутый в Mikrotik. Канал узкий и нестабильный по ширине. Основная идея, которая была реализована — обмен по принципу torrent-протокола копиями баз по ночам, когда нагрузка на сеть минимальна, а базы никто не трогает. В результате, каждое подразделение хранит копии двух соседних. Хорошая избыточность и географическая распределенность, что уже спасло однажды, когда сервер внезапно был убит ударом молнии.

Устанавливаем. По умолчанию я имею в виду сервер на базе deb-based Linux, хотя у syncthing есть и версии для Windows и Android. Для продакшена добавим репозитории из стабильной ветки и установим:

Теперь надо разобраться с демонами. Подобный софт не должен работать от имени root. Все созданные в результате синхронизации файлы должны принадлежать локальному пользователю. Поэтому есть два варианта — автоматический запуск как системный сервис либо в качестве пользовательского сервиса. Второй вариант будет работать только тогда, когда пользователь залогинился через ssh или авторизировался в локальной системе. Нас интересует системный демон и для этого лучше всего подойдет глубоко любимый общественностью systemd. Пути могут немного отличаться в разных дистрибутивах. Данный мануал применим к Debian и Ubuntu 16.04 Server. Для начала создаем юнит:

И вносим туда следующее содержимое:

[Unit]
Description=Syncthing — Open Source Continuous File Synchronization for %I
Documentation=man:syncthing(1)
After=network.target
Wants=syncthing-inotify@.service

[Service]
User=%i
ExecStart=/usr/bin/syncthing -no-browser -no-restart -logflags=0
Restart=on-failure
SuccessExitStatus=3 4
RestartForceExitStatus=3 4

Теперь остается лишь активировать сервис от имени нужного пользователя и можно настраивать ноду.

Интерфейс доступен по адресу 127.0.0.1:8384. Если вы хотите заходить с другой машины — придется поправить конфиг

/.config/syncthing/config.xml. Дальнейшие настройки предельно просты — добавляются доверенные ноды по уникальному ключу, а затем расшариваются нужные каталоги в режиме одно- или двусторонней синхронизации. При необходимости можно вручную указать ip адрес и порт ноды, без этого для установления связи будет использоваться сеть серверов-анонсеров и релеев. Более того, вы можете поднять анонсеры и релеи на своей инфраструктуре, что позволяет использовать ПО в рамках разветвленной локальной сети компании.

В любой момент можно удобно просмотреть список синхронизируемых файлов. Крайне полезный и удобный в настройке получился софт. Ресурсы потребляет достаточно экономно. прямо сейчас в режиме ожидания ест около 15 мегабайт RAM и доли процента процессорного времени. Общий объем данных в кластере в районе 500 ГБ.

Всем удачи в экспериментах. Прочной изоленты.

Update 1

ignoreDelete is an advanced folder setting that affects the handling of incoming index updates. When set, incoming updates with the delete flag set are ignored.

Некоторая проблема в том, что с точки зрения других нод, это будет рассинхроном.

Assume two devices, “Alice” and “Bob”, are sharing a folder. Bob has set ignoreDelete.

New and updated files are synchronized as usual between Alice and Bob. When Bob deletes a file, it is deleted for Alice as well. When Alice deletes a file, Bob ignores that update and does not delete the file.

In this state, Bob is fully up to date from his own point of view, as is Alice from her own point of view.

However from the point of view of Alice, who deleted a file and propagated the update, Bob is now out of date because he is yet to remove the file that was deleted.

From the point of view of Bob, who ignored the delete entry from Alice, Alice is now out of date because she is missing the file that was deleted

Update 2

Надо включить режим Introducer на раздающем сервере, тогда при добавлении папки на клиенте он на сервер добавится без подтверждений.

Источник