Docker sync mac os

docker-sync

Run your application at full speed while syncing your code for development, finally empowering you to utilize docker for development under OSX/Windows/Linux*

Developing with docker under OSX/ Windows is a huge pain, since sharing your code into containers will slow down the code-execution about 60 times (depends on the solution). Testing and working with a lot of the alternatives made us pick the best of those for each platform, and combine this in one single tool: docker-sync

• Support for OSX, Windows, Linux and FreeBSD
• Runs on Docker for Mac, Docker for Windows and Docker Toolbox
• Uses either native_osx, unison or rsync as possible strategies. The container performance is not influenced at all, see performance
• Very efficient due to the native_osx concept
• Without any dependencies on OSX when using (native_osx)
• Backgrounds as a daemon
• Supports multiple sync-end points and multiple projects at the same time
• Supports user-remapping on sync to avoid permission problems on the container
• Can be used with your docker-compose way or the integrated docker-compose way to start the stack and sync at the same time with one command
• Using overlays to keep your production docker-compose.yml untouched an portable
• Supports Linux* to use the same toolchain across all platforms, but maps on a native mount in linux (no sync)

Besides performance being the first priority for docker-sync, the second is, not forcing you into using a specific docker solution. Use docker-for-mac, docker toolbox, VirtualBox, VMware Fusion or Parallels, xhyve or whatever!

Documentation, Installation, Configuration

All the information, tutorials and best practises are provided in the Documentation

Источник

Michaël Perrin

Sustainable web development.

3 ways to get Docker for Mac faster on your Symfony app.

UPDATE (2017-05-05): article has been updated for docker-sync 0.4.

Docker is an amazing tool to set up your whole development platform locally (and elsewhere too).
As you may know, it has many benefits, including:

• Same, versioned and deterministic configuration accross environments and developers.
• Ease of use (adding any technical stack to the project is very easy, with just a few lines of configuration).
• Much better performance than a VM, at least on Linux (that’s the point of this article).
• Easy deployment.

However, you probably have noticed that running a Symfony app with Docker for Mac is very slow, almost unusable however fast is your Mac.
This article will explain several ways to make your Symfony runnable on Docker for Mac.

We are first going to set up a simple Symfony project to evaluate how the exposed solutions perform.
If you already have a Symfony project, you can skip directly to the solutions.

The performance tests have been measured on a 2016 MacBook Pro and are intended to show the variations depending on the solutions.

Demo project setup.

The example project will set up a very simple Symfony app with a typical Docker setup with the following containers:

• Nginx webserver.
• PHP 7.1.
• A simple container for the Composer binary.

The Symfony app will be stored in an app folder and the project file structure will look like this:

Setup the Symfony project

In the symfony-docker-test folder, run:

This needs the symfony command to be installed on your host, as described on the Symfony installation doc.

Define containers

Edit the docker-compose.yml file at the root of the project:

Setup the PHP container

Add the following Dockerfile file to the docker/php folder:

Add PHP configuration to the docker/php/php.ini file:

Setup the Vhost for Nginx

Edit the docker/nginx/app.conf file:

Run the project

Now that the project has been set up, simply run:

The app should now be available at http://127.0.0.1:8080 and you should see this kind of page:

So. everything is fine, expect one thing: the app is very slow for now 😩

Let’s run a simple benchmark for the prod and dev environments (you will mostly use the later):

• Prod: 17 seconds
• Dev: 129 seconds (ouch!)

Solution 1: Move vendors out of the shared directory

After several tests (as performed here: https://github.com/michaelperrin/docker-symfony-test), it turned out that
the bottleneck that slows down the application is the sharing of the vendor dir, that has a lot of files in it.

One efficient solution is to separate the vendor dir and make it only available in the container and not on the host.
Surprisingly, not sharing the cache directory doesn’t result in a big win, but it can be a second step of performance tuning.

To do so, edit your composer.json file in the app folder and add the config-dir parameter in the config entry:

Edit the app/autoload.php file in the app folder and change this line:

Add the /app-vendor folder as a volume for the php container in docker-compose.yml :

Make sure you have cleared your Symfony cache folder and install composer dependencies again by running this command:

• Prod: 2.8 seconds
• Dev: 16 seconds

Let’s see if we can do even better without cache sharing. Edit Symfony’s AppKernel.php file to change the getCacheDir method this way:

• Prod: 1.2 seconds
• Dev: 5 seconds

Not bad. But beware! The vendor files are now concealed in your container and won’t show anymore on the host. You won’t be able to debug the vendors, and autocomplete won’t be available in your IDE. My advice? Install first the dependencies in the standard directory, and then change again the composer.json file to make them installed in the container. This workaround is not as bad as it sounds if your dependencies don’t change often.

Solution 2: Docker sync

Docker-sync (http://docker-sync.io/) is a tool that makes use of rsync to synchronize volumes files between the host and your containers
instead of using Docker’s osxfs system.

Add a docker-sync.yml file at the root of the project:

Copy the docker-compose.yml file to docker-compose-dev.yml file and add these lines to the end:

Use a named volume for your app’s code by changing this:

Add now a Makefile file at the root directory, that will allow easy start / stop commands whatever the system the project is run on:

You can now start your project by running:

This will start your containers and the docker-sync daemon (answer yes to all questions the first time).

• Prod: 0.6 seconds
• Dev: 1.2 seconds

That’s a big win! Unfortunately, I experience several sync problems at times, with files not being synced from the host to the container, and with some user right issues on some files as well (chmod 777 to the rescue).

Solution 3: Docker’s cache system

The Docker team is aware of the slowness of Docker for Mac (see here and here)

The latest beta (aka. «Edge») versions of Docker have introduced some new ways to mount volumes.

If you have downloaded the edge version, simply change the docker-compose.yml file at the root of the project to add the :cached option to the volume share:

• Prod: 5.1 seconds
• Dev: 15.7 seconds

Not bad, but not as efficient as the other solutions. That can be enough for some projects though.
I can’t tell for now if there are downsides for this solution, but I will experiment it from now.

Conclusion

Good news! Docker on Mac can be usable for your Symfony project. There is no perfect solution though. Have a look to a summary of all the possibilities below and take your pick!

Dev benchmark	Prod benchmark	Pros	Cons
Default	129 seconds	17 seconds	Easy	Unusable
Solution 1: Non-shared vendors and cache	5 seconds	1.2 seconds	Fast	Vendors are not visible to host
Solution 2: Docker-sync	1.2 seconds	0.6 seconds	Very fast	Use of 3rd-party tool
Solution 3: Docker’s volume cache	15.7 seconds	5.1 seconds	Easy solution	Not as fast. Experimental for now.

Michaël Perrin

Read more posts by this author.

Share this post

Using Redis for session handling in Symfony 4.1+

Handling sessions outside of the local filesystem is getting even more usual in a cloud-based environment. Using Redis is…

Migrating users to a secure hashing algorithm in Symfony

Your app may use an old and unsecure hashing algorithm for storing passwords, like MD5 (without salt). This article…

Michaël Perrin © 2021 Proudly published with Ghost

Источник

Cut your Docker for Mac response times in half with docker-sync

I really do love the concept of Docker and containerization. I haven’t touched my MAMP dev setup for more than a year now and barely use my local PHP CLI. But there is and always was one problem: performance. Response times of a second for Laravel apps and 3-7 seconds for a larger WordPress stack are quite common. Thankfully, there is a solution for this problem, that does not require changing your whole tech stack: docker-sync.

What is docker-sync?

The Docker for Mac (and for Windows) performance problems have their roots in the OS file system layer between Docker and OS. On Linux, Docker can directly mount files and folders from file system, while on Mac Docker has to pass the request to the OS which takes care of writing the file to the disk. In case of macOS, OSXFS is the elephant in the room. While this does not sound like a big deal, it really is. Even a millisecond difference can pile up to half a second for your whole app, if you have a 500 source files which need to be read.

Thankfully, Docker is able to use native mounting if you put files into volumes, which are then handled by the Linux kernel. And this is exactly what docker-sync does. It creates a volume that holds all your app source files and makes it available to your app, which can read and write pretty fast to the volume. As volumes are usually not bound to the outside world (i.e. your file system), docker-sync implements different tools which take care of syncing the inside of the container with your host file system. This allows you to edit any files in your editor, they are synced into the volume and your app can access it.

docker-sync benchmarking

I benchmarked docker-sync because I wanted to look at hard numbers instead of guessing if the request was faster or not. The following two apps are based on Laravel and WordPress, while there are images from Bitnami used to run both. The main metric used here is the TTFB, or time to first byte, in milliseconds. I opened pages from both apps 10 times in a row, without specific page caching. Both apps are running with PHP 7.3.

App	docker-sync	Min	Max	Median	Difference
Laravel 5.8	off	1035ms	2158ms	1386ms
on	234ms	303ms	251ms	-81%
WordPress 5.2	off	2467ms	3942ms	2722ms
on	881ms	1589ms	1271ms	-53%

How to use docker-sync

I am pretty sure that you are hooked, huh? Saving up to 80% in response times sounds quite ridiculous, but. it’s true. So, how do we set up docker-sync now?

1. Install the tool

docker-sync is written in Ruby, so you can easily install it, even if you did not upgrade your Ruby version (v2.3 is shipped with macOS at the moment). To install it, run the following command:

Exit fullscreen mode

2. Add a new docker-sync.yml file to your project

This config file tells docker-sync which files to store and sync, which modes to use, and so on. Here is a basic config file you may just drop into your root folder:

Exit fullscreen mode

First, under the syncs option, you specify which volumes to create. Please note that the name of the volume ( your-app-files here), must be unique on your whole machine. The src option defines which files should be copied into the volume by default, you can exclude specific directories or files with the sync_excludes option. I excluded the Git, the PhpStorm and the node_modules folders here. Neither git nor the IDE folders have anything to do with the app itself, so they are not needed in the container. Node_modules are excluded because they are also not needed for regular PHP apps and thus can be removed, especially because it may take a lot of time until the 4596895 files and folders are synced.

You can find additional configuration params for volumes in the official documentation.

3. Add a docker-compose-dev.yml file

With the current versions of Docker Compose you are able to overwrite certain configurations of previous docker-compose.yml files. This is quite handy, because we can use the current docker-compose.yml file as it is, without any modifications. If you feel the urge to not use docker-sync at some time, you could start your app with the regular docker-compose up command.

Here is the example file that I used:

Exit fullscreen mode

This file tells Docker Compose to not use the regular mounts from the docker-compose.yml file, but instead mount the your-app-files volume into the containers. Notice that you have to add the volume to all containers that need access to the files, in this case both PHP and nginx. That’s basically it.

4. Start the docker-sync stack

There are actually two ways to start the stack with docker-sync:

1. by using docker-sync-stack or
2. by using docker-sync and docker-compose .

docker-sync-stack

Exit fullscreen mode

This command will first start docker-sync which creates the volume and prepares it for usage, and then runs the containers by calling
docker-compose -f docker-compose.yml -f docker-compose-dev.yml up , which starts Docker Compose with both docker-compose configuration files.

The downside here is, that both the sync and the Docker stack will run in the foreground. I personally dislike this as you are constantly in the need of opening multiple terminal windows for one app, but it’s fast and efficient.

To stop the stack, press CMD + C .

docker-sync + docker-compose

Exit fullscreen mode

This is the way I use at the moment, I may write a bash function or alias for that later. You can continue working in the terminal, I recommend Kitematic for fast log viewing of containers. (PS: Kitematic is built right into the Docker for Mac install package.)

How to solve permission problems

If you run into permission problems like me, with Laravel not being able to write logs and WordPress not being able to store uploads, you will likely have to specify another user ID. For my current Docker stack with Bitnami containers, I ran into issues because docker-sync be default uses the root user to store all files, while PHP FPM runs as the daemon user. This will lead to conflicts while saving files. To prevent this, you have to add sync_userid: ‘1’ to your docker-sync.yml config file, where 1 is the user ID of the daemon user inside the container.

After adding that line all problems went away instantly, and I was able to work with the setup.

Conclusion

It took quite some time to get the complete stack running. I find the documentation not that easy to understand, but I was able to set up a clean stack by looking into various examples provided by the project maintainer itself. After spending additional hours figuring out the permission issues I was able to run my apps with the docker-sync stack and gained a huge productivity boost. Pages are loading so fast right now, it’s incredible how much difference this little tool makes.

Many huge thanks to the maintainer Eugen Mayer and all contributors for publishing this awesome tool.

Источник