MJ Convento

/

Docker for local PHP development, two years in

2,195 words, about 10 min read

Someone joined the team on a Monday last month and had the whole stack running before lunch: PHP 7.2, nginx, MySQL 5.7, the Symfony app, the queue worker. Three commands, one of which was git clone. Two years ago the same onboarding was a day and a half of Homebrew taps, half an hour of my time, and a bug two weeks later because his mbstring had been compiled against a different ICU than mine.

We put the first docker-compose.yml in a client repository in January 2017. Two years in, I still think it was the right call, and I have a list of things it did not fix that is longer than the list of things it did.

What we were doing before

Two setups, both bad. Laptops ran PHP from Homebrew or from a PPA, which meant every machine had its own PHP patch version, its own extension set, and its own idea of where php.ini lived. When a client project needed 5.6 and the one next to it needed 7.1, you switched with a shell alias and forgot you had switched.

The other setup was a shared development server. One box, one nginx, one MySQL, one copy of PHP, and a directory per project. It solved consistency by making everyone consistent with everyone else, and it created a different problem: my broken migration was also your broken migration. Nobody could restart php-fpm without asking in the group chat first. Two people running composer install at the same time on that machine took long enough that you would go and get coffee.

Docker replaced both. Not because containers are elegant, but because the alternative was continuing to answer the question “does it work on your machine” with a shrug.

The compose file

This is close to the file we ship now, trimmed of one worker service and the mailhog container. The app is Symfony 3.4 on PHP 7.2 behind nginx.

version: '3.4'

services:
  nginx:
    image: nginx:1.15-alpine
    ports:
      - "8080:80"
    volumes:
      - ./:/var/www/app:cached
      - ./docker/nginx/app.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - php

  php:
    build:
      context: .
      dockerfile: docker/php/Dockerfile
      args:
        HOST_UID: '1000'
    volumes:
      - ./:/var/www/app:cached
      - app-var:/var/www/app/var
      - composer-cache:/tmp/composer
    environment:
      COMPOSER_HOME: /tmp/composer
      DATABASE_HOST: mysql
      PHP_IDE_CONFIG: serverName=docker
    depends_on:
      - mysql

  mysql:
    image: mysql:5.7
    command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
    ports:
      - "33061:3306"
    environment:
      MYSQL_ROOT_PASSWORD: root
      MYSQL_DATABASE: app
      MYSQL_USER: app
      MYSQL_PASSWORD: app
    volumes:
      - mysql-data:/var/lib/mysql
      - ./docker/mysql/initdb.d:/docker-entrypoint-initdb.d:ro

volumes:
  mysql-data:
  app-var:
  composer-cache:

A few of those choices took arguing about.

The file format version is 3.4 and not the newest. The Compose file reference carries a compatibility matrix: format 3.7 needs Docker Engine 18.06 or newer, 3.4 needs 17.09. Two people on Linux were still on a distro package older than that, and a compose file that refuses to parse on a colleague’s laptop is worse than a compose file without init. The current Docker Desktop for Mac release, 2.0.0.0 from 19 November, ships Docker 18.09.0 and Compose 1.23.1, and Compose 1.23.2 came out nine days later. The Mac side is never the constraint. The Linux side always is.

MySQL is the official image with the settings the image documentation defines: MYSQL_ROOT_PASSWORD, MYSQL_DATABASE, a project user, and the character set flags passed as arguments to mysqld through command. The data directory is a named volume on /var/lib/mysql, and the seed dump is bind mounted read only into /docker-entrypoint-initdb.d, which the entrypoint runs in alphabetical order on first boot only. That last word matters. Change the dump, and nothing happens until you docker-compose down -v and let it initialise again. We wasted an afternoon on that before someone read the paragraph.

Port 33061 on the host is there so that a GUI client can connect and so that two projects can be up at once. Three projects with 3306:3306 hardcoded is a lesson everyone learns exactly once.

Named volumes, bind mounts, and the number that won the argument

The volumes documentation calls named volumes the preferred mechanism for persisted data, and for the database that is not a debate: the data does not need to be readable from the host, and a volume managed by the engine is faster and easier to throw away. Application code is the opposite case. You want to edit it in your editor and have php-fpm see the change, which is a bind mount, and on macOS a bind mount is where the pain lives.

The compromise is visible in the compose file. Source is bind mounted. Symfony’s var/ directory, which is cache and logs and hundreds of generated PHP files, is a named volume mounted over the top of the bind mount. The composer cache is another named volume, pointed at by COMPOSER_HOME. What we did not move off the bind mount is vendor/, because my editor needs to index it for autocompletion, and living without that costs more than the seconds it saves.

The measurement that ended the discussion was composer install. On the shared development server, with two other people working on it, a clean install of the project’s dependencies took somewhere between two and a half and four minutes, and I stopped timing it because the variance was the point. In the container, with a warm cache in the named volume, it is 38 to 45 seconds. Cold, with nothing cached, it is about 2 minutes 20, which is roughly what it is anywhere. The team stopped objecting to Docker somewhere around the third time the fast number happened.

osxfs, and the flags that make it survivable

Bind mounts on macOS are slow, and Docker documents why rather than pretending otherwise. The performance tuning page explains that on Linux the container shares the host VFS directly and the guarantees cost nothing, while on macOS every file system action has to be passed synchronously between container and host. Their own example is go list ./... in a bind mounted source tree taking around 26 seconds. A PHP framework is the same shape of workload: thousands of small stats and opens, most of them from the autoloader.

Since Docker 17.04 there are two flags you can append to a mount, and Compose supports them on a service volume: cached, where the host’s view is authoritative and container reads may lag, and delegated, where the container’s view is authoritative and host reads may lag. Full consistency is the default.

On my machine, a cold Symfony cache warmup on a fully consistent bind mount took 41 seconds. With :cached on the source mount and var/ in a named volume, the same warmup is 9 seconds. Native, outside Docker, it is about 4. So we are still paying, and on the Linux laptops there is nothing to pay, which produces the mildly annoying situation where the test suite runs in 1 minute 5 on a ThinkPad and 3 minutes 40 on a newer MacBook.

Both flags are macOS only. Writing :cached in the file is harmless on Linux, which is the only reason it is committed rather than sitting in everyone’s local override file.

depends_on is not a readiness check

Our first CI-ish script did docker-compose up -d followed immediately by a migration, and got this:

In AbstractMySQLDriver.php line 36:
  An exception occurred in driver: SQLSTATE[HY000] [2002] Connection refused

This is documented behaviour, not a bug. Controlling startup order says it plainly: Compose starts containers in dependency order, but it does not wait until a container is ready, only until it is running. The docs suggest either making the application reconnect or wrapping the command in a polling script, and they are right that the real answer is the first one. For a local stack the second one is enough. We have a shell loop that calls mysqladmin ping until it answers or sixty seconds pass, and every command that touches the database goes through a Makefile target that runs it first.

One more thing if you are moving from format 2 to format 3: the condition form of depends_on does not exist in version 3 files. There is no declarative way to say “wait for healthy” in a v3 compose file, so stop looking for one.

What is still painful

File ownership on Linux. The php-fpm workers in the official image run as www-data, which is uid 33 in Debian, and my Linux account is 1000. Anything the container writes into the bind mount comes back owned by 33, and anything I create is unwritable by the container. On macOS you never see this, because the file sharing layer maps ownership for you, which means the person who introduces the bug is always on a Mac. The fix is a build argument and a usermod, and it is ugly:

FROM php:7.2-fpm

ARG HOST_UID=1000
RUN apt-get update && apt-get install -y libicu-dev 
    && docker-php-ext-install -j"$(nproc)" intl pdo_mysql opcache 
    && pecl install xdebug-2.6.1 
    && docker-php-ext-enable xdebug 
    && usermod -u "$HOST_UID" www-data && groupmod -g "$HOST_UID" www-data

The php image documentation gives you docker-php-ext-install, docker-php-ext-configure and docker-php-ext-enable, and tells you to pin PECL versions explicitly because PECL will happily pick a release your PHP cannot build. It also documents running the FPM variants under an arbitrary user with --user, which is the cleaner answer and which we have not adopted, because the uid has to come from somewhere and an environment variable in a committed .env is one more thing to explain.

Xdebug took two evenings. It is a reverse connection: the extension inside the container connects out to your IDE, so xdebug.remote_host has to name the host from inside the container. On the Mac that is host.docker.internal, which Docker for Mac documents as the recommended way to reach a service on the host from 18.03 onwards. On Linux that name does not resolve, so those machines set the docker0 gateway address instead, in a file that is not committed. Our dev config, which is copied into conf.d only in the development build stage:

zend_extension=xdebug.so
xdebug.remote_enable=1
xdebug.remote_host=host.docker.internal
xdebug.remote_port=9000
xdebug.remote_autostart=0
xdebug.idekey=PHPSTORM

Leaving remote_autostart at 0 means the debugger only engages when the browser extension sets the cookie, which keeps the extension from slowing down every request. And the default remote_port of 9000 is the same number php-fpm listens on, which confuses everyone for about ten minutes.

Production is still provisioned by hand

Here is the honest gap. None of this runs in production. Production is Ubuntu boxes with PHP installed from a package repository, nginx configured by a human, and a deploy that copies a release directory and symlinks it. We build images for laptops and CI and then deploy the old way.

Which means the parity claim is smaller than it sounds. We have made every developer’s environment identical to every other developer’s environment. We have not made the development environment identical to production, and twice this year a difference bit us: an opcache setting that was off locally and on in production, and a PHP patch version gap that changed how a date string parsed. The Debian based official image and an Ubuntu box with a third party PHP package are not the same build of PHP, and pretending they are is how you get a surprise on a Friday.

PHP 7.3.0 came out two days ago, along with 7.2.13. In the old world, trying it meant a spare VM and a morning. Now it is one line in one Dockerfile and a rebuild, and I will have an answer about whether our suite passes on 7.3 before the end of the week. That, more than anything about isolation or images, is the thing I would not give back.

What I want next is the production side: the same image built once in CI and shipped, rather than two descriptions of one environment drifting apart. That is a bigger fight than a compose file, it means arguing about the deploy pipeline, and I do not expect to win it this quarter.

Sources