Copying files to a container with Docker Compose

Question 1

I have a Dockerfile where I copy an existing directory (with content) to the container which works fine:

Dockerfile

FROM php:7.0-apache
COPY Frontend/ /var/www/html/aw3somevideo/
COPY Frontend/ /var/www/html/

RUN ls -al /var/www/html
RUN chown -R www-data:www-data /var/www/html 
RUN chmod -R 755 /var/www/html

But when I use a docker-compose.yml file there is only the directory aw3somevideo and inside aw3somevideo there is nothing.

docker-compose.yml:

 php:
    build: php/
    volumes:
      - ./Frontend/ :/var/www/html/
      - ./Frontend/index.php :/var/www/html/
    ports:
      - 8100:80

Maybe I do not understand the function of volumes and if that's the case please tell me how to copy my existing files to the container via a docker-compose.yml file.

Question 2

Given

    volumes:
      - /dir/on/host:/var/www/html

if /dir/on/host doesn't exist, it is created on the host and the empty content is mounted in the container at /var/www/html. Whatever content you had before in /var/www/html inside the container is inaccessible, until you unmount the volume; the new mount is hiding the old content.

Question 3

Updated for Docker 1.10

Docker 1.10 allows the definition of networks within the compose file. Here's the updated code

version: "2"

services:
  replica1:
    image: mongo:2.6.8
    container_name: replica1
    networks:
      - my-net
    ports:
      - "27018"
    environment:
      REPLICA2_URL: "http://replica2:27019"
  replica2:
    image: mongo:2.6.8
    container_name: replica2
    networks:
      - my-net
    ports:
      - "27019"
    environment:
      REPLICA1_URL: "http://replica1:27018"

networks:
  my-net:
    driver: bridge

Previous answer for Docker 1.9

As of Docker 1.9, the solution to this is to create a custom network and pass it to the docker-compose up command.

Create a network docker network create --driver bridge my-net
Reference that network as an environment variable (${NETWORK})in the docker-compose.yml files. Eg:

```

replica1:
  image: mongo:2.6.8
  container_name: replica1
  net: ${NETWORK}
  ports:
    - "27018"
  environment:
    REPLICA2_URL: "http://replica2:27019"

replica2:
  image: mongo:2.6.8
  container_name: replica2
  net: ${NETWORK}
  ports:
    - "27019"
  environment:
    REPLICA1_URL: "http://replica1:27018"

```

Note that replica1 in http://replica1:27018 will resolve to the ip address of the replica1 service (container). No need to hardcode ip addresses; An entry for replica1 is automatically added to the /etc/host of the replica2 container. Same goes for the replica1 container. Docker will add an entry for replica2 in its /etc/host file.

Call docker-compose, passing it the network you created NETWORK=my-net docker-compose up -d -f docker-compose.yml

I've created a bridge network above which only works within one node (host). Good for dev. If you need to get two nodes to talk to each other, you need to create an overlay network. Same principle though. You pass the network name to the docker-compose up command.

Question 4

[Refreshing this answer since it seems others have similar questions]

There are three kinds of volumes in docker:

Host volumes: these map a path from the host into the container with a bind mount. They have the short syntax /path/on/host:/path/in/container. Whatever exists on the host is what will be visible in the container, there's no merging of files or initialization from the image, and uid/gid's do not get any special mapping so you need to take care to allow the container uid/gid read and write access to this location (an exception is Docker for Mac with OSXFS). If the path on the host does not exist, docker will create an empty directory as root, and if it is a file, you can mount a single file into the container this way.
Named volumes: these have a name, instead of a host path as the source. They have the short syntax name:/path/in/container and in a compose file, you also need to define the named volume used in containers at the top level. By default, these are also a bind mount, but to a docker specific directory under /var/lib/docker/volumes that should be considered internal. However these defaults can be changed to allow things like NFS mounts, mounting disks, or even your own bind mounts to other locations. Named volumes also have a feature in docker, when they are new or empty and first used, docker copies the contents from the image into named volume before mounting it. This includes files, directories, uid/gid owners, and permissions. After that, they behave identical to a host volume, whatever is inside the volume overlays the image location.
Anonymous volumes: these only have a path inside the container. They are in the form /path/in/container and docker will create a default named volume with a guid as the name. They share the behaviors of named volumes, storing files under /var/lib/docker/volumes, initializing with the contents of the image, except they have a randomly generated guid that gives you no indication of how or even if they are being used. You can mount the volume in another container and inspect the contents, or you can find the container using the volume by inspecting each container to find the guid. If you create a container with the --rm flag, anonymous volumes will also be deleted automatically.
tmpfs: Wait, I said 3, and this is 4? That's because tmpfs isn't considered a volume, the syntax to mount it is different. The result is a pointer to an empty in memory filesystem. This is useful if you have temporary files you don't wish to save, they are relatively small, and you either need speed or want to be sure they aren't saved to disk.

In the OP's case:

/usr/app is mounted from the host, commonly used for development
/usr/app/node_modules is an anonymous volume initialized from the image

Why do this? Likely because you do not want to modify the node_modules directory on the host, particularly if there's platform specific data and you're running on Docker desktop where it's Mac/Win on the host and Linux in the container. It's also possible there's data in the image you want to get access to within the directory structure of the other volume mount.

Are there downsides to anonymous volumes? Two that I can think of:

If there's anything in /usr/app/node_modules that you want to reuse in a future container, you're unlikely to find the old volume. I tend to consider any data written to these as likely lost.
You'll often find the volumes on the host full of guids over time, and it's unclear which are in use and which can be deleted. Unused anonymous volumes are one of several causes of excessive disk use in docker.

For more details on docker volumes, see: https://docs.docker.com/storage/

Original answer:

The second one creates an anonymous volume. It will be listed in docker volume ls with a long unique id rather than a name. Docker-compose will be able to reuse this if you update your image, but it's easy to lose track of which volume belongs to what with those names, so I recommend always giving your volume a name.

Question 5

Aha! I fixed it. @Ijaz was totally correct - the RabbitMQ service takes a while to start, and my worker tries to connect before it's running.

I tried using a delay, but this failed when the RabbitMQ took longer than usual.

This is also indicative of a larger architectural problem - what happens if the queuing service (RabbitMQ in my case) goes offline during production? Right now, my entire site fails. There needs to be some built-in redundancy and polling.

As described this this related answer, we can use healthchecks in docker-compose 3+:

version: "3"

services:

  rabbitmq:
    image: rabbitmq
    command: rabbitmq-server
    expose:
      - 5672
      - 15672
    healthcheck:
      test: [ "CMD", "nc", "-z", "localhost", "5672" ]
      interval: 5s
      timeout: 15s
      retries: 1

  worker:
    image: worker
    restart: on-failure
    depends_on:
      - rabbitmq

Now, the worker container will restart a few times while the rabbitmq container stays unhealthy. rabbitmq immediately becomes healthy when nc -z localhost 5672 succeeds - i.e. when the queuing is live!

Question 6

All you need to do here is add context: . and dockerfile in your build section inside your docker-compose.yml file so that your service understands the complete directory structure.

# docker-compose.yml
version: "3"
services:
  webserver:
    build:
      context: .
      dockerfile: ./dockerfiles/webserver/Dockerfile
    image: webserver:php-apache

Copying files to a container with Docker Compose

Answers

docker,copy,docker-compose,volumes