Docker - working steps | Part-3 Docker File System, Docker Volumes

 To Understand Docker Volume, you need to go though below sequence.
Docker File System
Bind Mounts
Docker Volumes

1) Docker File System
A docker container runs the software stack defined in an image. Images are made of a set of read-only layers that work on a file system called the Union File System. When we start a new container, Docker adds a read-write layer on the top of the image layers allowing the container to run as though on a standard Linux file system. So, any file change inside the container creates a working copy in the read-write layer. However, when the container is stopped or deleted, that read-write layer is lost.

2) Bind Mounts
A Docker bind mount is a high-performance connection from the container to a directory on the host machine. It allows the host to share its own file system with the container, which can be made read-only or read-write.
This allows us to use a container to run tools that we don't want to install on our host, and yet still work with our host's files. For example, if we wanted to use a custom version of bash for a particular script, we might execute that script in a bash container, mounted to our current working directory:

e.g.,

$ docker run -v $(pwd):/var/opt/project bash:latest \ 
  bash -c "echo Hello > /var/opt/project/file.txt"

The –v option can be used for all forms of mounting and specifies, in this instance, the source on the host – the working directory in the output of $(pwd)  – and the target mount point in the container – /var/opt/project.

After running this command, we will find file.txt in the working directory of our host machine. This is a simple way to provide persistent files between invocations of a Docker container, though it is most useful for when the container is doing work on behalf of the host.
One good use case for this would be executing various versions of a language's build tools in Docker to avoid having conflicting installations on a developer machine.

We should note that $(pwd -W) is sometimes needed on Windows bash shells to provide the working directory in a form that the bash shell can pass to Docker.

3) Docker Volumes
A bind mount uses the host file system, but Docker volumes are native to Docker. Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure and OS of the host machine, volumes are completely managed by Docker. Volumes have several advantages over bind mounts:

• Volumes are easier to back up or migrate than bind mounts.
• You can manage volumes using Docker CLI commands or the Docker API.
• Volumes work on both Linux and Windows containers.
• Volumes can be more safely shared among multiple containers.
• Volume drivers let you store volumes on remote hosts or cloud providers, to encrypt the contents of volumes, or to add other functionality.

New volumes can have their content pre-populated by a container.
Volumes on Docker Desktop have much higher performance than bind mounts from Mac and Windows hosts.

In addition, volumes are often a better choice than persisting data in a container’s writable layer, because a volume does not increase the size of the containers using it, and the volume’s contents exist outside the lifecycle of a given container.

Let us do the following scenarios:

• Creating Volumes
• Listing Volumes
• Inspecting Volumes
• Removing Volumes
• Pruning Volumes
• Starting a Container with a Volume

Docker volumes nothing but directory in container and map with base machine. Like it is a network drive which mounted with your base machine.

e.g.,
Assume, you want to copy some files from base machine to containers and files from containers to base machine with regular interval/ daily basis, then docker volume can help you to fullfill this requirement. i.e. both way file movement. Even we can map container to container or container to base machine.

Note:
You need to declared the volume while lunchig the container.
for existing container, we can't declare a volume
Even if a container in stopped state, still we can access volumes

Mapping a volume in two ways:
1) Creating a volume between host to container
2) Creating a volume between container to container


[root@ip-172-31-38-12 ~]# docker images
REPOSITORY   TAG       IMAGE ID       CREATED       SIZE
ubuntu       latest    58db3edaf2be   4 weeks ago   77.8MB
[root@ip-172-31-38-12 ~]#
[root@ip-172-31-38-12 ~]# docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
[root@ip-172-31-38-12 ~]#
[root@ip-172-31-38-12 docker_class]# ls
dockerfile

-- create Docker file like below 

[root@ip-172-31-38-12 docker_class]# cat Dockerfile
FROM ubuntu
MAINTAINER gmohapat
VOLUME /data

-- Create image from dockerfile

[root@ip-172-31-38-12 docker_class]# docker build -t demoimagevol .
Sending build context to Docker daemon  2.048kB
Step 1/3 : FROM ubuntu
 ---> 58db3edaf2be
Step 2/3 : MAINTAINER gmohapat
 ---> Running in 7ad715c26e23
Removing intermediate container 7ad715c26e23
 ---> 54317c9383d4
Step 3/3 : VOLUME /data
 ---> Running in 123e7f1c699f
Removing intermediate container 123e7f1c699f
 ---> 44787e8aa6c0
Successfully built 44787e8aa6c0
Successfully tagged demoimagevol:latest
[root@ip-172-31-38-12 docker_class]#

[root@ip-172-31-38-12 docker_class]# docker images
REPOSITORY     TAG       IMAGE ID       CREATED          SIZE
demoimagevol   latest    44787e8aa6c0   55 seconds ago   77.8MB
ubuntu         latest    58db3edaf2be   4 weeks ago      77.8MB

Now we are good to lunch a container

[root@ip-172-31-38-12 docker_class]# docker run -it --name container9 demoimagevol /bin/bash

root@5a158a8f3ce2:/# ls
bin  boot  data  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var

root@5a158a8f3ce2:/#

Here you see, data volume is there. This 'data' directory is not normal directory. It is OS docker directory.
Let us go inside data volume and create some files and test below.

root@5a158a8f3ce2:/data# touch file1 file2 file3

root@5a158a8f3ce2:/data# ls
file1  file2  file3

Now, come-out from container with exit option.

root@5a158a8f3ce2:/data# exit
exit

[root@ip-172-31-38-12 docker_class]# docker ps -a
CONTAINER ID   IMAGE          COMMAND       CREATED         STATUS                     PORTS     NAMES
5a158a8f3ce2   demoimagevol   "/bin/bash"   4 minutes ago   Exited (0) 4 seconds ago             container9

[root@ip-172-31-38-12 docker_class]#

Now, Create a new contaier and map 'data' volume to new container

[root@ip-172-31-38-12 docker_class]# docker run -it --name container91 --privileged=true --volumes-from container9 ubuntu /bin/bash
root@204b1b533717:/# ls
bin  boot  data  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@204b1b533717:/# cd data/
root@204b1b533717:/data# ls
file1  file2  file3
root@204b1b533717:/data#

Here, you can see, data volume mapped from previous container and all there files are there in data volume.
Now, do manage some files and check that is reflected in container9.

[root@ip-172-31-38-12 docker_class]# docker ps -a
CONTAINER ID   IMAGE          COMMAND       CREATED          STATUS                      PORTS     NAMES
204b1b533717   ubuntu         "/bin/bash"   2 minutes ago    Exited (0) 14 seconds ago             container91
5a158a8f3ce2   demoimagevol   "/bin/bash"   15 minutes ago   Exited (0) 10 minutes ago             container9

[root@ip-172-31-38-12 docker_class]# docker start container9
container9
[root@ip-172-31-38-12 docker_class]# ls
Dockerfile
[root@ip-172-31-38-12 docker_class]# docker attach container9
root@5a158a8f3ce2:/# ls
bin  boot  data  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@5a158a8f3ce2:/# cd data/
root@5a158a8f3ce2:/data# ls
file1  file2
root@5a158a8f3ce2:/data#

Here you can see, in another container changes reflected.

Declare a volume using command line:

[root@ip-172-31-38-12 docker_class]# docker run -it --name container8 -v /backup ubuntu /bin/bash
root@d0912a3f8083:/# ls
backup  bin  boot  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@d0912a3f8083:/#

Now map the volume to host machine.

[root@ip-172-31-38-12 docker_class]# docker run -it --name container7 -v /home/ec2-user/newdir:/datavol --privileged=true ubuntu /bin/bash

root@494ef9cc4365:/# ls
bin  boot  datavol  dev  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@494ef9cc4365:/# 
root@494ef9cc4365:/# cd datavol/
root@494ef9cc4365:/datavol# touch samplefile1 samplefile2
root@494ef9cc4365:/datavol# ls
samplefile1  samplefile2
root@494ef9cc4365:/datavol# exit
exit

[root@ip-172-31-38-12 newdir]# cd /home/ec2-user/newdir
[root@ip-172-31-38-12 newdir]# ls -l
total 0
-rw-r--r-- 1 root root 0 Feb 26 19:41 samplefile1
-rw-r--r-- 1 root root 0 Feb 26 19:41 samplefile2
[root@ip-172-31-38-12 newdir]#

Yes, now we can see what files we created in container volume, we able to see in host machine file system.

In real time, can access log files, source files, artifacts, configuration files.