Sunday, April 10, 2016

@Docker cheat sheet


Installing Docker

Installing Docker on Ubuntu

sudo apt-get update
sudo apt-get install –y docker.io
sudo service docker status
docker -v
docker version
sudo service docker start
sudo docker info

Installing Docker on CentOs

yum install -y docker
systemctl status docker.service
systemctl start docker.service

Updating Docker

Add docker repo key to the local apt keychain

wget =q0- https://get.docker.com/gpg | apt-key add -

Add docker repo to apt sources

echo deb http://get.docker.com/ubuntu docker main > /etc/apt/sources.list.d/docker.list
apt-get update
apt-get install lxc-docker

Basic docker configuration

Viewing Docker socket

ls -l /run

Add user to docker group

sudo gpasswd -a vagrant docker
cat /etc/group

Configure docker deamon on a Ubuntu host to listen on a network port

netstat -tlp
service docker stop
docker -H 192.168.56.50:2375 -d &
netstat -tlp

Connect to docker Ubuntu host from centOs machine

Set env variable
export DOCKER_HOST=”tcp://192.168.56.50:2375”

Configure docker deamon on an Ubuntu host to listen on a network port and a unix socket

docker -H 192.168.56.50:2375 -H unix:///var/run/docker.sock -d &






Docker images

Pulling a Docker image

docker pull ubuntu
docker pull <image_name>:<tag_name>

Pulling all docker images

docker pull -a ubuntu
Note: Images are stored in /var/lib/docker/<storage driver>

Containers

Running a Docker container

Running an interactive Docker container

docker run -it ubuntu /bin/bash

List of running containers

docker ps

List of recently running containers

docker ps -a

Exit a container without killing it

Ctrl + p + q

Docker run with -d (detached)

docker run –d ubuntu /bin/bash -c “ping 8.8.8.8 -c 30”
<-c implies command>

Docker run with -d to restart unless stopped

docker run --restart=unless-stopped -d /bin/bash -c “ping 8.8.8.8 -c 30”

Set restart always on existing docker container

docker update --restart=always <container_id>

Update existing container to restart automatically

docker update --restart=always prime-numbers


Update existing container to not restart automatically


docker update --restart=no prime-numbers

Naming a docker container

docker run -it -v /test-vol --name=voltainer ubuntu:15.04 /bin/bash

Image layers

Display all layers for all images

docker images --tree

Location of images

/var/lib/docker/graph

Copying images to other hosts

docker commit <container_id> new_name
docker save -o $HOME/seacliffsand.tar seacliffsand

Starting up tar images

Peek inside the tar

tar -tf <tarfile>.tar

Import tar image

docker load -i <tarfile>.tar

docker attach

docker attach <container_id>

docker exec

docker exec –it <container_id> /bin/bash

Commands for working with containers

docker run -d

docker run -d –name=<container_name> <image_name>

docker run –d … -c (pass a shell command to bash shell of the container)
docker run -d ubuntu /bin/bash -c “ping 8.8.8.8 -c 30”
docker top to see the top running processes in a container
docker top <container_id>

cpu shares

docker run --cpu-shares=256

Assign specific amount of memory to a docker container

docker run -m=1g -it <image_name> /bin/bash

docker inspect to get all info about a container or image

docker inspect

docker start stop restart

start

docker run -it ubuntu:14.04 /bin/bash

 

detach

ctrl + p + q

 

stop

docker ps
docker stop <container_id or name>

kill

docker kill –s <posix_sig>


all containers run

docker ps -a

last container run

docker ps –l

start a closed container

docker start <container_id>
docker attach <container_id>

restart a running container

docker restart <container_id>

location of containers

/var/lib/docker/containers

deleting containers

docker rm <container_id or name>

delete a running container

docker rm -f <container_id or name>



Getting a shell in a container

nsenter – enter name space

docker inspect <container_id> | grep Pid
nsenter -m -u -n -p -i -t <Pid> /bin/bash

docker-enter
#to enter a running container
docker-enter <container_id>
docker exec (recommended way of getting a terminal inside container)
docker exec -it <container_id> /bin/bash



Building from a Dockerfile

Comment line – starts with a #

Example: Dockerfile

#Ubuntu based Hello World container
FROM ubuntu:15.04
RUN apt-get update
#Each RUN instruction creates a new layer
#To minimize the number of layers, combine the runs into fewer RUN lines
RUN apt-get install –y nginx
RUN apt-get install –y golang
CMD [“echo”, “Hello World”]

docker build command (-t = tag)

docker build -t helloworld:0.1 .
# dot (.) at the end implies build with the Docker file in pwd
docker build -t=”<tag>” .


Dockerfile ADD



ADD <LOCAL_SOURCE_FOLDER_PATH> <DOCKER_IMAGE_FOLDER_PATH>
# The ADD command is executed when the Docker image is being built. It is is not executed when the container is created.
# ADD - allows for the source to be a URL

Dockerfile COPY


COPY <LOCAL_SOURCE_FOLDER_PATH> <DOCKER_IMAGE_FOLDER_PATH>
# The COPY command is executed when the Docker container is created. It is is not executed when the container is being built.



Push images to docker hub

docker tag

docker tag <container_id> <username>/<reponame>:1.0
docker push <username>/<reponame>:1.0
#Enter username, password and email id for the docker hub account

docker history
docker history <container_id>
start the container of image built by docker build
docker run helloworld:0.1

Docker private registry

Starting a registry

docker run -d -p 5000:5000 registry
#The DNS name used to resolve our registry becomes a permanent part of the naming context of any repo that we push to our registry.

Accessing the private registry from browser

http://<hostname>:5000

Using a private registry

Pushing an image to the private registry

docker tag <image_id> <hostname>:5000/<image_name>
docker push <hostname>:5000/<image_name>



Configuring docker config for allowing insecure communication with the private registry

#In /etc/default/docker of the machine hosting the private registry
DOCKER_OPTS=”--insecure-registry <hostname>:5000”

#In /usr/lib/systemd/system/docker.service on CentOs client
ExecStart=/usr/bin/docker -d $OPTIONS $DOCKER_STORAGE_OPTIONS --insecure-registry <hostname>:5000
#Restart docker on CentOs after making the above change

Docker registry config settings

<TODO>

Running an image hosted in the private registry

docker run -d <hostname>:5000/<image_name>

Diving deeper with Dockerfile

The build cache

docker build -t=”build1” .
docker build -t=”build2” .
#if the docker deamon finds images that were built with the same instruction as the current one from the Dockerfile, it does not repeat the build step again. Instead, it picks up the image from the cache. When operating at scale, this speeds up builds significantly.

Dockerfile and Layers

docker images --tree
docker history <image_layer_id>


Exercise: Building a web server Dockerfile

Dockerfile: Convoluted example

#Simple web server
FROM ubuntu:15.04
RUN apt-get update
RUN apt-get install -y apache2
RUN apt-get install -y apache2-utils
RUN apt-get clean
EXPOSE 80
CMD [“apache2ctl”, “-D”, “FOREGROUND”]

Exercise: Run the web server container

docker run -d -p 80:80 webserver
docker ps
#connect from the browser: <hostname>:80

Reducing number of layers

Dockerfile: To reduce layers
#Simple web server
FROM ubuntu:15.04
RUN apt-get update && \
apt-get install -y apache2 && \
apt-get install -y apache2-utils && \
apt-get clean
EXPOSE 80
CMD [“apache2ctl”, “-D”, “FOREGROUND”]

The CMD instructions

#CMD executes only at runtime, run commands in containers at launch time. Only one CMD per dockerfile. If there are more, only the last one will be effective.
#RUN is a build-time instruction, adds layers to images, used to install apps.
#CMD shell form - Commands are expressed the same way as shell commands. Shell commands are automatically prepended by ‘/bin/sh –c’ if arguments are passed to the CMD.

#CMD Exec form – Pass arguments to CMD as json – [“command”, “arg1”]. Allows to work with containers that do not have shell. No shell features like variable expansion and no special characters.

The ENTRYPOINT instructions

#Preferred method of specifying the default app to run in the container. Cannot be overridden at runtime with normal commands.
#Any commands passed to “docker run … <command>” will be used as arguments to ENTRYPOINT
#In Dockerfile
ENTRYPOING [“echo”]
#Execute build and run commands
docker build –t=”hw2”
docker run hw2 Hello World
#the following passes /bin/bash as an argument to the hw2 container’s echo ENTRYPOINT.
docker run –it hw2 /bin/bash

#In Dockerfile for apache ENTRYPOINT
ENTRYPOINT[“apache2ctl”]
#Execute build and run commands (fires up the apache web server on the container)
docker build –t=”web2”
docker run –d –p 80:80 web2 –D FOREGROUND

To override ENTRYPOINT at runtime

<todo>
--entrypoint on the docker run command line

The ENV instructions

#Creating environment variables to the container
ENV var1=example1 var2=example2
#Using environment variables in the Dockerfile
ENV var1=ping var2=8.8.8.8
CMD $var1 $var2

Volumes

Create a volume on the container

docker run -it -v /test-vol --name=voltainer ubuntu:15.04 /bin/bash

Host mount: Map a local directory to a directory on the container

docker run -p 8080:8080 -v $HOME/jenkins_home:/var/jenkins_home jenkins:1.596.2 &

Volumes in a Dockerfile

#Host mounts are not possible from Dockerfile

FROM ubuntu:15.04
RUN apt-get update && apt-get install -y iputils-ping
VOLUME /data
ENTRYPOINT

Deleting a volume

docker rm -v <container>
#If we delete a container without specifying a -v, the container gets deleted, but the volume remains.

Docker Networking

#See what is on the network
ip a
#docker 0 is a network bridge or a virtual switch
#bridge utils is required for viewing whatever is running on docker 0
#apt-get install bridge-utils
#yum install bridge-utils
brctl show docker 0

Exercise:

docker run -it --name=net1 net-img
docker run -it --name=net2 net-img
#Each container gets one interface automatically attached to docker 0 bridge

#Inside our container
ip a
#loopback
#eth0 with a inet address

Exercise:

docker run -it --name=net2 net-img
docker inspect <container_id>
#See NetworkSettings for IPAddress, Gateway, bridge, etc.

ls -l /var/lib/docker/containers/<container_id>
#we are interested in hosts and resolv.conf
cat resolve.conf
#By default resolve.conf is a copy of /etc/resolve.conf on the Docker host
#We can override the resolve.conf by passing arguments on the docker run command line

cat hosts
#With recent versions of docker, it is allowed to change hosts and resolve on the fly.

Exercise to override resolve.conf:

docker run –dns=8.8.4.4 –name=dnstest net-img
docker inspect dnstest
#”Dns”: [“8.8.4.4”]

Exposing Ports

Exercise:

#Test for networking module
FROM ubuntu:15.04
RUN apt-get update && apt-get install -y iputils-ping traceroute apache2
EXPOSE 80
ENTRYPOINT [“apache2ctl”]
CMD [“-D”, “FOREGROUND”]

#build the container
docker build -t=”apache-img” .

#run the container – port 5001 on the docker host to port 80 on the container – any connection coming into the host on 5001 will be forwarded to port 80 on the container
docker run -d –p 5001:80 --name=web1 apache-img

#see the ports
docker port web1

#UDP
docker run -d -p 5002:80/udp --name=web2 apache-img

#What IP addresses are available on our docker host
ip -f inet a

#Specifying the host ip for port forwarding
docker run -d -p 192.168.56.50:5003:80 --name=web3 apache-img

#-P switch – map all exposed ports on a container, so all ports marked as exposed in the Dockerfile, to run them at high number ports on the docker host
Exercise:
FROM ubuntu:15.04
RUN apt-get update && apt-get install -y iputils-ping traceroute apache2
EXPOSE 80 500 600 700 800 900
ENTRYPOINT [“apache2ctl”]
CMD [“-D”, “FOREGROUND”]

#build
docker build -t=”throw-away” .

#run with –P
docker run -d –P --name=throw throw-away

#display ports
docker port throw

Linking containers


#Link between containers only. Not for communicating to the outside world.
#Define source with an alias
docker run --name=<source_alias> -d <image_name>

#Example
docker run --name=src -d seacliffsand

#Define receiver with an alias
docker run --name=<receiver_alias> --link=src:<source_alias> -it ubuntu /bin/bash

#Example:
docker run --name=rcvr --link=src:seacliffsand-src -it ubuntu /bin/bash

#Verify linkage
docker inspect rcvr

#Attach to the receiver container and review the environment variables
env | grep <source_alias>

#Example
env | grep SEACLIFFSAND_SRC
SEACLIFFSAND_SRC_PORT_50000_TCP_PROTO=tcp
SEACLIFFSAND_SRC_ENV_JENKINS_HOME=/var/jenkins_home
SEACLIFFSAND_SRC_PORT_50000_TCP_ADDR=172.17.0.2
SEACLIFFSAND_SRC_PORT_8080_TCP=tcp://172.17.0.2:8080
SEACLIFFSAND_SRC_NAME=/rcvr/seacliffsand-src
SEACLIFFSAND_SRC_ENV_JAVA_VERSION=7u65
SEACLIFFSAND_SRC_PORT_8080_TCP_ADDR=172.17.0.2
SEACLIFFSAND_SRC_ENV_JENKINS_VERSION=1.596.2
SEACLIFFSAND_SRC_PORT_8080_TCP_PORT=8080
SEACLIFFSAND_SRC_PORT_50000_TCP=tcp://172.17.0.2:50000
SEACLIFFSAND_SRC_PORT_8080_TCP_PROTO=tcp
SEACLIFFSAND_SRC_PORT_50000_TCP_PORT=50000
SEACLIFFSAND_SRC_PORT=tcp://172.17.0.2:8080
SEACLIFFSAND_SRC_ENV_JENKINS_UC=https://updates.jenkins-ci.org

#Docker adds the source alias to the /etc/hosts file of the receiver
cat /etc/hosts

#Example
cat /etc/hosts | grep seacliffsand-src
172.17.0.2      seacliffsand-src

#Only the receiver container knows about the source container’s networking config. The receiver can use the environment variables listed above to dynamically and programmatically configure itself.

#We can link multiple recipient containers to a single source container and a single recipient container to multiple sources.


Troubleshooting


Docker daemon logging

#Start daemon manually from the cli in debug, info, error, fatal mode
docker -d -l debug

#write logs to a file
docker -d -l debug >> logfile.txt

#Add the following line in etc/default/docker to change the logging level when docker is started as a service
DOCKER_OPTS=”--log-level=fatal”

Container logging

Display logs generated by the container’s PID1

docker logs <container_id>

Display the logs as a tail

docker logs <conainer_id> -f

#Suggested – If application level logs are needed by another system or if logs need to be kept (like most cases), mount a volume so that the application logs will persist outside the container.

Image troubleshooting

Intermediate images

#When there is an error while building a Dockerfile that will be an image with not tag
#Bring up that image using the image id
docker run --it <container_id> /bin/bash

The docker0 bridge

#Stop the docker service and check the ip address
service docker stop
ip a

#Delete the docker0 link
ip link del docker0

#Edit the docker config file etc/default/docker – bip refers to ‘bridge ip’
DOCKER_OPTS=--bip=150.150.0.1/24

#Turn on the docker service again
docker service start

#Run a container again
docker run -it ubuntuL15.04 /bin/bash

#Check ip address again – it will have ip address 150.150.0.X
ip a

Firewall config

IPTables on the docker host

Default value of the following is true
 --icc= inter container communication
 --iptables= decides whether docker will make any modifications to iptables rules

Check iptables

iptables -L -v

Stop docker and edit the docker config file /etc/default/docker – this should stop any inter container communication

DOCKER_OPTS=”--icc=false”

Docker will not be able to make any modifications to iptables rules and as a result icc will also be disabled

DOCKER_OPTS=”--iptables=false”

Vagrant provisioning

Use the following vagrant provisioning lines in a Vagrant file to create a new jenkins container on the Vagrant box at the time of provision. Vagrant will automatically ensure that the docker container created using this provisioning method will always run at the time of VM startup.
   config.vm.provision "shell", inline: <<-SHELL
      sudo chmod 777 -R /vagrant/jenkins_home
    SHELL
   config.vm.provision "docker" do |d|
                                 d.run "jenkins", args: "-p 8080:8080 -p 50000:50000 -v /vagrant/jenkins_home:/var/jenkins_home"
   end




Delete all dangling images

docker rmi $(docker images --quiet --filter "dangling=true")