目录

Docker轻量级容器探秘

参考资料

依赖前期技术

Cgroup/LXC

升级device-mapper

yum update -y device-mapper-libs

AUFS/BTRFS/OverlayFS

  1. aufs已经明确表示不会进入内核模块
  2. btrfs还不够成熟到生产环境用
  3. overlayfs在3.18的内核已经原生支持,目前稳定性不错
  4. zfs移植到linux也可以提供支持,但不是原生的(zfsonlinue)

docker实战

docker进程配置

/etc/sysconfig/docker

other_args="-D -s overlay --insecure-registry dl.dockerpool.com:5000"

或者

vi /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "insecure-registries" : ["hub.i.upyun.com:5443"]
}

docker仓库搜索

docker nodejs example

Dockerfile alpine 版本

FROM mhart/alpine-node:4
 
WORKDIR /app
COPY . .
 
# Install deps
RUN npm install --prod && \
    npm cache clean --force
 
# Expose ports
EXPOSE 3000
CMD ["node", "index.js"]

Dockerfile centos 版本

FROM blalor/centos
RUN yum install -y wget nodejs npm --enablerepo=epel
ADD . /srv/hello
WORKDIR /srv/hello
RUN npm install
EXPOSE 3000
CMD ["node","/srv/hello/index"]

/srv/hello

package.json

{
 "name": "docker-node-hello",
 "private": true,
 "version": "0.0.1",
 "description": "Node.js Hello world app on Ubuntu using docker",
 "dependencies": {
   "express": "4.x.x"
 }
}

index.js

var express = require('express')
var app = express()
 
app.get('/', function (req, res) {
 res.send('Hello World!')
})
 
var server = app.listen(3000, function () {
 
 var host = server.address().address
 var port = server.address().port
 
 console.log('Example app listening at http://%s:%s', host, port)
 
})

编译镜像

docker build -t "upyun/node-hello" --rm .

运行镜像

docker run -tid --name node --restart=always -p 3000:3000 upyun/node-hello
如果想要直接运行在后台, docker run -itd

常见问题收集

x509: failed to load system roots and no roots provided

  - tar: /etc/pki,/etc/ssl
  - rpm: ca-certificates

Unable to enable network bridge NAT: Iptables not found

--iptables=false
 
ln -snf /usr/local/iptables/sbin/iptables /sbin/iptables

Docker overlay2空间爆满

docker system df -v | less
docker system prune
docker volume rm $(docker volume ls -q)
docker volume ls
docker inspect edgex_log-data
 
for i in `docker system df -v|awk '/year/{print $3}'`;do docker rmi $i ;done

alpine APKINDEX temporary error

docker build -t 'opentalk/nginx' --rm .  --network=host

docker run 一次执行多条命令

docker run -tid -v /root/app:/root/app -p 5555:5555 robyn-python sh -c "prisma db push && python3 /root/app/app.py"

alpine包管理

#  查看所有包
apk search -v
 
#  查看所有含acf的包
apk search -v 'acf*'
 
#  查看描述中含NTP的包
apk search -v --description 'NTP'
 
#  包所有信息,涵盖cmd提供、安装大小、描述等
apk info -a zlib
 
#  包概要说明(描述、大小、官网)
apk info -v asterisk
 
#  包命令提供
apk info -P bind-tools
 
#  列出apline已安装的包
apk -vv info|sort
 
#  修改成清华的镜像
sed -i 's/dl-cdn.alpinelinux.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apk/repositories
#  阿里云镜像
sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
#  科大镜像
sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories
 
#  增加testing仓库(默认只有main和community)
sed -i '$a@testing http:# mirrors.tuna.tsinghua.edu.cn/alpine/edge/testing' /etc/apk/repositories

好用的docker收集

  • MySQL: 传统关系型数据库,事务性,复杂的SQL问题。
  • Redis:缓存、消息队列(基于分布式pub/sub,并不保证可靠)。
  • MongoDB:高伸缩性的场景,预期可扩展性的表结构。
  • Elasticsearch:全文搜索,提高模糊查询速度。
  • Kafka:消息队列(支持分组,支持at least once、at most once)

Dockerfile里添加软件包

$ cat Dockerfile
FROM alpine:latest
 
RUN apk update && apk add --no-cache \
    curl tree bind-tools tcpdump \
    strace sysstat iproute2 \
    util-linux
 
COPY bashrc ~/.bashrc

alpine的非root权限管理

RUN sed -r -i '/^[^r]/s@(.*):.*@\1:/sbin/nologin@g' /etc/passwd

alpine+ supervisor

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
 
[unix_http_server]
file=/tmp/supervisor.sock
username = upyun
password = upyun.com123
 
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock
username = upyun
password = upyun.com123
 
[supervisord]
nodaemon=true
user = root
 
[include]
files = /etc/supervisor.d/*.conf
 
[program:php]
command = /usr/bin/php-fpm5 -F
autostart = true
 
[program:nginx]
command = /usr/sbin/nginx -g "daemon off;"
autostart = true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
 
#[program:sshd]
#command = /usr/sbin/sshd -D
#autostart = true
如果容器里的supervisord设置了nofile rlimit限制,docker外面默认nofile=1024不够用
docker run --rm -itd --net=host -m 16g --ulimit nofile=131072  prismcdn/solar:1.0.1 start-server

alpine+ sshd

# Supervisord && cgroup for sshd                                                                                                                      
VOLUME ["/sys/fs/cgroup"]
 
RUN apk --update --no-cache add supervisor openrc openssh curl bind-tools \
    && /usr/bin/ssh-keygen -t rsa -b 4096 -P "" -f /root/.ssh/id_rsa \
    && /usr/bin/ssh-keygen -A \
    && /usr/bin/passwd -u root \
    && chmod 1777 /var/run/ /var/log/ \
    && mkdir -p /etc/supervisor.d

或者手动生成:

ssh-keygen -t rsa -N "" -f /etc/ssh/ssh_host_rsa_key
ssh-keygen -t ecdsa -N "" -f /etc/ssh/ssh_host_ecdsa_key
ssh-keygen -t ed25519 -N "" -f /etc/ssh/ssh_host_ed25519_key
rc-update add sshd

alpine-nginx

官方版

最小版

docker run -tid --name anolelive-nginx --restart=always -p 80:80 --dns=192.168.147.20 --dns=192.168.21.20 \
        -v /root/empirecms/upload/:/usr/html/ \
        -v /root/nginx.conf:/etc/nginx/nginx.conf \
        docker.io/smebberson/alpine-nginx 

alpine-fpm-php

官方版

最小版

docker run -itd  --restart=always --name anolelive-php7 -p 9000:9000 --dns=192.168.147.20 --dns=192.168.21.20 \
        -v /root/empirecms/upload/:/usr/html/ \
        docker.io/hermsi/alpine-fpm-php
 
 
docker run -itd  --restart=always --name upwan-php5 -v /root/upwan/:/usr/html/ -p 9000:9000 docker.io/webgoal/php-fpm-mysqli
docker run -itd  --restart=always --name upwan-php5 -v /root/upwan/:/usr/html/ -p 9000:9000 php56-fpm
docker run -itd --name phpfpm7 -v /html:/html -p 9001:9000 php71-fpm

wordpress

#!/bin/sh
OPTS="-itd --restart=always"
DIR="/disk/ssd1"
 
docker run $OPTS --name wp-nginx -p 80:80 -v $DIR/wordpress/:/usr/html/ -v `pwd`/nginx.conf:/etc/nginx/nginx.conf docker.io/smebberson/alpine-nginx
docker run $OPTS --name wp-php7  -p 9000:9000 -v $DIR/wordpress/:/usr/html/ docker.io/hermsi/alpine-fpm-php:7.2.8
docker run $OPTS --name wp-mysql -p 3306:3306 -v $DIR/wordpress_db:/var/lib/mysql -e MYSQL_ROOT_PASSWORD="5zgit%_g<e#O" percona/percona-server:5.7.22

mongodb

参考资料:

Dockerfile

FROM mongo:3.4
 
MAINTAINER <Alex Bejan> contact@bejanalex.com
 
ENV AUTH yes
ENV STORAGE_ENGINE wiredTiger
ENV JOURNALING yes
 
ADD run.sh /run.sh
ADD set_mongodb_password.sh /set_mongodb_password.sh
 
CMD ["/run.sh"]

run.sh

#!/bin/bash
set -m
 
mongodb_cmd="mongod --storageEngine $STORAGE_ENGINE"
cmd="$mongodb_cmd --httpinterface --rest --master"
[ "$AUTH" == "yes" ] && cmd="$cmd --auth"
[ "$JOURNALING" == "no" ] &&  cmd="$cmd --nojournal"
[ "$OPLOG_SIZE" != "" ] && cmd="$cmd --oplogSize $OPLOG_SIZE"
 
$cmd &
 
[ ! -f /data/db/.mongodb_password_set ] && /set_mongodb_password.sh
 
fg

set_mongodb_password.sh

#!/bin/bash
 
# Admin User
MONGODB_ADMIN_USER=${MONGODB_ADMIN_USER:-"admin"}
MONGODB_ADMIN_PASS=${MONGODB_ADMIN_PASS:-"upyun.com123"}
 
# Application Database User
MONGODB_APPLICATION_DATABASE=${MONGODB_APPLICATION_DATABASE:-"admin"}
MONGODB_APPLICATION_USER=${MONGODB_APPLICATION_USER:-"restapiuser"}
MONGODB_APPLICATION_PASS=${MONGODB_APPLICATION_PASS:-"upyun.com123"}
 
# Wait for MongoDB to boot
RET=1
while [[ RET -ne 0 ]]; do
    echo "=> Waiting for confirmation of MongoDB service startup..."
    sleep 5
    mongo admin --eval "help" >/dev/null 2>&1
    RET=$?
done
 
# Create the admin user
echo "=> Creating admin user with a password in MongoDB"
mongo admin --eval "db.createUser({user: '$MONGODB_ADMIN_USER', pwd: '$MONGODB_ADMIN_PASS', roles:[{role:'root',db:'admin'}]});"
 
sleep 3
 
# If we've defined the MONGODB_APPLICATION_DATABASE environment variable and it's a different database
# than admin, then create the user for that database.
# First it authenticates to Mongo using the admin user it created above.
# Then it switches to the REST API database and runs the createUser command 
# to actually create the user and assign it to the database.
if [ "$MONGODB_APPLICATION_DATABASE" != "admin" ]; then
    echo "=> Creating an ${MONGODB_APPLICATION_DATABASE} user with a password in MongoDB"
    mongo admin -u $MONGODB_ADMIN_USER -p $MONGODB_ADMIN_PASS << EOF
use $MONGODB_APPLICATION_DATABASE
db.createUser({user: '$MONGODB_APPLICATION_USER', pwd: '$MONGODB_APPLICATION_PASS', roles:[{role:'dbOwner', db:'$MONGODB_APPLICATION_DATABASE'}]})
EOF
fi
 
sleep 1
 
# If everything went well, add a file as a flag so we know in the future to not re-create the
# users if we're recreating the container (provided we're using some persistent storage)
echo "=> Done!"
touch /data/db/.mongodb_password_set
 
echo "========================================================================"
echo "You can now connect to the admin MongoDB server using:"
echo ""
echo "    mongo admin -u $MONGODB_ADMIN_USER -p $MONGODB_ADMIN_PASS --host <host> --port <port>"
echo ""
echo "Please remember to change the admin password as soon as possible!"
echo "========================================================================"

最简单的运行方式

#!/bin/sh
mkdir -p /data
docker run -itd --restart=always -m 1000M --memory-swap -1 -p 27017:27017 --dns=192.168.147.20 --dns=192.168.21.20 \
        -v /etc/resolv.conf:/etc/resolv.conf \
        -v /data/:/data/db \
        -e MONGODB_ADMIN_USER=admin \
        -e MONGODB_ADMIN_PASS=upyun.com123 \
        -e MONGODB_APPLICATION_DATABASE=nodebb \
        -e MONGODB_APPLICATION_USER=szb \
        alexpunct/mongo:3.4

galera-maria

主要是针对 K8S 容器化

maria/percona/mysql

FROM alpine:3.12
ENV SUPERVISOR_CONF_DIR=/etc/supervisor.d
ENV MYSQL_DATA_DIR=/opt/mysql/data
ENV MYSQL_USER=mysql
ENV MYSQL_STDOUT_FILE=/var/log/mysql-stdout.log
ENV MYSQL_STDERR_FILE=/var/log/mysql-stderr.log
 
RUN apk add --update --no-cache \
        inotify-tools py3-pip \
        supervisor \
        mysql \
        mysql-client
 
RUN mkdir -p $SUPERVISOR_CONF_DIR $MYSQL_DATA_DIR && \
    mysql_install_db --user=$MYSQL_USER --datadir=$MYSQL_DATA_DIR && \
    touch $MYSQL_STDOUT_FILE $MYSQL_STDERR_FILE
 
VOLUME $MYSQL_DATA_DIR
COPY config/mysql.ini $SUPERVISOR_CONF_DIR
 
ENTRYPOINT ["supervisord"]
CMD ["--nodaemon", "--configuration", "/etc/supervisord.conf"]

redis

elasticsearch

ES的开源监控

docker run -tid --name es-admin -p 4000:9100 mobz/elasticsearch-head:5

kafka

ceph

docker run -itd --restart=always -p 5000:5000 \
        -e CEPHMONS="$(awk -F= '/mon_host/{print $2}' /etc/ceph/ceph.conf)" \
        -e KEYRING="$(sudo cat /etc/ceph/ceph.client.admin.keyring)" \
crapworks/ceph-dash:latest
 
docker run -tid --restart=always -p=9128:9128 \
       -v /etc/ceph:/etc/ceph digitalocean/ceph_exporter

ignite

opensips

docker pull opensips/opensips:latest
docker run -tid --name=opensips --restart=always --network=host --privileged=true --cap-add=SYS_ADMIN opensips/opensips:latest

Tikv/Tidb