harbor 搭建

发表于 | 分类于 | 阅读次数:
字数统计: 3,008 字 | 阅读时长 ≈ 18 分钟

写在前面的话

每篇一句

临渊羡鱼，不如退而结网。

harbor搭建

harbor github
harbor各发布版本
harbor离线安装包harbor-offline-installer-v1.2.2.tgz

离线版安装，http

基本要求

python >= 2.7
Docker engine >= 1.10
Docker Compose >= 1.60

安装过程

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
root@controller:/data/harbor# mkdir  -p /data/harbor/
root@controller:/data/harbor# tar -xzvf harbor.v1.2.2.tar.gz 
root@controller:/data/harbor# ./install.sh
root@controller:/data/harbor# cat /etc/docker/daemon.json
{"registry-mirrors": ["http://8b3fddac.m.daocloud.io"],
"insecure-registries" : ["controller:80"]
}
root@controller:/lib/systemd/system# systemctl daemon-reload
root@controller:/lib/systemd/system# /etc/init.d/docker restart
[ ok ] Restarting docker (via systemctl): docker.service.
root@controller:/lib/systemd/system# systemctl status docker.service
● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2017-11-15 15:27:25 CST; 10s ago
     Docs: https://docs.docker.com
 Main PID: 33476 (dockerd)
    Tasks: 153
   Memory: 57.8M
      CPU: 3.966s
root@controller:/data/harbor# docker-compose stop
Stopping harbor-jobservice ... done
Stopping nginx ... done
Stopping harbor-ui ... done
Stopping harbor-db ... done
Stopping registry ... done
Stopping harbor-log ... done
root@controller:/data/harbor# docker-compose start
Starting log ... done
Starting adminserver ... done
Starting registry ... done
Starting ui ... done
Starting mysql ... done
Starting jobservice ... done
Starting proxy ... done
root@controller:/data/harbor# docker-compose ps
       Name                     Command               State                                Ports                               
------------------------------------------------------------------------------------------------------------------------------
harbor-adminserver   /harbor/harbor_adminserver       Up                                                                       
harbor-db            docker-entrypoint.sh mysqld      Up      3306/tcp                                                         
harbor-jobservice    /harbor/harbor_jobservice        Up                                                                       
harbor-log           /bin/sh -c crond && rm -f  ...   Up      127.0.0.1:1514->514/tcp                                          
harbor-ui            /harbor/harbor_ui                Up                                                                       
nginx                nginx -g daemon off;             Up      0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp, 0.0.0.0:80->80/tcp 
registry             /entrypoint.sh serve /etc/ ...   Up      5000/tcp

命令行登录

1
2
3
4
root@controller:/data/harbor# docker login controller:80
Username (admin): admin        
Password: 
Login Succeeded

往habor仓库push 镜像

1
2
3
4
5
6
7
8
9
10
11
root@controller:/data/harbor# docker tag  httpd controller:80/zzqproject/myhttpd:v1
root@controller:/data/harbor# docker push  controller:80/zzqproject/myhttpd:v1
The push refers to a repository [controller:80/zzqproject/myhttpd]
73ff04cf8df5: Pushed 
e70070ad9b01: Pushed 
10cd1ea92057: Pushed 
b7fac20846c1: Pushed 
5789daa1c37d: Pushed 
aa4c186185aa: Pushed 
5d6cbe0dbcf9: Pushed 
v1: digest: sha256:860090e16f4797126c632c138a4251133ee5af45a569d61c18e1547eddd956c2 size: 1780

离线版安装，https 方式 （建议使用该方式，特别是公网环境下）

自签名证书生成

注意如果生成自签名证书的host地址是IP，则需要编辑/etc/ssl/openssl.cnf，在[v3_ca]段落内添加。

Failed to tls handshake with 192.168.252.10 x509: cannot validate certificate for 192.168.252.10 because it doesn’t contain any IP SANs

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
root@controller:~# cd /root/crt/
root@controller:~/crt# cat /etc/ssl/openssl.cnf |grep -i subjectAltName
subjectAltName = IP:192.168.252.10
root@controller:/data/harbor# openssl genrsa -out private_key.pem 4096
Generating RSA private key, 4096 bit long modulus
...................................................................................................++
...........................++
e is 65537 (0x10001)
root@controller:/data/harbor# openssl req -new -x509 -key private_key.pem -out root.crt -days 3650
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:CN
State or Province Name (full name) [Some-State]:guangdong
Locality Name (eg, city) []:shenzhen
Organization Name (eg, company) [Internet Widgits Pty Ltd]:zzq
Organizational Unit Name (eg, section) []:zzq
Common Name (e.g. server FQDN or YOUR name) []:controller
Email Address []:zhaoqianzhang@yeah.net
root@controller:~/crt# ls -lrt
total 8
-rw-r--r-- 1 root root 3243 Nov 16 14:48 private_key.pem
-rw-r--r-- 1 root root 2147 Nov 16 14:48 root.crt

修改配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
root@controller:/data/harbor# cat harbor.cfg 
## Configuration file of Harbor
#The IP address or hostname to access admin UI and registry service.
#DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
hostname = controller
#The protocol for accessing the UI and token/notification service, by default it is http.
#It can be set to https if ssl is enabled on nginx.
ui_url_protocol = https
#The password for the root user of mysql db, change this before any production use.
db_password = root123
#Maximum number of job workers in job service  
max_job_workers = 8
#Determine whether or not to generate certificate for the registry's token.
#If the value is on, the prepare script creates new root cert and private key 
#for generating token to access the registry. If the value is off the default key/cert will be used.
#This flag also controls the creation of the notary signer's cert.
customize_crt = on
#The path of cert and key files for nginx, they are applied only the protocol is set to https
ssl_cert = /data/cert/server.crt
ssl_cert_key = /data/cert/server.key
#The path of secretkey storage
secretkey_path = /data
#Admiral's url, comment this attribute, or set its value to NA when Harbor is standalone
admiral_url = NA
#The password of the Clair's postgres database, only effective when Harbor is deployed with Clair.
#Please update it before deployment, subsequent update will cause Clair's API server and Harbor unable to access Clair's database.
clair_db_password = password
#NOTES: The properties between BEGIN INITIAL PROPERTIES and END INITIAL PROPERTIES
#only take effect in the first boot, the subsequent changes of these properties 
#should be performed on web ui
#************************BEGIN INITIAL PROPERTIES************************
#Email account settings for sending out password resetting emails.
#Email server uses the given username and password to authenticate on TLS connections to host and act as identity.
#Identity left blank to act as username.
email_identity = 
email_server = smtp.mydomain.com
email_server_port = 25
email_username = sample_admin@mydomain.com
email_password = abc
email_from = admin <sample_admin@mydomain.com>
email_ssl = false
##The initial password of Harbor admin, only works for the first time when Harbor starts. 
#It has no effect after the first launch of Harbor.
#Change the admin password from UI after launching Harbor.
harbor_admin_password = Harbor12345
##By default the auth mode is db_auth, i.e. the credentials are stored in a local database.
#Set it to ldap_auth if you want to verify a user's credentials against an LDAP server.
auth_mode = db_auth
#The url for an ldap endpoint.
ldap_url = ldaps://ldap.mydomain.com
#A user's DN who has the permission to search the LDAP/AD server. 
#If your LDAP/AD server does not support anonymous search, you should configure this DN and ldap_search_pwd.
#ldap_searchdn = uid=searchuser,ou=people,dc=mydomain,dc=com
#the password of the ldap_searchdn
#ldap_search_pwd = password
#The base DN from which to look up a user in LDAP/AD
ldap_basedn = ou=people,dc=mydomain,dc=com
#Search filter for LDAP/AD, make sure the syntax of the filter is correct.
#ldap_filter = (objectClass=person)
# The attribute used in a search to match a user, it could be uid, cn, email, sAMAccountName or other attributes depending on your LDAP/AD  
ldap_uid = uid 
#the scope to search for users, 1-LDAP_SCOPE_BASE, 2-LDAP_SCOPE_ONELEVEL, 3-LDAP_SCOPE_SUBTREE
ldap_scope = 3 
#Timeout (in seconds)  when connecting to an LDAP Server. The default value (and most reasonable) is 5 seconds.
ldap_timeout = 5
#Turn on or off the self-registration feature
self_registration = on
#The expiration time (in minute) of token created by token service, default is 30 minutes
token_expiration = 30
#The flag to control what users have permission to create projects
#The default value "everyone" allows everyone to creates a project. 
#Set to "adminonly" so that only admin user can create project.
project_creation_restriction = everyone
#Determine whether the job service should verify the ssl cert when it connects to a remote registry.
#Set this flag to off when the remote registry uses a self-signed or untrusted certificate.
verify_remote_cert = off
#************************END INITIAL PROPERTIES************************
#############

拷贝自签名证书到相应路径

1
2
3
4
mkdir -p  /data/cert/
cd /root/crt/
cp private_key.pem  /data/cert/server.key
cp root.crt  /data/cert/server.crt

harbor安装 （包含clair 镜像漏洞扫描模块）

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
root@controller:/data/harbor# ll
total 527684
drwxr-xr-x  4 root root      4096 Nov 16 14:24 ./
drwxr-xr-x 11 root root      4096 Nov 16 12:57 ../
drwxr-xr-x  2 root root      4096 Nov 16 11:06 bakyml/
drwxr-xr-x  4 root root      4096 Nov 16 12:55 common/
-rw-r--r--  1 root root      1163 Oct 20 16:52 docker-compose.clair.yml
-rw-r--r--  1 root root      1988 Oct 20 16:52 docker-compose.notary.yml
-rw-r--r--  1 root root      3191 Oct 20 16:52 docker-compose.yml
-rw-r--r--  1 root root      4304 Oct 20 16:52 harbor_1_1_0_template
-rw-r--r--  1 root root      4340 Nov 16 14:24 harbor.cfg
-rw-r--r--  1 root root 539885476 Oct 20 16:56 harbor.v1.2.2.tar.gz
-rwxr-xr-x  1 root root      5332 Oct 20 16:52 install.sh*
-rw-r--r--  1 root root    371640 Oct 20 16:52 LICENSE
-rw-r--r--  1 root root       482 Oct 20 16:52 NOTICE
-rwxr-xr-x  1 root root     17592 Oct 20 16:52 prepare*
-rwxr-xr-x  1 root root      4550 Oct 20 16:52 upgrade*
root@controller:/data/harbor# ./prepare --with-clair
Clearing the configuration file: ./common/config/clair/postgres_env
Clearing the configuration file: ./common/config/clair/config.yaml
Clearing the configuration file: ./common/config/clair/postgresql-init.d/README.md
Clearing the configuration file: ./common/config/db/env
Clearing the configuration file: ./common/config/jobservice/env
Clearing the configuration file: ./common/config/jobservice/app.conf
Clearing the configuration file: ./common/config/ui/env
Clearing the configuration file: ./common/config/ui/private_key.pem
Clearing the configuration file: ./common/config/ui/app.conf
Clearing the configuration file: ./common/config/adminserver/env
Clearing the configuration file: ./common/config/nginx/cert/server.key
Clearing the configuration file: ./common/config/nginx/cert/server.crt
Clearing the configuration file: ./common/config/nginx/nginx.conf
Clearing the configuration file: ./common/config/registry/config.yml
Clearing the configuration file: ./common/config/registry/root.crt
loaded secret from file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/app.conf
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
Copying offline data file for clair DB
Generated configuration file: ./common/config/clair/postgres_env
Generated configuration file: ./common/config/clair/config.yaml
The configuration files are ready, please use docker-compose to start the service.
root@controller:/data/harbor# ./install.sh --with-clair
[Step 0]: checking installation environment ...
Note: docker version: 17.06.1
Note: docker-compose version: 1.8.0
[Step 1]: loading Harbor images ...
Loaded image: vmware/harbor-ui:v1.2.2
Loaded image: vmware/notary-photon:server-0.5.0
Loaded image: vmware/nginx-photon:1.11.13
Loaded image: vmware/registry:2.6.2-photon
Loaded image: photon:1.0
Loaded image: vmware/notary-photon:signer-0.5.0
Loaded image: vmware/harbor-adminserver:v1.2.2
Loaded image: vmware/harbor-log:v1.2.2
Loaded image: vmware/harbor-db:v1.2.2
Loaded image: vmware/harbor-jobservice:v1.2.2
Loaded image: vmware/harbor-notary-db:mariadb-10.1.10
Loaded image: vmware/clair:v2.0.1-photon
Loaded image: vmware/postgresql:9.6.4-photon
[Step 2]: preparing environment ...
Clearing the configuration file: ./common/config/clair/postgres_env
Clearing the configuration file: ./common/config/clair/config.yaml
Clearing the configuration file: ./common/config/clair/postgresql-init.d/README.md
Clearing the configuration file: ./common/config/db/env
Clearing the configuration file: ./common/config/jobservice/env
Clearing the configuration file: ./common/config/jobservice/app.conf
Clearing the configuration file: ./common/config/ui/env
Clearing the configuration file: ./common/config/ui/private_key.pem
Clearing the configuration file: ./common/config/ui/app.conf
Clearing the configuration file: ./common/config/adminserver/env
Clearing the configuration file: ./common/config/nginx/cert/server.key
Clearing the configuration file: ./common/config/nginx/cert/server.crt
Clearing the configuration file: ./common/config/nginx/nginx.conf
Clearing the configuration file: ./common/config/registry/config.yml
Clearing the configuration file: ./common/config/registry/root.crt
loaded secret from file: /data/secretkey
Generated configuration file: ./common/config/nginx/nginx.conf
Generated configuration file: ./common/config/adminserver/env
Generated configuration file: ./common/config/ui/env
Generated configuration file: ./common/config/registry/config.yml
Generated configuration file: ./common/config/db/env
Generated configuration file: ./common/config/jobservice/env
Generated configuration file: ./common/config/jobservice/app.conf
Generated configuration file: ./common/config/ui/app.conf
Generated certificate, key file: ./common/config/ui/private_key.pem, cert file: ./common/config/registry/root.crt
Copying offline data file for clair DB
Generated configuration file: ./common/config/clair/postgres_env
Generated configuration file: ./common/config/clair/config.yaml
The configuration files are ready, please use docker-compose to start the service.
[Step 3]: checking existing instance of Harbor ...
Note: stopping existing Harbor instance ...
Stopping harbor-jobservice ... done
Stopping nginx ... done
Stopping harbor-ui ... done
Stopping clair ... done
Stopping harbor-adminserver ... done
Stopping clair-db ... done
Stopping registry ... done
Stopping harbor-db ... done
Stopping harbor-log ... done
Removing harbor-jobservice ... done
Removing nginx ... done
Removing harbor-ui ... done
Removing clair ... done
Removing harbor-adminserver ... done
Removing clair-db ... done
Removing registry ... done
Removing harbor-db ... done
Removing harbor-log ... done
Removing network harbor_harbor
Removing network harbor_harbor-clair
[Step 4]: starting Harbor ...
WARNING: The Docker Engine you're using is running in swarm mode.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.
To deploy your application across the swarm, use the bundle feature of the Docker experimental build.
More info:
https://docs.docker.com/compose/bundles
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-clair" with the default driver
Creating harbor-log
Creating registry
Creating harbor-db
Creating harbor-adminserver
Creating clair-db
Creating clair
Creating harbor-ui
Creating harbor-jobservice
Creating nginx
✔ ----Harbor has been installed and started successfully.----
Now you should be able to visit the admin portal at https://controller. 
For more details, please visit https://github.com/vmware/harbor .

命令行登录(ubuntu16.04 harbor 本机)

1
2
3
4
5
6
7
8
9
10
root@controller:/data/harbor# cat /etc/docker/daemon.json
{"registry-mirrors": ["http://8b3fddac.m.daocloud.io"],
"insecure-registries" : ["controller"],
"insecure-registries" : ["192.168.252.10"]
}
systemctl restart docker.service
root@controller:/data/harbor# docker login  192.168.252.10 
Username (admin): admin
Password: 
Login Succeeded

命令行登录(centos7.4 client 客户端)

1
2
3
4
5
6
7
[root@docker_host ~]# cat /usr/lib/systemd/system/docker.service |grep insecure
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.252.10
[root@docker_host ~]# systemctl  restart docker
[root@docker_host ~]# docker login  192.168.252.10
Username (admin): admin
Password: 
Login Succeeded

解决域名登陆失败

1、证书不匹配，注意在生成证书crt时Common Name 要使用FQDN（也就是harbor host的域名即域名）
https://docs.docker.com/registry/insecure/#use-self-signed-certificates

1
2
3
4
5
6
7
8
[root@docker_host ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.252.10 controller
[root@docker_host ~]# docker login  controller
Username: admin
Password: 
Error response from daemon: Get https://controller/v2/: x509: certificate is valid for zhangzhaoqian, not controller

2、证书不是认证的机构颁发的

1
2
3
4
[root@docker_host ~]# docker login  controller
Username: admin
Password: 
Error response from daemon: Get https://controller/v2/: x509: certificate signed by unknown authority

命令行登录(centos7.4 client 客户端),支持域名和ip都行

多个不验证仓库格式： –insecure-registry IP1:PORT –insecure-registry IP2:PORT –insecure-registry IP3:PORT

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@docker_host ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.252.10 controller
[root@docker_host ~]# cat /usr/lib/systemd/system/docker.service |grep -i insecure
ExecStart=/usr/bin/dockerd --insecure-registry controller --insecure-registry 192.168.252.10
[root@docker_host ~]# systemctl  restart docker
Warning: docker.service changed on disk. Run 'systemctl daemon-reload' to reload units.
[root@docker_host ~]# systemctl daemon-reload
[root@docker_host ~]# systemctl  restart docker
[root@docker_host ~]# docker login 192.168.252.10
Username (admin): admin
Password: 
Login Succeeded
[root@docker_host ~]# docker login controller
Username (admin): admin
Password: 
Login Succeeded

命令行登录(ubuntu16.04 harbor 本机)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
root@controller:/data/harbor# docker-compose -f  docker-compose.yml  -f docker-compose.clair.yml  stop
Stopping harbor-jobservice ... done
Stopping nginx ... done
Stopping harbor-ui ... done
Stopping clair ... done
Stopping harbor-adminserver ... done
Stopping clair-db ... done
Stopping registry ... done
Stopping harbor-db ... done
Stopping harbor-log ... done
root@controller:/etc/docker/certs.d/controller# cat /etc/docker/daemon.json
{"registry-mirrors": ["http://8b3fddac.m.daocloud.io"],
"insecure-registries" : ["controller"],
"insecure-registries" : ["192.168.252.10"]
}
root@controller:/etc/docker/certs.d/controller# /etc/init.d/docker restart
[ ok ] Restarting docker (via systemctl): docker.service.
root@controller:/data/harbor# docker-compose -f  docker-compose.yml  -f docker-compose.clair.yml  start
Starting log ... done
Starting adminserver ... done
Starting mysql ... done
Starting registry ... done
Starting ui ... done
Starting jobservice ... done
Starting proxy ... done
Starting postgres ... done
Starting clair ... done
root@controller:/etc/docker/certs.d/controller# docker login controller
Username (admin): admin
Password: 
Login Succeeded
root@controller:/etc/docker/certs.d/controller# docker login 192.168.252.10
Username (admin): admin
Password: 
Login Succeeded

同步配置

master主： controller 192.168.252.10
slave从： docker_host 192.168.252.20

主，刚刚已经搭建好了，https方式。
从，使用http参照前面。

效果图

master 登录首页

master 镜像扫描

master 新建复制目标

master 新建复制规则


master 同步镜像到slave 记录

slave 在复制规则建立前

slave 自动同步成功

slave http 方式wireshark 抓包，可以看到账号密码、镜像等信息，不安全。

harborwithtsl.pcapng
harbor-http.pcapng

----纸上得来终觉浅绝知此事要躬行----
最好的赞赏是您的阅读！
0%