Etcd 服务发现与 Registrator 配置指南

Etcd 服务发现与 Registrator 配置指南

本文档介绍如何使用 etcd 和 Registrator 构建基于 Docker 的自动服务发现系统。etcd 作为服务注册中心,Registrator 自动监控 Docker 容器并注册服务信息。

📋 前置要求

  • Docker 20.10+ 已安装
  • Docker Compose V2(推荐)或 docker-compose 1.29+
  • 至少 3 个节点用于 etcd 集群(生产环境推荐)

⚠️ 重要说明

Registrator 项目状态:Registrator 项目已经不再积极维护。对于新项目,建议考虑以下替代方案:

  • Consul + Consul Registrator:更现代的服务发现方案
  • Kubernetes:内置服务发现机制
  • Traefik:自动服务发现和负载均衡
  • 自定义脚本:基于 Docker Events API 实现

本文档保留 Registrator 配置作为参考,但建议新项目使用更现代的方案。

🏗️ 架构说明

1
2
3
4
5
6
7
8
9
10
11
12
13
┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│ Node 1 │ │ Node 2 │ │ Node 3 │
│ │ │ │ │ │
│ etcd-1 │◄───►│ etcd-2 │◄───►│ etcd-3 │
│ registrator│ │ registrator│ │ registrator│
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└───────────────────┴───────────────────┘

┌──────▼──────┐
│ etcd │
│ Cluster │
└─────────────┘

📦 项目结构

1
2
mkdir -p etcd-registrator-setup
cd etcd-registrator-setup

🐳 Docker Compose 配置

创建 docker-compose.yml 文件:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
version: '3.8'

services:
# etcd 节点 1
etcd:
container_name: etcd-node1
image: quay.io/coreos/etcd:v3.5.9
environment:
ETCD_NAME: etcd-node1
ETCD_DATA_DIR: /etcd-data
ETCD_ADVERTISE_CLIENT_URLS: "http://${ETCD_IP:-192.168.3.170}:2379"
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS: "http://${ETCD_IP:-192.168.3.170}:2380"
ETCD_LISTEN_PEER_URLS: "http://0.0.0.0:2380"
ETCD_INITIAL_CLUSTER_TOKEN: etcd-cluster-token
ETCD_INITIAL_CLUSTER: "etcd-node1=http://${ETCD_IP:-192.168.3.170}:2380,etcd-node2=http://${ETCD_NODE1_IP:-192.168.3.21}:2380,etcd-node3=http://${ETCD_NODE2_IP:-192.168.3.33}:2380"
ETCD_INITIAL_CLUSTER_STATE: new
ETCDCTL_API: "3"
volumes:
- etcd-node1-data:/etcd-data
ports:
- "2379:2379"
- "2380:2380"
networks:
- etcd-network
restart: unless-stopped

# etcd 节点 2(需要在对应节点上运行)
etcd_node2:
container_name: etcd-node2
image: quay.io/coreos/etcd:v3.5.9
environment:
ETCD_NAME: etcd-node2
ETCD_DATA_DIR: /etcd-data
ETCD_ADVERTISE_CLIENT_URLS: "http://${ETCD_NODE1_IP:-192.168.3.21}:2379"
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS: "http://${ETCD_NODE1_IP:-192.168.3.21}:2380"
ETCD_LISTEN_PEER_URLS: "http://0.0.0.0:2380"
ETCD_INITIAL_CLUSTER_TOKEN: etcd-cluster-token
ETCD_INITIAL_CLUSTER: "etcd-node1=http://${ETCD_IP:-192.168.3.170}:2380,etcd-node2=http://${ETCD_NODE1_IP:-192.168.3.21}:2380,etcd-node3=http://${ETCD_NODE2_IP:-192.168.3.33}:2380"
ETCD_INITIAL_CLUSTER_STATE: new
ETCDCTL_API: "3"
volumes:
- etcd-node2-data:/etcd-data
ports:
- "2379:2379"
- "2380:2380"
networks:
- etcd-network
restart: unless-stopped

# etcd 节点 3(需要在对应节点上运行)
etcd_node3:
container_name: etcd-node3
image: quay.io/coreos/etcd:v3.5.9
environment:
ETCD_NAME: etcd-node3
ETCD_DATA_DIR: /etcd-data
ETCD_ADVERTISE_CLIENT_URLS: "http://${ETCD_NODE2_IP:-192.168.3.33}:2379"
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS: "http://${ETCD_NODE2_IP:-192.168.3.33}:2380"
ETCD_LISTEN_PEER_URLS: "http://0.0.0.0:2380"
ETCD_INITIAL_CLUSTER_TOKEN: etcd-cluster-token
ETCD_INITIAL_CLUSTER: "etcd-node1=http://${ETCD_IP:-192.168.3.170}:2380,etcd-node2=http://${ETCD_NODE1_IP:-192.168.3.21}:2380,etcd-node3=http://${ETCD_NODE2_IP:-192.168.3.33}:2380"
ETCD_INITIAL_CLUSTER_STATE: new
ETCDCTL_API: "3"
volumes:
- etcd-node3-data:/etcd-data
ports:
- "2379:2379"
- "2380:2380"
networks:
- etcd-network
restart: unless-stopped

# Registrator 节点 1
registrator:
container_name: registrator-node1
image: gliderlabs/registrator:latest
command:
- "-ttl=60"
- "-ttl-refresh=30"
- "-ip"
- "${ETCD_IP:-192.168.3.170}"
- "etcd://${ETCD_IP:-192.168.3.170}:2379/services"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
depends_on:
- etcd
networks:
- etcd-network
restart: unless-stopped

# Registrator 节点 2
registrator_node2:
container_name: registrator-node2
image: gliderlabs/registrator:latest
command:
- "-ttl=60"
- "-ttl-refresh=30"
- "-ip"
- "${ETCD_NODE1_IP:-192.168.3.21}"
- "etcd://${ETCD_NODE1_IP:-192.168.3.21}:2379/services"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
depends_on:
- etcd_node2
networks:
- etcd-network
restart: unless-stopped

# Registrator 节点 3
registrator_node3:
container_name: registrator-node3
image: gliderlabs/registrator:latest
command:
- "-ttl=60"
- "-ttl-refresh=30"
- "-ip"
- "${ETCD_NODE2_IP:-192.168.3.33}"
- "etcd://${ETCD_NODE2_IP:-192.168.3.33}:2379/services"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
depends_on:
- etcd_node3
networks:
- etcd-network
restart: unless-stopped

volumes:
etcd-node1-data:
etcd-node2-data:
etcd-node3-data:

networks:
etcd-network:
driver: bridge

🚀 部署脚本

deploy_manager.sh(主节点)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#!/bin/bash

# 配置节点 IP(根据实际情况修改)
export ETCD_IP=192.168.3.170
export ETCD_NODE1_IP=192.168.3.21
export ETCD_NODE2_IP=192.168.3.33

# 启动 etcd 和 registrator
docker compose up -d etcd registrator

# 等待服务启动
sleep 5

# 检查服务状态
docker compose ps

deploy_node2.sh(节点 2)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#!/bin/bash

# 配置节点 IP
export ETCD_IP=192.168.3.170
export ETCD_NODE1_IP=192.168.3.21
export ETCD_NODE2_IP=192.168.3.33

# 启动 etcd 节点 2 和对应的 registrator
docker compose up -d etcd_node2 registrator_node2

# 等待服务启动
sleep 5

# 检查服务状态
docker compose ps

deploy_node3.sh(节点 3)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#!/bin/bash

# 配置节点 IP
export ETCD_IP=192.168.3.170
export ETCD_NODE1_IP=192.168.3.21
export ETCD_NODE2_IP=192.168.3.33

# 启动 etcd 节点 3 和对应的 registrator
docker compose up -d etcd_node3 registrator_node3

# 等待服务启动
sleep 5

# 检查服务状态
docker compose ps

设置执行权限:

1
chmod +x deploy_*.sh

🐍 Python 客户端示例

使用 etcd3-py(推荐)

安装依赖:

1
pip install etcd3

etcd_client_demo.py(使用 etcd3-py):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
#!/usr/bin/env python3
"""
etcd 服务发现客户端示例
使用 etcd3-py 库(推荐)
"""

import etcd3
import json
from typing import List, Dict

class ServiceDiscovery:
def __init__(self, host: str = '192.168.3.170', port: int = 2379):
"""
初始化 etcd 客户端

Args:
host: etcd 服务器地址
port: etcd 服务器端口
"""
self.client = etcd3.client(host=host, port=port)
self.service_prefix = '/services/'

def list_services(self) -> List[str]:
"""
列出所有已注册的服务

Returns:
服务名称列表
"""
services = set()
for value, metadata in self.client.get_prefix(self.service_prefix):
if metadata:
# 从 key 中提取服务名称
key = metadata.key.decode('utf-8')
service_name = key.replace(self.service_prefix, '').split('/')[0]
services.add(service_name)
return list(services)

def get_service_instances(self, service_name: str) -> List[Dict]:
"""
获取指定服务的所有实例

Args:
service_name: 服务名称

Returns:
服务实例列表,每个实例包含地址和端口信息
"""
instances = []
service_path = f"{self.service_prefix}{service_name}/"

for value, metadata in self.client.get_prefix(service_path):
if value and metadata:
try:
# Registrator 存储的格式通常是 JSON
service_info = json.loads(value.decode('utf-8'))
instances.append({
'key': metadata.key.decode('utf-8'),
'value': service_info,
'address': service_info.get('Addr', ''),
'port': service_info.get('Port', '')
})
except json.JSONDecodeError:
# 如果不是 JSON,直接使用原始值
instances.append({
'key': metadata.key.decode('utf-8'),
'value': value.decode('utf-8')
})

return instances

def watch_service(self, service_name: str, callback):
"""
监听服务变化

Args:
service_name: 服务名称
callback: 回调函数,接收 (event_type, key, value) 参数
"""
service_path = f"{self.service_prefix}{service_name}/"

def watch_callback(event):
event_type = 'PUT' if isinstance(event, etcd3.events.PutEvent) else 'DELETE'
key = event.key.decode('utf-8') if event.key else ''
value = event.value.decode('utf-8') if hasattr(event, 'value') and event.value else ''
callback(event_type, key, value)

# 开始监听
for event in self.client.watch_prefix(service_path):
watch_callback(event)


def main():
"""示例用法"""
sd = ServiceDiscovery(host='192.168.3.170', port=2379)

# 列出所有服务
print("已注册的服务:")
services = sd.list_services()
for service in services:
print(f" - {service}")

# 获取特定服务的实例
if services:
service_name = services[0]
print(f"\n服务 '{service_name}' 的实例:")
instances = sd.get_service_instances(service_name)
for instance in instances:
print(f" - {instance}")

# 监听服务变化(示例)
def on_service_change(event_type, key, value):
print(f"\n服务变化: {event_type} - {key}")
if value:
print(f" 值: {value}")

print("\n开始监听服务变化(按 Ctrl+C 停止)...")
try:
if services:
sd.watch_service(services[0], on_service_change)
except KeyboardInterrupt:
print("\n停止监听")


if __name__ == '__main__':
main()

使用传统 etcd 库(兼容旧代码)

如果必须使用旧的 python-etcd 库:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
#!/usr/bin/env python3
"""
etcd 服务发现客户端示例(兼容旧版)
使用 python-etcd 库
"""

try:
import etcd
except ImportError:
print("请安装 python-etcd: pip install python-etcd")
exit(1)

class ServiceDiscoveryLegacy:
def __init__(self, host: str = '192.168.3.170', port: int = 2379):
self.client = etcd.Client(host=host, port=port)
self.service_path = '/services/'

def list_services(self):
"""列出所有服务"""
try:
directory = self.client.read(self.service_path, recursive=True)
services = set()

if directory and hasattr(directory, 'children'):
for child in directory.children:
# 提取服务名称
relative_path = child.key.replace(self.service_path, '')
service_name = relative_path.split('/')[0]
services.add(service_name)

return list(services)
except etcd.EtcdKeyNotFound:
return []

def get_service_instances(self, service_name: str):
"""获取服务实例"""
req_path = f"{self.service_path}{service_name}/"

try:
directory = self.client.get(req_path)
instances = []

if directory and hasattr(directory, 'children'):
for child in directory.children:
instances.append({
'key': child.key,
'value': child.value
})

return instances
except etcd.EtcdKeyNotFound:
return []


if __name__ == '__main__':
sd = ServiceDiscoveryLegacy(host='192.168.3.170', port=2379)

services = sd.list_services()
print(f"已注册的服务: {services}")

if services:
instances = sd.get_service_instances(services[0])
print(f"服务 '{services[0]}' 的实例:")
for instance in instances:
print(f" {instance['key']}: {instance['value']}")

🔍 验证和测试

检查 etcd 集群状态

1
2
3
4
5
# 使用 etcdctl(在容器内)
docker exec etcd-node1 etcdctl --endpoints=http://localhost:2379 endpoint health

# 或使用本地 etcdctl
etcdctl --endpoints=http://192.168.3.170:2379 endpoint health

查看注册的服务

1
2
3
4
5
# 列出所有键
docker exec etcd-node1 etcdctl --endpoints=http://localhost:2379 get --prefix /services

# 或使用 Python 脚本
python etcd_client_demo.py

测试服务注册

启动一个测试容器,Registrator 会自动注册:

1
2
3
4
5
6
docker run -d \
--name test-service \
-p 8080:8080 \
-e SERVICE_NAME=test-service \
-e SERVICE_8080_NAME=test-service \
nginx:alpine

🔒 安全建议

  1. 启用 etcd 认证:生产环境必须启用 TLS 和认证
  2. 限制网络访问:使用防火墙限制 etcd 端口访问
  3. 定期备份:定期备份 etcd 数据
  4. 监控告警:监控 etcd 集群健康状态

📚 参考资源

🔄 迁移建议

如果考虑迁移到更现代的方案:

  1. Consul:功能更丰富,社区活跃
  2. Kubernetes:如果使用 K8s,内置服务发现
  3. Traefik:自动服务发现和负载均衡
  4. 自定义方案:基于 Docker Events API 实现
0%