CTFd集群搭建whale

# CTFd搭建教程

> 本次安装使用Docker Swarm集群，针对中大型比赛建议使用本教程，从机只需要docker join加入swarm集群即可

## 安装docker

```shell
curl -fsSL https://download.imxbt.cn/getdocker -o get-docker.sh
DOWNLOAD_URL=https://mirrors.ustc.edu.cn/docker-ce sh get-docker.sh
```

### 添加镜像加速源

```shell
echo '{"registry-mirrors": ["加速源"]}' | sudo tee /etc/docker/daemon.json
```

## 安装CTFd和whale

```d
# Git拉取CTFd
mkdir -p /opt/
cd /opt
git clone https://mirror.ghproxy.com/https://github.com/CTFd/CTFd.git
# 拉取whale
cd CTFd/CTFd/plugins
git clone https://mirror.ghproxy.com/https://github.com/frankli0324/ctfd-whale.git ctfd-whale
cd ctfd-whale
```

### 配置docker-compose

> 这部分根据自己实际情况配置

```shell
cd /opt/CTFd
nano docker-compose.yml
```

### 修改Dockerfile

```shell
cd /opt/CTFd
nano Dockerfile
```

在 `WORKDIR /opt/CTFd` 下一行新增换源内容，以加快拉取速度

```dockerfile
RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources
```

在 `pip install` 处后面添加(2处)

```
-i https://pypi.tuna.tsinghua.edu.cn/simple 
```

### 编译并运行CTFd

```shell
docker compose build
docker compose up -d
docker network connect frpcadmin <CTFd容器id>
```

全部拉取后提示started即可通过8000端口访问CTFd,也可以自行配置NGINX实现80/443端口访问

## 初始化

首先在CTFd的机器上初始化一个Docker Swarm

```shell
docker swarm init
docker node update --label-add='name=ctfd_frp_containers' $(docker node ls -q)
docker node update --label-add "name=linux-1" $(docker node ls -q)
```

访问CTFd web页面之后一切设置完毕后到管理面板进行设置，找到whale

![image-20240817150832068](https://download.imxbt.cn/upload/202408171508322187b00d08f34f04f5602ebd26763ef2a1d.png)

出现如图所示即是whale插件安装成功

### （单机）编辑docker-compose

```shell
cd /opt/CTFd
nano docker-compose.yml
```

在services.ctfd.volumes中加入

```yaml
- /var/run/docker.sock:/var/run/docker.sock
```

使其可以映射到docker的API

### （集群）使Docker Swarm可以被连接

> 此方案的缺点是如果远程访问没有TLS的话会有安全风险，建议内网访问不要开放到公网

```
systemctl edit docker.service
```

在如下行

```shell
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
```

追加内容

```shell
 tcp://0.0.0.0:2375
```

重启应用即可

```shell
systemctl daemon-reload && systemctl restart docker
#放行ufw端口
ufw allow 2375
```

### 编辑frp

在 `/opt/CTFd/conf/frp/` 中新建一个 `frps.ini和frpc.ini` 并填入如下内容

> frps.ini

```ini
[common]
bind_addr = 0.0.0.0 #这里做演示使用的IP，为了安全请及时更换
bind_port = 7897
token = ctfdmaster
```

> frpc.ini

```ini
[common]
server_addr = frps $这里代指frps的docker IP
server_port = 7897
token = ctfdmaster
admin_addr = 0.0.0.0 #这里做演示使用的IP，为了安全请及时更换
admin_port = 7400
admin_user = user
admin_pwd = pwd
```

#### 编辑docker-compose

在CTFd的docker-compose.yml中找到networks追加

```yaml
    frp_admin:
        internal: true
    frp_containers:
        driver: overlay
        internal: true
        attachable: true
```

在services中追加如下

```yaml
  frps:
    image: snowdreamtech/frps:0.30.0
    restart: always
    volumes:
       - ./conf/frp/frps.ini:/etc/frp/frps.ini
    entrypoint:
       - /usr/bin/frps
       - '-c'
       - /etc/frp/frps.ini
    ports:
       - '34000-35000:34000-35000' #题目端口范围
    networks:
       frp_admin:
       default:
       
  frpc:
    image: snowdreamtech/frpc:0.30.0
    restart: always
    ports:
       - 7400:7400
    volumes:
        - ./conf/frp/frpc.ini:/etc/frp/frpc.ini #配置文件在CTFd/conf/frp/frpc.ini中
    entrypoint:
        - /usr/bin/frpc
        - '-c'
        - /etc/frp/frpc.ini    
    depends_on:
        - frps
    networks:
        frp_admin:
        frp_containers:
```

并在services.ctfd中修改如下

```yaml
services:
    ...
    ctfd:
        ...
        depends_on:
            ...
            - frpc
        networks:
            ...
            frp_admin:
```

重启docker compose容器即可

### 配置API

回到whale设置页面，在Docker页面的API URL中设置

```
tcp://172.17.0.1:2375
```

即可连接到Docker API

在frp页面的API URL中填写如下即可连接到frpc，如果没有配置admin账密直接填写 `http://frpc:7400`

```
http://user:pwd@frpc:7400
```

**Direct IP Address**填写服务器IP或者域名即可，direct minimum port和direct maximum port是转发用的端口范围，要和docker-compose中的frps配置的端口范围相匹配

在Frpc config template中填写，即可完成连接Frp

```ini
[common]
server_addr = frps
server_port = 7897
token = ctfdmaster
```

全部配置完成之后，DockerAPI和FRP API不可连接的报错理应是消失的，如下图

![image-20240818112936538](https://download.imxbt.cn/upload/20240818112936793d60cd6724439a2006457c9697777d289.png)

> 此配置默认使用直连方式暴露docker题目，所以在添加题目的时候动态容器的Frp Redirect Type应该指定为 Direct

### 无法启动容器

无法启动容器的报错：`The network ctfd_frp-containers cannot be used with services. Only networks scoped to the swarm can be used, such as those created with the overlay driver.`

请确保你在docker-compose中正确设置frp-containers这个network网络名称，如果不行请执行如下命令重建swarm网络

```shell
docker network rm ctfd_frp-containers
docker network create --driver overlay --attachable ctfd_frp-containers
```

## 成品配置

### docker-compose.yml

```yaml
version: '3'

services:
  frps:
    image: snowdreamtech/frps:0.30.0
    restart: always
    volumes:
       - ./conf/frp/frps.ini:/etc/frp/frps.ini
    entrypoint:
       - /usr/bin/frps
       - '-c'
       - /etc/frp/frps.ini
    ports:
       - '34000-35000:34000-35000' #题目端口范围
    networks:
       frp_admin:
       default:

frpc:
    image: snowdreamtech/frpc:0.30.0
    restart: always
    ports:
       - 7400:7400
    volumes:
        - ./conf/frp/frpc.ini:/etc/frp/frpc.ini #配置文件在CTFd/conf/frp/frpc.ini中
    entrypoint:
        - /usr/bin/frpc
        - '-c'
        - /etc/frp/frpc.ini
    depends_on:
        - frps
    networks:
        frp_admin:
        frp_containers:

ctfd:
    build: .
    user: root
    restart: always
    ports:
      - "8000:8000"
    environment:
      - UPLOAD_FOLDER=/var/uploads
      - DATABASE_URL=mysql+pymysql://ctfd:ctfd@db/ctfd
      - REDIS_URL=redis://cache:6379
      - WORKERS=16
      - LOG_FOLDER=/var/log/CTFd
      - ACCESS_LOG=-
      - ERROR_LOG=-
      - REVERSE_PROXY=true
    volumes:
      - .data/CTFd/logs:/var/log/CTFd
      - .data/CTFd/uploads:/var/uploads
      - .:/opt/CTFd:ro
    depends_on:
      - db
      - frpc
    networks:
        frp_admin:
        default:
        internal:
        frp_containers:

db:
    image: mariadb:10.11
    restart: always
    environment:
      - MARIADB_ROOT_PASSWORD=ctfd
      - MARIADB_USER=ctfd
      - MARIADB_PASSWORD=ctfd
      - MARIADB_DATABASE=ctfd
      - MARIADB_AUTO_UPGRADE=1
    volumes:
      - .data/mysql:/var/lib/mysql
    networks:
        internal:
    # This command is required to set important mariadb defaults
    command: [mysqld, --character-set-server=utf8mb4, --collation-server=utf8mb4_unicode_ci, --wait_timeout=28800, --log-warnings=0]

cache:
    image: redis:4
    restart: always
    volumes:
    - .data/redis:/data
    networks:
        internal:

networks:
    default:
    internal:
        internal: true
    frp_admin:
        internal: true
    frp_containers:
        driver: overlay
        internal: true
        attachable: true
```

### Dockerfile

```dockerfile
FROM python:3.11-slim-bookworm AS build

WORKDIR /opt/CTFd
RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources #换apt源

# hadolint ignore=DL3008
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        build-essential \
        libffi-dev \
        libssl-dev \
        git \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* \
    && python -m venv /opt/venv

ENV PATH="/opt/venv/bin:$PATH"

COPY . /opt/CTFd

RUN pip install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
    && for d in CTFd/plugins/*; do \
        if [ -f "$d/requirements.txt" ]; then \
            pip install --no-cache-dir -r "$d/requirements.txt" -i https://pypi.tuna.tsinghua.edu.cn/simple ; \
        fi; \
    done;

FROM python:3.11-slim-bookworm AS release
WORKDIR /opt/CTFd

# hadolint ignore=DL3008
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        libffi8 \
        libssl3 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

COPY --chown=1001:1001 . /opt/CTFd

RUN useradd \
    --no-log-init \
    --shell /bin/bash \
    -u 1001 \
    ctfd \
    && mkdir -p /var/log/CTFd /var/uploads \
    && chown -R 1001:1001 /var/log/CTFd /var/uploads /opt/CTFd \
    && chmod +x /opt/CTFd/docker-entrypoint.sh

COPY --chown=1001:1001 --from=build /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

USER 1001
EXPOSE 8000
ENTRYPOINT ["/opt/CTFd/docker-entrypoint.sh"]
```

### frpc.ini

```ini
[common]
server_addr = frps
server_port = 7897
token = token
admin_addr = 0.0.0.0
admin_port = 7400
admin_user = user
admin_pwd = pwd
```

### frps.ini

```ini
[common]
bind_addr = 0.0.0.0
bind_port = 7897
token = token
```

咸鱼社

CTFd集群搭建whale

发表新评论