SfTian

CTFd集群搭建whale

CTFd搭建教程

本次安装使用Docker Swarm集群，针对中大型比赛建议使用本教程，从机只需要docker join加入swarm集群即可

安装docker

curl -fsSL https://download.imxbt.cn/getdocker -o get-docker.sh
DOWNLOAD_URL=https://mirrors.ustc.edu.cn/docker-ce sh get-docker.sh

添加镜像加速源

echo '{"registry-mirrors": ["加速源"]}' | sudo tee /etc/docker/daemon.json

安装CTFd和whale

# Git拉取CTFd
mkdir -p /opt/
cd /opt
git clone https://mirror.ghproxy.com/https://github.com/CTFd/CTFd.git
# 拉取whale
cd CTFd/CTFd/plugins
git clone https://mirror.ghproxy.com/https://github.com/frankli0324/ctfd-whale.git ctfd-whale
cd ctfd-whale

配置docker-compose

这部分根据自己实际情况配置

cd /opt/CTFd
nano docker-compose.yml

修改Dockerfile

cd /opt/CTFd
nano Dockerfile

在 WORKDIR /opt/CTFd 下一行新增换源内容，以加快拉取速度

RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources

在 pip install 处后面添加(2处)

-i https://pypi.tuna.tsinghua.edu.cn/simple

编译并运行CTFd

docker compose build
docker compose up -d
docker network connect frpcadmin <CTFd容器id>

全部拉取后提示started即可通过8000端口访问CTFd,也可以自行配置NGINX实现80/443端口访问

初始化

首先在CTFd的机器上初始化一个Docker Swarm

docker swarm init
docker node update --label-add='name=ctfd_frp_containers' $(docker node ls -q)
docker node update --label-add "name=linux-1" $(docker node ls -q)

访问CTFd web页面之后一切设置完毕后到管理面板进行设置，找到whale

image-20240817150832068

出现如图所示即是whale插件安装成功

（单机）编辑docker-compose

cd /opt/CTFd
nano docker-compose.yml

在services.ctfd.volumes中加入

- /var/run/docker.sock:/var/run/docker.sock

使其可以映射到docker的API

（集群）使Docker Swarm可以被连接

此方案的缺点是如果远程访问没有TLS的话会有安全风险，建议内网访问不要开放到公网

systemctl edit docker.service

在如下行

ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

追加内容

 tcp://0.0.0.0:2375

重启应用即可

systemctl daemon-reload && systemctl restart docker
#放行ufw端口
ufw allow 2375

编辑frp

在 /opt/CTFd/conf/frp/ 中新建一个 frps.ini和frpc.ini 并填入如下内容

frps.ini

[common]
bind_addr = 0.0.0.0 #这里做演示使用的IP，为了安全请及时更换
bind_port = 7897
token = ctfdmaster

frpc.ini

[common]
server_addr = frps $这里代指frps的docker IP
server_port = 7897
token = ctfdmaster
admin_addr = 0.0.0.0 #这里做演示使用的IP，为了安全请及时更换
admin_port = 7400
admin_user = user
admin_pwd = pwd

编辑docker-compose

在CTFd的docker-compose.yml中找到networks追加

    frp_admin:
        internal: true
    frp_containers:
        driver: overlay
        internal: true
        attachable: true

在services中追加如下

  frps:
    image: snowdreamtech/frps:0.30.0
    restart: always
    volumes:
       - ./conf/frp/frps.ini:/etc/frp/frps.ini
    entrypoint:
       - /usr/bin/frps
       - '-c'
       - /etc/frp/frps.ini
    ports:
       - '34000-35000:34000-35000' #题目端口范围
    networks:
       frp_admin:
       default:

  frpc:
    image: snowdreamtech/frpc:0.30.0
    restart: always
    ports:
       - 7400:7400
    volumes:
        - ./conf/frp/frpc.ini:/etc/frp/frpc.ini #配置文件在CTFd/conf/frp/frpc.ini中
    entrypoint:
        - /usr/bin/frpc
        - '-c'
        - /etc/frp/frpc.ini    
    depends_on:
        - frps
    networks:
        frp_admin:
        frp_containers:

并在services.ctfd中修改如下

services:
    ...
    ctfd:
        ...
        depends_on:
            ...
            - frpc
        networks:
            ...
            frp_admin:

重启docker compose容器即可

配置API

回到whale设置页面，在Docker页面的API URL中设置

tcp://172.17.0.1:2375

即可连接到Docker API

在frp页面的API URL中填写如下即可连接到frpc，如果没有配置admin账密直接填写 http://frpc:7400

http://user:pwd@frpc:7400

Direct IP Address填写服务器IP或者域名即可，direct minimum port和direct maximum port是转发用的端口范围，要和docker-compose中的frps配置的端口范围相匹配

在Frpc config template中填写，即可完成连接Frp

[common]
server_addr = frps
server_port = 7897
token = ctfdmaster

全部配置完成之后，DockerAPI和FRP API不可连接的报错理应是消失的，如下图

image-20240818112936538

此配置默认使用直连方式暴露docker题目，所以在添加题目的时候动态容器的Frp Redirect Type应该指定为 Direct

无法启动容器

无法启动容器的报错：The network ctfd_frp-containers cannot be used with services. Only networks scoped to the swarm can be used, such as those created with the overlay driver.

请确保你在docker-compose中正确设置frp-containers这个network网络名称，如果不行请执行如下命令重建swarm网络

docker network rm ctfd_frp-containers
docker network create --driver overlay --attachable ctfd_frp-containers

成品配置

docker-compose.yml

version: '3'

services:
  frps:
    image: snowdreamtech/frps:0.30.0
    restart: always
    volumes:
       - ./conf/frp/frps.ini:/etc/frp/frps.ini
    entrypoint:
       - /usr/bin/frps
       - '-c'
       - /etc/frp/frps.ini
    ports:
       - '34000-35000:34000-35000' #题目端口范围
    networks:
       frp_admin:
       default:

  frpc:
    image: snowdreamtech/frpc:0.30.0
    restart: always
    ports:
       - 7400:7400
    volumes:
        - ./conf/frp/frpc.ini:/etc/frp/frpc.ini #配置文件在CTFd/conf/frp/frpc.ini中
    entrypoint:
        - /usr/bin/frpc
        - '-c'
        - /etc/frp/frpc.ini
    depends_on:
        - frps
    networks:
        frp_admin:
        frp_containers:

  ctfd:
    build: .
    user: root
    restart: always
    ports:
      - "8000:8000"
    environment:
      - UPLOAD_FOLDER=/var/uploads
      - DATABASE_URL=mysql+pymysql://ctfd:ctfd@db/ctfd
      - REDIS_URL=redis://cache:6379
      - WORKERS=16
      - LOG_FOLDER=/var/log/CTFd
      - ACCESS_LOG=-
      - ERROR_LOG=-
      - REVERSE_PROXY=true
    volumes:
      - .data/CTFd/logs:/var/log/CTFd
      - .data/CTFd/uploads:/var/uploads
      - .:/opt/CTFd:ro
    depends_on:
      - db
      - frpc
    networks:
        frp_admin:
        default:
        internal:
        frp_containers:

  db:
    image: mariadb:10.11
    restart: always
    environment:
      - MARIADB_ROOT_PASSWORD=ctfd
      - MARIADB_USER=ctfd
      - MARIADB_PASSWORD=ctfd
      - MARIADB_DATABASE=ctfd
      - MARIADB_AUTO_UPGRADE=1
    volumes:
      - .data/mysql:/var/lib/mysql
    networks:
        internal:
    # This command is required to set important mariadb defaults
    command: [mysqld, --character-set-server=utf8mb4, --collation-server=utf8mb4_unicode_ci, --wait_timeout=28800, --log-warnings=0]

  cache:
    image: redis:4
    restart: always
    volumes:
    - .data/redis:/data
    networks:
        internal:

networks:
    default:
    internal:
        internal: true
    frp_admin:
        internal: true
    frp_containers:
        driver: overlay
        internal: true
        attachable: true

Dockerfile

FROM python:3.11-slim-bookworm AS build

WORKDIR /opt/CTFd
RUN sed -i 's/deb.debian.org/mirrors.ustc.edu.cn/g' /etc/apt/sources.list.d/debian.sources #换apt源 

# hadolint ignore=DL3008
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        build-essential \
        libffi-dev \
        libssl-dev \
        git \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/* \
    && python -m venv /opt/venv

ENV PATH="/opt/venv/bin:$PATH"

COPY . /opt/CTFd

RUN pip install --no-cache-dir -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple \
    && for d in CTFd/plugins/*; do \
        if [ -f "$d/requirements.txt" ]; then \
            pip install --no-cache-dir -r "$d/requirements.txt" -i https://pypi.tuna.tsinghua.edu.cn/simple ; \
        fi; \
    done;


FROM python:3.11-slim-bookworm AS release
WORKDIR /opt/CTFd

# hadolint ignore=DL3008
RUN apt-get update \
    && apt-get install -y --no-install-recommends \
        libffi8 \
        libssl3 \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

COPY --chown=1001:1001 . /opt/CTFd

RUN useradd \
    --no-log-init \
    --shell /bin/bash \
    -u 1001 \
    ctfd \
    && mkdir -p /var/log/CTFd /var/uploads \
    && chown -R 1001:1001 /var/log/CTFd /var/uploads /opt/CTFd \
    && chmod +x /opt/CTFd/docker-entrypoint.sh

COPY --chown=1001:1001 --from=build /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"

USER 1001
EXPOSE 8000
ENTRYPOINT ["/opt/CTFd/docker-entrypoint.sh"]

frpc.ini

[common]
server_addr = frps
server_port = 7897
token = token
admin_addr = 0.0.0.0
admin_port = 7400
admin_user = user
admin_pwd = pwd

frps.ini

[common]
bind_addr = 0.0.0.0
bind_port = 7897
token = token

发表新评论