Skip to main content

Deployment Example (AMD)

This document uses CentOS/RedHat as the deployment example. Kylin V10 and Huawei EulerOS can also follow this guide. Differences for Debian-based systems are noted separately.

1. Environment Preparation​

  1. Prepare a Linux server with the minimum configuration requirements of 4-core CPU, 16 GB memory, and 40 GB data disk (excluding the system disk). See more recommendations, and ensure that ports 8880, 38880, and 38881 are not occupied.

  2. Log in to the server as the root user.

  3. If the server has a data disk, mount the data disk to /data before deployment, or modify the default storage path during installation to the path where your data disk is mounted.

  4. Disable Firewalld (Debian / Ubuntu do not install this service by default, so you can skip this step).

    systemctl stop firewalld && systemctl disable firewalld

  5. Disable SELinux (Debian / Ubuntu do not enable this feature by default, so you can skip this step).

    setenforce 0
    sed -i s/"^SELINUX=.*$"/"SELINUX=disabled"/g /etc/selinux/config

  6. Adjust MaxMapCount.

    sysctl -w vm.max_map_count=262144
    grep -q '^vm.max_map_count' /etc/sysctl.d/99-sysctl.conf && sed -i 's/^vm.max_map_count.*/vm.max_map_count=262144/' /etc/sysctl.d/99-sysctl.conf || echo 'vm.max_map_count=262144' >> /etc/sysctl.d/99-sysctl.conf

  7. Adjust SysFileNr.

    sysctl -w fs.file-max=2048000
    grep -q '^fs.file-max' /etc/sysctl.d/99-sysctl.conf && sed -i 's/^fs.file-max.*/fs.file-max=2048000/' /etc/sysctl.d/99-sysctl.conf || echo 'fs.file-max=2048000' >> /etc/sysctl.d/99-sysctl.conf

  8. Adjust IPv4Forward.

    sysctl -w net.ipv4.ip_forward=1
    grep -q '^net.ipv4.ip_forward' /etc/sysctl.d/99-sysctl.conf && sed -i 's/^net.ipv4.ip_forward.*/net.ipv4.ip_forward=1/' /etc/sysctl.d/99-sysctl.conf || echo 'net.ipv4.ip_forward=1' >> /etc/sysctl.d/99-sysctl.conf

  9. If the operating system is Debian 10 or later, first run iptables --version to check whether iptables is available.

    If iptables is not installed, install it with the following reference commands (the server must be able to access the Internet):

    apt-get update
    apt install -y iptables

2. Docker Installation​

  1. Download the Docker package.

    wget https://pdpublic.mingdao.com/private-deployment/offline/common/docker-28.5.2.tgz

  2. Extract the package and move the files to the binary directory.

    tar -zxvf docker-28.5.2.tgz
    mv -f docker/* /usr/local/bin/

  3. Create the Docker configuration file.

    The default Docker data directory is /data/docker. To change the default data directory, modify the data-root value in the configuration file.

    mkdir -p  /etc/docker/
    cat > /etc/docker/daemon.json <<EOF
    {
      "registry-mirrors": ["https://uvlkeb6d.mirror.aliyuncs.com"],
      "data-root": "/data/docker",
      "max-concurrent-downloads": 10,
      "exec-opts": ["native.cgroupdriver=cgroupfs"],
      "storage-driver": "overlay2",
      "default-address-pools":[{"base":"172.80.0.0/16","size":24}]
    }
    EOF

  4. Configure systemd to manage Docker.

    cat > /etc/systemd/system/docker.service <<EOF
    [Unit]
    Description=Docker
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=notify
    ExecStart=/usr/local/bin/dockerd
    ExecReload=/bin/kill -s HUP \$MAINPID
    LimitNOFILE=102400
    LimitNPROC=infinity
    LimitCORE=0
    TimeoutStartSec=0
    Delegate=yes
    KillMode=process
    Restart=on-failure
    StartLimitBurst=3
    StartLimitInterval=60s
    [Install]
    WantedBy=multi-user.target
    EOF

  5. Start Docker.

    systemctl daemon-reload && systemctl start docker && systemctl enable docker

3. Standalone Installation​

  1. Download images (Offline Resources).

    docker pull registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-hdp:1.2.0
    docker pull registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-core:7.4.0
    docker pull registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-sc:3.2.0
    docker pull registry.cn-hangzhou.aliyuncs.com/mdpublic/mingdaoyun-flink:1.19.720

  2. Download the manager.

    wget https://pdpublic.mingdao.com/private-deployment/7.4.0/mingdaoyun_private_deployment_captain_linux_amd64.tar.gz

  3. Create a directory and extract the manager to the new directory.

    mkdir /usr/local/MDPrivateDeployment/
    tar -zxvf mingdaoyun_private_deployment_captain_linux_amd64.tar.gz -C /usr/local/MDPrivateDeployment/

  4. Enter the manager directory and start the manager.

    cd /usr/local/MDPrivateDeployment/
    bash ./service.sh start

  5. After startup, access http://{Server IP}:38881 in a browser and set the system access address, then select the HDP product for initialization (the initialization process takes about 5 minutes).

  6. After deployment is complete, it is recommended to configure the service to start on boot. See How to Configure Startup on Boot.