Skip to main content

How to extend the dependencies of code blocks

Standalone Mode​

Dependency Library Persistence​

The following operations must be performed while the service is running and need to be done only once during the first setup. Subsequent installation of extension libraries does not require this operation.

  1. Create a directory to mount the dependency libraries (the actual directory can be customized), such as:

    mkdir -p /data/mingdao/script/volume/command/package/python-3.6/
    mkdir -p /data/mingdao/script/volume/command/package/nodejs-10.18.0/

  2. Retrieve pre-installed dependency libraries:

    docker cp $(docker ps | grep command | awk '{print $1}'):/usr/local/lib/python3.6/site-packages/ /data/mingdao/script/volume/command/package/python-3.6/
    docker cp $(docker ps | grep command | awk '{print $1}'):/usr/local/node-10.18.0/lib/node_modules/ /data/mingdao/script/volume/command/package/nodejs-10.18.0/

  3. Modify docker-compose.yaml to mount the dependency library directories from the host to the container:

    services:
      command:
        volumes:
        - ./volume/command/package/python-3.6/site-packages/:/usr/local/lib/python3.6/site-packages/
        - ./volume/command/package/nodejs-10.18.0/node_modules/:/usr/local/node-10.18.0/lib/node_modules/

  4. Execute the following command in the root directory of the manager to restart the service:

    bash ./service.sh restartall

Installing Extension Libraries Online​

Reminder
  • Ensure the "Dependency Library Persistence" step is completed first. The container's dependency library directory must be persistently mounted to the host. Otherwise, extension libraries will need to be reinstalled after the container restarts.
  • Ensure the server has internet access for online installation of extension libraries.

Python​

Using the example of installing the python-dateutil extension library:

  1. Enter the command container:

    docker exec -it $(docker ps | grep command | awk '{print $1}') bash

  2. Install the python-dateutil extension library inside the command container:

    pip3 install --target=/usr/local/lib/python3.6/site-packages/ python-dateutil

After installation, the newly installed extension library can be found in the default host path /data/mingdao/script/volume/command/package/python-3.6/site-packages/.

JavaScript (based on Nodejs)​

Using the example of installing the dayjs extension library:

  1. Enter the command container:

    docker exec -it $(docker ps | grep command | awk '{print $1}') bash

  2. Install the dayjs extension library inside the container:

    /usr/local/node-10.18.0/bin/npm -g install dayjs

After installation, the newly installed extension library can be found in the default host path /data/mingdao/script/volume/command/package/nodejs-10.18.0/node_modules/.

Installing Extension Libraries Offline​

Reminder
  • Ensure the "Dependency Library Persistence" step is completed first. The container's dependency library directory must be persistently mounted to the host. Otherwise, extension libraries will need to be reinstalled after the container restarts.
  • This method can be used if the server does not have internet access or if the extension libraries are provided as offline package files.

Python​

Using the example of installing the pycryptodome extension library:

  1. In an environment with internet access, retrieve the offline package file for the extension library:

    pip3 download pycryptodome -d "/package"
    • The offline package file for the pycryptodome dependency library will be generated in the /package directory.

  2. Upload the offline package file to the server hosting the HAP service.

  3. Copy the offline package file into the container and install it:

    1. Copy the pycryptodome dependency library offline file to the /tmp directory inside the command container:

      docker cp pycryptodome-3.15.0-cp35-abi3-manylinux1_x86_64.whl $(docker ps | grep command | awk '{print $1}'):/tmp

    2. Enter the command container:

      docker exec -it $(docker ps | grep command | awk '{print $1}') bash

    3. Install the pycryptodome extension library offline package inside the container:

      pip3 install --target=/usr/local/lib/python3.6/site-packages/ /tmp/pycryptodome-3.15.0-cp35-abi3-manylinux1_x86_64.whl

JavaScript (based on Nodejs)​

Using the example of installing the dayjs extension library:

  1. In an environment with internet access, retrieve the offline package file for the extension library:

    npm install dayjs -save
    • A node_modules/dayjs dependency library directory will be generated in the current directory.

  2. Package the dayjs extension library directory:

    cd node_modules
    tar czf dayjs.tar.gz dayjs

  3. Upload the packaged file dayjs.tar.gz to the server hosting the HAP service.

  4. Extract the offline package file to the persistently mounted path for dependency libraries:

    tar xf dayjs.tar.gz -C /data/mingdao/script/volume/command/package/nodejs-10.18.0/node_modules
    • If the default data path for HAP was modified, adjust the extraction path according to the actual environment setup.

Cluster Mode​

Dependency Library Persistence​

  1. Create the dependency library mount directory on the first microservice server:

    mkdir -p /data/mingdao/script/volume/command/package/python-3.6/site-packages
    mkdir -p /data/mingdao/script/volume/command/package/nodejs-10.18.0/node_modules

  2. Retrieve the prebuilt Python dependency libraries to the local directory:

    kubectl cp $(kubectl get pod|grep command-|awk 'NR==1{print $1}'):/usr/local/lib/python3.6/site-packages /data/mingdao/script/volume/command/package/python-3.6/site-packages

  3. Retrieve the prebuilt JavaScript dependency libraries to the local directory:

    kubectl cp $(kubectl get pod|grep command-|awk 'NR==1{print $1}'):/usr/local/node-10.18.0/lib/node_modules /data/mingdao/script/volume/command/package/nodejs-10.18.0/node_modules

  4. Package and archive the /data/mingdao/script/volume/command/ directory on the current server, then synchronize and distribute it to all other microservice nodes in the cluster.

  5. On each microservice node, extract the archived file. Ensure that the following paths are established and the internal file contents are identical on every node:

    /data/mingdao/script/volume/command/package/python-3.6/site-packages
    /data/mingdao/script/volume/command/package/nodejs-10.18.0/node_modules

Configure Dependency Library Mounts​

  1. Edit the service.yaml configuration file for the microservice (typically located in the /data/mingdao/script/kubernetes/ path of the first microservice node).

    Add mount configurations for the command service under its deployment:

            volumeMounts:
            - mountPath: /usr/local/lib/python3.6/site-packages
              name: python-volume
            - mountPath: /usr/local/node-10.18.0/lib/node_modules
              name: node-volume
          volumes:
          - name: python-volume
            hostPath:
              path: /data/mingdao/script/volume/command/package/python-3.6/site-packages
              type: Directory
          - name: node-volume
            hostPath:
              path: /data/mingdao/script/volume/command/package/nodejs-10.18.0/node_modules
              type: Directory
    1. Modify the instance count for the command service:
      • Adjust the replicas value to match or exceed the number of current microservice nodes.
      • For example, if there are 3 microservice nodes, set the replica count to 3 or above.

  2. Apply the updated configuration file to activate the mounts:

    kubectl apply -f service.yaml

  3. Check the status of the command service:

    kubectl get pod | grep command
    • After the update, the command service will perform a rolling restart. Wait 1-2 minutes until all command instances restart and their status displays as 2/2, indicating normal operation.

Online Installation of Extension Libraries​

Reminder
  • Ensure the "Dependency Library Persistence" step is completed to persist the container dependency library directories onto the host machine; otherwise, extension libraries will need to be reinstalled after container restarts.
  • Online installation of extension libraries requires the server to have internet access.

Python​

In a cluster environment, since the command service uses multi-instance (Pod) deployment, you must ensure that the extension library is correctly installed in each Pod instance. The following example script runs in the default namespace by default, executing installation commands across all relevant Pods:

Example: Installing the python-dateutil extension library

for pod in $(kubectl get pods -l app=command -n default -o jsonpath='{.items[*].metadata.name}'); do
    echo "Processing Pod: $pod"
    kubectl exec -it $pod -n default -- pip3 install -i https://mirrors.aliyun.com/pypi/simple/ --target=/usr/local/lib/python3.6/site-packages/ python-dateutil
done

JavaScript (Nodejs-based)​

In a cluster environment, since the command service uses multi-instance (Pod) deployment, you must ensure that the extension library is correctly installed in each Pod instance. The following example script runs in the default namespace by default, executing installation commands across all relevant Pods:

Example: Installing the dayjs extension library

for pod in $(kubectl get pods -l app=command -n default -o jsonpath='{.items[*].metadata.name}'); do
    echo "Processing Pod: $pod"
    kubectl exec -it $pod -n default -- /usr/local/node-10.18.0/bin/npm -g install dayjs --registry=https://registry.npmmirror.com
done

Offline Installation of Extension Libraries​

Reminder
  • Ensure the "Dependency Library Persistence" step is completed to persist the container dependency library directories onto the host machine; otherwise, extension libraries will need to be reinstalled after container restarts.
  • Use offline installation methods when the server cannot access the internet or if the provided extension libraries come as offline package files.

Python​

Example: Installing the pycryptodome extension library

  1. Retrieve the offline file for the extension library in an environment with internet access:

    pip3 download pycryptodome -d "/package" -i https://mirrors.aliyun.com/pypi/simple/
    • The /package directory will contain the offline files for the pycryptodome dependency library.

  2. Upload the offline library file (e.g., pycryptodome-3.15.0-cp35-abi3-manylinux1_x86_64.whl) to a temporary directory on the first microservice node.

  3. Execute the following script on the first microservice node.

    Since the command service in the cluster uses multi-instance (Pod) deployment, you must ensure the offline library is distributed and installed across all instances. The following example script runs in the default namespace by default, distributing and installing the offline library file across all relevant Pods:

    # Update the FILE_NAME variable value based on the actual offline file name
    FILE_NAME="pycryptodome-3.15.0-cp35-abi3-manylinux1_x86_64.whl"    
    for pod in $(kubectl get pods -l app=command -n default -o jsonpath='{.items[*].metadata.name}'); do
       echo "Processing Pod: $pod"
       kubectl cp ./$FILE_NAME $pod:/tmp/ -n default
       kubectl exec -it $pod -n default -- pip3 install --target=/usr/local/lib/python3.6/site-packages/ /tmp/$FILE_NAME
    done

JavaScript (Nodejs-based)​

Example: Installing the dayjs extension library

  1. Retrieve the offline file for the extension library in an environment with internet access:

    npm install dayjs -save  --registry=https://registry.npmmirror.com
    • The node_modules/dayjs directory will be generated containing the dayjs dependency library.

  2. Package the dayjs extension library directory:

    cd node_modules
    tar czf dayjs.tar.gz dayjs

  3. Upload the packaged file dayjs.tar.gz to a temporary directory on the first microservice node.

  4. Execute the following script on the first microservice node.

    Since the command service in the cluster uses multi-instance (Pod) deployment, you must ensure the extension library is correctly distributed and installed across all instances. The following example script runs in the default namespace by default, distributing and extracting the library across all relevant Pods:

    # Update the NODE_FILE variable value based on the actual offline package file name
    NODE_FILE="dayjs.tar.gz"
    for pod in $(kubectl get pods -l app=command -n default -o jsonpath='{.items[*].metadata.name}'); do
       echo "Processing Pod: $pod"
       kubectl cp ./$NODE_FILE $pod:/tmp/ -n default
       kubectl exec -it $pod -n default -- tar xf /tmp/$NODE_FILE -C /usr/local/node-10.18.0/lib/node_modules
    done