KubeEdge Sedna运行联合推理实例-en

Following the previous post, KubeEdge, EdgeMesh, and Sedna are installed. Next we run the joint inference example from the official docs.

Official doc: Helmet detection — joint inference

If EdgeMesh is wired correctly, the example should work by following the doc.

Data and models

• Small model on the edge

mkdir -p /data/little-model
cd /data/little-model
wget <https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/helmet-detection-inference/little-model.tar.gz>
tar -zxvf little-model.tar.gz

• Large model on the cloud

mkdir -p /data/big-model
cd /data/big-model
wget <https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/helmet-detection-inference/big-model.tar.gz>
tar -zxvf big-model.tar.gz

• Images
  • Small-model worker: kubeedge/sedna-example-joint-inference-helmet-detection-little:v0.3.0
  • Large-model worker: kubeedge/sedna-example-joint-inference-helmet-detection-big:v0.3.0

git clone <https://github.com/kubeedge/sedna.git>
./examples/build_image.sh joint_inference   # append joint_inference to build only joint-inference images; omit to build federated learning, etc.

If builds are slow, I added the following to joint-inference-helmet-detection-big.Dockerfile and joint-inference-helmet-detection-little.Dockerfile:

RUN sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list
RUN apt-get clean
RUN pip config set global.index-url <http://mirrors.aliyun.com/pypi/simple>
RUN pip config set install.trusted-host mirrors.aliyun.com
RUN pip install --upgrade pip

to point apt and pip at Aliyun mirrors.

Create the joint inference service

(All kubectl steps run on the cloud.)

• Large-model Model on the cloud

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind:  Model
metadata:
  name: helmet-detection-inference-big-model
  namespace: default
spec:
  url: "/data/big-model/yolov3_darknet.pb"
  format: "pb"
EOF

• Small-model Model on the edge

kubectl create -f - <<EOF
apiVersion: sedna.io/v1alpha1
kind: Model
metadata:
  name: helmet-detection-inference-little-model
  namespace: default
spec:
  url: "/data/little-model/yolov3_resnet18.pb"
  format: "pb"
EOF

On the edge, create a directory for inference outputs:

mkdir -p /joint_inference/output

On the cloud, set CLOUD_NODE and EDGE_NODE:

CLOUD_NODE="cloud-node-name"
EDGE_NODE="edge-node-name"

Create the JointInferenceService (I swapped images to a domestic mirror). Example manifest:

kind: JointInferenceService
metadata:
  name: helmet-detection-inference-example
  namespace: default
spec:
  edgeWorker:
    model:
      name: "helmet-detection-inference-little-model"
    hardExampleMining:
      name: "IBT"
      parameters:
        - key: "threshold_img"
          value: "0.9"
        - key: "threshold_box"
          value: "0.9"
    template:
      spec:
        nodeName: $EDGE_NODE
        dnsPolicy: ClusterFirstWithHostNet
        containers:
        - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/kubeedge/sedna-example-joint-inference-helmet-detection-little:v0.3.0
          imagePullPolicy: IfNotPresent
          name:  little-model
          env:  # user defined environments
          - name: input_shape
            value: "416,736"
          - name: "video_url"
            value: "rtsp://localhost/video"
          - name: "all_examples_inference_output"
            value: "/data/output"
          - name: "hard_example_cloud_inference_output"
            value: "/data/hard_example_cloud_inference_output"
          - name: "hard_example_edge_inference_output"
            value: "/data/hard_example_edge_inference_output"
          resources:  # user defined resources
            requests:
              memory: 64M
              cpu: 100m
            limits:
              memory: 2Gi
          volumeMounts:
            - name: outputdir
              mountPath: /data/
        volumes:   # user defined volumes
          - name: outputdir
            hostPath:
              # user must create the directory in host
              path: /joint_inference/output
              type: Directory
  cloudWorker:
    model:
      name: "helmet-detection-inference-big-model"
    template:
      spec:
        nodeName: $CLOUD_NODE
        dnsPolicy: ClusterFirstWithHostNet
        containers:
          - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/kubeedge/sedna-example-joint-inference-helmet-detection-big:v0.3.0
            name:  big-model
            imagePullPolicy: IfNotPresent
            env:  # user defined environments
              - name: "input_shape"
                value: "544,544"
            resources:  # user defined resources
              requests:
                memory: 2Gi
EOF

Simulate an RTSP stream on the edge

1. Install the open-source streaming server EasyDarwin.
2. Start EasyDarwin.
3. Download the sample video.
4. Push a stream to a URL the inference service can reach (e.g. rtsp://localhost/video).

(The doc link for EasyDarwin-linux-8.1.0-1901141151.tar.gz was dead; I found and downloaded it elsewhere.)

cd EasyDarwin-linux-8.1.0-1901141151
./start.sh

mkdir -p /data/video
cd /data/video
wget <https://kubeedge.obs.cn-north-1.myhuaweicloud.com/examples/helmet-detection-inference/video.tar.gz>
tar -zxvf video.tar.gz

ffmpeg -re -i /data/video/video.mp4 -vcodec libx264 -f rtsp rtsp://localhost/video

When everything is healthy, pods stay Running, and you can inspect results under the output path from the JointInferenceService spec (e.g. /joint_inference/output).