gRPC python sending image + meta data

Admin 1
Viewed 3850

I need to pass an image (numpy array), additional meta-information about the image like height, width, etc. from the python grpc client to python grpc server.

I need to run this method.

import numpy as np 

def predict(img, w, h):
    # some operations
    return img.shape[2], np.mean(img)

I looked into the documentation, but there is no compatible datatype for numpy array in protobuf.

https://developers.google.com/protocol-buffers/docs/proto3

image_procedure.proto

syntax = "proto3";

// input image, width, height
message Image {
    image_type image = 1;
    int32 width = 2;
    int32 height = 3;
}

// output prediction

message Prediction {
    int32 channel = 4;
    float mean = 5;
}

// service
service ImageProcedure {
    rpc ImageMeanWH(Image) returns (Prediction) {}
}

How can I send the image and other related data to the server and get the response?

grpc python
2 Answers

You can always encode your numpy array as base64 string and pass it to the server.

Your .proto file should look something like this:

syntax = "proto3";

// input image, width, height
message B64Image {
    string b64image = 1;
    int32 width = 2;
    int32 height = 3;
}

// output prediction

message Prediction {
    int32 channel = 4;
    float mean = 5;
}

// service
service ImageProcedure {
    rpc ImageMeanWH(B64Image) returns (Prediction) {}
}

Once you compile your .proto file with

python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. image_procedure.proto

there will be two .py files generated for you:

image_procedure_pb2.py

image_procedure_pb2_grpc.py

In the client code, generate a request message by encoding the image with base64.

frame = np.random.randint(0,255, (416,416,3), dtype=np.uint8) # dummy rgb image

data = base64.b64encode(frame)


# create a valid request message
image_req = image_procedure_pb2.B64Image(b64image = data, width = 416, height = 416)

In your procedure, just decode the base64 and run any operation you want.

import numpy as np 
import base64

def predict(b64img, w, h):
    b64decoded = base64.b64decode(b64img)

    imgarr = np.frombuffer(b64decoded, dtype=np.uint8).reshape(w, h, -1)

    return imgarr.shape[2], np.mean(imgarr)

I extended the code into a tutorial, you can check it out here: https://github.com/zabir-nabil/simple-gRPC

user_46220460

Seems to be an old topic but hopefully other people could still benefit from the answer.

In my case I have a C++ GUI application written with Qt to display the image (server). Client side I have Matlab (Mex/C++) and Python.

The protobuf file (Proto3) looks like:

message DataElement {
    string name = 1;
    ElementType type = 2;
    repeated int32 dimension = 3;
    repeated int32 offset = 4;        
    bytes data = 5;
    int32 viewIndex = 6;
}
message EmptyReply {}
service ImageServer {
    rpc setImage(DataElement) returns (EmptyReply) {}
}

My python client looks like:

class SetImageClient:
    def __init__(self):
        self.channel = grpc.insecure_channel('localhost:50061')
        self.stub = ImageProtoBuf_pb2_grpc.ImageServerStub(self.channel)    

    def setImage(self, viewIndex, name, offset, ndarray):
        dataElement = ImageProtoBuf_pb2.DataElement(viewIndex = viewIndex, 
            name = name, offset=offset, dimension=ndarray.shape, data=ndarray.tobytes())
        self.stub.setImage(dataElement)

I don't have python server side code (as I mentioned I use C++ server). but basically it should be just a reverse process. Once you received data (in the format of DataElement), you can convert to numpy.ndarray by:

ndarray1 = np.frombuffer(dataElement.data, dtype='uint8')
ndarray1 = np.reshape(ndarray1, dataElement.dimension)

And you should be good to go.

user_88344640
Related