We are trying to find a universal data model, which can be applied everywhere.
I think there is no doubt that protobuf is good in the microservice architecture as a protocol among processes of different platforms (Windows, Linux, etc.) and languages (Java, C#, C++, Python, etc.)
But how about using protobuf within the process?
To make it simple and easy to illustrate the problems, let say I am making a C# gRPC service and GUI (gRPC client).
After some study, it seems that it is not suitable from what I can see.
- Stability: It looks like that protobuf is still in a changing phase. For example, removing Optional keyword in proto3, but adding it back again in proto release 3.15.
- Data type: The data types in proto are not fully compatible with common data types. For example, decimal. We need to define another complex data type (What's the best way to represent System.Decimal in Protocol Buffers?) and doing the conversion.
- Conversion: We need conversion before we can utilize it in the related language. You cannot add the self-defined proto decimal directly in c#.
- Repeated conversion: Conversion is not one off, but back and forth. Let say we have this proto object passing through 5 functions and need to have some calculations on the decimal field at each function. That means, we will need to convert the decimal field in the proto object to C# decimal in each function, have the calculation to get the result, convert and assign the result back to the decimal field in the proto object, then pass the proto object to the next function.
- GUI control binding: We cannot bind the proto field (for those without an matched type in C#). Even we can specify the conversion to do so in the control somehow, (Indeed, I am not sure if we can/it is good to do so) in simple control, like textbox. It may not be easy for complicated control like datagridview because there may be different built-in editors for different native data types. If we use proto data types, that means we need to write customized editors for them. And also, we may need to define other behaviors, like how to sort them.
Using the auto-generated proto class within the process seems not a good idea to me as the reasons listed above. The above case only focus on C# service and client. When it comes to different languages, the cases should be more complicated.
Although .net soap services are slow and wordy (if you look at the wsdl definition), one thing I appreciate very much is that both the service and the client are using the same object with native data types, which can be used directly without any issue. The conversion is done in the communication directly and automatically.
The way I can think of at the moment is that:
- Using proto as a communication protocol only
- Write a class (use native/built-in data types) for each proto message type and our own conversion logic. Doing so because I cannot find any framework/lib to do so.
- After receiving the proto object, convert it directly to an object with native/built-in types before further processing. (the benefit here is that even there is a major change in proto spec, we only need to change the conversion logic only without affecting other layers).
Am I on the right track? What is common/best practice to resolve the problems listed above?
Any help is highly appreciated!!!