Context: I have a .net webapi which streams a video frame-by-frame using the following SignalR streaming method:
public async IAsyncEnumerable<byte[]> CameraStream(
string cameraPath,
int delay,
[EnumeratorCancellation] CancellationToken cancellationToken)
{
// omitted opencv video capture code to process ip cam livestream...
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
bytes = SomeMethodToGetCurrentFrameAsJpgByteArray(cameraPath)
yield return bytes;
await Task.Delay(delay, cancellationToken);
}
}
In my blazor client, I create a hubconnection and open the stream as follows:
public void OpenCamera()
{
cancellationTokenSource = new CancellationTokenSource();
var stream = HubConnection?.StreamAsync<byte[]>(
"CameraStream", "", 30, cancellationTokenSource.Token
);
_ = InvokeAsync(async () =>
{
await foreach (var frame in stream)
{
var base64 = Convert.ToBase64String(frame);
await cameraStreamInterop.DrawPixelsAsync($"cameraOutput{CameraId}",base64);
StateHasChanged();
}
});
}
cancellationtokensource is needed to stop the stream. cameraStreamInterop calls the following JavaScript function:
export function drawPixels(cameraId, imageBytes) {
const element = document.getElementById(cameraId) // canvas
const ctx = element.getContext("2d")
var img = new Image()
img.src = "data:image/jpg;base64, " + imageBytes
img.onload = function () {
console.log("Image Onload")
ctx.drawImage(img, 0, 0, element.width, element.height)
};
img.onerror = function (err) {
console.log("Img Onerror:", err)
};
}
this all works great, but after the first frame is loaded in the canvas, it doesn't change even though the js function is still logging "Image Onload" and the streamed frames are different. Calling StateHasChanged() doesn't do much neither.
Question: How am I able to update the canvas every frame?
Bonus: Is SignalR (using MessagePack) and sending the data frame-by-frame a good practice or is there a way to improve this?
Any tips would be great. Thanks in advance!