The final requirement is to create a system which can stream video footage via a UNIX IPC socket from Python to Rust. The Python script has exclusive access to the camera/video. I'm pretty new to Rust.
I have tried an approach wherein the control flows as follows:
- In Python, as the video flows in, strip each frame out of it to convert it into a NumPy array
- Then, convert it into a string. Then to bytes.
- Send it over a UNIX IPC socket.
- At the receiving end, convert it back into string.
- Hopefully parse it to get a usable array and make an image from it.
Sender:
import socket, os, cv2, numpy
# print all the string without truncating
numpy.set_printoptions(threshold=numpy.inf)
# declare the camera socket and unlink path if already in use
camera_socket_path = '/home/user/exps/test.sock'
try:
os.unlink(camera_socket_path)
except OSError:
pass
# create and listen on the socket
camera_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
camera_socket.bind(camera_socket_path)
camera_socket.listen()
conn, addr = camera_socket.accept()
vidcap = cv2.VideoCapture('example.mp4')
success, image = vidcap.read()
print(str(image))
while success:
conn.send(bytes(str(image), 'utf-8'))
success, image = vidcap.read()
break # after sending a frame, for testing
conn.close()
Receiver:
use std::os::unix::net::UnixStream;
use std::io::prelude::*;
use image::RgbImage;
use ndarray::Array3;
fn array_to_image(arr: Array3<u8>) -> RgbImage {
assert!(arr.is_standard_layout());
let (height, width, _) = arr.dim();
let raw = arr.into_raw_vec();
RgbImage::from_raw(width as u32, height as u32, raw)
.expect("container should have the right size for the image dimensions")
}
fn main() {
// create a standard UNIX IPX socket stream
let mut stream = UnixStream::connect("/home/user/exps/test.sock").unwrap();
loop {
// 2074139 is the length of the string printed at sender :D
// I thought it might work out but it didn't obviously
// the docs suggest an exponential of 2. default was 1024
let mut buf = [0; 2074139];
let count = stream.read(&mut buf).unwrap();
let response = String::from_utf8(buf[..2074139].to_vec()).unwrap();
// write a parser function to convert the string to Array3 iterating through line()
// let pic = array_to_image(convert(response));
println!("{}", response);
break; // get a single frame, for testing
}
}
Both the strings look nothing like each other. In the final version, I hope to create a stream using which I can continuously do this for incoming video stream from a camera, with something like cv2.VideoCapture(). What is the list of stuff to be done to achieve it?