I am trying to make an OBS-like app using Flutter.
I was trying to use Flutter engine to draw widgets onto the video frames, along with screen, with separated layers.
I came up with a bad way, which:
- use
RenderRepaintBoundaryto get images of specific area. - use
ffmpegto compose these.pngseries into video withh.264encoding. - (then maybe use
.mp4files to publish as video stream??)
, which is baaad in real-time performance and efficiency apparently.
(relevant code)
// some_page.dart
int index = 0;
Future<void> onTestPressed() async {
int i = 0;
while (i++ < 600) {
try {
var render = repaintKey.currentContext!.findRenderObject()
as RenderRepaintBoundary;
double dpr = window.devicePixelRatio;
var byteData = await render
.toImage(pixelRatio: dpr)
.then((image) => image.toByteData(format: ImageByteFormat.png));
var tempDir = await getTemporaryDirectory();
var fileName = '${tempDir.path}/frame_${index++}';
var bytes = byteData!.buffer.asUint8List();
var file = File(fileName);
if (!file.existsSync()) {
file.createSync();
}
await file.writeAsBytes(bytes);
// OpenFile.open(fileName);
} catch (e) {
if (kDebugMode) {
print(e);
}
}
}
}
I know that Flutter uses Skia as its graphic engine, and could I use Skia ability (by drawing widgets) somehow so as to produce video frames more directly?
Thank you.