Flutter desktop custom game loop appears to drop frames

Viewed 667

I'm testing a basic game loop for Flutter Desktop using the code below but it is not entirely smooth. It sort of seems to very briefly pause every half second or so and jump forward. I suspect that some frames are being dropped but there is nothing intensive going on so I can't see why.

The first thing I though of was that perhaps I need to adjust my update() function to allow for the exact microsecond delta since the last frame - but that didn't fix it regardless of whether I used the timestamp provided by the gameLoop() callback or measured it directly from DateTime.now().

I'm running on a fast gaming laptop so hardware isn't the problem. I am starting to think it could be something in the Flutter framework itself. Perhaps it drops a frame because it is garbage collecting? Or perhaps it skips a frame if it doesn't detect it has changed?

By comparison I created a similar loop in javascript which ran perfectly smoothly. The image used in both the Flutter and Javascript tests was a test 60wx72h png in the local directory.

I'd appreciate any pointers as to whether this is something in my code or something in the Flutter framework.

By the way, there is a discussion of this issue on the Flutter group which explores some other approaches to this problem. See flutter-dev.

Flutter/Dart:

import 'dart:typed_data';
import 'dart:ui';
import 'package:flutter/material.dart' hide Image;
import 'package:flutter/services.dart';

Canvas canvas;
Sprite sprite;
PictureRecorder pictureRecorder;
SceneBuilder builder;

main() async {
  WidgetsFlutterBinding.ensureInitialized();
  WidgetsBinding.instance.window.onBeginFrame = gameLoop;
  sprite = Sprite();
  WidgetsBinding.instance.window.scheduleFrame();
}

void gameLoop(Duration now) {
  pictureRecorder = PictureRecorder();
  canvas = Canvas(pictureRecorder, Rect.fromLTWH(
      0.0, 0.0, WidgetsBinding.instance.window.physicalSize.width, WidgetsBinding.instance.window.physicalSize.height));
  sprite.update();
  sprite.render();
  builder = SceneBuilder();
  builder.addPicture(Offset.zero, pictureRecorder.endRecording());
  WidgetsBinding.instance.window.render(builder.build());
  WidgetsBinding.instance.window.scheduleFrame();
}

class Sprite {
  double x=0, y=400, dx=8, dy=0;
  String imageName = 'assets/bobR.png';
  Image image;

  void update(){
    x+= dx;
    y+= dy;
    if((x<0 && dx<0)||(x>1000 && dx>0)) dx=-dx;
    if((y<0 && dy<0)||(y>1000 && dy>0)) dy=-dy;
  }

  void render(){
    if (image!=null) canvas.drawImage(image, Offset(x,y), Paint());
  }

  Sprite(){
    loadAssets();
  }

  void loadAssets(){
    rootBundle.load(imageName).then((byteData){
      Uint8List lst = Uint8List.view(byteData.buffer);
      instantiateImageCodec(lst).then((codec){
        codec.getNextFrame().then((frameInfo){
          image = frameInfo.image;
        });
      });
    });
  }

}

Javascript equivalent:

<!DOCTYPE html>
<html>
<head>
    <script type="application/javascript">
        "use strict";
        let canvas;
        let context;
        let image = new Image();
        let x=0,y=0;
        let dx=4,dy=0;
        let turnX, turnY;
        image.src = "bobR.png";
        window.onload = init;

        function resize(){
            canvas.width  = window.innerWidth;
            canvas.height = window.innerHeight;
            turnX = canvas.width - image.width;
            turnY = canvas.height - image.height;
        }

        function init(){
            canvas = document.getElementById('canvas');
            context = canvas.getContext('2d');
            window.onresize = resize;
            resize();
            window.requestAnimationFrame(gameLoop);
    }

        function gameLoop(timeStamp){
            context.clearRect(0, 0, canvas.width, canvas.height);
            update();
            render();
            window.requestAnimationFrame(gameLoop);
    }

        function update(){
            x += dx;
            y += dy;
            if ((x<0 && dx<0) || (x>turnX && dx>0)) dx = -dx;
            if ((y<0 && dy<0) || (y>turnY && dy>0)) dy = -dy;
        }

        function render(){
            context.drawImage(image, x, y);
        }
        
    </script>
<body>
  <canvas id="canvas">To play this game, use a more up to date browser</canvas>
</body>
</html>

EDIT

I tried using the Flame game engine but still get the same slightly unsmooth motion compared to Javascript. Code is below:

import 'dart:ui';
import 'package:flutter/material.dart';
import 'package:flame/game.dart';
import 'package:flame/flame.dart';
import 'package:flame/position.dart';
import 'package:flame/sprite.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  TestGame game = TestGame();
  runApp(game.widget);
}

class TestGame extends Game {
  Size screenSize;
  Player player;

  TestGame() {
    initialize();
  }

  void initialize() async {
    Flame.images.load('bobR.png');
    player = Player();
    resize(await Flame.util.initialDimensions());
  }

  void render(Canvas canvas) {
    Rect bgRect = Rect.fromLTWH(0, 0, screenSize.width, screenSize.height);
    Paint bgPaint = Paint();
    bgPaint.color = Colors.white;
    canvas.drawRect(bgRect, bgPaint);
    player.render(canvas);
  }

  void update(double t) {
    player.update(t);
  }

  void resize(Size size) {
    screenSize = size;
  }

}

class Player {

  Position p = Position(0,400); double dx = 8, dy = 0;
  Sprite sprite;
  Player() {
    sprite = Sprite('bobR.png');
  }

  void render(Canvas c) {
    sprite.renderPosition(c, p);
  }

  void update(double t) {
    p.x += dx;
    p.y += dy;
    if ((p.x < 0 && dx < 0) || (p.x > 1000 && dx > 0))
      dx = -dx;
    if ((p.y < 0 && dy < 0) || (p.y > 1000 && dy > 0))
      dy = -dy;
  }
}
2 Answers

Allocations in tight loops (such as gameLoop) can create pressure on the garbage collector.

Try moving your allocations out of your game loop by setting up an initialization routine that only runs once and reuse the objects created.

The JavaScript implementation does just that, which is why it runs smoothly.

I recommend you check out Flame https://flame-engine.org/docs/#/ which is a minimal game engine on top of Flutter.

Try drawing using the Canvas.drawRawAtlas method An extended example can be found here I also recorded a video, as I am interested in checking what Flutter is capable of and for which games it is suitable. At 500 sprites, some GPU processes after Flutter Engine start to go beyond 16 ms in one frame.

Related