How to prevent SliverAppBar from fading out background image?

I have the following SliverAppBar inside a CustomScrollView:

  SliverAppBar(
    stretch: true,
    floating: false,
    pinned: true,
    expandedHeight: 300,
    flexibleSpace: FlexibleSpaceBar(
      stretchModes: [StretchMode.zoomBackground],
      background: (
        Image.asset("assets/$bgimg", fit: BoxFit.cover)
      ),
    ),
  ),

It works as expected, however, as the users scrolls back up over the SliverAppBar, the background image (placed inside a FlexibleSpaceBar) fades away. The actual fading apparently happens inside the actual FlexibleSpaceBar Widget, as peeking into the source files gives this :

final double opacity = 1.0 - Interval(fadeStart, fadeEnd).transform(t);

Video: https://imgur.com/gallery/3TWJ0lO

I don't want the SliverAppBar to turn blue when the user scrolls back up. How do you prevent this behaviour and makes the background stay persistently at all times? Am I looking at the wrong widget?

I have tried setting the color property of SliverAppBar to transparent, but it still fades out.