Flutter StaggeredGridView rearrange crossAxisCount and position of tiles in Grid

I want to use the StaggeredGridView to create a dynamic Grid, were the items don't have a static size but take the size of it's content. To accomplish dynamic sizing I use the StaggeredTile.fit(1) and that also works.
But I want to give the ability to rearrange the number of items in one row. For that I set the crossAxisCount to a number of 1-3 and that should either place 1, 2, or 3 items in a row. But as you can see in the images below the items don't get rearranged to fit them next to each other but only the size of the tiles changes.

I use a CupertinoSlidingSegmentedControl to change the value between 1 to 3 and the variable that holds this value is called segmentedControlGroupValue. Below you can find the code fot the SliverStaggeredGrid and the CupertinoSlidingSegmentedControl:

SliverStaggeredGrid.countBuilder(
  mainAxisSpacing: kSmallPadding,
  crossAxisCount: segmentedControlGroupValue ?? 1,
  itemCount: 6,
  staggeredTileBuilder: (int index) => StaggeredTile.fit(1),
  itemBuilder: (BuildContext context, int i) => Card(
    child: Container(
      color: Colors.green,
      padding: EdgeInsets.all(kMediumPadding),
      child: Text(
        i.toString(),
      ),
    ),
  ),
),

CupertinoSlidingSegmentedControl(
  groupValue: segmentedControlGroupValue,
  padding: const EdgeInsets.all(6),
  children: const {
    1: Icon(Icons.menu),
    2: Icon(
      QuaderFont.th_large_1,
      size: 16,
    ),
    3: Icon(Icons.apps)
  },
  onValueChanged: (value) => setState(() {
    segmentedControlGroupValue = value as int?;
  }),
),

When I set crossAxisCount to 2 and restart the app the grid gets arranged with two items in one row but it is not possible to programmatically change this value while running the app. Or did I just implement it wrong?