ASP.net razor pages and multi-select, posting all items in select

I have a form which has two selects, one on the left, one on the right. One on the left to list the items to add, two buttons, one to add and one to remove items to another select using a bit of jQuery. Both selects have the multiple="multiple" attributes set.

The problem is, when I submit the form, only items selected in the right hand select are posted in the form data, which is expected. I have added the following bit of jQuery to select all of the items in the select list on submit, but it still only sends the data of the single selected item before the submit button is pressed:

$("#pageForm").submit(function (e) {
     $("#controlGradesList option").prop("selected", "selected");
});

This is my form markup:

<form method="post" id="pageForm">

    @Html.HiddenFor(x => x.Data.Ref)

    <div class="row">
        <div class="col-sm-5">
            <h3>All grades</h3>
            <div class="form-group">
                <input type="text" id="filterBox" class="form-control" placeholder="Filter grades" />
            </div>
            <div class="form-group">
                <select class="form-control" size="20" multiple="multiple" id="gradesList" asp-items="@(new SelectList(Model.Data.AvailableGrades, "Ref", "Display"))">
                </select>
            </div>
        </div>

        <div class="col-sm-2">
            <div class="push-down-arrows">
                <div class="form-group">
                    <input type="button" class="btn btn-primary btn-block" id="addButton" value=">>" title="Add grade to control group" />
                </div>
                <div class="form-group">
                    <input type="button" class="btn btn-primary btn-block" id="removeButton" value="<<" title="Remove grade from control group" />
                </div>
            </div>
        </div>

        <div class="col-sm-5">
            <h3>Associated grades</h3>
            <div class="grades-padding-top">
                <div class="form-group">
                    <select class="form-control" size="20" multiple="multiple" id="controlGradesList" asp-for="Data.AssociatedGradeRefs" asp-items="@(new SelectList(Model.Data.AssociatedGrades, "Ref", "Display"))">
                    </select>
                </div>
            </div>
        </div>

    </div>

    <hr />

    <button class="btn btn-primary" type="submit"><i class="fa fa-save">&nbsp;</i>Save</button>
    | <a href="#" onclick="window.history.go(-1);">Back to previous page</a>

</form>

What am I missing here?