How to create a typescript complaint yup schema where at least one prop is a union type?

The issue:

import * as yup from 'yup';

interface WithUnion {
  boolOrString: boolean | string;
}

const testSchema = yup.object().require().shape<WithUnion>({
  boolOrString: yup.mixed().oneOf([yup.boolean().required(), yup.string().require()])
})

Throws the following error:

Type 'ObjectSchema<object & { boolOrString: BooleanSchema<boolean> | StringSchema<string>; }>' is not assignable to type 'ObjectSchema<{ boolOrString: string | boolean; }>'.
  The types of 'fields.boolOrString' are incompatible between these types.
    Type 'Schema<BooleanSchema<boolean> | StringSchema<string>>' is not assignable to type 'Schema<string | boolean>'.
      Type 'BooleanSchema<boolean> | StringSchema<string>' is not assignable to type 'string | boolean'.
        Type 'BooleanSchema<boolean>' is not assignable to type 'string | boolean'.
          Type 'BooleanSchema<boolean>' is not assignable to type 'string'.

Is there an alternative approach?