setFieldError is overwritten when using the validation methods

I am using custom input handler like setFieldError for setting errors. After setting custom error, when every another field was changed, it removes the error message applied using setFieldError. Setting the third parameter of setFieldValue to false is not working. I don't want to hit handleSubmit until the error is resolved.

<Formik
    initialValues={initialValues}
    validationSchema={validationSchemaState}
    onSubmit={handleSubmit}
    enableReinitialize
></Formik>

await axios
      .post(url)
      .then((res) => {
        setFieldValue(field.name, res.data.response, false);
      })
      .catch((error) => {
        setFieldError(field.name, error);
      });