Jenkins Build With Parameters shows old and new parameter fields

My Jenkins shared pipeline, used by dozens of repos internally, has several parameter fields defined. We have several repos that initially used another library, or their own ad-hoc pipelines, but then imported the shared library.

In some of these cases, the "Build With Parameters" UI shows both the old and new parameters. We have re-run the pipelines several times, but the old fields won't go away.

If we create a new branch, only the new fields are shown.

I am thinking there's an xml file on the controller that we have to wipe, but I'm not sure where.

Has anyone seen this? What's the resolution?

Update 1

I went into the jobs folder for a branch showing this, and backed-up the config.xml file:

> cd /mnt/data/jenkins/jobs/<org>/jobs/workflows/jobs/<repo>/branches/jenkins-test/

> mv config.xml config.xml-bak

Upon re-running the job, it re-creates the config.xml and it has the same issue: old and new fields still showing.

So I deleted the entire jenkins-test branch folder. The job doesn't go away in the UI. I made a test commit to the branch in Github, Jenkins picked it up and ran it. It automatically re-created the jenkins-test folder on the controller, and upon re-trying Build With Parameters, it still shows the old and new params.

Got to keep digging...

Update 2

I created https://issues.jenkins.io/browse/JENKINS-69652 for this.

update 3

I confirmed the commit that triggered the build is the right commit in the source repo, and it is pulling the correct commit of the pipeline library.

Update 4

Interesting. I Replayed a run, and entered a dummy pipeline:

pipeline {
    agent none
    stages {
        stage ('hello') {
           steps {
               echo 'hello!'
           } 
        }
    }
}

Going back to the Build With Parameters UI, it now shows only the old parameter fields. Where is this coming from? Where is it stuck??