How to use the LDAP form login and Form Login on Symfony 5.3 with the new Authenticator-based Security system?

I get an strange behavior when I enable the new authentication based security.

I find that chain of form_login and form_login_ldap won't work together. Both on their own work, either json_login and json_login_ldap.

Here is my security.yaml

security:

    # https://symfony.com/doc/current/security/experimental_authenticators.html
    enable_authenticator_manager: true

    password_hashers:
        # this internal class is used by Symfony to represent in-memory users
        # (the 'InMemoryUser' class was introduced in Symfony 5.3.
        # In previous versions it was called 'User')
        Symfony\Component\Security\Core\User\InMemoryUser: 'auto'
        App\Entity\User: 'auto'

    # https://symfony.com/doc/current/security.html#where-do-users-come-from-user-providers
    providers:
        users_in_memory:
            memory:
                users:
                    john: { password: '$2y$13$zrK3oibk8KK3PxO/bkFrduFj61jsPbEBZlAiaAaK2rKlanv2E4UhG', roles: [ 'ROLE_ADMIN' ] }
                    jane: { password: '$2y$13$zrK3oibk8KK3PxO/bkFrduFj61jsPbEBZlAiaAaK2rKlanv2E4UhG', roles: [ 'ROLE_ADMIN', 'ROLE_SUPER_ADMIN' ] }
        users:
            entity:
                # the class of the entity that represents users
                class: 'App\Entity\User'
                # the property to query by - e.g. username, email, etc
                property: 'username'

        my_ldap:
            ldap:
                service: Symfony\Component\Ldap\Ldap
                base_dn: '##########'
                search_dn: '##########'
                search_password: '########'
                default_roles: ["ROLE_USER"]
                filter:  "########"
                uid_key: "########"

        all_users:
            chain:
                providers: [ 'users_in_memory', 'users', 'my_ldap' ]

        # used to reload user from session & other features (e.g. switch_user)
        app_user_provider:
            entity:
                class: App\Entity\User
                property: username
                # optional: if you're using multiple Doctrine entity
                # managers, this option defines which one to use
                # manager_name: 'customer'

    firewalls:
        dev:
            pattern: ^/(_(profiler|wdt)|css|images|js)/
            security: false
        main:
            lazy:           true
            entry_point:    form_login
            provider:       all_users

            form_login:
                login_path: login
                check_path: login
                default_target_path: login_success

            form_login_ldap:
                login_path: login
                check_path: login
                service: Symfony\Component\Ldap\Ldap
                dn_string: '##########'
                default_target_path: login_success

            logout:
                path: app_logout
                # where to redirect after logout
                target: login

    # Easy way to control access for large sections of your site
    # Note: Only the *first* access control that matches will be used
    access_control:
        # - { path: ^/admin, roles: ROLE_ADMIN }
        # - { path: ^/profile, roles: ROLE_USER }

Maybe someone has an idea? Setting enable_authenticator_manager to false and removing entry_point will make it work. Looks like the support method for form_login_ldap returns always false.