Spring Security combining SAML and backdoor Login

I have an application that is successfully authenticating against an SSO provider via SAML. However there are a handful of users who need to access the site via a regular login box. I would like the flow to be for normal users to be redirected to the SSO provider, but give the option of a special page for other users (maybe directly to the login page) and have them hit a local database.

I'm having trouble combining the JavaConfigs for these two usecases. Here is my config for SAML:

    protected void configure(final HttpSecurity http) throws Exception {
        http
        .authorizeRequests()
        .antMatchers("/admin/**").hasAuthority(Authority.ADMIN.getAuthority())
        .and()
        .apply(SAMLConfigurer.saml())
        .userDetailsService(accountService)
        .serviceProvider()
        .keyStore()
        .storeFilePath("saml/keystore.jks")
        .password(this.password)
        .keyname(this.keyAlias)
        .keyPassword(this.password)
        .and()
        .protocol("https")
        .hostname(String.format("%s:%s", host, this.port))
        .basePath("/")
        .and()
        .identityProvider()
        .metadataFilePath(this.metadataUrl);
    }

This works fine, users are redirected immediately and all is well. So if I had a regular login box alone, something like this would work fine:

    @Override
    protected void configure(HttpSecurity http) throws Exception {
        http
        .csrf().disable()
        .authorizeRequests()
        .antMatchers("/admin").hasAuthority(Authority.ADMIN.getAuthority()).and()
        .formLogin().loginPage("/login").successHandler(successHandler).loginProcessingUrl("/login").and()
        .logout().logoutRequestMatcher(new AntPathRequestMatcher("/logout", "GET"));
    }

    @Autowired
    public void configureGlobal(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(accountService).passwordEncoder(passwordEncoder);
    }

But if I try to combine these two in any way, they seem to step on each other. My first attempt was using the @Order annotation, but if I give the SAML order first priority, the login form does not submit (shows a 405 error). If I give the login order first priority, everyone gets redirected to the login page instead of the SSO page. How can I combine these so that the login page works if users specifically hit it, but everything else gets redirected to SSO?