Angular 4 router: direct browser navigation restarts app

When I use the browser to directly navigate to a route the app re-bootstraps and then it takes me to the correct page/component. When I navigate through a button for example it takes me there directly without re-bootstrapping the app.

Is this expected behavior? How can I prevent this?

The problem is we use identity server for our authentication. It requires a callback url, which is treated like a direct browser navigation and the app re-bootstraps.

Our app-routing.module looks like this:

const routes: Routes = [
  { path: '', redirectTo: '/dashboard', pathMatch: 'full' },
  { path: 'dashboard', component: DashboardComponent, canActivate: [RouteGuardService] },
  { path: 'users', component: UsersMainComponent},
  { path: 'vendors', component: VendorMainComponent},
  { path: 'invoices', component: InvoicesMainComponent},
  { path: 'offers' , component: EsdMainComponent},
  { path: 'settings' , component: SettingsMainComponent},
  { path: 'callback' , component: CallbackComponent},
  { path: 'createvendor' , component: CreateVendorsComponent, canActivate: [RouteGuardService]},
  { path: 'customerdashboard' , component: CustomerDashboardComponent},
];

@NgModule({
  imports: [RouterModule.forRoot(routes)],
  exports: [RouterModule],
  providers: [RouteGuardService]
})
export class AppRoutingModule { }

The callback path is the callback url for identity server.