How to do type checking in Golang fast?

I'm parsing a Go application using the following piece of code. Basically, I recursively visit all packages from a top-level directory and parse each package using parser.ParseDir() function.

readPackages is a helper function that walks all package directories starting from path and for each directory, it calls the given function with given base path and the relative directory.

import (
  "go/ast"
  "go/importer"
  "go/parser"
  "go/token"
  "go/types"
  "path/filepath"
  "strings"
)

func Parse(path string) error {
  fset := token.NewFileSet()

  moduleName := getModuleName()

  return readPackages(path, func(baseDir, relDir string) error {
    pkgDir := filepath.Join(baseDir, relDir)
    importPath := filepath.Join(moduleName, relDir)

    pkgs, err := parser.ParseDir(fset, pkgDir, nil, parser.AllErrors)
    if err != nil {
      return err
    }

    for pkgName, pkg := range pkgs {
      checker := &types.Config{
        IgnoreFuncBodies: true,
        FakeImportC:      true,
        Error:            nil,
        Importer:         importer.ForCompiler(fset, "source", nil),
      }

      files := make([]*goast.File, 0)
      for fileName, file := range pkg.Files {
        if strings.HasSuffix(fileName, "_test.go") {
          continue
        }
        files = append(files, file)
      }

      info := &gotypes.Info{
        Defs:  map[*goast.Ident]gotypes.Object{},
        Uses:  map[*goast.Ident]gotypes.Object{},
        Types: map[goast.Expr]gotypes.TypeAndValue{},
      }

      _, err = checker.Check(importPath, fset, files, info)
      if err != nil {
        return err
      }

      for fileName, file := range pkg.Files {
        if strings.HasSuffix(fileName, "_test.go") {
          continue
        }
        // Process file AST ...
      }
    }

    return nil
  })
}

When running this Parse function on a real application (a service that imports http, gRPC, etc.), it takes a very very long time for type checking to complete. I tried using the importer.Default() which does not resolve none built-in packages.

My question is what's the correct or an efficient way of type checking ast.Files in Go? How does the Go compiler itself do this so fast? Why is it taking so long using importer.ForCompiler?