Using Application Insights with ILoggerFactory

I'm trying to log exceptions to Application Insights. I succeeded in doing this by calling TelemetryClient.TrackException directly. However, I would like to abstract away from this in my code in case I'd want to log to other platforms in the future, so I would like to stick to only the ILogger interface.

I found that you can use ILoggerFactory.AddApplicationInsights (as implemented here) but no matter what I did, I don't see the logs showing up in ApplicationInsights with this.

Below is my code:

Startup.cs

    IConfigurationRoot Configuration { get; set; }
    ILoggerFactory LoggerFactory { get; set; }
    IServiceProvider ServiceProvider { get; set; }

    public Startup( IHostingEnvironment hostingEnvironment, ILoggerFactory loggerFactory )
    {
        this.LoggerFactory = loggerFactory;
        string configurationFilePath = "abc.json";

        this.Configuration = new ConfigurationBuilder()
            .SetBasePath( hostingEnvironment.ContentRootPath )
            .AddJsonFile( configurationFilePath, optional: true, reloadOnChange: true )
            .AddEnvironmentVariables()
            .Build();
    }

    public void Configure(
        IApplicationBuilder applicationBuilder,
        IHostingEnvironment hostingEnvironment,
        ILoggerFactory loggerFactory,
        IServiceProvider serviceProvider )
    {
        this.ServiceProvider = serviceProvider;
        loggerFactory.AddApplicationInsights( serviceProvider );
        applicationBuilder.UseMvc();
    }

    public void ConfigureServices( IServiceCollection services )
    {
        services.AddApplicationInsightsTelemetry( this.Configuration );
        services.AddMvc( .... // A bunch of options here ... )
    }

Then, I try to use this in my controller like this:

    ILogger<XController> Logger { get; set; }

    public XController( ILogger<XController> logger )
    {
        this.Logger = logger;
    }

    [HttpPost]
    [Route( "v3.0/abcd" )]
    public async Task PostEvent( [FromBody] XEvent xEvent )
    {
        this.Logger.LogError( 0, new Exception( "1234" ), "1234" );
    }

However, I don't see any exceptions associated with the request at all. If I replace the Logger.LogError line with TelemetryClient.TrackException (and create the TelemetryClient first), then I can see the exception without any issues.

I don't know what I'm doing wrong. Could anyone help?