How can we avoid that every package in our .NET Core 3.1 project, is checked at every NuGet source during Azure DevOps pipeline?

user_2823537 0
Viewed 335

In our .NET Core 3.1 project (REST API), we've multiple NuGet packages. General packages comes from the nuget.org source, some custom made packages are retrieved from a private source.

In Azure DevOps, we've a build pipeline with a task to restore the NuGet packages. Here we saw that every packages was checked with every source. A general package such as Swashbuckle.AspNetCore.SwaggerGen was also searched on our private source. Due to the amount of requests from DevOps, the first attempt of the pipeline was interpreted as a DOS attack on our system. When the failed run was started again, the task succeeds without any error.

- task: DotNetCoreCLI@2
  displayName: dotnet restore
  inputs:
    command: 'restore'
    projects: '**/*.sln'
    feedsToUse: 'config'
    nugetConfigPath: 'src/NuGet.config'

In the tasks detail, we see the below message returning for every package.

GET private_source/nuget/FindPackagesById()?id='xunit.analyzers'&semVerLevel=2.0.0 Retrying 'FindPackagesByIdAsyncCore' for source 'private_source/nuget/FindPackagesById()?id='Microsoft.AspNetCore.Mvc.Razor'&semVerLevel=2.0.0'. An error occurred while sending the request. The response ended prematurely.

How can we avoid that every package in our solution, is checked at every NuGet source? Or what can we change to get a successfull build the first time?

nuget-package-restore azure-pipelines nuget .net-core azure-devops
3 Answers

NuGet recently introduced the feature, called Package Source Mapping: https://devblogs.microsoft.com/nuget/introducing-package-source-mapping/

Here's the nuget.config snippet from the blog post:

<!-- Define my package sources, nuget.org and contoso.com. -->
<!-- `clear` ensures no additional sources are inherited from another config file. -->
<packageSources>
  <clear />
  <!-- `key` can be any identifier for your source. -->
  <add key="nuget.org" value="https://api.nuget.org/v3/index.json" />
  <add key="contoso.com" value="https://contoso.com/packages/" />
</packageSources>

<!-- Define mappings by adding package ID patterns beneath the target source. -->
<!-- Contoso.* packages will be restored from contoso.com, everything else from nuget.org. -->
<packageSourceMapping>
  <!-- key value for <packageSource> should match key values from <packageSources> element -->
  <packageSource key="nuget.org">
    <package pattern="*" />
  </packageSource>
  <packageSource key="contoso.com">
    <package pattern="Contoso.*" />
  </packageSource>
</packageSourceMapping>

Regarding the error message:

An error occurred while sending the request. The response ended prematurely.

This suggests there's something wrong with the server or networking. A good nuget server should return HTTP 404 for packages that doesn't exist on it. Implementing package source mapping might not solve your restore problem.

user_28430650

I suggest creating an Azure Devops Artifacts feed having upstream source both from nuget.org and your private feed. There is no other way you can use multiple sources to do partial restore.

user_578640

How can we avoid that every package in our .NET Core 3.1 project, is checked at every NuGet source during Azure DevOps pipeline?

I am afraid there is no such out of box way to resolve this restriction.

That's because no matter how we set the resource, when we restore the package for the first time, nuget.exe will iterate over each resource for every package. This problem will be alleviated when we run the pipeline again, because it is from nuget.org The packages will be cached in our private feed. When we restore again, it will be retrieved from the private feed first：

Check my previous thread for some more details.

Besides, If you want to avoid this problem the first time, you can try not to restore the entire .sln file, you can choose package.configs for the specify reject:

enter image description here

user_74607770
Related