Edit

Share via

What are upstream sources?

Azure DevOps Services | Azure DevOps Server | Azure DevOps Server 2022 | Azure DevOps Server 2020

Azure Artifacts upstream sources allow developers to store packages from various origins in a single feed, including packages published to the feed and those installed from public registries like NuGet.org or npmjs.com. Once upstream sources are enabled, any package installed from an upstream source is automatically saved to your feed.

Note

To save packages from upstreams, you must have the Feed and Upstream Reader (Collaborator) role or higher. See Feed roles and permissions for more details.

Why use upstream sources?

Enabling upstream sources offers several advantages for managing your product’s dependencies within a single feed:

  • Simplicity: Storing all your packages in a single feed simplifies your configuration files like NuGet.config, npmrc, or settings.xml. With just one feed in your config file, you reduce setup complexity and minimize errors.

  • Consistent Builds: Your feed resolves package requests in a defined order, helping ensure predictable and reliable builds across environments.

  • Package Integrity: Your feed retains metadata about packages saved from upstream sources, allowing you to verify their authenticity and ensure you're using the original versions, not copies or potentially malicious versions.

  • Reliability: Packages installed from upstream sources are automatically saved to your feed. This ensures continued access even if the upstream source becomes temporarily unavailable due to maintenance or other issues so you can continue developing and building with confidence.

Best practices for package consumers

To take full advantage of the benefits of upstream sources as a package consumer, follow these best practices:

Use a single feed in your configuration file

In order for your feed to provide a deterministic restore, make sure that your configuration file (such as nuget.config or npmrc) references only one feed with upstream sources enabled.

  • Examples:

    registry=https://pkgs.dev.azure.com/fabrikam/_packaging/FabrikamFiber/npm/registry/
always-auth=true

    <packageSources>
  <clear />
  <add key="FabrikamFiber" value="https://pkgs.dev.azure.com/fabrikam/_packaging/FabrikamFiber/nuget/v3/index.json" />
</packageSources>

    Note

    NuGet compiles several configuration files to determine the complete set of options to apply. Using <clear /> ensures that all other package sources specified in higher-level configuration files are ignored.

Order your upstream sources intentionally

If you're using only public registries like NuGet.org or npmjs.com, the order of upstream sources doesn’t affect behavior. Requests to the feed follow the sequence outlined in the search order section.

However, when managing multiple sources, such as a combination of feeds and public registries, each upstream source is searched in the order defined in the feed’s configuration settings. In these cases, we recommend placing the public registries first in the list of upstream sources.

In some unique scenarios, some organizations modify open-source software (OSS) packages to address security concerns, enhance functionality, or meet specific internal requirements that require rebuilding the package internally rather than obtaining it directly from a public repository. If your organization follows this practice, place the upstream source containing these customized OSS packages before other public registries. This ensures your customized versions are used instead of the public ones.

Best practices for feed owners and package publishers

To ensure your feed can be easily configured as an upstream source, follow these best practices:

Use the default view

All newly created feeds use the @Local view by default. This view includes:

  • Packages published directly to the feed.
  • Packages saved from upstream sources.

If you want to use other views such as a view for newly released package versions, you can promote your packages to the @Release view and then make that view available to your target consumers. See Feed views for more details.

Construct a package graph

To construct a package graph, simply connect to the feed's default view and install the package you want to share. Once a package is saved to the default view, users who want to consume it will be able to resolve the package graph and install the desired version. Packages from upstream sources are displayed based on the configured view for the corresponding upstream source. See How upstreams construct the set of available packages for more details.

Search order

For public package managers that support multiple feeds, such as NuGet and Maven, the order in which feeds are queried can sometimes be unclear or nondeterministic. For example, NuGet sends parallel queries to all the feeds in the configuration file and processes responses in a first-in, first-out (FIFO) manner, which can lead to inconsistent results.

Azure Artifacts upstream sources eliminate this uncertainty by enforcing a structured search order, by searching the feed and its upstream sources in the following order:

  1. Packages that have been published directly to the feed.

  2. Packages that have been saved from an upstream source.

  3. Packages available from upstream sources. Each upstream source is searched in the order it's listed in the feed's configuration.

Note

Azure Artifacts does not support searching for packages in upstream sources using the NuGet Package Explorer in Visual Studio.

Save packages from upstream sources

When an upstream source is enabled on your feed, Azure Artifacts automatically saves a copy of any package installed by a collaborator or higher from that upstream source.

For example, you can install packages directly from the upstream source using a command like npm install express. Alternatively, packages might be installed as part of dependency resolution, so installing express would also save its dependencies, such as accepts.

Upstream sources offer a critical safeguard for your consumers and infrastructure. If the public registry experiences downtime, maintenance, or becomes temporarily unavailable, you can still retrieve the necessary packages from your feed and continue your development.

Note

Custom upstream sources are only supported for npm packages.

Override packages from upstream sources

When upstream sources are enabled in your feed, you cannot publish a package version that already exists in one of those upstream sources. For example, if the NuGet.org upstream is enabled, you won’t be able to publish Newtonsoft.Json 10.0.3 to your feed, since that version is already available on NuGet.org.

To override a package version from an upstream source:

  1. Disable the relevant upstream source.

  2. Publish your desired package version to the feed.

  3. Re-enable the upstream source.

This workflow ensures you can publish the desired version while maintaining the integrity of your upstream sources.

Note

Package versions are immutable. Saved packages remain in the feed even if the upstream source is disabled or removed.

Upstream sources health status

If a feed has a failing upstream source, metadata for packages using the same protocol can no longer be refreshed. To check the health status of your upstream sources, follow these steps:

  1. Sign in to your Azure DevOps organization, and navigate to your project.

  2. Select Artifacts, then select your feed from the dropdown menu.

  3. Select the gear icon gear icon to open Feed settings, then select Upstream sources.

    A screenshot showing the upstream sources last sync up status.

  4. If any failures occur, a warning message will be displayed. Select the Failed status to view detailed information, including the cause of the failure and steps to resolve it.

    A screenshot displaying details of the sync up failure.

Note

For public registries like NuGet.org, there is typically a 3-6 hour delay between when a package is pushed to the public registry and when it becomes available for download. This delay depends on job timing and data propagation. However, when the upstream source is an Azure Artifacts feed, the latency is usually no more than a few minutes.

Related content