Project-specific override for Cargo

user_453616 0
Viewed 552

I primarily want to use Debian's Rust packages, rather than fetching some random code from the wider Internet (I'm old-fashioned, I know, let's not get into that part). To this end, my ~/.cargo/config.toml looks like

[net]
offline = true

[source]

[source.crates-io]
replace-with = "debian"

[source.debian]
directory = "/usr/share/cargo/registry"

This works great after I install the librust-*-dev packages that I desire. However, in some specific projects, I'd like to lift this rule and tell Cargo "hey, you can in fact go wild and get whatever you want from crates.io". According to the Cargo book, a project-specific /project/.cargo/config.toml should take precedence over my user one. Assume this project-specific .cargo/config.toml:

[net]
offline = false

[source]

[source.crates-io]

I'm still not able to cargo build a project with dependencies from outside of my replacement source. If for example, I make a Cargo.toml that depends on yew (a randomly chosen crate that I know isn't available in my replacement source) I get

$ cargo build
error: no matching package found
searched package name: `yew`

What am I misunderstanding about Cargo's sources, replacement and per-project overrides?

rust-cargo dependency-management
3 Answers

It looks as if you are suffering from this issue: https://github.com/rust-lang/cargo/issues/8687

You would like to unset a config key on a upper-level config.toml but this is not supported.

I've played a bit with the config, and the only way I got it to work was to overwrite in the project-local config.toml the properties that were set in the upper-level config.toml.

In your case your upper-level config.toml specifies replace-with, so you have to overwrite that. But you can't overwrite it with crates-io, which is the registry you want to use, because that is exactly the registry with the replace-with key.

So until the above issue gets acted upon, we have to, essentially, use a mirror, both in the config and as an actual registry to download from:

[net]
offline = false

[source]

[source.crates-io]
replace-with = "crates-io-mirror"

[source.crates-io-mirror]
registry = "https://gitlab.com/integer32llc/crates.io-index"

As we both tested, it seems it's not possible to reuse the normal crates.io registry url because that is already defined and will fail with:

error: source crates-io-mirror defines source registry https://github.com/rust-lang/crates.io-index, but that source is already defined by crates-io note: Sources are not allowed to be defined multiple times.

So instead the URL above is an actual mirror server of crates.io. Then you can run cargo build successfully in the local project.

user_41088030

The answer suggested by @blackgreen is one possible workaround for the underlying problem until issues 10045 and 10057 (or a combination thereof) are solved. Another, perhaps slightly less ugly, workaround follows below for those who need it.

I ended up working around the problem using UnionFS (I guess the more modern OverlayFS should work well too).

I simply add

[source.crates-io]
replace-with = "union"

[source.union]
directory = "/home/gspr/.cargo-overlay/union-registry"

to my ~/.cargo/config.toml and then do

unionfs -o ro /usr/share/cargo/registry:/home/gspr/.cargo-overlay/local-registry /home/gspr/.cargo-overlay/union-registry

Now /home/gspr/.cargo-overlay/union-registry reflects the union of /usr/share/cargo/registry and /home/gspr/.cargo-overlay/local-registry, with priority to the former in case of conflicts.

So what goes in ~/.cargo-overlay/local-registry? Individual extra crates, in the same way as in Debian's /usr/share/cargo/registry. That is to say, directories named cratename-version as they are distributed by upstream – but with a single extra file, namely .cargo-checksum.json added to them. The content of that extra file can be extracted from the crates.io index as follows.

Suppose we have cloned the crates.io index into ~/.cargo-overlay/crates.io-index, i.e.

git clone https://github.com/rust-lang/crates.io-index.git ~/.cargo-overlay/crates.io-index

Then suppose we've extracted a crate foo at version 0.1.2 into ~/.cargo-overlay/local-registry/foo-0.1.2. We can generate the missing .cargo-checksum.json like so:

cd ~/.cargo-overlay
index_file=$(find crates.io-index -type f -name foo)
cksum=$(jq -r "select(.name == \"foo\" and .vers == \"0.1.2\" ) | .cksum" ${index_file})
jo package="${cksum}" files="{}" > local-registry/foo-0.1.2/.cargo-checksum.json
user_4536160

The recently released Cargo 1.56 adds a feature that should let one do what my question asks for: patch tables can now be specified in a project-specific .cargo/config.toml, which means that [patch] stanzas can now be introduced outside of Cargo.toml. That should do the trick! I haven't yet verified this, as I am stuck with an older Cargo for a little while still.

user_4536160
Related