How does stack resolve dependencies?

Question

How does stack resolve dependecy conflicts?

I just started off with Haskell and I have few questions on how stack resolves dependencies.

Let's say my project requires lib A and lib B.

Internally, lib A requires lib X-1.9.0 and lib B requires lib X-2.0.0, how would stack resolve this?

stack documentation says they use snapshots to resolve conflicts, how does that work? Does it mean authors of lib A and lib B decide on a version of lib X which works with both of them? If so, what happens when I use a newer version of lib A or lib B or if either of them are not in the snapshots?

How are snapshots actually made?
Stack by default installs packages globally. What happens when a Project A requires lib Y-1.0 and Project B requries lib Y-1.1? How does this gets taken care of?
How does one use packages at "stackage.org"?

I was trying to install beam-core and google took me to https://www.stackage.org/package/beam-core where there's no mention of the command which installs it or what is the latest version. I could not find the version number anywhere expect in github releases.

With both pip and npm, it's quite straight forward and all the information on how to install and use is available on package's page. For example both,

https://pypi.org/project/bencode.py/

https://www.npmjs.com/package/projects

contains version number and install command, even though they are quite obvious.
I often get errors related to 'stack-configuration' when I try to install a package. I don't what 'stack-configuration' is? What does all these errors mean and how to resolve them in context with all the above questions?

Performing stack install beam-core or stack repl --package beam-core --package beam-sqlite --package sqlite-simple --package beam-migrate --package text results in

`Users/username/Documents/beam-learn/beam-learn.cabal was modified manually. Ignoring   /Users/username/Documents/beam-learn/package.yaml in favor of the cabal file.
If you want to use the package.yaml file instead of the cabal file,
then please delete the cabal file.
Stack has not been tested with GHC versions above 8.6, and using 8.8.2, this may fail
Stack has not been tested with Cabal versions above 2.4, but version 3.0.1.0 was found, this may fail

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for hashable-1.2.7.0:
    base-4.13.0.0 from stack configuration does not match >=4.4 && <4.13  (latest matching version is 4.12.0.0)
needed due to beam-core-0.8.1.0 -> hashable-1.2.7.0

Some different approaches to resolving this:

  * Set 'allow-newer: true' in /Users/username/.stack/config.yaml to ignore all version constraints and build anyway.

  * Build requires unattainable version of base. Since base is a part of GHC, you most likely need to use a different GHC version with the matching base.

Plan construction failed.`

user_7203016 · Answer

For question #1:

Stack is designed around the concept that, for a given Stack project, only one version of a given package will be used. So, if you have a project that requires libraries A and B, and each of them depend on different versions of library X, then you cannot build your project as-is with Stack.

Snapshots are constructed by building collections of versions of packages (with exactly one version per package) such that all inter-package dependencies can be satisfied. This is done by the Stackage "curators" as described here using the curator tool. The curator tool uses the index of packages available on Hackage to construct a set of versions of packages (exactly one version per package) that are compatible in the sense that all package interdependencies are satisfied.

So, the library authors don't need to decide on a version of X that works with both. Rather, they need to specify a range of versions of X that their package works with, and the curator tool selects the most recent version of X that works with both their packages, as well as everyone else's packages that depend on X or on which X has a dependency.

If you want to use a newer version of library A or B that isn't in the snapshot, you add it as an extra dependency in your build plan (i.e., in the extra-deps section of your stack.yaml file). If the new version can't be built with the snapshot's version of X, you need to add an extra dependency for X too. If that breaks other packages and you can't find a set of extra dependencies that resolves all conflicts, you're out of luck.

In practice, because most packages have relatively generous ranges of dependencies and, for actively maintained packages, those ranges are generally kept up to date with newer compatible dependency versions, you don't often run into unresolvable conflicts, but it does happen.

For question #2:

Stack doesn't really install packages globally. It installs snapshot packages in a global cache (on Linux, in the directory ~/.stack) organized by snapshot. So, multiple versions can be installed in this cache under different snapshots, and the project will use whichever version is appropriate for the project's selected snapshot.

For question #3:

On the Stackage page for beam-core, you can see that the most recent LTS snapshot that contains it is lts-14.27. You can create a new project using this resolver with:

$ stack new --resolver lts-14.27 my-beam-project

To add beam-core to your project, edit my-beam-project/package.yaml and add a dependency:

dependencies:
- base >= 4.7 && < 5
- beam-core           # <-- add this

Now, run stack build in your my-beam-project directory:

$ cd my-beam-project
$ stack build

It will build beam-core and all its dependencies which takes a few minutes, unless you've built beam-core for this snapshot before.

You can fiddle around with beam-core by running stack ghci in your project:

$ stack ghci
...
Configuring GHCi with the following packages: my-beam-project
GHCi, version 8.6.5: http://www.haskell.org/ghc/  :? for help
[1 of 2] Compiling Lib              ( /u/buhr/src/overflow/my-beam-project/src/Lib.hs, interpreted )
[2 of 2] Compiling Main             ( /u/buhr/src/overflow/my-beam-project/app/Main.hs, interpreted )
Ok, two modules loaded.
Loaded GHCi configuration from /tmp/haskell-stack-ghci/e5a80991/ghci-script
*Main Lib> import Database.Beam
*Main Lib Database.Beam> :t fieldName
fieldName
  :: Functor f =>
     (text-1.2.3.1:Data.Text.Internal.Text
      -> f text-1.2.3.1:Data.Text.Internal.Text)
     -> TableField table ty -> f (TableField table ty)
*Main Lib Database.Beam>

and, of course, you can add code in src/Lib.hs and/or app/Main.hs using the beam-core package.

For question #4:

As noted in a comment, I believe the problem you're encountering is that the most recent LTS is lts-15.3, and beam-core is not currently being built for this snapshot.

Because beam-core was built for previous snapshots but isn't being built for the current snapshot, there's probably a good reason it's been left out. In this case, it looks like the maintainer has not upgraded it to work with the latest GHC versions. Specifically, the latest version of beam-core-0.8.0.0 requires a version of hashable < 1.3, but the latest version satisfying that constraint is hashable-1.2.7.0 which requires base < 4.13. And, while it's far from obvious, that means it doesn't work with GHC 8.8, only GHC 8.6, so you have to go back to a GHC 8.6-series Stackage snapshot.

How does stack resolve dependencies?

1 Answers