How do I inform conda to install a later version of apache-beam?

One possible reason why your command is not able to give you the latest version is because it is not available when you specify the cf201901 label to conda forge, which you can see on the website:

But also when you try to specify the version explicitly:

$ conda install -c conda-forge/label/cf201901 apache-beam=2.22
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - apache-beam=2.22

Current channels:

  - https://conda.anaconda.org/conda-forge/label/cf201901/linux-64
  - https://conda.anaconda.org/conda-forge/label/cf201901/noarch
  - https://repo.anaconda.com/pkgs/main/linux-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/linux-64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

So try to leave out the label and specify the version you want:

 conda install -c conda-forge apache-beam=2.22

I am suspecting however that you already had added conda-forge to your default channels, since your output is already showing the apache-beam is pulled from the conda-forge channel without a label.

So if above still does not work, you have conflicting package (versions) in your current environment that conda is not able to resolve for apache-beam version 2.22

This can be solved if you just create a new environment for the project for which you need apache-beam:

conda create -n <some name> -c conda-forge apache-beam

this will create an environment that you can activate with

conda activate <some name>

The answer from @FlyingTeller accurately described the problem and a resolution but no harm in sharing a bit of extra info.

I neglected to say that I'm building my conda env in a docker image and that docker image gets rebuilt in a CI pipeline. In order to ensure I have a repeatable build I have an environment.yaml file in my repo and it is THAT which I use to build the conda env.

I have had to make changes to that environment.yaml file at various times over the past month (I have installed sh, apache-beam, pytest, pytest-xdict, ipython, behave...and I think that is the order in which I installed them) and my strategy for doing so is to:

1. docker run myimage:most-recently-built-tag
2. conda install the required packages
3. export a new version of environment.yaml
4. commit new version environment.yaml to the repo

In doing so I got myself into a position where the version of apache-beam that got installed was 2.16.0 and conflicting dependencies meant I wasn't able to upgrade apache-beam.

I created a new environment and issued conda install sh apache-beam pytest pytest-xdict ipython behave which successfully resolved all dependencies and installed apache-beam=2.2.0.

Hence I think my strategy, when I want to make changes to environment.yaml, should be:

1. create a new conda env
2. conda install all the required packages in a single command
3. export a new environment.yaml and commit it to the repo so it can be used to build my docker image in the CI pipeline

Comments on anything above would be welcomed.

UPDATE. Here's my Dockerfile that builds a conda environment from which I can export environment.yaml

FROM debian:buster-slim

RUN apt-get update && apt-get install curl gnupg -y

RUN curl https://repo.anaconda.com/pkgs/misc/gpgkeys/anaconda.asc | gpg --dearmor > conda.gpg && \
        install -o root -g root -m 644 conda.gpg /usr/share/keyrings/conda-archive-keyring.gpg && \
        gpg --keyring /usr/share/keyrings/conda-archive-keyring.gpg --no-default-keyring \
        --fingerprint 34161F5BF5EB1D4BFBBB8F0A8AEB4F8B29D82806 && \
        echo "deb [arch=amd64 signed-by=/usr/share/keyrings/conda-archive-keyring.gpg] https://repo.anaconda.com/pkgs/misc/debrepo/conda stable main" \
        > /etc/apt/sources.list.d/conda.list

RUN MINICONDA_VERSION=4.9.2 && \
        CONDA_VERSION='4.9.*' && \
        CONDA_DIR=/opt/conda && \
        cd /tmp && \
        curl -O https://repo.anaconda.com/miniconda/Miniconda3-py37_${MINICONDA_VERSION}-Linux-x86_64.sh && \
        /bin/bash Miniconda3-py37_${MINICONDA_VERSION}-Linux-x86_64.sh -f -b -p $CONDA_DIR && \
        rm Miniconda3-py37_${MINICONDA_VERSION}-Linux-x86_64.sh && \
        $CONDA_DIR/bin/conda config --system --set auto_update_conda false && \
        $CONDA_DIR/bin/conda config --system --set show_channel_urls true && \
        $CONDA_DIR/bin/conda config --system --remove channels defaults && \
        $CONDA_DIR/bin/conda config --system --add channels main && \
        $CONDA_DIR/bin/conda config --system --set env_prompt '({name}) ' && \
        $CONDA_DIR/bin/conda config --system --append envs_dirs /opt/conda/envs/ && \
        $CONDA_DIR/bin/conda config --system --append pkgs_dirs /opt/conda/pkgs/ && \
        $CONDA_DIR/bin/conda update --quiet --yes --all conda="${CONDA_VERSION}" && \
        $CONDA_DIR/bin/conda config --system --append channels conda-forge  && \
        $CONDA_DIR/bin/conda create -n py3 python=3.7 

RUN bash -c "source /opt/conda/bin/activate /opt/conda/envs/py3 && conda install apache-beam sh pytest pytest-xdist ipython behave"

RUN echo "source /opt/conda/bin/activate /opt/conda/envs/py3" >> /root/.bashrc

and generate my environment.yaml by running:

docker build . -t conda-beam && \
  docker run --entrypoint bash conda-beam:latest -c \
  "source /opt/conda/bin/activate /opt/conda/envs/py3 \
  && conda env export"

which, when I ran it just now, returned

name: py3
channels:
  - main
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - apache-beam=2.22.0=py37h8f50634_0
  - apipkg=1.5=py37_0
  - arrow-cpp=0.15.1=py37h7cd5009_5
  - attrs=20.3.0=pyhd3eb1b0_0
  - backcall=0.2.0=py_0
  - behave=1.2.6=py37h89c1867_1003
  - blas=1.0=mkl
  - boost-cpp=1.71.0=h7b6447c_0
  - brotli=1.0.9=he6710b0_2
  - brotlipy=0.7.0=py37h27cfd23_1003
  - bzip2=1.0.8=h7b6447c_0
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2020.12.8=h06a4308_0
  - certifi=2020.12.5=py37h06a4308_0
  - cffi=1.14.4=py37h261ae71_0
  - chardet=4.0.0=py37h06a4308_1003
  - crcmod=1.7=py37hc8dfbb8_1003
  - cryptography=3.3.1=py37h3c74f83_0
  - cython=0.29.21=py37h2531618_0
  - decorator=4.4.2=py_0
  - dill=0.3.1.1=py37_1
  - docopt=0.6.2=py37_0
  - double-conversion=3.1.5=he6710b0_1
  - execnet=1.7.1=py_0
  - fastavro=0.23.5=py37h7b6447c_0
  - future=0.18.2=py37_1
  - gflags=2.2.2=he6710b0_0
  - glog=0.4.0=he6710b0_0
  - grpc-cpp=1.26.0=hf8bcb03_0
  - grpcio=1.31.0=py37hf8bcb03_0
  - httplib2=0.17.0=py37hc8dfbb8_1
  - icu=58.2=he6710b0_3
  - idna=2.10=py_0
  - importlib-metadata=2.0.0=py_1
  - importlib_metadata=2.0.0=1
  - iniconfig=1.1.1=py_0
  - intel-openmp=2020.2=254
  - ipython=7.19.0=py37hb070fc8_0
  - ipython_genutils=0.2.0=pyhd3eb1b0_1
  - jedi=0.18.0=py37h06a4308_0
  - krb5=1.18.2=h173b8e3_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libboost=1.71.0=h97c9712_0
  - libedit=3.1.20191231=h14c3975_1
  - libevent=2.1.8=h1ba5d50_1
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.1.0=hdf63c60_0
  - libprotobuf=3.11.2=hd408876_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - lz4-c=1.8.1.2=h14c3975_0
  - mkl=2020.2=256
  - mkl-service=2.3.0=py37he8ac12f_0
  - mkl_fft=1.2.0=py37h23d657b_0
  - mkl_random=1.1.1=py37h0573a6f_0
  - mock=2.0.0=py37_0
  - more-itertools=8.6.0=pyhd3eb1b0_0
  - ncurses=6.2=he6710b0_1
  - numpy=1.19.2=py37h54aff64_0
  - numpy-base=1.19.2=py37hfa32c7d_0
  - oauth2client=4.1.3=py_0
  - openssl=1.1.1i=h27cfd23_0
  - packaging=20.8=pyhd3eb1b0_0
  - pandas=1.2.0=py37ha9443f7_0
  - parse=1.18.0=pyh9f0ad1d_0
  - parse_type=0.5.2=py37h89c1867_3
  - parso=0.7.0=py_0
  - pbr=5.5.1=py_0
  - pexpect=4.8.0=pyhd3eb1b0_3
  - pickleshare=0.7.5=pyhd3eb1b0_1003
  - pip=20.3.3=py37h06a4308_0
  - pluggy=0.13.1=py37_0
  - prompt-toolkit=3.0.8=py_0
  - protobuf=3.11.2=py37he6710b0_0
  - ptyprocess=0.7.0=pyhd3eb1b0_2
  - py=1.10.0=pyhd3eb1b0_0
  - pyarrow=0.15.1=py37h0573a6f_0
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycparser=2.20=py_2
  - pydot=1.3.0=py37_0
  - pygments=2.7.3=pyhd3eb1b0_0
  - pykerberos=1.2.1=py37h680d80a_2
  - pymongo=3.11.2=py37h2531618_0
  - pyopenssl=20.0.1=pyhd3eb1b0_1
  - pyparsing=2.4.7=py_0
  - pysocks=1.7.1=py37_1
  - pytest=6.2.1=py37h06a4308_0
  - pytest-forked=1.3.0=py_0
  - pytest-xdist=2.2.0=pyhd3eb1b0_0
  - python=3.7.9=h7579374_0
  - python-avro=1.9.2.1=py37hc8dfbb8_1
  - python-dateutil=2.8.1=py_0
  - python-hdfs=2.5.8=py_0
  - python_abi=3.7=1_cp37m
  - pytz=2020.5=pyhd3eb1b0_0
  - pyyaml=5.3.1=py37h7b6447c_1
  - re2=2020.11.01=h2531618_1
  - readline=8.0=h7b6447c_0
  - requests=2.25.1=pyhd3eb1b0_0
  - requests-kerberos=0.12.0=py37_0
  - rsa=4.7=pyhd3eb1b0_0
  - setuptools=51.1.2=py37h06a4308_3
  - sh=1.14.1=py37h06a4308_0
  - six=1.11.0=py37_1
  - snappy=1.1.8=he6710b0_0
  - sqlite=3.33.0=h62c20be_0
  - thrift-cpp=0.11.0=h02b749d_3
  - tk=8.6.10=hbc83047_0
  - toml=0.10.1=py_0
  - traitlets=5.0.5=py_0
  - typing-extensions=3.7.4.3=0
  - typing_extensions=3.7.4.3=py_0
  - uriparser=0.9.3=he6710b0_1
  - urllib3=1.26.2=pyhd3eb1b0_0
  - wcwidth=0.2.5=py_0
  - wheel=0.36.2=pyhd3eb1b0_0
  - xz=5.2.5=h7b6447c_0
  - yaml=0.2.5=h7b6447c_0
  - zipp=3.4.0=pyhd3eb1b0_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.3.7=h0b5b093_0
prefix: /opt/conda/envs/py3