Github Actions stays stuck on a task and finally fail on timeout

Question

For the ruptures python package, we are using Github Actions to run our tests on each PR that should be merged on master. It worked beautifully up until now. But since few days, this Gh Actions just stays stuck and finally fails on a timeout (360 minutes).

Please, find here an example of the symptoms on a particular run.

Please, find here the Gh Actions code that performs the tests. It has a strategy matrix on several python versions and os.

name: Run tests

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    strategy:
      matrix:
        python-version: [3.6, 3.7, 3.8, 3.9]
        os: [ubuntu-latest, windows-latest, macos-latest]
        exclude:
          - os: windows-latest
            python-version: 3.9
    runs-on: ${{ matrix.os }}
    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install ruptures
      run: |
        python -m pip install --upgrade pip
        python -m pip install .[test]
    - name: Test with pytest
      run: |
        python -m pytest

From what I can see, there is no error except just it stays stuck. From what I noticed, it stays stuck always on the first elements of the strategy matrix combination, i.e. for us it is python v3.6 (which is the lowest in the matrix possibilities). But again, not sure it is linked to that since no error, just a timeout with no specific message about something going wrong (unless I did not looked to the right place).

Since I thought it might be linked to Gh getting lost with too many combination jobs on the matrix, I tried specifying max-parallel: 2 in the strategy. But it did not solved anything.

For the same reason, I tried removing from the strategy matrix the python v3.6 in order to see if the python v3.6 was the particular issue. But it does not solve anything and the Gh Actions just stays stuck on the Python v3.7 instead of v3.6. See here to see the run.

Tests run smoothly locally.

I don't get what cause this behaviour, even more since this Gh Actions used to run without any problem for several months.

user_15742241 · Answer

Thanks to jidicula, I was able to investigate and find a solution.

Bottom line :

It has nothing to do with Github Actions
It is entirely related to pip

Solutions :

Instead of upgrading pip using python -m pip install --upgrade pip, set manually the version to the last working version python -m pip install pip==21.0.1. In details :

In our particular case, upgrading pip would give the v21.1 version
The last version of pip that allowed the Gh Actions to run smoothly was v21.0.1. So I just now set manually pip to this particular value

Explanations :

It seems like there is an issue with the new (v21.1) pip version

In the failing Gh Actions, there was the following message :

Processing /home/runner/work/ruptures/ruptures
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behaviour before it becomes the default.
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.

Nevertheless, once pip is upgraded to v21.1, adding or not adding the --use-feature=in-tree-build did not solved anything and pip just stayed stuck.

Related also the the use of --use-feature=in-tree-build I found this issue on pip repo. This very last issue has a comment that proposes to do

python setup.py bdist_wheel
pip install dist/*.whl

instead of pip install . which is not a good solution in our particular case since we are making use of the possibility to specify a [options.extras_require] like for instance python -m pip install .[test]. But it might help in your case !

Finally, for more details on the underlying bug in pip, please find here the issue grouping several related behaviours.

Github Actions stays stuck on a task and finally fail on timeout

1 Answers