dvc.exceptions.CyclicGraphError: Pipeline has a cycle involving: load_extract_save

user_13355130 0
Viewed 38
stages:
  load_extract_save: 
    cmd: python src/stage_01_load_extract_save.py --config=config/config.yaml
    deps:
      - config/config.yaml
      - src/utils/all_utils.py
      - src/stage_01_load_extract_save.py
      - artifacts/data
    outs:
      - artifacts/data
      - artifacts/clean_data/X.npy
      - artifacts/clean_data/Y.npy

  train_test_split_save:
    cmd: python src/stage_02_train_test_split_save.py --config=config/config.yaml --params=params.yaml
    deps:
      - artifacts/clean_data/X.npy
      - artifacts/clean_data/Y.npy
      - src/utils/all_utils.py
      - params.yaml
      - config/config.yaml
      - src/stage_02_train_test_split_save.py
    outs:
      - artifacts/train_data/X_train.npy
      - artifacts/train_data/Y_train.npy
      - artifacts/test_data/X_test.npy
      - artifacts/test_data/Y_test.npy
  
  train_model:
    cmd:  python src/stage_03_train.py --config=config/config.yaml --params=params.yaml
    deps:
      - artifacts/train_data/X_train.npy
      - artifacts/train_data/Y_train.npy
      - artifacts/test_data/X_test.npy
      - artifacts/test_data/Y_test.npy
      - src/stage_03_train.py
      - src/utils/all_utils.py
      - config/config.yaml
      - params.yaml
    outs:
      - artifacts/checkpoints
      - artifacts/model
  
  metrics:
    cmd: python src/stage_04_metrics.py --config=config/config.yaml
    deps:
      - src/stage_04_metrics.py
      - config/config.yaml
      - src/utils/all_utils.py
      - artifacts/test_data/X_test.npy
      - artifacts/test_data/Y_test.npy
      - artifacts/checkpoints
      - artifacts/model
    outs:
      - confusion_matrix.png

This is my DVC.yaml.

I have created Github workflow to reproduce it, but whenever I run it it gives me the following error - ... ERROR: Pipeline has a cycle involving: load_extract_save.

The error looks like this.

dvc python
1 Answers

Stage load_extract_save both outputs and depends on the same path (artifacts/data). That's a cycle.

Pipeline structures should be directed acyclical graphs, otherwise dvc repro could execute that stage over and over forever.

user_7619630
Related