Context: I'm currently using Python to a code a data-reduction pipeline for a large astronomical imaging system. The main pipeline class passes experimental data through a number of discrete processing 'stages'.
The stages are written in separate .py files which constitute a package. A list of available stages is generated at runtime so the user can choose which stages to run the data through. The aim of this approach is to allow the user to create additional stages in the future.
Issue: All of the pipeline configuration parameters and data structures are (currently) located within the main pipeline class. Is there a simple way to access these from within the stages which are imported at runtime?
My current best attempt seems 'wrong' and somewhat primitive, as it uses circular imports and class variables. Is there perhaps a way for a pipeline instance to pass a reference to itself as an argument to each of the stages it calls?
This is my first time coding a large python project and my lack of design knowledge is really showing.
Any help would be greatly appreciated.