Using Typer and Hydra together

user_12400214 0
Viewed 368

I have a simple Typer application:

import typer

app = typer.Typer()

@app.command()
def say_hi():
    print("Hi")

@app.callback()
def main():
    pass

if __name__ == "__main__":
    app()

I would like to use Hydra for the configuration management of the app, however I am not sure how to do that without losing the ability to override the config from the CLI.

My first attempt was:

import hydra
import typer
from omegaconf import DictConfig, OmegaConf

app = typer.Typer()

@app.command()
def say_hi():
    print("Hi")

@app.callback()
@hydra.main(config_path="conf", config_name="config")
def main(cfg: DictConfig) -> None:
    print(OmegaConf.to_yaml(cfg))

if __name__ == "__main__":
    app()

But I get an error saying:

RuntimeError: Type not yet supported: <class 'omegaconf.dictconfig.DictConfig'>

If I remove the DictConfig type annotation I get an error that cfg is missing.

I saw in Hydra docs the Compose API that allows to initialize the config without decorators:

@app.callback()
def main() -> None:
    with initialize(config_path="conf", job_name="test_app"):
        cfg = compose(config_name="config")
        print(OmegaConf.to_yaml(cfg))

but It seems that I can't override config from the command line in this case as those values are not recognized by the Typer app.

Any recommendations how it can be resolved?

typer fb-hydra command-line-interface python
1 Answers

The compose function accepts an optional list of override strings:

with initialize(config_path="conf", job_name="test_app"):
    cfg = compose(config_name="config", overrides=["db=mysql", "db.user=me"])

You will need to get a list of override strings from the command line, then pass that list to compose. Here is an example using Typer; similar patterns could work for e.g. argparse or click. (Disclaimer: I am not a Typer expert)

from typing import List, Optional

import typer
from omegaconf import OmegaConf, DictConfig

from hydra import compose, initialize

app = typer.Typer()

def my_compose(overrides: Optional[List[str]]) -> DictConfig:
    with initialize(config_path="conf", job_name="test_app"):
        return compose(config_name="config", overrides=overrides)

@app.command()
def say_hi(overrides: Optional[List[str]] = typer.Argument(None)):
    print("HI!")
    print(f"Got {overrides=}")
    cfg = my_compose(overrides)
    print("\nHydra config:")
    print(OmegaConf.to_yaml(cfg))

@app.command()
def say_bye(overrides: Optional[List[str]] = typer.Argument(None)):
    cfg = my_compose(overrides)
    ...
    print("BYE!")

if __name__ == "__main__":
    app()

$ python my_app.py say-hi +foo=bar +baz=qux
HI!
Got overrides=('+foo=bar', '+baz=qux')

Hydra config:
foo: bar
baz: qux

$ python my_app.py say-bye
BYE!
user_42563460
Related