Framework

mlopus.artschema.framework.A

Invariant TypeVar bound to object.

Type of artifact

mlopus.artschema.framework.D

Invariant TypeVar bound to mlopus.artschema.framework.Dumper.

Type of Dumper

mlopus.artschema.framework.L

Invariant TypeVar bound to mlopus.artschema.framework.Loader.

Type of Loader

pydantic model mlopus.artschema.Dumper

Bases: BaseModel, BaseParamsMixin, ABC, Generic[A]

Base class for artifact dumpers.

class Config

Bases: Config

Class-level config.

abstract _dump(path, artifact)

Save artifact to path as file or dir.

Parameters:

• path (Path) – When this method is called, it is guaranteed that this path doesn’t exist yet and that it’s parent is a dir.

• artifact (TypeVar(A, bound= object)) – An instance of Artifact.

Return type:

None

abstract _verify(path)

Verify the path.

Parameters:

path (Path) – Path where an instance of Artifact is supposed to have been dumped, downloaded or placed.

Raises:

AssertionError – Unless path is a file or dir in the expected structure.

Return type:

None

dump(path, artifact, overwrite=False)

Save artifact to path as file or dir.

If possible, also saves a file with this dumper’s conf.

Parameters:

• path (Path | str) – Target path.

• artifact (Union[TypeVar(A, bound= object), dict]) –

  • An instance of Artifact.

  • A dict that can be parsed into an Artifact (in case Artifact is a Pydantic model)

• overwrite (bool) – Overwrite the path if exists.

Return type:

None

maybe_save_conf(path, strict)

If path is a dir, save a file with this dumper’s conf.

Parameters:

• path (Path) –

• strict (bool) –

Raises:

• FileNotFoundError – If path doesn’t exist.

• FileExistsError – If strict is True and a dumper conf file already exists in path.

verify(path)

Verify the path.

Parameters:

path (Path | str) – Path where an instance of Artifact is supposed to have been dumped, downloaded or placed.

Raises:

AssertionError – Unless path is a file or dir in the expected structure.

Return type:

None

property Artifact: Type[A]

Artifact type used by this dumper.

Returns:

Type of A

pydantic model mlopus.artschema.Loader

Bases: BaseModel, BaseParamsMixin, ABC, Generic[A, D]

Base class for artifact loaders.

abstract _load(path, dumper)

Load artifact from path.

Parameters:

• path (Path) – Path to artifact file or dir.

• dumper (TypeVar(D, bound= Dumper)) –

  • If path is a dir containing a dumper conf file, this param will be an instance of Dumper equivalent to the one that was originally used to save the artifact.

  • Otherwise, it will be a Dumper initialized with empty params.

Return type:

Union[TypeVar(A, bound= object), dict]

Returns:

• An instance of Artifact.

• A dict that can be parsed into an Artifact (in case Artifact is a Pydantic model)

load(path, dry_run=False)

Load artifact from path.

As a side effect, this will use a Dumper to verify() the path. If path is a dir containing a dumper conf file, the used Dumper is equivalent to the one that was originally used to save the artifact. Otherwise, it’s a Dumper initialized with empty params.

Parameters:

• path (Path | str) – Path to artifact file or dir.

• dry_run (bool) – Just verify the artifact path.

Return type:

Union[TypeVar(A, bound= object), Path]

Returns:

• If dry_run is True, the same path.

• Otherwise, an instance of Artifact.

property Artifact: Type[A]

Artifact type used by this loader.

Returns:

Type of A

property Dumper: Type[D]

Dumper type used by this loader.

Returns:

Type of D

pydantic model mlopus.artschema.Schema

Bases: BaseModel, BaseParamsMixin, Generic[A, D, L]

Base class for artifact schemas.

Serves for putting together the types of Artifact, Dumper and Loader.

Example:

class Artifact(pydantic.BaseModel):
    some_data: dict[str, str]


class Dumper(mlopus.artschema.Dumper[Artifact]):
    encoding: str = "UTF-8"

    def _dump(self, path: Path, artifact: A) -> None:
        # save `artifact.some_data` inside `path` using `self.encoding`


class Loader(mlopus.artschema.Loader[Artifact, Dumper]):
    max_files: int | None = None

    def _load(self, path: Path, dumper: Dumper) -> Artifact:
        # load instance of `Artifact` from `path` using `self.max_files` and `dumper.encoding`


class Schema(mlopus.artschema.Schema[Artifact, Dumper, Loader]):
    pass  # No methods needed here, but the type params are important!


# Instantiate
artifact = Artifact(some_data={...})

# Dump
dumper = Schema().get_dumper(artifact, encoding="...")
dumper(path)

# Load
loader = Schema().get_loader(max_files=3)
loader(path)  # Returns: Artifact

# Combine with MlflowApi
with mlopus.mlflow \
    .get_api(...) \
    .get_exp(...) \
    .start_run(...):

    run.log_artifact(dumper, path_in_run="foobar")

run.load_artifact(loader, path_in_run="foobar")
# Same applies when using `log_model_version` and `load_model_artifact`

get_dumper(artifact, dumper=None, **dumper_kwargs)

Get a dumper callback.

Parameters:

• artifact (Union[TypeVar(A, bound= object), dict, Path]) –

  • An instance of Artifact

  • A Path to a file or dir containing a pre-dumped Artifact

  • A dict that can be parsed into an Artifact (in case Artifact is a Pydantic model)

• dumper (Union[TypeVar(D, bound= Dumper), dict, None]) –

  Custom Dumper configuration. Defaults to an empty dict.

  • An instance of Dumper

  • A dict that can be parsed into a Dumper

• dumper_kwargs –

  Keyword arguments for instantiating a Dumper.

  Incompatible with the dumper param.

Return type:

Union[Callable[[Path], None], Path]

Returns:

• If artifact is a Path: The same Path after being verified by the configured Dumper.

• Otherwise: A callback that accepts a Path and uses the configured Dumper to dump the provided Artifact on it.

get_loader(loader=None, dry_run=False, **loader_kwargs)

Get a loader callback.

Parameters:

• loader (Union[TypeVar(L, bound= Loader), dict, None]) –

  Custom Loader configuration. Defaults to an empty dict.

  • An instance of Loader

  • A dict that can be parsed into a Loader

• loader_kwargs –

  Keyword arguments for instantiating a Loader.

  Incompatible with the loader param.

• dry_run (bool) –

  See Loader.load.dry_run.

Return type:

Callable[[Path], Union[TypeVar(A, bound= object), Path]]

Returns:

• If dry_run is True: A callback that accepts a Path, verifies it and returns it.

• Otherwise: A callback that accepts a Path and uses the configured Loader to load and return an Artifact

property Artifact: Type[A]

Artifact type used by this schema.

Returns:

Type of A

property Dumper: Type[D]

Dumper type used by this schema.

Returns:

Type of D

property Loader: Type[L]

Loader type used by this schema.

Returns:

Type of L