Artifacts Catalog

pydantic model mlopus.artschema.ArtifactsCatalog

Bases: BaseModel

Base class for artifact catalogs.

Useful for type-safe loading/downloading/exporting artifacts based on parsed application settings.

Example settings:

foo:
    schema: package.module:Schema  # Schema specified explicitly by fully qualified class name
    subject:
        run_id: 12345678
        path_in_run: foo
bar:
    schema: default  # Schema obtained by alias from model version tags or parent model tags
    subject:
        model_name: foo
        model_version: 3

Example usage:

# Load the YAML settings above
artifact_specs: dict = ...

# Declare an artifact catalog
class ArtifactsCatalog(mlopus.artschema.ArtifactsCatalog):
    foo: FooArtifact
    bar: BarArtifact

# Cache all artifacts and metadata and verify their files using the specified schemas
ArtifactsCatalog.download(mlflow_api, artifact_specs)

# Load all artifacts using the specified schemas
artifacts_catalog = ArtifactsCatalog.load(mlflow_api, artifact_specs)

artifacts_catalog.foo  # `FooArtifact`
artifacts_catalog.bar  # `BarArtifact`

In the example above, artifact_specs is implicitly parsed into a mapping of str to LoadArtifactSpec, while the subject values of foo and bar are parsed into RunArtifact and ModelVersionArtifact, respectively.

classmethod download(cls, mlflow_api, artifact_specs, verify=True)

Cache artifacts and metadata and verify the files against the schemas.

See also

• LoadArtifactSpec.download()

Parameters:

• mlflow_api (BaseMlflowApi) –

• artifact_specs (Dict[str, LoadArtifactSpec]) –

• verify (bool) –

  Use schemas for verification after download.

  See verify().

Return type:

Dict[str, Path]

classmethod export(cls, mlflow_api, artifact_specs, target, verify=True)

Export artifacts and metadata caches while preserving cache structure.

See also

• LoadArtifactSpec.export()

Parameters:

• mlflow_api (BaseMlflowApi) –

• artifact_specs (Dict[str, LoadArtifactSpec]) –

• target (Path | str) – Cache export target path.

• verify (bool) –

  Use schemas for verification after export.

  See verify().

Return type:

None

classmethod load(cls, mlflow_api, artifact_specs)

Load artifacts from specs using their respective schemas.

See also

• LoadArtifactSpec.load()

Parameters:

• mlflow_api (BaseMlflowApi) –

• artifact_specs (Dict[str, LoadArtifactSpec]) –

classmethod verify(mlflow_api, artifact_specs)

Validate the artifact specs against this catalog and Python environment.

New in version 1.4.

• Assert that the Python package requirements are met for each artifact schema in the specs.

• Assert that the return type of each artifact schema matches the expected type in the catalog.

Parameters:

• mlflow_api (BaseMlflowApi) –

• artifact_specs (Dict[str, LoadArtifactSpec]) –

Return type:

None

pydantic model mlopus.artschema.ArtifactsCatalogWithMetadata

Bases: ArtifactsCatalog

Variant of ArtifactsCatalog with added metadata to each field.

A condition for usage is that every field must be typed as LoadedArtifact[T], were T is the actual artifact type returned by the configured artifact schema.

New in version 1.4.

Example usage:

# Declare an artifact catalog
class ArtifactsCatalog(mlopus.artschema.ArtifactsCatalogWithMetadata):
    foo: LoadedArtifact[Foo]
    bar: LoadedArtifact[Bar]

# Load all artifacts using the specified schemas
artifacts_catalog = ArtifactsCatalog.load(mlflow_api, artifact_specs)

artifacts_catalog.foo.value  # `Foo(...)` instance
artifacts_catalog.foo.meta.run.params  # Params of the artifact's source run on MLFlow

pydantic model mlopus.artschema.LoadedArtifact

Bases: BaseModel, Generic[T]

Loaded artifact with metadata.

New in version 1.4.

field meta: LoadArtifactSpec [Required]

field value: TypeVar(T) [Required]