feast.infra.materialization package
Subpackages
Submodules
feast.infra.materialization.batch_materialization_engine module
- class feast.infra.materialization.batch_materialization_engine.BatchMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]
Bases:
ABC
The interface that Feast uses to control the compute system that handles batch materialization.
- abstract materialize(registry: BaseRegistry, tasks: List[MaterializationTask]) List[MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo.
- Parameters:
registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.
- Returns:
A list of materialization jobs representing each task.
- abstract teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]
Tears down all cloud resources used by the materialization engine for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
fvs – Feature views whose corresponding infrastructure should be deleted.
entities – Entities whose corresponding infrastructure should be deleted.
- abstract update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]
Prepares cloud resources required for batch materialization for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
views_to_delete – Feature views whose corresponding infrastructure should be deleted.
views_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.
entities_to_delete – Entities whose corresponding infrastructure should be deleted.
entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.
- class feast.infra.materialization.batch_materialization_engine.MaterializationJob[source]
Bases:
ABC
A MaterializationJob represents an ongoing or executed process that materializes data as per the definition of a materialization task.
- abstract error() BaseException | None [source]
- abstract status() MaterializationJobStatus [source]
- task: MaterializationTask
- class feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus(value)[source]
Bases:
Enum
An enumeration.
- AVAILABLE = 3
- CANCELLED = 6
- CANCELLING = 5
- ERROR = 4
- RUNNING = 2
- SUCCEEDED = 7
- WAITING = 1
- class feast.infra.materialization.batch_materialization_engine.MaterializationTask(project: str, feature_view: BatchFeatureView | StreamFeatureView | FeatureView, start_time: datetime, end_time: datetime, tqdm_builder: Callable[[int], tqdm])[source]
Bases:
object
A MaterializationTask represents a unit of data that needs to be materialized from an offline store to an online store.
- feature_view: BatchFeatureView | StreamFeatureView | FeatureView
feast.infra.materialization.local_engine module
- class feast.infra.materialization.local_engine.LocalMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]
Bases:
BatchMaterializationEngine
- materialize(registry, tasks: List[MaterializationTask]) List[MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo.
- Parameters:
registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.
- Returns:
A list of materialization jobs representing each task.
- teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]
Tears down all cloud resources used by the materialization engine for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
fvs – Feature views whose corresponding infrastructure should be deleted.
entities – Entities whose corresponding infrastructure should be deleted.
- update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]
Prepares cloud resources required for batch materialization for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
views_to_delete – Feature views whose corresponding infrastructure should be deleted.
views_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.
entities_to_delete – Entities whose corresponding infrastructure should be deleted.
entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.
- class feast.infra.materialization.local_engine.LocalMaterializationEngineConfig(*, type: Literal['local'] = 'local')[source]
Bases:
FeastConfigBaseModel
Batch Materialization Engine config for local in-process engine
- class feast.infra.materialization.local_engine.LocalMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: BaseException | NoneType = None)[source]
Bases:
MaterializationJob
- error() BaseException | None [source]
- status() MaterializationJobStatus [source]
- task: MaterializationTask
feast.infra.materialization.snowflake_engine module
- class feast.infra.materialization.snowflake_engine.SnowflakeMaterializationEngine(*, repo_config: RepoConfig, offline_store: OfflineStore, online_store: OnlineStore, **kwargs)[source]
Bases:
BatchMaterializationEngine
- generate_snowflake_materialization_query(repo_config: RepoConfig, fv_latest_mapped_values_sql: str, feature_view: BatchFeatureView | FeatureView, project: str) str [source]
- materialize(registry, tasks: List[MaterializationTask]) List[MaterializationJob] [source]
Materialize data from the offline store to the online store for this feature repo.
- Parameters:
registry – The registry for the current feature store.
tasks – A list of individual materialization tasks.
- Returns:
A list of materialization jobs representing each task.
- materialize_to_external_online_store(repo_config: RepoConfig, materialization_sql: str, feature_view: StreamFeatureView | FeatureView, tqdm_builder: Callable[[int], tqdm]) None [source]
- materialize_to_snowflake_online_store(repo_config: RepoConfig, materialization_sql: str, feature_view: BatchFeatureView | FeatureView, project: str) None [source]
- teardown_infra(project: str, fvs: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities: Sequence[Entity])[source]
Tears down all cloud resources used by the materialization engine for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
fvs – Feature views whose corresponding infrastructure should be deleted.
entities – Entities whose corresponding infrastructure should be deleted.
- update(project: str, views_to_delete: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], views_to_keep: Sequence[BatchFeatureView | StreamFeatureView | FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity])[source]
Prepares cloud resources required for batch materialization for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
views_to_delete – Feature views whose corresponding infrastructure should be deleted.
views_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.
entities_to_delete – Entities whose corresponding infrastructure should be deleted.
entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.
- class feast.infra.materialization.snowflake_engine.SnowflakeMaterializationEngineConfig(*, type: Literal['snowflake.engine'] = 'snowflake.engine', config_path: str | None = '/home/docs/.snowsql/config', account: str | None = None, user: str | None = None, password: str | None = None, role: str | None = None, warehouse: str | None = None, authenticator: str | None = None, database: StrictStr, schema: str | None = 'PUBLIC')[source]
Bases:
FeastConfigBaseModel
Batch Materialization Engine config for Snowflake Snowpark Python UDFs
- database: StrictStr
Snowflake database name
- class feast.infra.materialization.snowflake_engine.SnowflakeMaterializationJob(job_id: str, status: feast.infra.materialization.batch_materialization_engine.MaterializationJobStatus, error: BaseException | NoneType = None)[source]
Bases:
MaterializationJob
- error() BaseException | None [source]
- status() MaterializationJobStatus [source]
- task: MaterializationTask