feast.infra package
Subpackages
- feast.infra.offline_stores package
- Subpackages
- feast.infra.offline_stores.contrib package
- Subpackages
- Submodules
- feast.infra.offline_stores.contrib.athena_repo_configuration module
- feast.infra.offline_stores.contrib.mssql_repo_configuration module
- feast.infra.offline_stores.contrib.postgres_repo_configuration module
- feast.infra.offline_stores.contrib.spark_repo_configuration module
- feast.infra.offline_stores.contrib.trino_repo_configuration module
- Module contents
- feast.infra.offline_stores.contrib package
- Submodules
- feast.infra.offline_stores.bigquery module
BigQueryOfflineStore
BigQueryOfflineStoreConfig
BigQueryRetrievalJob
BigQueryRetrievalJob.full_feature_names
BigQueryRetrievalJob.metadata
BigQueryRetrievalJob.on_demand_feature_views
BigQueryRetrievalJob.persist()
BigQueryRetrievalJob.supports_remote_storage_export()
BigQueryRetrievalJob.to_bigquery()
BigQueryRetrievalJob.to_remote_storage()
BigQueryRetrievalJob.to_sql()
arrow_schema_to_bq_schema()
block_until_done()
get_http_client_info()
- feast.infra.offline_stores.bigquery_source module
BigQueryLoggingDestination
BigQueryOptions
BigQuerySource
BigQuerySource.created_timestamp_column
BigQuerySource.date_partition_column
BigQuerySource.description
BigQuerySource.field_mapping
BigQuerySource.from_proto()
BigQuerySource.get_table_column_names_and_types()
BigQuerySource.get_table_query_string()
BigQuerySource.name
BigQuerySource.owner
BigQuerySource.query
BigQuerySource.source_datatype_to_feast_value_type()
BigQuerySource.table
BigQuerySource.tags
BigQuerySource.timestamp_field
BigQuerySource.to_proto()
BigQuerySource.validate()
SavedDatasetBigQueryStorage
- feast.infra.offline_stores.file module
- feast.infra.offline_stores.file_source module
FileLoggingDestination
FileOptions
FileSource
FileSource.create_filesystem_and_path()
FileSource.created_timestamp_column
FileSource.date_partition_column
FileSource.description
FileSource.field_mapping
FileSource.file_format
FileSource.from_proto()
FileSource.get_table_column_names_and_types()
FileSource.get_table_query_string()
FileSource.name
FileSource.owner
FileSource.path
FileSource.s3_endpoint_override
FileSource.source_datatype_to_feast_value_type()
FileSource.tags
FileSource.timestamp_field
FileSource.to_proto()
FileSource.validate()
SavedDatasetFileStorage
- feast.infra.offline_stores.offline_store module
- feast.infra.offline_stores.offline_utils module
FeatureViewQueryContext
FeatureViewQueryContext.created_timestamp_column
FeatureViewQueryContext.date_partition_column
FeatureViewQueryContext.entities
FeatureViewQueryContext.entity_selections
FeatureViewQueryContext.features
FeatureViewQueryContext.field_mapping
FeatureViewQueryContext.max_event_timestamp
FeatureViewQueryContext.min_event_timestamp
FeatureViewQueryContext.name
FeatureViewQueryContext.table_subquery
FeatureViewQueryContext.timestamp_field
FeatureViewQueryContext.ttl
assert_expected_columns_in_entity_df()
build_point_in_time_query()
get_entity_df_timestamp_bounds()
get_expected_join_keys()
get_feature_view_query_context()
get_offline_store_from_config()
get_pyarrow_schema_from_batch_source()
get_temp_entity_table_name()
infer_event_timestamp_from_entity_df()
- feast.infra.offline_stores.redshift module
RedshiftOfflineStore
RedshiftOfflineStoreConfig
RedshiftOfflineStoreConfig.cluster_id
RedshiftOfflineStoreConfig.database
RedshiftOfflineStoreConfig.iam_role
RedshiftOfflineStoreConfig.region
RedshiftOfflineStoreConfig.require_cluster_and_user_or_workgroup()
RedshiftOfflineStoreConfig.s3_staging_location
RedshiftOfflineStoreConfig.type
RedshiftOfflineStoreConfig.user
RedshiftOfflineStoreConfig.workgroup
RedshiftRetrievalJob
RedshiftRetrievalJob.full_feature_names
RedshiftRetrievalJob.metadata
RedshiftRetrievalJob.on_demand_feature_views
RedshiftRetrievalJob.persist()
RedshiftRetrievalJob.supports_remote_storage_export()
RedshiftRetrievalJob.to_redshift()
RedshiftRetrievalJob.to_remote_storage()
RedshiftRetrievalJob.to_s3()
- feast.infra.offline_stores.redshift_source module
RedshiftLoggingDestination
RedshiftOptions
RedshiftSource
RedshiftSource.created_timestamp_column
RedshiftSource.database
RedshiftSource.date_partition_column
RedshiftSource.description
RedshiftSource.field_mapping
RedshiftSource.from_proto()
RedshiftSource.get_table_column_names_and_types()
RedshiftSource.get_table_query_string()
RedshiftSource.name
RedshiftSource.owner
RedshiftSource.query
RedshiftSource.schema
RedshiftSource.source_datatype_to_feast_value_type()
RedshiftSource.table
RedshiftSource.tags
RedshiftSource.timestamp_field
RedshiftSource.to_proto()
RedshiftSource.validate()
SavedDatasetRedshiftStorage
- feast.infra.offline_stores.snowflake module
SnowflakeOfflineStore
SnowflakeOfflineStoreConfig
SnowflakeOfflineStoreConfig.Config
SnowflakeOfflineStoreConfig.account
SnowflakeOfflineStoreConfig.authenticator
SnowflakeOfflineStoreConfig.blob_export_location
SnowflakeOfflineStoreConfig.config_path
SnowflakeOfflineStoreConfig.convert_timestamp_columns
SnowflakeOfflineStoreConfig.database
SnowflakeOfflineStoreConfig.password
SnowflakeOfflineStoreConfig.role
SnowflakeOfflineStoreConfig.schema_
SnowflakeOfflineStoreConfig.storage_integration_name
SnowflakeOfflineStoreConfig.type
SnowflakeOfflineStoreConfig.user
SnowflakeOfflineStoreConfig.warehouse
SnowflakeRetrievalJob
SnowflakeRetrievalJob.full_feature_names
SnowflakeRetrievalJob.metadata
SnowflakeRetrievalJob.on_demand_feature_views
SnowflakeRetrievalJob.persist()
SnowflakeRetrievalJob.supports_remote_storage_export()
SnowflakeRetrievalJob.to_arrow_batches()
SnowflakeRetrievalJob.to_pandas_batches()
SnowflakeRetrievalJob.to_remote_storage()
SnowflakeRetrievalJob.to_snowflake()
SnowflakeRetrievalJob.to_spark_df()
SnowflakeRetrievalJob.to_sql()
- feast.infra.offline_stores.snowflake_source module
SavedDatasetSnowflakeStorage
SnowflakeLoggingDestination
SnowflakeOptions
SnowflakeSource
SnowflakeSource.created_timestamp_column
SnowflakeSource.database
SnowflakeSource.date_partition_column
SnowflakeSource.description
SnowflakeSource.field_mapping
SnowflakeSource.from_proto()
SnowflakeSource.get_table_column_names_and_types()
SnowflakeSource.get_table_query_string()
SnowflakeSource.name
SnowflakeSource.owner
SnowflakeSource.query
SnowflakeSource.schema
SnowflakeSource.source_datatype_to_feast_value_type()
SnowflakeSource.table
SnowflakeSource.tags
SnowflakeSource.timestamp_field
SnowflakeSource.to_proto()
SnowflakeSource.validate()
- Module contents
- Subpackages
- feast.infra.online_stores package
- Subpackages
- feast.infra.online_stores.contrib package
- Subpackages
- Submodules
- feast.infra.online_stores.contrib.cassandra_repo_configuration module
- feast.infra.online_stores.contrib.hazelcast_repo_configuration module
- feast.infra.online_stores.contrib.hbase_repo_configuration module
- feast.infra.online_stores.contrib.mysql_repo_configuration module
- feast.infra.online_stores.contrib.postgres module
- feast.infra.online_stores.contrib.postgres_repo_configuration module
- feast.infra.online_stores.contrib.rockset_repo_configuration module
- Module contents
- feast.infra.online_stores.contrib package
- Submodules
- feast.infra.online_stores.bigtable module
- feast.infra.online_stores.datastore module
DatastoreOnlineStore
DatastoreOnlineStoreConfig
DatastoreTable
DatastoreTable.project
DatastoreTable.name
DatastoreTable.project_id
DatastoreTable.namespace
DatastoreTable.from_infra_object_proto()
DatastoreTable.from_proto()
DatastoreTable.namespace
DatastoreTable.project
DatastoreTable.project_id
DatastoreTable.teardown()
DatastoreTable.to_infra_object_proto()
DatastoreTable.to_proto()
DatastoreTable.update()
get_http_client_info()
- feast.infra.online_stores.dynamodb module
DynamoDBOnlineStore
DynamoDBOnlineStoreConfig
DynamoDBTable
DynamoDBTable.name
DynamoDBTable.region
DynamoDBTable.endpoint_url
DynamoDBTable._dynamodb_client
DynamoDBTable._dynamodb_resource
DynamoDBTable.endpoint_url
DynamoDBTable.from_infra_object_proto()
DynamoDBTable.from_proto()
DynamoDBTable.region
DynamoDBTable.teardown()
DynamoDBTable.to_infra_object_proto()
DynamoDBTable.to_proto()
DynamoDBTable.update()
- feast.infra.online_stores.helpers module
- feast.infra.online_stores.online_store module
- feast.infra.online_stores.redis module
- feast.infra.online_stores.snowflake module
SnowflakeOnlineStore
SnowflakeOnlineStoreConfig
SnowflakeOnlineStoreConfig.Config
SnowflakeOnlineStoreConfig.account
SnowflakeOnlineStoreConfig.authenticator
SnowflakeOnlineStoreConfig.config_path
SnowflakeOnlineStoreConfig.database
SnowflakeOnlineStoreConfig.password
SnowflakeOnlineStoreConfig.role
SnowflakeOnlineStoreConfig.schema_
SnowflakeOnlineStoreConfig.type
SnowflakeOnlineStoreConfig.user
SnowflakeOnlineStoreConfig.warehouse
- feast.infra.online_stores.sqlite module
- Module contents
- Subpackages
- feast.infra.registry package
- Submodules
- feast.infra.registry.base_registry module
BaseRegistry
BaseRegistry.apply_data_source()
BaseRegistry.apply_entity()
BaseRegistry.apply_feature_service()
BaseRegistry.apply_feature_view()
BaseRegistry.apply_materialization()
BaseRegistry.apply_saved_dataset()
BaseRegistry.apply_user_metadata()
BaseRegistry.apply_validation_reference()
BaseRegistry.commit()
BaseRegistry.delete_data_source()
BaseRegistry.delete_entity()
BaseRegistry.delete_feature_service()
BaseRegistry.delete_feature_view()
BaseRegistry.delete_saved_dataset()
BaseRegistry.delete_validation_reference()
BaseRegistry.get_data_source()
BaseRegistry.get_entity()
BaseRegistry.get_feature_service()
BaseRegistry.get_feature_view()
BaseRegistry.get_infra()
BaseRegistry.get_on_demand_feature_view()
BaseRegistry.get_request_feature_view()
BaseRegistry.get_saved_dataset()
BaseRegistry.get_stream_feature_view()
BaseRegistry.get_user_metadata()
BaseRegistry.get_validation_reference()
BaseRegistry.list_data_sources()
BaseRegistry.list_entities()
BaseRegistry.list_feature_services()
BaseRegistry.list_feature_views()
BaseRegistry.list_on_demand_feature_views()
BaseRegistry.list_project_metadata()
BaseRegistry.list_request_feature_views()
BaseRegistry.list_saved_datasets()
BaseRegistry.list_stream_feature_views()
BaseRegistry.list_validation_references()
BaseRegistry.proto()
BaseRegistry.refresh()
BaseRegistry.to_dict()
BaseRegistry.update_infra()
- feast.infra.registry.file module
- feast.infra.registry.gcs module
- feast.infra.registry.registry module
FeastObjectType
FeastObjectType.DATA_SOURCE
FeastObjectType.ENTITY
FeastObjectType.FEATURE_SERVICE
FeastObjectType.FEATURE_VIEW
FeastObjectType.ON_DEMAND_FEATURE_VIEW
FeastObjectType.REQUEST_FEATURE_VIEW
FeastObjectType.STREAM_FEATURE_VIEW
FeastObjectType.get_objects_from_registry()
FeastObjectType.get_objects_from_repo_contents()
Registry
Registry.apply_data_source()
Registry.apply_entity()
Registry.apply_feature_service()
Registry.apply_feature_view()
Registry.apply_materialization()
Registry.apply_saved_dataset()
Registry.apply_user_metadata()
Registry.apply_validation_reference()
Registry.cached_registry_proto
Registry.cached_registry_proto_created
Registry.cached_registry_proto_ttl
Registry.clone()
Registry.commit()
Registry.delete_data_source()
Registry.delete_entity()
Registry.delete_feature_service()
Registry.delete_feature_view()
Registry.delete_validation_reference()
Registry.get_data_source()
Registry.get_entity()
Registry.get_feature_service()
Registry.get_feature_view()
Registry.get_infra()
Registry.get_on_demand_feature_view()
Registry.get_request_feature_view()
Registry.get_saved_dataset()
Registry.get_stream_feature_view()
Registry.get_user_metadata()
Registry.get_validation_reference()
Registry.list_data_sources()
Registry.list_entities()
Registry.list_feature_services()
Registry.list_feature_views()
Registry.list_on_demand_feature_views()
Registry.list_project_metadata()
Registry.list_request_feature_views()
Registry.list_saved_datasets()
Registry.list_stream_feature_views()
Registry.list_validation_references()
Registry.proto()
Registry.refresh()
Registry.teardown()
Registry.update_infra()
get_registry_store_class_from_scheme()
get_registry_store_class_from_type()
- feast.infra.registry.registry_store module
- feast.infra.registry.s3 module
- feast.infra.registry.sql module
FeastMetadataKeys
SqlRegistry
SqlRegistry.apply_data_source()
SqlRegistry.apply_entity()
SqlRegistry.apply_feature_service()
SqlRegistry.apply_feature_view()
SqlRegistry.apply_materialization()
SqlRegistry.apply_saved_dataset()
SqlRegistry.apply_user_metadata()
SqlRegistry.apply_validation_reference()
SqlRegistry.commit()
SqlRegistry.delete_data_source()
SqlRegistry.delete_entity()
SqlRegistry.delete_feature_service()
SqlRegistry.delete_feature_view()
SqlRegistry.delete_validation_reference()
SqlRegistry.get_data_source()
SqlRegistry.get_entity()
SqlRegistry.get_feature_service()
SqlRegistry.get_feature_view()
SqlRegistry.get_infra()
SqlRegistry.get_on_demand_feature_view()
SqlRegistry.get_request_feature_view()
SqlRegistry.get_saved_dataset()
SqlRegistry.get_stream_feature_view()
SqlRegistry.get_user_metadata()
SqlRegistry.get_validation_reference()
SqlRegistry.list_data_sources()
SqlRegistry.list_entities()
SqlRegistry.list_feature_services()
SqlRegistry.list_feature_views()
SqlRegistry.list_on_demand_feature_views()
SqlRegistry.list_project_metadata()
SqlRegistry.list_request_feature_views()
SqlRegistry.list_saved_datasets()
SqlRegistry.list_stream_feature_views()
SqlRegistry.list_validation_references()
SqlRegistry.proto()
SqlRegistry.refresh()
SqlRegistry.teardown()
SqlRegistry.update_infra()
SqlRegistryConfig
- Module contents
- feast.infra.transformation_servers package
- feast.infra.utils package
- Subpackages
- Submodules
- feast.infra.utils.aws_utils module
AthenaError
AthenaQueryError
AthenaStatementNotFinishedError
AthenaTableNameTooLong
RedshiftStatementNotFinishedError
delete_api_gateway()
delete_lambda_function()
delete_redshift_table()
delete_s3_directory()
download_s3_directory()
drop_temp_table()
execute_athena_query()
execute_athena_query_and_unload_to_s3()
execute_athena_query_async()
execute_redshift_query_and_unload_to_s3()
execute_redshift_statement()
execute_redshift_statement_async()
get_account_id()
get_athena_data_client()
get_athena_query_result()
get_bucket_and_key()
get_first_api_gateway()
get_lambda_function()
get_redshift_data_client()
get_redshift_statement_result()
get_s3_resource()
list_s3_files()
temporarily_upload_arrow_table_to_redshift()
temporarily_upload_df_to_redshift()
unload_athena_query_to_df()
unload_athena_query_to_pa()
unload_redshift_query_to_df()
unload_redshift_query_to_pa()
update_lambda_function_environment()
upload_arrow_table_to_athena()
upload_arrow_table_to_redshift()
upload_df_to_athena()
upload_df_to_redshift()
upload_df_to_s3()
wait_for_athena_execution()
wait_for_redshift_statement()
- feast.infra.utils.hbase_utils module
HbaseConstants
HbaseUtils
HbaseUtils.conn
HbaseUtils.host
HbaseUtils.port
HbaseUtils.timeout
HbaseUtils.batch()
HbaseUtils.check_if_table_exist()
HbaseUtils.close_conn()
HbaseUtils.create_table()
HbaseUtils.create_table_with_default_cf()
HbaseUtils.delete_table()
HbaseUtils.print_table()
HbaseUtils.put()
HbaseUtils.row()
HbaseUtils.rows()
main()
- Module contents
Submodules
feast.infra.aws module
- class feast.infra.aws.AwsProvider(config: RepoConfig)[source]
Bases:
PassthroughProvider
- get_feature_server_endpoint() str | None [source]
Returns endpoint for the feature server, if it exists.
- teardown_infra(project: str, tables: Sequence[FeatureView], entities: Sequence[Entity]) None [source]
Tears down all cloud resources for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
tables – Feature views whose corresponding infrastructure should be deleted.
entities – Entities whose corresponding infrastructure should be deleted.
- update_infra(project: str, tables_to_delete: Sequence[FeatureView], tables_to_keep: Sequence[FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity], partial: bool)[source]
Reconciles cloud resources with the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
tables_to_delete – Feature views whose corresponding infrastructure should be deleted.
tables_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.
entities_to_delete – Entities whose corresponding infrastructure should be deleted.
entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.
partial – If true, tables_to_delete and tables_to_keep are not exhaustive lists, so infrastructure corresponding to other feature views should be not be touched.
feast.infra.gcp module
- class feast.infra.gcp.GcpProvider(config: RepoConfig)[source]
Bases:
PassthroughProvider
This class only exists for backwards compatibility.
feast.infra.infra_object module
- class feast.infra.infra_object.Infra(infra_objects: ~typing.List[~feast.infra.infra_object.InfraObject] = <factory>)[source]
Bases:
object
Represents the set of infrastructure managed by Feast.
- Parameters:
infra_objects – A list of InfraObjects, each representing one infrastructure object.
- classmethod from_proto(infra_proto: Infra)[source]
Returns an Infra object created from a protobuf representation.
- infra_objects: List[InfraObject]
- class feast.infra.infra_object.InfraObject(name: str)[source]
Bases:
ABC
Represents a single infrastructure object (e.g. online store table) managed by Feast.
- abstract static from_infra_object_proto(infra_object_proto: InfraObject) Any [source]
Returns an InfraObject created from a protobuf representation.
- Parameters:
infra_object_proto – A protobuf representation of an InfraObject.
- Raises:
FeastInvalidInfraObjectType – The type of InfraObject could not be identified.
- static from_proto(infra_object_proto: Any) Any [source]
Converts a protobuf representation of a subclass to an object of that subclass.
- Parameters:
infra_object_proto – A protobuf representation of an InfraObject.
- Raises:
FeastInvalidInfraObjectType – The type of InfraObject could not be identified.
feast.infra.key_encoding_utils module
- feast.infra.key_encoding_utils.serialize_entity_key(entity_key: EntityKey, entity_key_serialization_version=1) bytes [source]
Serialize entity key to a bytestring so it can be used as a lookup key in a hash table.
We need this encoding to be stable; therefore we cannot just use protobuf serialization here since it does not guarantee that two proto messages containing the same data will serialize to the same byte string[1].
[1] https://developers.google.com/protocol-buffers/docs/encoding
- feast.infra.key_encoding_utils.serialize_entity_key_prefix(entity_keys: List[str]) bytes [source]
Serialize keys to a bytestring, so it can be used to prefix-scan through items stored in the online store using serialize_entity_key.
This encoding is a partial implementation of serialize_entity_key, only operating on the keys of entities, and not the values.
feast.infra.local module
- class feast.infra.local.LocalProvider(config: RepoConfig)[source]
Bases:
PassthroughProvider
This class only exists for backwards compatibility.
- plan_infra(config: RepoConfig, desired_registry_proto: Registry) Infra [source]
Returns the Infra required to support the desired registry.
- Parameters:
config – The RepoConfig for the current FeatureStore.
desired_registry_proto – The desired registry, in proto form.
feast.infra.passthrough_provider module
- class feast.infra.passthrough_provider.PassthroughProvider(config: RepoConfig)[source]
Bases:
Provider
The passthrough provider delegates all operations to the underlying online and offline stores.
- property batch_engine: BatchMaterializationEngine
- get_historical_features(config: RepoConfig, feature_views: List[FeatureView], feature_refs: List[str], entity_df: DataFrame | str, registry: BaseRegistry, project: str, full_feature_names: bool) RetrievalJob [source]
Retrieves the point-in-time correct historical feature values for the specified entity rows.
- Parameters:
config – The config for the current feature store.
feature_views – A list containing all feature views that are referenced in the entity rows.
feature_refs – The features to be retrieved.
entity_df – A collection of rows containing all entity columns on which features need to be joined, as well as the timestamp column used for point-in-time joins. Either a pandas dataframe can be provided or a SQL query.
registry – The registry for the current feature store.
project – Feast project to which the feature views belong.
full_feature_names – If True, feature names will be prefixed with the corresponding feature view name, changing them from the format “feature” to “feature_view__feature” (e.g. “daily_transactions” changes to “customer_fv__daily_transactions”).
- Returns:
A RetrievalJob that can be executed to get the features.
- ingest_df(feature_view: FeatureView, df: DataFrame)[source]
Persists a dataframe to the online store.
- Parameters:
feature_view – The feature view to which the dataframe corresponds.
df – The dataframe to be persisted.
- ingest_df_to_offline_store(feature_view: FeatureView, table: Table)[source]
Persists a dataframe to the offline store.
- Parameters:
feature_view – The feature view to which the dataframe corresponds.
df – The dataframe to be persisted.
- materialize_single_feature_view(config: RepoConfig, feature_view: FeatureView, start_date: datetime, end_date: datetime, registry: BaseRegistry, project: str, tqdm_builder: Callable[[int], tqdm]) None [source]
Writes latest feature values in the specified time range to the online store.
- Parameters:
config – The config for the current feature store.
feature_view – The feature view to materialize.
start_date – The start of the time range.
end_date – The end of the time range.
registry – The registry for the current feature store.
project – Feast project to which the objects belong.
tqdm_builder – A function to monitor the progress of materialization.
- property offline_store
- offline_write_batch(config: RepoConfig, feature_view: FeatureView, data: Table, progress: Callable[[int], Any] | None) None [source]
- online_read(config: RepoConfig, table: FeatureView, entity_keys: List[EntityKey], requested_features: List[str] = None) List [source]
Reads features values for the given entity keys.
- Parameters:
config – The config for the current feature store.
table – The feature view whose feature values should be read.
entity_keys – The list of entity keys for which feature values should be read.
requested_features – The list of features that should be read.
- Returns:
A list of the same length as entity_keys. Each item in the list is a tuple where the first item is the event timestamp for the row, and the second item is a dict mapping feature names to values, which are returned in proto format.
- property online_store
- online_write_batch(config: RepoConfig, table: FeatureView, data: List[Tuple[EntityKey, Dict[str, Value], datetime, datetime | None]], progress: Callable[[int], Any] | None) None [source]
Writes a batch of feature rows to the online store.
If a tz-naive timestamp is passed to this method, it is assumed to be UTC.
- Parameters:
config – The config for the current feature store.
table – Feature view to which these feature rows correspond.
data – A list of quadruplets containing feature data. Each quadruplet contains an entity key, a dict containing feature values, an event timestamp for the row, and the created timestamp for the row if it exists.
progress – Function to be called once a batch of rows is written to the online store, used to show progress.
- retrieve_feature_service_logs(feature_service: FeatureService, start_date: datetime, end_date: datetime, config: RepoConfig, registry: BaseRegistry) RetrievalJob [source]
Reads logged features for the specified time window.
- Parameters:
feature_service – The feature service whose logs should be retrieved.
start_date – The start of the window.
end_date – The end of the window.
config – The config for the current feature store.
registry – The registry for the current feature store.
- Returns:
A RetrievalJob that can be executed to get the feature service logs.
- retrieve_saved_dataset(config: RepoConfig, dataset: SavedDataset) RetrievalJob [source]
Reads a saved dataset.
- Parameters:
config – The config for the current feature store.
dataset – A SavedDataset object containing all parameters necessary for retrieving the dataset.
- Returns:
A RetrievalJob that can be executed to get the saved dataset.
- teardown_infra(project: str, tables: Sequence[FeatureView], entities: Sequence[Entity]) None [source]
Tears down all cloud resources for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
tables – Feature views whose corresponding infrastructure should be deleted.
entities – Entities whose corresponding infrastructure should be deleted.
- update_infra(project: str, tables_to_delete: Sequence[FeatureView], tables_to_keep: Sequence[FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity], partial: bool)[source]
Reconciles cloud resources with the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
tables_to_delete – Feature views whose corresponding infrastructure should be deleted.
tables_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.
entities_to_delete – Entities whose corresponding infrastructure should be deleted.
entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.
partial – If true, tables_to_delete and tables_to_keep are not exhaustive lists, so infrastructure corresponding to other feature views should be not be touched.
- write_feature_service_logs(feature_service: FeatureService, logs: Table | str, config: RepoConfig, registry: BaseRegistry)[source]
Writes features and entities logged by a feature server to the offline store.
The schema of the logs table is inferred from the specified feature service. Only feature services with configured logging are accepted.
- Parameters:
feature_service – The feature service to be logged.
logs – The logs, either as an arrow table or as a path to a parquet directory.
config – The config for the current feature store.
registry – The registry for the current feature store.
feast.infra.provider module
- class feast.infra.provider.Provider(config: RepoConfig)[source]
Bases:
ABC
A provider defines an implementation of a feature store object. It orchestrates the various components of a feature store, such as the offline store, online store, and materialization engine. It is configured through a RepoConfig object.
- get_feature_server_endpoint() str | None [source]
Returns endpoint for the feature server, if it exists.
- abstract get_historical_features(config: RepoConfig, feature_views: List[FeatureView], feature_refs: List[str], entity_df: DataFrame | str, registry: BaseRegistry, project: str, full_feature_names: bool) RetrievalJob [source]
Retrieves the point-in-time correct historical feature values for the specified entity rows.
- Parameters:
config – The config for the current feature store.
feature_views – A list containing all feature views that are referenced in the entity rows.
feature_refs – The features to be retrieved.
entity_df – A collection of rows containing all entity columns on which features need to be joined, as well as the timestamp column used for point-in-time joins. Either a pandas dataframe can be provided or a SQL query.
registry – The registry for the current feature store.
project – Feast project to which the feature views belong.
full_feature_names – If True, feature names will be prefixed with the corresponding feature view name, changing them from the format “feature” to “feature_view__feature” (e.g. “daily_transactions” changes to “customer_fv__daily_transactions”).
- Returns:
A RetrievalJob that can be executed to get the features.
- ingest_df(feature_view: FeatureView, df: DataFrame)[source]
Persists a dataframe to the online store.
- Parameters:
feature_view – The feature view to which the dataframe corresponds.
df – The dataframe to be persisted.
- ingest_df_to_offline_store(feature_view: FeatureView, df: Table)[source]
Persists a dataframe to the offline store.
- Parameters:
feature_view – The feature view to which the dataframe corresponds.
df – The dataframe to be persisted.
- abstract materialize_single_feature_view(config: RepoConfig, feature_view: FeatureView, start_date: datetime, end_date: datetime, registry: BaseRegistry, project: str, tqdm_builder: Callable[[int], tqdm]) None [source]
Writes latest feature values in the specified time range to the online store.
- Parameters:
config – The config for the current feature store.
feature_view – The feature view to materialize.
start_date – The start of the time range.
end_date – The end of the time range.
registry – The registry for the current feature store.
project – Feast project to which the objects belong.
tqdm_builder – A function to monitor the progress of materialization.
- abstract online_read(config: RepoConfig, table: FeatureView, entity_keys: List[EntityKey], requested_features: List[str] | None = None) List[Tuple[datetime | None, Dict[str, Value] | None]] [source]
Reads features values for the given entity keys.
- Parameters:
config – The config for the current feature store.
table – The feature view whose feature values should be read.
entity_keys – The list of entity keys for which feature values should be read.
requested_features – The list of features that should be read.
- Returns:
A list of the same length as entity_keys. Each item in the list is a tuple where the first item is the event timestamp for the row, and the second item is a dict mapping feature names to values, which are returned in proto format.
- abstract online_write_batch(config: RepoConfig, table: FeatureView, data: List[Tuple[EntityKey, Dict[str, Value], datetime, datetime | None]], progress: Callable[[int], Any] | None) None [source]
Writes a batch of feature rows to the online store.
If a tz-naive timestamp is passed to this method, it is assumed to be UTC.
- Parameters:
config – The config for the current feature store.
table – Feature view to which these feature rows correspond.
data – A list of quadruplets containing feature data. Each quadruplet contains an entity key, a dict containing feature values, an event timestamp for the row, and the created timestamp for the row if it exists.
progress – Function to be called once a batch of rows is written to the online store, used to show progress.
- plan_infra(config: RepoConfig, desired_registry_proto: Registry) Infra [source]
Returns the Infra required to support the desired registry.
- Parameters:
config – The RepoConfig for the current FeatureStore.
desired_registry_proto – The desired registry, in proto form.
- abstract retrieve_feature_service_logs(feature_service: FeatureService, start_date: datetime, end_date: datetime, config: RepoConfig, registry: BaseRegistry) RetrievalJob [source]
Reads logged features for the specified time window.
- Parameters:
feature_service – The feature service whose logs should be retrieved.
start_date – The start of the window.
end_date – The end of the window.
config – The config for the current feature store.
registry – The registry for the current feature store.
- Returns:
A RetrievalJob that can be executed to get the feature service logs.
- abstract retrieve_saved_dataset(config: RepoConfig, dataset: SavedDataset) RetrievalJob [source]
Reads a saved dataset.
- Parameters:
config – The config for the current feature store.
dataset – A SavedDataset object containing all parameters necessary for retrieving the dataset.
- Returns:
A RetrievalJob that can be executed to get the saved dataset.
- abstract teardown_infra(project: str, tables: Sequence[FeatureView], entities: Sequence[Entity])[source]
Tears down all cloud resources for the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
tables – Feature views whose corresponding infrastructure should be deleted.
entities – Entities whose corresponding infrastructure should be deleted.
- abstract update_infra(project: str, tables_to_delete: Sequence[FeatureView], tables_to_keep: Sequence[FeatureView], entities_to_delete: Sequence[Entity], entities_to_keep: Sequence[Entity], partial: bool)[source]
Reconciles cloud resources with the specified set of Feast objects.
- Parameters:
project – Feast project to which the objects belong.
tables_to_delete – Feature views whose corresponding infrastructure should be deleted.
tables_to_keep – Feature views whose corresponding infrastructure should not be deleted, and may need to be updated.
entities_to_delete – Entities whose corresponding infrastructure should be deleted.
entities_to_keep – Entities whose corresponding infrastructure should not be deleted, and may need to be updated.
partial – If true, tables_to_delete and tables_to_keep are not exhaustive lists, so infrastructure corresponding to other feature views should be not be touched.
- abstract write_feature_service_logs(feature_service: FeatureService, logs: Table | Path, config: RepoConfig, registry: BaseRegistry)[source]
Writes features and entities logged by a feature server to the offline store.
The schema of the logs table is inferred from the specified feature service. Only feature services with configured logging are accepted.
- Parameters:
feature_service – The feature service to be logged.
logs – The logs, either as an arrow table or as a path to a parquet directory.
config – The config for the current feature store.
registry – The registry for the current feature store.
- feast.infra.provider.get_provider(config: RepoConfig) Provider [source]