feast package
Subpackages
- feast.diff package
- feast.dqm package
- feast.infra package
- Subpackages
- feast.infra.offline_stores package
- Subpackages
- Submodules
- feast.infra.offline_stores.bigquery module
- feast.infra.offline_stores.bigquery_source module
- feast.infra.offline_stores.file module
- feast.infra.offline_stores.file_source module
- feast.infra.offline_stores.offline_store module
- feast.infra.offline_stores.offline_utils module
- feast.infra.offline_stores.redshift module
- feast.infra.offline_stores.redshift_source module
- feast.infra.offline_stores.snowflake module
- feast.infra.offline_stores.snowflake_source module
- Module contents
- feast.infra.online_stores package
- feast.infra.utils package
- feast.infra.offline_stores package
- Submodules
- feast.infra.aws module
- feast.infra.gcp module
- feast.infra.infra_object module
- feast.infra.key_encoding_utils module
- feast.infra.local module
- feast.infra.passthrough_provider module
- feast.infra.provider module
- Module contents
- Subpackages
- feast.loaders package
- feast.protos package
Submodules
feast.base_feature_view module
- class feast.base_feature_view.BaseFeatureView(*, name: str, features: Optional[List[feast.field.Field]] = None, description: str = '', tags: Optional[Dict[str, str]] = None, owner: str = '')[source]
Bases:
abc.ABC
A BaseFeatureView defines a logical group of features.
- features
The list of features defined as part of this base feature view.
- Type
List[feast.field.Field]
- projection
The feature view projection storing modifications to be applied to this base feature view at retrieval time.
- created_timestamp
The time when the base feature view was created.
- Type
optional
- last_updated_timestamp
The time when the base feature view was last updated.
- Type
optional
- created_timestamp: Optional[datetime.datetime]
- ensure_valid()[source]
Validates the state of this feature view locally.
- Raises
ValueError – The feature view is invalid.
- features: List[feast.field.Field]
- last_updated_timestamp: Optional[datetime.datetime]
- abstract property proto_class: Type[proto.message.Message]
- set_projection(feature_view_projection: feast.feature_view_projection.FeatureViewProjection) None [source]
Sets the feature view projection of this base feature view to the given projection.
- Parameters
feature_view_projection – The feature view projection to be set.
- Raises
ValueError – The name or features of the projection do not match.
- with_name(name: str)[source]
Returns a renamed copy of this base feature view. This renamed copy should only be used for query operations and will not modify the underlying base feature view.
- Parameters
name – The name to assign to the copy.
- with_projection(feature_view_projection: feast.feature_view_projection.FeatureViewProjection)[source]
Returns a copy of this base feature view with the feature view projection set to the given projection.
- Parameters
feature_view_projection – The feature view projection to assign to the copy.
- Raises
ValueError – The name or features of the projection do not match.
feast.cli module
- class feast.cli.NoOptionDefaultFormat(name: Optional[str], context_settings: Optional[Dict[str, Any]] = None, callback: Optional[Callable[[...], Any]] = None, params: Optional[List[click.core.Parameter]] = None, help: Optional[str] = None, epilog: Optional[str] = None, short_help: Optional[str] = None, options_metavar: Optional[str] = '[OPTIONS]', add_help_option: bool = True, no_args_is_help: bool = False, hidden: bool = False, deprecated: bool = False)[source]
Bases:
click.core.Command
feast.constants module
feast.data_format module
- class feast.data_format.AvroFormat(schema_json: str)[source]
Bases:
feast.data_format.StreamFormat
Defines the Avro streaming data format that encodes data in Avro format
- class feast.data_format.FileFormat[source]
Bases:
abc.ABC
Defines an abtract file forma used to encode feature data in files
- class feast.data_format.ParquetFormat[source]
Bases:
feast.data_format.FileFormat
Defines the Parquet data format
- class feast.data_format.ProtoFormat(class_path: str)[source]
Bases:
feast.data_format.StreamFormat
Defines the Protobuf data format
feast.data_source module
- class feast.data_source.DataSource(*, event_timestamp_column: Optional[str] = None, created_timestamp_column: Optional[str] = None, field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', name: Optional[str] = None, timestamp_field: Optional[str] = None)[source]
Bases:
abc.ABC
DataSource that can be used to source features.
- Parameters
name – Name of data source, which should be unique within a project
timestamp_field (optional) – (Deprecated) Event timestamp column used for point in time joins of feature values.
created_timestamp_column (optional) – Timestamp column indicating when the row was created, used for deduplicating rows.
field_mapping (optional) – A dictionary mapping of column names in this data source to feature names in a feature table or view. Only used for feature columns, not entity or timestamp columns.
date_partition_column (optional) – Timestamp column used for partitioning.
description (optional) –
tags (optional) – A dictionary of key-value pairs to store arbitrary metadata.
owner (optional) – The owner of the data source, typically the email of the primary maintainer.
timestamp_field – Event timestamp field used for point in time joins of feature values.
- abstract static from_proto(data_source: feast.core.DataSource_pb2.DataSource) Any [source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- abstract static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- abstract to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.data_source.KafkaOptions(bootstrap_servers: str, message_format: feast.data_format.StreamFormat, topic: str)[source]
Bases:
object
DataSource Kafka options used to source features from Kafka messages
- classmethod from_proto(kafka_options_proto: feast.core.DataSource_pb2.KafkaOptions)[source]
Creates a KafkaOptions from a protobuf representation of a kafka option
- Parameters
kafka_options_proto – A protobuf representation of a DataSource
- Returns
Returns a BigQueryOptions object based on the kafka_options protobuf
- class feast.data_source.KafkaSource(*args, name: Optional[str] = None, event_timestamp_column: Optional[str] = '', bootstrap_servers: Optional[str] = None, message_format: Optional[feast.data_format.StreamFormat] = None, topic: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = '', batch_source: Optional[feast.data_source.DataSource] = None)[source]
Bases:
feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.data_source.KinesisOptions(record_format: feast.data_format.StreamFormat, region: str, stream_name: str)[source]
Bases:
object
DataSource Kinesis options used to source features from Kinesis records
- classmethod from_proto(kinesis_options_proto: feast.core.DataSource_pb2.KinesisOptions)[source]
Creates a KinesisOptions from a protobuf representation of a kinesis option
- Parameters
kinesis_options_proto – A protobuf representation of a DataSource
- Returns
Returns a KinesisOptions object based on the kinesis_options protobuf
- class feast.data_source.KinesisSource(*args, name: Optional[str] = None, event_timestamp_column: Optional[str] = '', created_timestamp_column: Optional[str] = '', record_format: Optional[feast.data_format.StreamFormat] = None, region: Optional[str] = '', stream_name: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = '', batch_source: Optional[feast.data_source.DataSource] = None)[source]
Bases:
feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.data_source.PushSource(*args, name: Optional[str] = None, batch_source: Optional[feast.data_source.DataSource] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
A source that can be used to ingest features on request
- batch_source: feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.data_source.RequestDataSource(*args, **kwargs)[source]
Bases:
feast.data_source.RequestSource
- schema: List[feast.field.Field]
- class feast.data_source.RequestSource(*args, name: Optional[str] = None, schema: Optional[Union[Dict[str, feast.value_type.ValueType], List[feast.field.Field]]] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
RequestSource that can be used to provide input features for on demand transforms
- schema
Schema mapping from the input feature name to a ValueType
- Type
List[feast.field.Field]
- owner
The owner of the request data source, typically the email of the primary maintainer.
- Type
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- schema: List[feast.field.Field]
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
feast.driver_test_data module
- class feast.driver_test_data.EventTimestampType(value)[source]
Bases:
enum.Enum
An enumeration.
- TZ_AWARE_FIXED_OFFSET = 2
- TZ_AWARE_US_PACIFIC = 3
- TZ_AWARE_UTC = 1
- TZ_NAIVE = 0
- feast.driver_test_data.create_customer_daily_profile_df(customers, start_date, end_date) pandas.core.frame.DataFrame [source]
Example df generated by this function:
event_timestamp | customer_id | current_balance | avg_passenger_count | lifetime_trip_count | created ||------------------+-------------+-----------------+---------------------+---------------------+------------------| | 2021-03-17 19:31 | 1010 | 0.889188 | 0.049057 | 412 | 2021-03-24 19:38 | | 2021-03-18 19:31 | 1010 | 0.979273 | 0.212630 | 639 | 2021-03-24 19:38 | | 2021-03-19 19:31 | 1010 | 0.976549 | 0.176881 | 70 | 2021-03-24 19:38 | | 2021-03-20 19:31 | 1010 | 0.273697 | 0.325012 | 68 | 2021-03-24 19:38 | | 2021-03-21 19:31 | 1010 | 0.438262 | 0.313009 | 192 | 2021-03-24 19:38 | | | … | … | … | … | | | 2021-03-19 19:31 | 1001 | 0.738860 | 0.857422 | 344 | 2021-03-24 19:38 | | 2021-03-20 19:31 | 1001 | 0.848397 | 0.745989 | 106 | 2021-03-24 19:38 | | 2021-03-21 19:31 | 1001 | 0.301552 | 0.185873 | 812 | 2021-03-24 19:38 | | 2021-03-22 19:31 | 1001 | 0.943030 | 0.561219 | 322 | 2021-03-24 19:38 | | 2021-03-23 19:31 | 1001 | 0.354919 | 0.810093 | 273 | 2021-03-24 19:38 |
- feast.driver_test_data.create_driver_hourly_stats_df(drivers, start_date, end_date) pandas.core.frame.DataFrame [source]
Example df generated by this function:
event_timestamp | driver_id | conv_rate | acc_rate | avg_daily_trips | created ||------------------+-----------+-----------+----------+-----------------+------------------| | 2021-03-17 19:31 | 5010 | 0.229297 | 0.685843 | 861 | 2021-03-24 19:34 | | 2021-03-17 20:31 | 5010 | 0.781655 | 0.861280 | 769 | 2021-03-24 19:34 | | 2021-03-17 21:31 | 5010 | 0.150333 | 0.525581 | 778 | 2021-03-24 19:34 | | 2021-03-17 22:31 | 5010 | 0.951701 | 0.228883 | 570 | 2021-03-24 19:34 | | 2021-03-17 23:31 | 5010 | 0.819598 | 0.262503 | 473 | 2021-03-24 19:34 | | | … | … | … | … | | | 2021-03-24 16:31 | 5001 | 0.061585 | 0.658140 | 477 | 2021-03-24 19:34 | | 2021-03-24 17:31 | 5001 | 0.088949 | 0.303897 | 618 | 2021-03-24 19:34 | | 2021-03-24 18:31 | 5001 | 0.096652 | 0.747421 | 480 | 2021-03-24 19:34 | | 2021-03-17 19:31 | 5005 | 0.142936 | 0.707596 | 466 | 2021-03-24 19:34 | | 2021-03-17 19:31 | 5005 | 0.142936 | 0.707596 | 466 | 2021-03-24 19:34 |
- feast.driver_test_data.create_field_mapping_df(start_date, end_date) pandas.core.frame.DataFrame [source]
Example df generated by this function: | event_timestamp | column_name | created | |------------------+-------------+------------------| | 2021-03-17 19:00 | 99 | 2021-03-24 19:38 | | 2021-03-17 19:00 | 22 | 2021-03-24 19:38 | | 2021-03-17 19:00 | 7 | 2021-03-24 19:38 | | 2021-03-17 19:00 | 45 | 2021-03-24 19:38 |
- feast.driver_test_data.create_global_daily_stats_df(start_date, end_date) pandas.core.frame.DataFrame [source]
Example df generated by this function:
event_timestamp | num_rides | avg_ride_length | created ||------------------+-------------+-----------------+------------------| | 2021-03-17 19:00 | 99 | 0.889188 | 2021-03-24 19:38 | | 2021-03-18 19:00 | 52 | 0.979273 | 2021-03-24 19:38 | | 2021-03-19 19:00 | 66 | 0.976549 | 2021-03-24 19:38 | | 2021-03-20 19:00 | 84 | 0.273697 | 2021-03-24 19:38 | | 2021-03-21 19:00 | 89 | 0.438262 | 2021-03-24 19:38 | | | … | … | | | 2021-03-24 19:00 | 54 | 0.738860 | 2021-03-24 19:38 | | 2021-03-25 19:00 | 58 | 0.848397 | 2021-03-24 19:38 | | 2021-03-26 19:00 | 69 | 0.301552 | 2021-03-24 19:38 | | 2021-03-27 19:00 | 63 | 0.943030 | 2021-03-24 19:38 | | 2021-03-28 19:00 | 79 | 0.354919 | 2021-03-24 19:38 |
- feast.driver_test_data.create_location_stats_df(locations, start_date, end_date) pandas.core.frame.DataFrame [source]
Example df generated by this function:
event_timestamp | location_id | temperature | created |
- feast.driver_test_data.create_orders_df(customers, drivers, start_date, end_date, order_count, locations=None) pandas.core.frame.DataFrame [source]
Example df generated by this function (if locations):
order_id | driver_id | customer_id | origin_id | destination_id | order_is_success | event_timestamp |
feast.entity module
- class feast.entity.Entity(*args, name: Optional[str] = None, value_type: feast.value_type.ValueType = ValueType.UNKNOWN, description: str = '', join_key: Optional[str] = None, tags: Optional[Dict[str, str]] = None, owner: str = '', join_keys: Optional[List[str]] = None)[source]
Bases:
object
An entity defines a collection of entities for which features can be defined. An entity can also contain associated metadata.
- value_type
The type of the entity, such as string or float.
- join_key
A property that uniquely identifies different entities within the collection. The join_key property is typically used for joining entities with their associated features. If not specified, defaults to the name.
- Type
- created_timestamp
The time when the entity was created.
- Type
Optional[datetime.datetime]
- last_updated_timestamp
The time when the entity was last updated.
- Type
Optional[datetime.datetime]
- join_keys
A list of property that uniquely identifies different entities within the collection. This is meant to replace the join_key parameter, but currently only supports a list of size one.
- Type
List[str]
- created_timestamp: Optional[datetime.datetime]
- classmethod from_proto(entity_proto: feast.core.Entity_pb2.Entity)[source]
Creates an entity from a protobuf representation of an entity.
- Parameters
entity_proto – A protobuf representation of an entity.
- Returns
An Entity object based on the entity protobuf.
- is_valid()[source]
Validates the state of this entity locally.
- Raises
ValueError – The entity does not have a name or does not have a type.
- last_updated_timestamp: Optional[datetime.datetime]
- to_proto() feast.core.Entity_pb2.Entity [source]
Converts an entity object to its protobuf representation.
- Returns
An EntityProto protobuf.
- value_type: feast.value_type.ValueType
feast.errors module
- exception feast.errors.ConflictingFeatureViewNames(feature_view_name: str)[source]
Bases:
Exception
- exception feast.errors.EntityTimestampInferenceException(expected_column_name: str)[source]
Bases:
Exception
- exception feast.errors.ExperimentalFeatureNotEnabled(feature_flag_name: str)[source]
Bases:
Exception
- exception feast.errors.FeastClassImportError(module_name: str, class_name: str)[source]
Bases:
Exception
- exception feast.errors.FeastEntityDFMissingColumnsError(expected, missing)[source]
Bases:
Exception
- exception feast.errors.FeastExtrasDependencyImportError(extras_type: str, nested_error: str)[source]
Bases:
Exception
- exception feast.errors.FeastFeatureServerTypeInvalidError(feature_server_type: str)[source]
Bases:
Exception
- exception feast.errors.FeastFeatureServerTypeSetError(feature_server_type: str)[source]
Bases:
Exception
- exception feast.errors.FeastInvalidBaseClass(class_name: str, class_type: str)[source]
Bases:
Exception
- exception feast.errors.FeastJoinKeysDuringMaterialization(source: str, join_key_columns: Set[str], source_columns: Set[str])[source]
Bases:
Exception
- exception feast.errors.FeastModuleImportError(module_name: str, class_name: str)[source]
Bases:
Exception
- exception feast.errors.FeastOfflineStoreUnsupportedDataSource(offline_store_name: str, data_source_name: str)[source]
Bases:
Exception
- exception feast.errors.FeastOnlineStoreInvalidName(online_store_class_name: str)[source]
Bases:
Exception
- exception feast.errors.FeastOnlineStoreUnsupportedDataSource(online_store_name: str, data_source_name: str)[source]
Bases:
Exception
- exception feast.errors.FeastProviderLoginError[source]
Bases:
Exception
Error class that indicates a user has not authenticated with their provider.
- exception feast.errors.FeatureNameCollisionError(feature_refs_collisions: List[str], full_feature_names: bool)[source]
Bases:
Exception
- exception feast.errors.IncompatibleRegistryStoreClass(actual_class: str, expected_class: str)[source]
Bases:
Exception
- exception feast.errors.RegistryInferenceFailure(repo_obj_type: str, specific_issue: str)[source]
Bases:
Exception
- exception feast.errors.RequestDataNotFoundInEntityDfException(feature_name, feature_view_name)[source]
feast.feast_object module
feast.feature module
- class feast.feature.Feature(name: str, dtype: feast.value_type.ValueType, labels: Optional[Dict[str, str]] = None)[source]
Bases:
object
A Feature represents a class of serveable feature.
- Parameters
name – Name of the feature.
dtype – The type of the feature, such as string or float.
labels (optional) – User-defined metadata in dictionary form.
- property dtype: feast.value_type.ValueType
Gets the data type of this feature.
- classmethod from_proto(feature_proto: feast.core.Feature_pb2.FeatureSpecV2)[source]
- Parameters
feature_proto – FeatureSpecV2 protobuf object
- Returns
Feature object
- property name
Gets the name of this feature.
feast.feature_server module
- class feast.feature_server.PushFeaturesRequest(*, push_source_name: str, df: dict, allow_registry_cache: bool = True)[source]
Bases:
pydantic.main.BaseModel
- class feast.feature_server.WriteToFeatureStoreRequest(*, feature_view_name: str, df: dict, allow_registry_cache: bool = True)[source]
Bases:
pydantic.main.BaseModel
- feast.feature_server.get_app(store: feast.feature_store.FeatureStore)[source]
feast.feature_service module
- class feast.feature_service.FeatureService(*args, name: Optional[str] = None, features: Optional[List[Union[feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView]]] = None, tags: Dict[str, str] = None, description: str = '', owner: str = '')[source]
Bases:
object
A feature service defines a logical group of features from one or more feature views. This group of features can be retrieved together during training or serving.
- feature_view_projections
A list containing feature views and feature view projections, representing the features in the feature service.
- created_timestamp
The time when the feature service was created.
- Type
Optional[datetime.datetime]
- last_updated_timestamp
The time when the feature service was last updated.
- Type
Optional[datetime.datetime]
- created_timestamp: Optional[datetime.datetime] = None
- feature_view_projections: List[feast.feature_view_projection.FeatureViewProjection]
- classmethod from_proto(feature_service_proto: feast.core.FeatureService_pb2.FeatureService)[source]
Converts a FeatureServiceProto to a FeatureService object.
- Parameters
feature_service_proto – A protobuf representation of a FeatureService.
- last_updated_timestamp: Optional[datetime.datetime] = None
feast.feature_store module
- class feast.feature_store.FeatureStore(repo_path: Optional[str] = None, config: Optional[feast.repo_config.RepoConfig] = None)[source]
Bases:
object
A FeatureStore object is used to define, create, and retrieve features.
- Parameters
repo_path (optional) – Path to a feature_store.yaml used to configure the feature store.
config (optional) – Configuration object used to configure the feature store.
- apply(objects: Union[feast.data_source.DataSource, feast.entity.Entity, feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView, feast.request_feature_view.RequestFeatureView, feast.feature_service.FeatureService, List[Union[feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView, feast.request_feature_view.RequestFeatureView, feast.entity.Entity, feast.feature_service.FeatureService, feast.data_source.DataSource]]], objects_to_delete: Optional[List[Union[feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView, feast.request_feature_view.RequestFeatureView, feast.entity.Entity, feast.feature_service.FeatureService, feast.data_source.DataSource]]] = None, partial: bool = True)[source]
Register objects to metadata store and update related infrastructure.
The apply method registers one or more definitions (e.g., Entity, FeatureView) and registers or updates these objects in the Feast registry. Once the apply method has updated the infrastructure (e.g., create tables in an online store), it will commit the updated registry. All operations are idempotent, meaning they can safely be rerun.
- Parameters
objects – A single object, or a list of objects that should be registered with the Feature Store.
objects_to_delete – A list of objects to be deleted from the registry and removed from the provider’s infrastructure. This deletion will only be performed if partial is set to False.
partial – If True, apply will only handle the specified objects; if False, apply will also delete all the objects in objects_to_delete, and tear down any associated cloud resources.
- Raises
ValueError – The ‘objects’ parameter could not be parsed properly.
Examples
Register an Entity and a FeatureView.
>>> from feast import FeatureStore, Entity, FeatureView, Feature, ValueType, FileSource, RepoConfig >>> from datetime import timedelta >>> fs = FeatureStore(repo_path="feature_repo") >>> driver = Entity(name="driver_id", value_type=ValueType.INT64, description="driver id") >>> driver_hourly_stats = FileSource( ... path="feature_repo/data/driver_stats.parquet", ... timestamp_field="event_timestamp", ... created_timestamp_column="created", ... ) >>> driver_hourly_stats_view = FeatureView( ... name="driver_hourly_stats", ... entities=["driver_id"], ... ttl=timedelta(seconds=86400 * 1), ... batch_source=driver_hourly_stats, ... ) >>> fs.apply([driver_hourly_stats_view, driver]) # register entity and feature view
- config: feast.repo_config.RepoConfig
- create_saved_dataset(from_: feast.infra.offline_stores.offline_store.RetrievalJob, name: str, storage: feast.saved_dataset.SavedDatasetStorage, tags: Optional[Dict[str, str]] = None, feature_service: Optional[feast.feature_service.FeatureService] = None) feast.saved_dataset.SavedDataset [source]
Execute provided retrieval job and persist its outcome in given storage. Storage type (eg, BigQuery or Redshift) must be the same as globally configured offline store. After data successfully persisted saved dataset object with dataset metadata is committed to the registry. Name for the saved dataset should be unique within project, since it’s possible to overwrite previously stored dataset with the same name.
- Returns
SavedDataset object with attached RetrievalJob
- Raises
ValueError if given retrieval job doesn't have metadata –
- delete_feature_service(name: str)[source]
Deletes a feature service.
- Parameters
name – Name of feature service.
- Raises
FeatureServiceNotFoundException – The feature view could not be found.
- delete_feature_view(name: str)[source]
Deletes a feature view.
- Parameters
name – Name of feature view.
- Raises
FeatureViewNotFoundException – The feature view could not be found.
- static ensure_request_data_values_exist(needed_request_data: Set[str], needed_request_fv_features: Set[str], request_data_features: Dict[str, List[Any]])[source]
- get_data_source(name: str) feast.data_source.DataSource [source]
Retrieves the list of data sources from the registry.
- Parameters
name – Name of the data source.
- Returns
The specified data source.
- Raises
DataSourceObjectNotFoundException – The data source could not be found.
- get_entity(name: str, allow_registry_cache: bool = False) feast.entity.Entity [source]
Retrieves an entity.
- Parameters
name – Name of entity.
allow_registry_cache – (Optional) Whether to allow returning this entity from a cached registry
- Returns
The specified entity.
- Raises
EntityNotFoundException – The entity could not be found.
- get_feature_server_endpoint() Optional[str] [source]
Returns endpoint for the feature server, if it exists.
- get_feature_service(name: str, allow_cache: bool = False) feast.feature_service.FeatureService [source]
Retrieves a feature service.
- Parameters
name – Name of feature service.
allow_cache – Whether to allow returning feature services from a cached registry.
- Returns
The specified feature service.
- Raises
FeatureServiceNotFoundException – The feature service could not be found.
- get_feature_view(name: str, allow_registry_cache: bool = False) feast.feature_view.FeatureView [source]
Retrieves a feature view.
- Parameters
name – Name of feature view.
allow_registry_cache – (Optional) Whether to allow returning this entity from a cached registry
- Returns
The specified feature view.
- Raises
FeatureViewNotFoundException – The feature view could not be found.
- get_historical_features(entity_df: Union[pandas.core.frame.DataFrame, str], features: Union[List[str], feast.feature_service.FeatureService], full_feature_names: bool = False) feast.infra.offline_stores.offline_store.RetrievalJob [source]
Enrich an entity dataframe with historical feature values for either training or batch scoring.
This method joins historical feature data from one or more feature views to an entity dataframe by using a time travel join.
Each feature view is joined to the entity dataframe using all entities configured for the respective feature view. All configured entities must be available in the entity dataframe. Therefore, the entity dataframe must contain all entities found in all feature views, but the individual feature views can have different entities.
Time travel is based on the configured TTL for each feature view. A shorter TTL will limit the amount of scanning that will be done in order to find feature data for a specific entity key. Setting a short TTL may result in null values being returned.
- Parameters
entity_df (Union[pd.DataFrame, str]) – An entity dataframe is a collection of rows containing all entity columns (e.g., customer_id, driver_id) on which features need to be joined, as well as a event_timestamp column used to ensure point-in-time correctness. Either a Pandas DataFrame can be provided or a string SQL query. The query must be of a format supported by the configured offline store (e.g., BigQuery)
features – The list of features that should be retrieved from the offline store. These features can be specified either as a list of string feature references or as a feature service. String feature references must have format “feature_view:feature”, e.g. “customer_fv:daily_transactions”.
full_feature_names – If True, feature names will be prefixed with the corresponding feature view name, changing them from the format “feature” to “feature_view__feature” (e.g. “daily_transactions” changes to “customer_fv__daily_transactions”).
- Returns
RetrievalJob which can be used to materialize the results.
- Raises
ValueError – Both or neither of features and feature_refs are specified.
Examples
Retrieve historical features from a local offline store.
>>> from feast import FeatureStore, RepoConfig >>> import pandas as pd >>> fs = FeatureStore(repo_path="feature_repo") >>> entity_df = pd.DataFrame.from_dict( ... { ... "driver_id": [1001, 1002], ... "event_timestamp": [ ... datetime(2021, 4, 12, 10, 59, 42), ... datetime(2021, 4, 12, 8, 12, 10), ... ], ... } ... ) >>> retrieval_job = fs.get_historical_features( ... entity_df=entity_df, ... features=[ ... "driver_hourly_stats:conv_rate", ... "driver_hourly_stats:acc_rate", ... "driver_hourly_stats:avg_daily_trips", ... ], ... ) >>> feature_data = retrieval_job.to_df()
- static get_needed_request_data(grouped_odfv_refs: List[Tuple[feast.on_demand_feature_view.OnDemandFeatureView, List[str]]], grouped_request_fv_refs: List[Tuple[feast.request_feature_view.RequestFeatureView, List[str]]]) Tuple[Set[str], Set[str]] [source]
- get_on_demand_feature_view(name: str) feast.on_demand_feature_view.OnDemandFeatureView [source]
Retrieves a feature view.
- Parameters
name – Name of feature view.
- Returns
The specified feature view.
- Raises
FeatureViewNotFoundException – The feature view could not be found.
- get_online_features(features: Union[List[str], feast.feature_service.FeatureService], entity_rows: List[Dict[str, Any]], full_feature_names: bool = False) feast.online_response.OnlineResponse [source]
Retrieves the latest online feature data.
Note: This method will download the full feature registry the first time it is run. If you are using a remote registry like GCS or S3 then that may take a few seconds. The registry remains cached up to a TTL duration (which can be set to infinity). If the cached registry is stale (more time than the TTL has passed), then a new registry will be downloaded synchronously by this method. This download may introduce latency to online feature retrieval. In order to avoid synchronous downloads, please call refresh_registry() prior to the TTL being reached. Remember it is possible to set the cache TTL to infinity (cache forever).
- Parameters
features – The list of features that should be retrieved from the online store. These features can be specified either as a list of string feature references or as a feature service. String feature references must have format “feature_view:feature”, e.g. “customer_fv:daily_transactions”.
entity_rows – A list of dictionaries where each key-value is an entity-name, entity-value pair.
full_feature_names – If True, feature names will be prefixed with the corresponding feature view name, changing them from the format “feature” to “feature_view__feature” (e.g. “daily_transactions” changes to “customer_fv__daily_transactions”).
- Returns
OnlineResponse containing the feature data in records.
- Raises
Exception – No entity with the specified name exists.
Examples
Retrieve online features from an online store.
>>> from feast import FeatureStore, RepoConfig >>> fs = FeatureStore(repo_path="feature_repo") >>> online_response = fs.get_online_features( ... features=[ ... "driver_hourly_stats:conv_rate", ... "driver_hourly_stats:acc_rate", ... "driver_hourly_stats:avg_daily_trips", ... ], ... entity_rows=[{"driver_id": 1001}, {"driver_id": 1002}, {"driver_id": 1003}, {"driver_id": 1004}], ... ) >>> online_response_dict = online_response.to_dict()
- get_saved_dataset(name: str) feast.saved_dataset.SavedDataset [source]
Find a saved dataset in the registry by provided name and create a retrieval job to pull whole dataset from storage (offline store).
If dataset couldn’t be found by provided name SavedDatasetNotFound exception will be raised.
Data will be retrieved from globally configured offline store.
- Returns
SavedDataset with RetrievalJob attached
- Raises
- list_data_sources(allow_cache: bool = False) List[feast.data_source.DataSource] [source]
Retrieves the list of data sources from the registry.
- Parameters
allow_cache – Whether to allow returning data sources from a cached registry.
- Returns
A list of data sources.
- list_entities(allow_cache: bool = False) List[feast.entity.Entity] [source]
Retrieves the list of entities from the registry.
- Parameters
allow_cache – Whether to allow returning entities from a cached registry.
- Returns
A list of entities.
- list_feature_services() List[feast.feature_service.FeatureService] [source]
Retrieves the list of feature services from the registry.
- Returns
A list of feature services.
- list_feature_views(allow_cache: bool = False) List[feast.feature_view.FeatureView] [source]
Retrieves the list of feature views from the registry.
- Parameters
allow_cache – Whether to allow returning entities from a cached registry.
- Returns
A list of feature views.
- list_on_demand_feature_views(allow_cache: bool = False) List[feast.on_demand_feature_view.OnDemandFeatureView] [source]
Retrieves the list of on demand feature views from the registry.
- Returns
A list of on demand feature views.
- list_request_feature_views(allow_cache: bool = False) List[feast.request_feature_view.RequestFeatureView] [source]
Retrieves the list of feature views from the registry.
- Parameters
allow_cache – Whether to allow returning entities from a cached registry.
- Returns
A list of feature views.
- materialize(start_date: datetime.datetime, end_date: datetime.datetime, feature_views: Optional[List[str]] = None) None [source]
Materialize data from the offline store into the online store.
This method loads feature data in the specified interval from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving.
- Parameters
start_date (datetime) – Start date for time range of data to materialize into the online store
end_date (datetime) – End date for time range of data to materialize into the online store
feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.
Examples
Materialize all features into the online store over the interval from 3 hours ago to 10 minutes ago.
>>> from feast import FeatureStore, RepoConfig >>> from datetime import datetime, timedelta >>> fs = FeatureStore(repo_path="feature_repo") >>> fs.materialize( ... start_date=datetime.utcnow() - timedelta(hours=3), end_date=datetime.utcnow() - timedelta(minutes=10) ... ) Materializing... ...
- materialize_incremental(end_date: datetime.datetime, feature_views: Optional[List[str]] = None) None [source]
Materialize incremental new data from the offline store into the online store.
This method loads incremental new feature data up to the specified end time from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving. The start time of the interval materialized is either the most recent end time of a prior materialization or (now - ttl) if no such prior materialization exists.
- Parameters
end_date (datetime) – End date for time range of data to materialize into the online store
feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.
- Raises
Exception – A feature view being materialized does not have a TTL set.
Examples
Materialize all features into the online store up to 5 minutes ago.
>>> from feast import FeatureStore, RepoConfig >>> from datetime import datetime, timedelta >>> fs = FeatureStore(repo_path="feature_repo") >>> fs.materialize_incremental(end_date=datetime.utcnow() - timedelta(minutes=5)) Materializing... ...
- push(push_source_name: str, df: pandas.core.frame.DataFrame, allow_registry_cache: bool = True)[source]
Push features to a push source. This updates all the feature views that have the push source as stream source. :param push_source_name: The name of the push source we want to push data to. :param df: the data being pushed. :param allow_registry_cache: whether to allow cached versions of the registry.
- refresh_registry()[source]
Fetches and caches a copy of the feature registry in memory.
Explicitly calling this method allows for direct control of the state of the registry cache. Every time this method is called the complete registry state will be retrieved from the remote registry store backend (e.g., GCS, S3), and the cache timer will be reset. If refresh_registry() is run before get_online_features() is called, then get_online_features() will use the cached registry instead of retrieving (and caching) the registry itself.
Additionally, the TTL for the registry cache can be set to infinity (by setting it to 0), which means that refresh_registry() will become the only way to update the cached registry. If the TTL is set to a value greater than 0, then once the cache becomes stale (more time than the TTL has passed), a new cache will be downloaded synchronously, which may increase latencies if the triggering method is get_online_features().
- property registry: feast.registry.Registry
Gets the registry of this feature store.
- repo_path: pathlib.Path
- serve(host: str, port: int, no_access_log: bool) None [source]
Start the feature consumption server locally on a given port.
feast.feature_view module
- class feast.feature_view.FeatureView(*args, name: Optional[str] = None, entities: Optional[Union[List[feast.entity.Entity], List[str]]] = None, ttl: Optional[Union[google.protobuf.duration_pb2.Duration, datetime.timedelta]] = None, batch_source: Optional[feast.data_source.DataSource] = None, stream_source: Optional[feast.data_source.DataSource] = None, features: Optional[List[feast.feature.Feature]] = None, tags: Optional[Dict[str, str]] = None, online: bool = True, description: str = '', owner: str = '', schema: Optional[List[feast.field.Field]] = None, source: Optional[feast.data_source.DataSource] = None)[source]
Bases:
feast.base_feature_view.BaseFeatureView
A FeatureView defines a logical group of features.
- ttl
The amount of time this group of features lives. A ttl of 0 indicates that this group of features lives forever. Note that large ttl’s or a ttl of 0 can result in extremely computationally intensive queries.
- Type
Optional[datetime.timedelta]
- batch_source
The batch source of data where this group of features is stored. This is optional ONLY if a push source is specified as the stream_source, since push sources contain their own batch sources. This is deprecated in favor of source.
- Type
optional
- stream_source
The stream source of data where this group of features is stored. This is deprecated in favor of source.
- Type
optional
- schema
The schema of the feature view, including feature, timestamp, and entity columns.
- Type
List[feast.field.Field]
- features
The list of features defined as part of this feature view. Each feature should also be included in the schema.
- Type
List[feast.field.Field]
- source
The source of data for this group of features. May be a stream source, or a batch source. If a stream source, the source should contain a batch_source for backfills & batch materialization.
- Type
optional
- batch_source: feast.data_source.DataSource
- ensure_valid()[source]
Validates the state of this feature view locally.
- Raises
ValueError – The feature view does not have a name or does not have entities.
- features: List[feast.field.Field]
- classmethod from_proto(feature_view_proto: feast.core.FeatureView_pb2.FeatureView)[source]
Creates a feature view from a protobuf representation of a feature view.
- Parameters
feature_view_proto – A protobuf representation of a feature view.
- Returns
A FeatureViewProto object based on the feature view protobuf.
- materialization_intervals: List[Tuple[datetime.datetime, datetime.datetime]]
- property most_recent_end_time: Optional[datetime.datetime]
Retrieves the latest time up to which the feature view has been materialized.
- Returns
The latest time, or None if the feature view has not been materialized.
- property proto_class: Type[feast.core.FeatureView_pb2.FeatureView]
- schema: List[feast.field.Field]
- source: Optional[feast.data_source.DataSource]
- stream_source: Optional[feast.data_source.DataSource]
- to_proto() feast.core.FeatureView_pb2.FeatureView [source]
Converts a feature view object to its protobuf representation.
- Returns
A FeatureViewProto protobuf.
- ttl: Optional[datetime.timedelta]
- with_join_key_map(join_key_map: Dict[str, str])[source]
Returns a copy of this feature view with the join key map set to the given map. This join_key mapping operation is only used as part of query operations and will not modify the underlying FeatureView.
- Parameters
join_key_map – A map of join keys in which the left is the join_key that corresponds with the feature data and the right corresponds with the entity data.
Examples
Join a location feature data table to both the origin column and destination column of the entity data.
- temperatures_feature_service = FeatureService(
name=”temperatures”, features=[
- location_stats_feature_view
.with_name(“origin_stats”) .with_join_key_map(
{“location_id”: “origin_id”}
),
- location_stats_feature_view
.with_name(“destination_stats”) .with_join_key_map(
{“location_id”: “destination_id”}
),
],
)
feast.feature_view_projection module
- class feast.feature_view_projection.FeatureViewProjection(name: str, name_alias: Optional[str], features: List[feast.field.Field], join_key_map: Dict[str, str] = {})[source]
Bases:
object
A feature view projection represents a selection of one or more features from a single feature view.
- features
The list of features represented by the feature view projection.
- Type
List[feast.field.Field]
- join_key_map
A map to modify join key columns during retrieval of this feature view projection.
- features: List[feast.field.Field]
feast.field module
- class feast.field.Field(*, name: str, dtype: Union[feast.types.ComplexFeastType, feast.types.PrimitiveFeastType], tags: Optional[Dict[str, str]] = None)[source]
Bases:
object
A Field represents a set of values with the same structure.
- dtype
The type of the field, such as string or float.
- dtype: Union[feast.types.ComplexFeastType, feast.types.PrimitiveFeastType]
- classmethod from_feature(feature: feast.feature.Feature)[source]
Creates a Field object from a Feature object.
- Parameters
feature – Feature object to convert.
feast.flags module
feast.flags_helper module
- feast.flags_helper.enable_aws_lambda_feature_server(repo_config: feast.repo_config.RepoConfig) bool [source]
- feast.flags_helper.enable_on_demand_feature_views(repo_config: feast.repo_config.RepoConfig) bool [source]
- feast.flags_helper.feature_flag_enabled(repo_config: feast.repo_config.RepoConfig, flag_name: str) bool [source]
feast.importer module
- feast.importer.import_class(module_name: str, class_name: str, class_type: Optional[str] = None)[source]
Dynamically loads and returns a class from a module.
- Parameters
module_name – The name of the module.
class_name – The name of the class.
class_type – Optional name of a base class of the class.
- Raises
FeastInvalidBaseClass – If the class name does not end with the specified suffix.
FeastModuleImportError – If the module cannot be imported.
FeastClassImportError – If the class cannot be imported.
feast.inference module
- feast.inference.update_data_sources_with_inferred_event_timestamp_col(data_sources: List[feast.data_source.DataSource], config: feast.repo_config.RepoConfig) None [source]
- feast.inference.update_entities_with_inferred_types_from_feature_views(entities: List[feast.entity.Entity], feature_views: List[feast.feature_view.FeatureView], config: feast.repo_config.RepoConfig) None [source]
Infers the types of the entities by examining the schemas of feature view batch sources.
- Parameters
entities – The entities to be updated.
feature_views – A list containing feature views associated with the entities.
config – The config for the current feature store.
- feast.inference.update_feature_views_with_inferred_features(fvs: List[feast.feature_view.FeatureView], entities: List[feast.entity.Entity], config: feast.repo_config.RepoConfig) None [source]
Infers the set of features associated to each FeatureView and updates the FeatureView with those features. Inference occurs through considering each column of the underlying data source as a feature except columns that are associated with the data source’s timestamp columns and the FeatureView’s entity columns.
- Parameters
fvs – The feature views to be updated.
entities – A list containing entities associated with the feature views.
config – The config for the current feature store.
feast.names module
feast.on_demand_feature_view module
- class feast.on_demand_feature_view.OnDemandFeatureView(*args, name: Optional[str] = None, features: Optional[List[feast.feature.Feature]] = None, sources: Optional[Dict[str, Union[feast.feature_view.FeatureView, feast.feature_view_projection.FeatureViewProjection, feast.data_source.RequestSource]]] = None, udf: Optional[method] = None, inputs: Optional[Dict[str, Union[feast.feature_view.FeatureView, feast.feature_view_projection.FeatureViewProjection, feast.data_source.RequestSource]]] = None, schema: Optional[List[feast.field.Field]] = None, description: str = '', tags: Optional[Dict[str, str]] = None, owner: str = '')[source]
Bases:
feast.base_feature_view.BaseFeatureView
[Experimental] An OnDemandFeatureView defines a logical group of features that are generated by applying a transformation on a set of input sources, such as feature views and request data sources.
- features
The list of features in the output of the on demand feature view.
- Type
List[feast.field.Field]
- source_feature_view_projections
A map from input source names to actual input sources with type FeatureViewProjection.
- source_request_sources
A map from input source names to the actual input sources with type RequestSource.
- Type
- udf
The user defined transformation function, which must take pandas dataframes as inputs.
- Type
method
- owner
The owner of the on demand feature view, typically the email of the primary maintainer.
- Type
- features: List[feast.field.Field]
- classmethod from_proto(on_demand_feature_view_proto: feast.core.OnDemandFeatureView_pb2.OnDemandFeatureView)[source]
Creates an on demand feature view from a protobuf representation.
- Parameters
on_demand_feature_view_proto – A protobuf representation of an on-demand feature view.
- Returns
A OnDemandFeatureView object based on the on-demand feature view protobuf.
- get_request_data_schema() Dict[str, feast.value_type.ValueType] [source]
- get_transformed_features_df(df_with_features: pandas.core.frame.DataFrame, full_feature_names: bool = False) pandas.core.frame.DataFrame [source]
- infer_features()[source]
Infers the set of features associated to this feature view from the input source.
- Raises
RegistryInferenceFailure – The set of features could not be inferred.
- property proto_class: Type[feast.core.OnDemandFeatureView_pb2.OnDemandFeatureView]
- source_feature_view_projections: Dict[str, feast.feature_view_projection.FeatureViewProjection]
- source_request_sources: Dict[str, feast.data_source.RequestSource]
- to_proto() feast.core.OnDemandFeatureView_pb2.OnDemandFeatureView [source]
Converts an on demand feature view object to its protobuf representation.
- Returns
A OnDemandFeatureViewProto protobuf.
- udf: method
- feast.on_demand_feature_view.on_demand_feature_view(*args, features: Optional[List[feast.feature.Feature]] = None, sources: Optional[Dict[str, Union[feast.feature_view.FeatureView, feast.data_source.RequestSource]]] = None, inputs: Optional[Dict[str, Union[feast.feature_view.FeatureView, feast.data_source.RequestSource]]] = None, schema: Optional[List[feast.field.Field]] = None, description: str = '', tags: Optional[Dict[str, str]] = None, owner: str = '')[source]
Creates an OnDemandFeatureView object with the given user function as udf.
- Parameters
features (deprecated) – The list of features in the output of the on demand feature view, after the transformation has been applied.
sources (optional) – A map from input source names to the actual input sources, which may be feature views, feature view projections, or request data sources. These sources serve as inputs to the udf, which will refer to them by name.
inputs (optional) – A map from input source names to the actual input sources, which may be feature views, feature view projections, or request data sources. These sources serve as inputs to the udf, which will refer to them by name.
schema (optional) – The list of features in the output of the on demand feature view, after the transformation has been applied.
description (optional) – A human-readable description.
tags (optional) – A dictionary of key-value pairs to store arbitrary metadata.
owner (optional) – The owner of the on demand feature view, typically the email of the primary maintainer.
feast.online_response module
- class feast.online_response.OnlineResponse(online_response_proto: feast.serving.ServingService_pb2.GetOnlineFeaturesResponse)[source]
Bases:
object
Defines an online response in feast.
feast.proto_json module
feast.registry module
- class feast.registry.FeastObjectType(value)[source]
Bases:
enum.Enum
An enumeration.
- DATA_SOURCE = 'data source'
- ENTITY = 'entity'
- FEATURE_SERVICE = 'feature service'
- FEATURE_VIEW = 'feature view'
- ON_DEMAND_FEATURE_VIEW = 'on demand feature view'
- REQUEST_FEATURE_VIEW = 'request feature view'
- static get_objects_from_registry(registry: feast.registry.Registry, project: str) Dict[feast.registry.FeastObjectType, List[Any]] [source]
- static get_objects_from_repo_contents(repo_contents: feast.repo_contents.RepoContents) Dict[feast.registry.FeastObjectType, List[Any]] [source]
- class feast.registry.Registry(registry_config: Optional[feast.repo_config.RegistryConfig], repo_path: Optional[pathlib.Path])[source]
Bases:
object
Registry: A registry allows for the management and persistence of feature definitions and related metadata.
- apply_data_source(data_source: feast.data_source.DataSource, project: str, commit: bool = True)[source]
Registers a single data source with Feast
- Parameters
data_source – A data source that will be registered
project – Feast project that this data source belongs to
commit – Whether to immediately commit to the registry
- apply_entity(entity: feast.entity.Entity, project: str, commit: bool = True)[source]
Registers a single entity with Feast
- Parameters
entity – Entity that will be registered
project – Feast project that this entity belongs to
commit – Whether the change should be persisted immediately
- apply_feature_service(feature_service: feast.feature_service.FeatureService, project: str, commit: bool = True)[source]
Registers a single feature service with Feast
- Parameters
feature_service – A feature service that will be registered
project – Feast project that this entity belongs to
- apply_feature_view(feature_view: feast.base_feature_view.BaseFeatureView, project: str, commit: bool = True)[source]
Registers a single feature view with Feast
- Parameters
feature_view – Feature view that will be registered
project – Feast project that this feature view belongs to
commit – Whether the change should be persisted immediately
- apply_materialization(feature_view: feast.feature_view.FeatureView, project: str, start_date: datetime.datetime, end_date: datetime.datetime, commit: bool = True)[source]
Updates materialization intervals tracked for a single feature view in Feast
- Parameters
feature_view – Feature view that will be updated with an additional materialization interval tracked
project – Feast project that this feature view belongs to
start_date (datetime) – Start date of the materialization interval to track
end_date (datetime) – End date of the materialization interval to track
commit – Whether the change should be persisted immediately
- apply_saved_dataset(saved_dataset: feast.saved_dataset.SavedDataset, project: str, commit: bool = True)[source]
Registers a single entity with Feast
- Parameters
saved_dataset – SavedDataset that will be added / updated to registry
project – Feast project that this dataset belongs to
commit – Whether the change should be persisted immediately
- cached_registry_proto: Optional[feast.core.Registry_pb2.Registry] = None
- cached_registry_proto_created: Optional[datetime.datetime] = None
- cached_registry_proto_ttl: datetime.timedelta
- clone() feast.registry.Registry [source]
- delete_data_source(name: str, project: str, commit: bool = True)[source]
Deletes a data source or raises an exception if not found.
- Parameters
name – Name of data source
project – Feast project that this data source belongs to
commit – Whether the change should be persisted immediately
- delete_entity(name: str, project: str, commit: bool = True)[source]
Deletes an entity or raises an exception if not found.
- Parameters
name – Name of entity
project – Feast project that this entity belongs to
commit – Whether the change should be persisted immediately
- delete_feature_service(name: str, project: str, commit: bool = True)[source]
Deletes a feature service or raises an exception if not found.
- Parameters
name – Name of feature service
project – Feast project that this feature service belongs to
commit – Whether the change should be persisted immediately
- delete_feature_view(name: str, project: str, commit: bool = True)[source]
Deletes a feature view or raises an exception if not found.
- Parameters
name – Name of feature view
project – Feast project that this feature view belongs to
commit – Whether the change should be persisted immediately
- get_data_source(name: str, project: str, allow_cache: bool = False) feast.data_source.DataSource [source]
Retrieves a data source.
- Parameters
name – Name of data source
project – Feast project that this data source belongs to
allow_cache – Whether to allow returning this data source from a cached registry
- Returns
Returns either the specified data source, or raises an exception if none is found
- get_entity(name: str, project: str, allow_cache: bool = False) feast.entity.Entity [source]
Retrieves an entity.
- Parameters
name – Name of entity
project – Feast project that this entity belongs to
allow_cache – Whether to allow returning this entity from a cached registry
- Returns
Returns either the specified entity, or raises an exception if none is found
- get_feature_service(name: str, project: str, allow_cache: bool = False) feast.feature_service.FeatureService [source]
Retrieves a feature service.
- Parameters
name – Name of feature service
project – Feast project that this feature service belongs to
allow_cache – Whether to allow returning this feature service from a cached registry
- Returns
Returns either the specified feature service, or raises an exception if none is found
- get_feature_view(name: str, project: str, allow_cache: bool = False) feast.feature_view.FeatureView [source]
Retrieves a feature view.
- Parameters
name – Name of feature view
project – Feast project that this feature view belongs to
allow_cache – Allow returning feature view from the cached registry
- Returns
Returns either the specified feature view, or raises an exception if none is found
- get_infra(project: str, allow_cache: bool = False) feast.infra.infra_object.Infra [source]
Retrieves the stored Infra object.
- Parameters
project – Feast project that the Infra object refers to
allow_cache – Whether to allow returning this entity from a cached registry
- Returns
The stored Infra object.
- get_on_demand_feature_view(name: str, project: str, allow_cache: bool = False) feast.on_demand_feature_view.OnDemandFeatureView [source]
Retrieves an on demand feature view.
- Parameters
name – Name of on demand feature view
project – Feast project that this on demand feature view belongs to
allow_cache – Whether to allow returning this on demand feature view from a cached registry
- Returns
Returns either the specified on demand feature view, or raises an exception if none is found
- get_saved_dataset(name: str, project: str, allow_cache: bool = False) feast.saved_dataset.SavedDataset [source]
Retrieves a saved dataset.
- Parameters
name – Name of dataset
project – Feast project that this dataset belongs to
allow_cache – Whether to allow returning this dataset from a cached registry
- Returns
Returns either the specified SavedDataset, or raises an exception if none is found
- list_data_sources(project: str, allow_cache: bool = False) List[feast.data_source.DataSource] [source]
Retrieve a list of data sources from the registry
- Parameters
project – Filter data source based on project name
allow_cache – Whether to allow returning data sources from a cached registry
- Returns
List of data sources
- list_entities(project: str, allow_cache: bool = False) List[feast.entity.Entity] [source]
Retrieve a list of entities from the registry
- Parameters
allow_cache – Whether to allow returning entities from a cached registry
project – Filter entities based on project name
- Returns
List of entities
- list_feature_services(project: str, allow_cache: bool = False) List[feast.feature_service.FeatureService] [source]
Retrieve a list of feature services from the registry
- Parameters
allow_cache – Whether to allow returning entities from a cached registry
project – Filter entities based on project name
- Returns
List of feature services
- list_feature_views(project: str, allow_cache: bool = False) List[feast.feature_view.FeatureView] [source]
Retrieve a list of feature views from the registry
- Parameters
allow_cache – Allow returning feature views from the cached registry
project – Filter feature views based on project name
- Returns
List of feature views
- list_on_demand_feature_views(project: str, allow_cache: bool = False) List[feast.on_demand_feature_view.OnDemandFeatureView] [source]
Retrieve a list of on demand feature views from the registry
- Parameters
project – Filter on demand feature views based on project name
allow_cache – Whether to allow returning on demand feature views from a cached registry
- Returns
List of on demand feature views
- list_request_feature_views(project: str, allow_cache: bool = False) List[feast.request_feature_view.RequestFeatureView] [source]
Retrieve a list of request feature views from the registry
- Parameters
allow_cache – Allow returning feature views from the cached registry
project – Filter feature views based on project name
- Returns
List of feature views
- list_saved_datasets(project: str, allow_cache: bool = False) List[feast.saved_dataset.SavedDataset] [source]
Retrieves a list of all saved datasets in specified project
- Parameters
project – Feast project
allow_cache – Whether to allow returning this dataset from a cached registry
- Returns
Returns the list of SavedDatasets
- refresh()[source]
Refreshes the state of the registry cache by fetching the registry state from the remote registry store.
- to_dict(project: str) Dict[str, List[Any]] [source]
Returns a dictionary representation of the registry contents for the specified project.
For each list in the dictionary, the elements are sorted by name, so this method can be used to compare two registries.
- Parameters
project – Feast project to convert to a dict
- update_infra(infra: feast.infra.infra_object.Infra, project: str, commit: bool = True)[source]
Updates the stored Infra object.
- Parameters
infra – The new Infra object to be stored.
project – Feast project that the Infra object refers to
commit – Whether the change should be persisted immediately
feast.registry_store module
- class feast.registry_store.NoopRegistryStore[source]
- class feast.registry_store.RegistryStore[source]
Bases:
abc.ABC
A registry store is a storage backend for the Feast registry.
feast.repo_config module
- class feast.repo_config.FeastBaseModel(**extra_data: Any)[source]
Bases:
pydantic.main.BaseModel
Feast Pydantic Configuration Class
- class feast.repo_config.FeastConfigBaseModel[source]
Bases:
pydantic.main.BaseModel
Feast Pydantic Configuration Class
- class feast.repo_config.RegistryConfig(*, registry_store_type: pydantic.types.StrictStr = None, path: pydantic.types.StrictStr, cache_ttl_seconds: pydantic.types.StrictInt = 600, **extra_data: Any)[source]
Bases:
feast.repo_config.FeastBaseModel
Metadata Store Configuration. Configuration that relates to reading from and writing to the Feast registry.
- cache_ttl_seconds: pydantic.types.StrictInt
The cache TTL is the amount of time registry state will be cached in memory. If this TTL is exceeded then the registry will be refreshed when any feature store method asks for access to registry state. The TTL can be set to infinity by setting TTL to 0 seconds, which means the cache will only be loaded once and will never expire. Users can manually refresh the cache by calling feature_store.refresh_registry()
- Type
- path: pydantic.types.StrictStr
Path to metadata store. Can be a local path, or remote object storage path, e.g. a GCS URI
- Type
- class feast.repo_config.RepoConfig(*, registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig] = 'data/registry.db', project: pydantic.types.StrictStr, provider: pydantic.types.StrictStr, online_store: Any = None, offline_store: Any = None, feature_server: Any = None, flags: Any = None, repo_path: pathlib.Path = None, go_feature_retrieval: bool = False, **data: Any)[source]
Bases:
feast.repo_config.FeastBaseModel
Repo config. Typically loaded from feature_store.yaml
- feature_server: Optional[Any]
Feature server configuration (optional depending on provider)
- Type
FeatureServerConfig
- flags: Any
Feature flags for experimental features (optional)
- Type
Flags
- offline_store: Any
Offline store configuration (optional depending on provider)
- Type
OfflineStoreConfig
- online_store: Any
Online store configuration (optional depending on provider)
- Type
OnlineStoreConfig
- project: pydantic.types.StrictStr
Feast project id. This can be any alphanumeric string up to 16 characters. You can have multiple independent feature repositories deployed to the same cloud provider account, as long as they have different project ids.
- Type
- registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig]
Path to metadata store. Can be a local path, or remote object storage path, e.g. a GCS URI
- Type
- repo_path: Optional[pathlib.Path]
- write_to_path(repo_path: pathlib.Path)[source]
- feast.repo_config.load_repo_config(repo_path: pathlib.Path) feast.repo_config.RepoConfig [source]
feast.repo_contents module
- class feast.repo_contents.RepoContents(data_sources: List[feast.data_source.DataSource], feature_views: List[feast.feature_view.FeatureView], on_demand_feature_views: List[feast.on_demand_feature_view.OnDemandFeatureView], request_feature_views: List[feast.request_feature_view.RequestFeatureView], entities: List[feast.entity.Entity], feature_services: List[feast.feature_service.FeatureService])[source]
Bases:
tuple
Represents the objects in a Feast feature repo.
- property data_sources
Alias for field number 0
- property entities
Alias for field number 4
- property feature_services
Alias for field number 5
- property feature_views
Alias for field number 1
- property on_demand_feature_views
Alias for field number 2
- property request_feature_views
Alias for field number 3
feast.repo_operations module
- feast.repo_operations.apply_total(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path, skip_source_validation: bool)[source]
- feast.repo_operations.apply_total_with_repo_instance(store: feast.feature_store.FeatureStore, project: str, registry: feast.registry.Registry, repo: feast.repo_contents.RepoContents, skip_source_validation: bool)[source]
- feast.repo_operations.cli_check_repo(repo_path: pathlib.Path)[source]
- feast.repo_operations.get_ignore_files(repo_root: pathlib.Path, ignore_paths: List[str]) Set[pathlib.Path] [source]
Get all ignore files that match any of the user-defined ignore paths
- feast.repo_operations.get_repo_files(repo_root: pathlib.Path) List[pathlib.Path] [source]
Get the list of all repo files, ignoring undesired files & directories specified in .feastignore
- feast.repo_operations.is_valid_name(name: str) bool [source]
A name should be alphanumeric values and underscores but not start with an underscore
- feast.repo_operations.log_infra_changes(views_to_keep: Set[feast.feature_view.FeatureView], views_to_delete: Set[feast.feature_view.FeatureView])[source]
- feast.repo_operations.parse_repo(repo_root: pathlib.Path) feast.repo_contents.RepoContents [source]
Collects unique Feast object definitions from the given feature repo.
Specifically, if an object foo has already been added, bar will still be added if (bar == foo), but not if (bar is foo). This ensures that import statements will not result in duplicates, but defining two equal objects will.
- feast.repo_operations.plan(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path, skip_source_validation: bool)[source]
- feast.repo_operations.py_path_to_module(path: pathlib.Path) str [source]
- feast.repo_operations.read_feastignore(repo_root: pathlib.Path) List[str] [source]
Read .feastignore in the repo root directory (if exists) and return the list of user-defined ignore paths
- feast.repo_operations.registry_dump(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path)[source]
For debugging only: output contents of the metadata registry
- feast.repo_operations.teardown(repo_config: feast.repo_config.RepoConfig, repo_path: pathlib.Path)[source]
feast.request_feature_view module
- class feast.request_feature_view.RequestFeatureView(name: str, request_data_source: feast.data_source.RequestSource, description: str = '', tags: Optional[Dict[str, str]] = None, owner: str = '')[source]
Bases:
feast.base_feature_view.BaseFeatureView
[Experimental] A RequestFeatureView defines a logical group of features that should be available as an input to an on demand feature view at request time.
- request_source
The request source that specifies the schema and features of the request feature view.
- features
The list of features defined as part of this request feature view.
- Type
List[feast.field.Field]
- owner
The owner of the request feature view, typically the email of the primary maintainer.
- Type
- features: List[feast.field.Field]
- classmethod from_proto(request_feature_view_proto: feast.core.RequestFeatureView_pb2.RequestFeatureView)[source]
Creates a request feature view from a protobuf representation.
- Parameters
request_feature_view_proto – A protobuf representation of an request feature view.
- Returns
A RequestFeatureView object based on the request feature view protobuf.
- property proto_class: Type[feast.core.RequestFeatureView_pb2.RequestFeatureView]
- request_source: feast.data_source.RequestSource
feast.saved_dataset module
- class feast.saved_dataset.SavedDataset(name: str, features: List[str], join_keys: List[str], storage: feast.saved_dataset.SavedDatasetStorage, full_feature_names: bool = False, tags: Optional[Dict[str, str]] = None, feature_service_name: Optional[str] = None)[source]
Bases:
object
- as_reference(profiler: feast.dqm.profilers.profiler.Profiler) feast.saved_dataset.ValidationReference [source]
- created_timestamp: Optional[datetime.datetime] = None
- static from_proto(saved_dataset_proto: feast.core.SavedDataset_pb2.SavedDataset)[source]
Converts a SavedDatasetProto to a SavedDataset object.
- Parameters
saved_dataset_proto – A protobuf representation of a SavedDataset.
- get_profile(profiler: feast.dqm.profilers.profiler.Profiler) feast.dqm.profilers.profiler.Profile [source]
- last_updated_timestamp: Optional[datetime.datetime] = None
- max_event_timestamp: Optional[datetime.datetime] = None
- min_event_timestamp: Optional[datetime.datetime] = None
- to_proto() feast.core.SavedDataset_pb2.SavedDataset [source]
Converts a SavedDataset to its protobuf representation.
- Returns
A SavedDatasetProto protobuf.
- with_retrieval_job(retrieval_job: RetrievalJob) SavedDataset [source]
- class feast.saved_dataset.SavedDatasetStorage[source]
Bases:
object
- static from_proto(storage_proto: feast.core.SavedDataset_pb2.SavedDatasetStorage) feast.saved_dataset.SavedDatasetStorage [source]
- abstract to_data_source() feast.data_source.DataSource [source]
- class feast.saved_dataset.ValidationReference(dataset: feast.saved_dataset.SavedDataset, profiler: feast.dqm.profilers.profiler.Profiler)[source]
Bases:
object
- dataset: feast.saved_dataset.SavedDataset
- property profile: feast.dqm.profilers.profiler.Profile
- profiler: feast.dqm.profilers.profiler.Profiler
feast.transformation_server module
- class feast.transformation_server.TransformationServer(fs: feast.feature_store.FeatureStore)[source]
Bases:
feast.protos.feast.serving.TransformationService_pb2_grpc.TransformationServiceServicer
- feast.transformation_server.start_server(store: feast.feature_store.FeatureStore, port: int)[source]
feast.type_map module
- feast.type_map.bq_to_feast_value_type(bq_type_as_str: str) feast.value_type.ValueType [source]
- feast.type_map.feast_value_type_to_pandas_type(value_type: feast.value_type.ValueType) Any [source]
- feast.type_map.feast_value_type_to_python_type(field_value_proto: feast.types.Value_pb2.Value) Any [source]
Converts field value Proto to Dict and returns each field’s Feast Value Type value in their respective Python value.
- Parameters
field_value_proto – Field value Proto
- Returns
Python native type representation/version of the given field_value_proto
- feast.type_map.pa_to_feast_value_type(pa_type_as_str: str) feast.value_type.ValueType [source]
- feast.type_map.python_type_to_feast_value_type(name: str, value: Optional[Any] = None, recurse: bool = True, type_name: Optional[str] = None) feast.value_type.ValueType [source]
Finds the equivalent Feast Value Type for a Python value. Both native and Pandas types are supported. This function will recursively look for nested types when arrays are detected. All types must be homogenous.
- Parameters
name – Name of the value or field
value – Value that will be inspected
recurse – Whether to recursively look for nested types in arrays
- Returns
Feast Value Type
- feast.type_map.python_values_to_feast_value_type(name: str, values: Any, recurse: bool = True) feast.value_type.ValueType [source]
- feast.type_map.python_values_to_proto_values(values: List[Any], feature_type: feast.value_type.ValueType = ValueType.UNKNOWN) List[feast.types.Value_pb2.Value] [source]
- feast.type_map.redshift_to_feast_value_type(redshift_type_as_str: str) feast.value_type.ValueType [source]
- feast.type_map.snowflake_python_type_to_feast_value_type(snowflake_python_type_as_str: str) feast.value_type.ValueType [source]
- feast.type_map.spark_schema_to_np_dtypes(dtypes: List[Tuple[str, str]]) Iterator[numpy.dtype] [source]
- feast.type_map.spark_to_feast_value_type(spark_type_as_str: str) feast.value_type.ValueType [source]
feast.types module
- class feast.types.Array(base_type: Union[feast.types.PrimitiveFeastType, feast.types.ComplexFeastType])[source]
Bases:
feast.types.ComplexFeastType
An Array represents a list of types.
- base_type
The base type of the array.
- base_type: Union[feast.types.PrimitiveFeastType, feast.types.ComplexFeastType]
- to_value_type() feast.value_type.ValueType [source]
Converts a ComplexFeastType object to the corresponding ValueType enum.
- class feast.types.ComplexFeastType[source]
Bases:
abc.ABC
A ComplexFeastType represents a structured type that is recognized by Feast.
- abstract to_value_type() feast.value_type.ValueType [source]
Converts a ComplexFeastType object to the corresponding ValueType enum.
- class feast.types.PrimitiveFeastType(value)[source]
Bases:
enum.Enum
A PrimitiveFeastType represents a primitive type in Feast.
Note that these values must match the values in /feast/protos/types/Value.proto.
- BOOL = 7
- BYTES = 1
- FLOAT32 = 6
- FLOAT64 = 5
- INT32 = 3
- INT64 = 4
- INVALID = 0
- STRING = 2
- UNIX_TIMESTAMP = 8
- to_value_type() feast.value_type.ValueType [source]
Converts a PrimitiveFeastType object to the corresponding ValueType enum.
- feast.types.from_value_type(value_type: feast.value_type.ValueType) Union[feast.types.ComplexFeastType, feast.types.PrimitiveFeastType] [source]
Converts a ValueType enum to a Feast type.
- Parameters
value_type – The ValueType to be converted.
- Raises
ValueError – The conversion could not be performed.
feast.usage module
- class feast.usage.AlwaysSampler[source]
Bases:
feast.usage.Sampler
- class feast.usage.FnCall(fn_name: str, id: str, start: datetime.datetime, end: Union[datetime.datetime, NoneType] = None, parent_id: Union[str, NoneType] = None)[source]
Bases:
object
- end: Optional[datetime.datetime] = None
- start: datetime.datetime
- class feast.usage.RatioSampler(ratio)[source]
Bases:
feast.usage.Sampler
- MAX_COUNTER = 4294967295
- property priority
- class feast.usage.UsageContext[source]
Bases:
object
- call_stack: List[feast.usage.FnCall]
- completed_calls: List[feast.usage.FnCall]
- sampler: feast.usage.Sampler = <feast.usage.AlwaysSampler object>
- feast.usage.log_exceptions(*args, **attrs)[source]
Function decorator that track errors and send them to Feast Developers
- feast.usage.log_exceptions_and_usage(*args, **attrs)[source]
This function decorator enables three components: 1. Error tracking 2. Usage statistic collection 3. Time profiling
This data is being collected, anonymized and sent to Feast Developers. All events from nested decorated functions are being grouped into single event to build comprehensive context useful for profiling and error tracking.
- Usage example (will result in one output event):
@log_exceptions_and_usage def fn(…):
nested()
@log_exceptions_and_usage(attr=’value’) def nested(…):
deeply_nested()
@log_exceptions_and_usage(attr2=’value2’, sample=RateSampler(rate=0.1)) def deeply_nested(…):
…
feast.utils module
- feast.utils.make_tzaware(t: datetime.datetime) datetime.datetime [source]
We assume tz-naive datetimes are UTC
- feast.utils.to_naive_utc(ts: datetime.datetime) datetime.datetime [source]
feast.value_type module
- class feast.value_type.ValueType(value)[source]
Bases:
enum.Enum
Feature value type. Used to define data types in Feature Tables.
- BOOL = 7
- BOOL_LIST = 17
- BYTES = 1
- BYTES_LIST = 11
- DOUBLE = 5
- DOUBLE_LIST = 15
- FLOAT = 6
- FLOAT_LIST = 16
- INT32 = 3
- INT32_LIST = 13
- INT64 = 4
- INT64_LIST = 14
- NULL = 19
- STRING = 2
- STRING_LIST = 12
- UNIX_TIMESTAMP = 8
- UNIX_TIMESTAMP_LIST = 18
- UNKNOWN = 0
feast.version module
feast.wait module
- feast.wait.wait_retry_backoff(retry_fn: Callable[[], Tuple[Any, bool]], timeout_secs: int = 0, timeout_msg: Optional[str] = 'Timeout while waiting for retry_fn() to return True', max_interval_secs: int = 60) Any [source]
Repeatedly try calling given retry_fn until it returns a True boolean success flag. Waits with a exponential backoff between retries until timeout when it throws TimeoutError. :param retry_fn: Callable that returns a result and a boolean success flag. :param timeout_secs: timeout in seconds to give up retrying and throw TimeoutError,
or 0 to retry perpetually.
- Parameters
timeout_msg – Message to use when throwing TimeoutError.
max_interval_secs – max wait in seconds to wait between retries.
- Returns
Returned Result from retry_fn() if success flag is True.
Module contents
- class feast.BigQuerySource(*, event_timestamp_column: Optional[str] = '', table: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = None, query: Optional[str] = None, name: Optional[str] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = None)[source]
Bases:
feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL
- property query
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- property table
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.Entity(*args, name: Optional[str] = None, value_type: feast.value_type.ValueType = ValueType.UNKNOWN, description: str = '', join_key: Optional[str] = None, tags: Optional[Dict[str, str]] = None, owner: str = '', join_keys: Optional[List[str]] = None)[source]
Bases:
object
An entity defines a collection of entities for which features can be defined. An entity can also contain associated metadata.
- value_type
The type of the entity, such as string or float.
- join_key
A property that uniquely identifies different entities within the collection. The join_key property is typically used for joining entities with their associated features. If not specified, defaults to the name.
- Type
- created_timestamp
The time when the entity was created.
- Type
Optional[datetime.datetime]
- last_updated_timestamp
The time when the entity was last updated.
- Type
Optional[datetime.datetime]
- join_keys
A list of property that uniquely identifies different entities within the collection. This is meant to replace the join_key parameter, but currently only supports a list of size one.
- Type
List[str]
- created_timestamp: Optional[datetime.datetime]
- classmethod from_proto(entity_proto: feast.core.Entity_pb2.Entity)[source]
Creates an entity from a protobuf representation of an entity.
- Parameters
entity_proto – A protobuf representation of an entity.
- Returns
An Entity object based on the entity protobuf.
- is_valid()[source]
Validates the state of this entity locally.
- Raises
ValueError – The entity does not have a name or does not have a type.
- last_updated_timestamp: Optional[datetime.datetime]
- to_proto() feast.core.Entity_pb2.Entity [source]
Converts an entity object to its protobuf representation.
- Returns
An EntityProto protobuf.
- value_type: feast.value_type.ValueType
- class feast.Feature(name: str, dtype: feast.value_type.ValueType, labels: Optional[Dict[str, str]] = None)[source]
Bases:
object
A Feature represents a class of serveable feature.
- Parameters
name – Name of the feature.
dtype – The type of the feature, such as string or float.
labels (optional) – User-defined metadata in dictionary form.
- property dtype: feast.value_type.ValueType
Gets the data type of this feature.
- classmethod from_proto(feature_proto: feast.core.Feature_pb2.FeatureSpecV2)[source]
- Parameters
feature_proto – FeatureSpecV2 protobuf object
- Returns
Feature object
- property name
Gets the name of this feature.
- class feast.FeatureService(*args, name: Optional[str] = None, features: Optional[List[Union[feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView]]] = None, tags: Dict[str, str] = None, description: str = '', owner: str = '')[source]
Bases:
object
A feature service defines a logical group of features from one or more feature views. This group of features can be retrieved together during training or serving.
- feature_view_projections
A list containing feature views and feature view projections, representing the features in the feature service.
- created_timestamp
The time when the feature service was created.
- Type
Optional[datetime.datetime]
- last_updated_timestamp
The time when the feature service was last updated.
- Type
Optional[datetime.datetime]
- created_timestamp: Optional[datetime.datetime] = None
- feature_view_projections: List[feast.feature_view_projection.FeatureViewProjection]
- classmethod from_proto(feature_service_proto: feast.core.FeatureService_pb2.FeatureService)[source]
Converts a FeatureServiceProto to a FeatureService object.
- Parameters
feature_service_proto – A protobuf representation of a FeatureService.
- last_updated_timestamp: Optional[datetime.datetime] = None
- class feast.FeatureStore(repo_path: Optional[str] = None, config: Optional[feast.repo_config.RepoConfig] = None)[source]
Bases:
object
A FeatureStore object is used to define, create, and retrieve features.
- Parameters
repo_path (optional) – Path to a feature_store.yaml used to configure the feature store.
config (optional) – Configuration object used to configure the feature store.
- apply(objects: Union[feast.data_source.DataSource, feast.entity.Entity, feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView, feast.request_feature_view.RequestFeatureView, feast.feature_service.FeatureService, List[Union[feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView, feast.request_feature_view.RequestFeatureView, feast.entity.Entity, feast.feature_service.FeatureService, feast.data_source.DataSource]]], objects_to_delete: Optional[List[Union[feast.feature_view.FeatureView, feast.on_demand_feature_view.OnDemandFeatureView, feast.request_feature_view.RequestFeatureView, feast.entity.Entity, feast.feature_service.FeatureService, feast.data_source.DataSource]]] = None, partial: bool = True)[source]
Register objects to metadata store and update related infrastructure.
The apply method registers one or more definitions (e.g., Entity, FeatureView) and registers or updates these objects in the Feast registry. Once the apply method has updated the infrastructure (e.g., create tables in an online store), it will commit the updated registry. All operations are idempotent, meaning they can safely be rerun.
- Parameters
objects – A single object, or a list of objects that should be registered with the Feature Store.
objects_to_delete – A list of objects to be deleted from the registry and removed from the provider’s infrastructure. This deletion will only be performed if partial is set to False.
partial – If True, apply will only handle the specified objects; if False, apply will also delete all the objects in objects_to_delete, and tear down any associated cloud resources.
- Raises
ValueError – The ‘objects’ parameter could not be parsed properly.
Examples
Register an Entity and a FeatureView.
>>> from feast import FeatureStore, Entity, FeatureView, Feature, ValueType, FileSource, RepoConfig >>> from datetime import timedelta >>> fs = FeatureStore(repo_path="feature_repo") >>> driver = Entity(name="driver_id", value_type=ValueType.INT64, description="driver id") >>> driver_hourly_stats = FileSource( ... path="feature_repo/data/driver_stats.parquet", ... timestamp_field="event_timestamp", ... created_timestamp_column="created", ... ) >>> driver_hourly_stats_view = FeatureView( ... name="driver_hourly_stats", ... entities=["driver_id"], ... ttl=timedelta(seconds=86400 * 1), ... batch_source=driver_hourly_stats, ... ) >>> fs.apply([driver_hourly_stats_view, driver]) # register entity and feature view
- config: feast.repo_config.RepoConfig
- create_saved_dataset(from_: feast.infra.offline_stores.offline_store.RetrievalJob, name: str, storage: feast.saved_dataset.SavedDatasetStorage, tags: Optional[Dict[str, str]] = None, feature_service: Optional[feast.feature_service.FeatureService] = None) feast.saved_dataset.SavedDataset [source]
Execute provided retrieval job and persist its outcome in given storage. Storage type (eg, BigQuery or Redshift) must be the same as globally configured offline store. After data successfully persisted saved dataset object with dataset metadata is committed to the registry. Name for the saved dataset should be unique within project, since it’s possible to overwrite previously stored dataset with the same name.
- Returns
SavedDataset object with attached RetrievalJob
- Raises
ValueError if given retrieval job doesn't have metadata –
- delete_feature_service(name: str)[source]
Deletes a feature service.
- Parameters
name – Name of feature service.
- Raises
FeatureServiceNotFoundException – The feature view could not be found.
- delete_feature_view(name: str)[source]
Deletes a feature view.
- Parameters
name – Name of feature view.
- Raises
FeatureViewNotFoundException – The feature view could not be found.
- static ensure_request_data_values_exist(needed_request_data: Set[str], needed_request_fv_features: Set[str], request_data_features: Dict[str, List[Any]])[source]
- get_data_source(name: str) feast.data_source.DataSource [source]
Retrieves the list of data sources from the registry.
- Parameters
name – Name of the data source.
- Returns
The specified data source.
- Raises
DataSourceObjectNotFoundException – The data source could not be found.
- get_entity(name: str, allow_registry_cache: bool = False) feast.entity.Entity [source]
Retrieves an entity.
- Parameters
name – Name of entity.
allow_registry_cache – (Optional) Whether to allow returning this entity from a cached registry
- Returns
The specified entity.
- Raises
EntityNotFoundException – The entity could not be found.
- get_feature_server_endpoint() Optional[str] [source]
Returns endpoint for the feature server, if it exists.
- get_feature_service(name: str, allow_cache: bool = False) feast.feature_service.FeatureService [source]
Retrieves a feature service.
- Parameters
name – Name of feature service.
allow_cache – Whether to allow returning feature services from a cached registry.
- Returns
The specified feature service.
- Raises
FeatureServiceNotFoundException – The feature service could not be found.
- get_feature_view(name: str, allow_registry_cache: bool = False) feast.feature_view.FeatureView [source]
Retrieves a feature view.
- Parameters
name – Name of feature view.
allow_registry_cache – (Optional) Whether to allow returning this entity from a cached registry
- Returns
The specified feature view.
- Raises
FeatureViewNotFoundException – The feature view could not be found.
- get_historical_features(entity_df: Union[pandas.core.frame.DataFrame, str], features: Union[List[str], feast.feature_service.FeatureService], full_feature_names: bool = False) feast.infra.offline_stores.offline_store.RetrievalJob [source]
Enrich an entity dataframe with historical feature values for either training or batch scoring.
This method joins historical feature data from one or more feature views to an entity dataframe by using a time travel join.
Each feature view is joined to the entity dataframe using all entities configured for the respective feature view. All configured entities must be available in the entity dataframe. Therefore, the entity dataframe must contain all entities found in all feature views, but the individual feature views can have different entities.
Time travel is based on the configured TTL for each feature view. A shorter TTL will limit the amount of scanning that will be done in order to find feature data for a specific entity key. Setting a short TTL may result in null values being returned.
- Parameters
entity_df (Union[pd.DataFrame, str]) – An entity dataframe is a collection of rows containing all entity columns (e.g., customer_id, driver_id) on which features need to be joined, as well as a event_timestamp column used to ensure point-in-time correctness. Either a Pandas DataFrame can be provided or a string SQL query. The query must be of a format supported by the configured offline store (e.g., BigQuery)
features – The list of features that should be retrieved from the offline store. These features can be specified either as a list of string feature references or as a feature service. String feature references must have format “feature_view:feature”, e.g. “customer_fv:daily_transactions”.
full_feature_names – If True, feature names will be prefixed with the corresponding feature view name, changing them from the format “feature” to “feature_view__feature” (e.g. “daily_transactions” changes to “customer_fv__daily_transactions”).
- Returns
RetrievalJob which can be used to materialize the results.
- Raises
ValueError – Both or neither of features and feature_refs are specified.
Examples
Retrieve historical features from a local offline store.
>>> from feast import FeatureStore, RepoConfig >>> import pandas as pd >>> fs = FeatureStore(repo_path="feature_repo") >>> entity_df = pd.DataFrame.from_dict( ... { ... "driver_id": [1001, 1002], ... "event_timestamp": [ ... datetime(2021, 4, 12, 10, 59, 42), ... datetime(2021, 4, 12, 8, 12, 10), ... ], ... } ... ) >>> retrieval_job = fs.get_historical_features( ... entity_df=entity_df, ... features=[ ... "driver_hourly_stats:conv_rate", ... "driver_hourly_stats:acc_rate", ... "driver_hourly_stats:avg_daily_trips", ... ], ... ) >>> feature_data = retrieval_job.to_df()
- static get_needed_request_data(grouped_odfv_refs: List[Tuple[feast.on_demand_feature_view.OnDemandFeatureView, List[str]]], grouped_request_fv_refs: List[Tuple[feast.request_feature_view.RequestFeatureView, List[str]]]) Tuple[Set[str], Set[str]] [source]
- get_on_demand_feature_view(name: str) feast.on_demand_feature_view.OnDemandFeatureView [source]
Retrieves a feature view.
- Parameters
name – Name of feature view.
- Returns
The specified feature view.
- Raises
FeatureViewNotFoundException – The feature view could not be found.
- get_online_features(features: Union[List[str], feast.feature_service.FeatureService], entity_rows: List[Dict[str, Any]], full_feature_names: bool = False) feast.online_response.OnlineResponse [source]
Retrieves the latest online feature data.
Note: This method will download the full feature registry the first time it is run. If you are using a remote registry like GCS or S3 then that may take a few seconds. The registry remains cached up to a TTL duration (which can be set to infinity). If the cached registry is stale (more time than the TTL has passed), then a new registry will be downloaded synchronously by this method. This download may introduce latency to online feature retrieval. In order to avoid synchronous downloads, please call refresh_registry() prior to the TTL being reached. Remember it is possible to set the cache TTL to infinity (cache forever).
- Parameters
features – The list of features that should be retrieved from the online store. These features can be specified either as a list of string feature references or as a feature service. String feature references must have format “feature_view:feature”, e.g. “customer_fv:daily_transactions”.
entity_rows – A list of dictionaries where each key-value is an entity-name, entity-value pair.
full_feature_names – If True, feature names will be prefixed with the corresponding feature view name, changing them from the format “feature” to “feature_view__feature” (e.g. “daily_transactions” changes to “customer_fv__daily_transactions”).
- Returns
OnlineResponse containing the feature data in records.
- Raises
Exception – No entity with the specified name exists.
Examples
Retrieve online features from an online store.
>>> from feast import FeatureStore, RepoConfig >>> fs = FeatureStore(repo_path="feature_repo") >>> online_response = fs.get_online_features( ... features=[ ... "driver_hourly_stats:conv_rate", ... "driver_hourly_stats:acc_rate", ... "driver_hourly_stats:avg_daily_trips", ... ], ... entity_rows=[{"driver_id": 1001}, {"driver_id": 1002}, {"driver_id": 1003}, {"driver_id": 1004}], ... ) >>> online_response_dict = online_response.to_dict()
- get_saved_dataset(name: str) feast.saved_dataset.SavedDataset [source]
Find a saved dataset in the registry by provided name and create a retrieval job to pull whole dataset from storage (offline store).
If dataset couldn’t be found by provided name SavedDatasetNotFound exception will be raised.
Data will be retrieved from globally configured offline store.
- Returns
SavedDataset with RetrievalJob attached
- Raises
- list_data_sources(allow_cache: bool = False) List[feast.data_source.DataSource] [source]
Retrieves the list of data sources from the registry.
- Parameters
allow_cache – Whether to allow returning data sources from a cached registry.
- Returns
A list of data sources.
- list_entities(allow_cache: bool = False) List[feast.entity.Entity] [source]
Retrieves the list of entities from the registry.
- Parameters
allow_cache – Whether to allow returning entities from a cached registry.
- Returns
A list of entities.
- list_feature_services() List[feast.feature_service.FeatureService] [source]
Retrieves the list of feature services from the registry.
- Returns
A list of feature services.
- list_feature_views(allow_cache: bool = False) List[feast.feature_view.FeatureView] [source]
Retrieves the list of feature views from the registry.
- Parameters
allow_cache – Whether to allow returning entities from a cached registry.
- Returns
A list of feature views.
- list_on_demand_feature_views(allow_cache: bool = False) List[feast.on_demand_feature_view.OnDemandFeatureView] [source]
Retrieves the list of on demand feature views from the registry.
- Returns
A list of on demand feature views.
- list_request_feature_views(allow_cache: bool = False) List[feast.request_feature_view.RequestFeatureView] [source]
Retrieves the list of feature views from the registry.
- Parameters
allow_cache – Whether to allow returning entities from a cached registry.
- Returns
A list of feature views.
- materialize(start_date: datetime.datetime, end_date: datetime.datetime, feature_views: Optional[List[str]] = None) None [source]
Materialize data from the offline store into the online store.
This method loads feature data in the specified interval from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving.
- Parameters
start_date (datetime) – Start date for time range of data to materialize into the online store
end_date (datetime) – End date for time range of data to materialize into the online store
feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.
Examples
Materialize all features into the online store over the interval from 3 hours ago to 10 minutes ago.
>>> from feast import FeatureStore, RepoConfig >>> from datetime import datetime, timedelta >>> fs = FeatureStore(repo_path="feature_repo") >>> fs.materialize( ... start_date=datetime.utcnow() - timedelta(hours=3), end_date=datetime.utcnow() - timedelta(minutes=10) ... ) Materializing... ...
- materialize_incremental(end_date: datetime.datetime, feature_views: Optional[List[str]] = None) None [source]
Materialize incremental new data from the offline store into the online store.
This method loads incremental new feature data up to the specified end time from either the specified feature views, or all feature views if none are specified, into the online store where it is available for online serving. The start time of the interval materialized is either the most recent end time of a prior materialization or (now - ttl) if no such prior materialization exists.
- Parameters
end_date (datetime) – End date for time range of data to materialize into the online store
feature_views (List[str]) – Optional list of feature view names. If selected, will only run materialization for the specified feature views.
- Raises
Exception – A feature view being materialized does not have a TTL set.
Examples
Materialize all features into the online store up to 5 minutes ago.
>>> from feast import FeatureStore, RepoConfig >>> from datetime import datetime, timedelta >>> fs = FeatureStore(repo_path="feature_repo") >>> fs.materialize_incremental(end_date=datetime.utcnow() - timedelta(minutes=5)) Materializing... ...
- push(push_source_name: str, df: pandas.core.frame.DataFrame, allow_registry_cache: bool = True)[source]
Push features to a push source. This updates all the feature views that have the push source as stream source. :param push_source_name: The name of the push source we want to push data to. :param df: the data being pushed. :param allow_registry_cache: whether to allow cached versions of the registry.
- refresh_registry()[source]
Fetches and caches a copy of the feature registry in memory.
Explicitly calling this method allows for direct control of the state of the registry cache. Every time this method is called the complete registry state will be retrieved from the remote registry store backend (e.g., GCS, S3), and the cache timer will be reset. If refresh_registry() is run before get_online_features() is called, then get_online_features() will use the cached registry instead of retrieving (and caching) the registry itself.
Additionally, the TTL for the registry cache can be set to infinity (by setting it to 0), which means that refresh_registry() will become the only way to update the cached registry. If the TTL is set to a value greater than 0, then once the cache becomes stale (more time than the TTL has passed), a new cache will be downloaded synchronously, which may increase latencies if the triggering method is get_online_features().
- property registry: feast.registry.Registry
Gets the registry of this feature store.
- repo_path: pathlib.Path
- serve(host: str, port: int, no_access_log: bool) None [source]
Start the feature consumption server locally on a given port.
- class feast.FeatureView(*args, name: Optional[str] = None, entities: Optional[Union[List[feast.entity.Entity], List[str]]] = None, ttl: Optional[Union[google.protobuf.duration_pb2.Duration, datetime.timedelta]] = None, batch_source: Optional[feast.data_source.DataSource] = None, stream_source: Optional[feast.data_source.DataSource] = None, features: Optional[List[feast.feature.Feature]] = None, tags: Optional[Dict[str, str]] = None, online: bool = True, description: str = '', owner: str = '', schema: Optional[List[feast.field.Field]] = None, source: Optional[feast.data_source.DataSource] = None)[source]
Bases:
feast.base_feature_view.BaseFeatureView
A FeatureView defines a logical group of features.
- ttl
The amount of time this group of features lives. A ttl of 0 indicates that this group of features lives forever. Note that large ttl’s or a ttl of 0 can result in extremely computationally intensive queries.
- Type
Optional[datetime.timedelta]
- batch_source
The batch source of data where this group of features is stored. This is optional ONLY if a push source is specified as the stream_source, since push sources contain their own batch sources. This is deprecated in favor of source.
- Type
optional
- stream_source
The stream source of data where this group of features is stored. This is deprecated in favor of source.
- Type
optional
- schema
The schema of the feature view, including feature, timestamp, and entity columns.
- Type
List[feast.field.Field]
- features
The list of features defined as part of this feature view. Each feature should also be included in the schema.
- Type
List[feast.field.Field]
- source
The source of data for this group of features. May be a stream source, or a batch source. If a stream source, the source should contain a batch_source for backfills & batch materialization.
- Type
optional
- batch_source: feast.data_source.DataSource
- created_timestamp: Optional[datetime.datetime]
- ensure_valid()[source]
Validates the state of this feature view locally.
- Raises
ValueError – The feature view does not have a name or does not have entities.
- features: List[feast.field.Field]
- classmethod from_proto(feature_view_proto: feast.core.FeatureView_pb2.FeatureView)[source]
Creates a feature view from a protobuf representation of a feature view.
- Parameters
feature_view_proto – A protobuf representation of a feature view.
- Returns
A FeatureViewProto object based on the feature view protobuf.
- last_updated_timestamp: Optional[datetime.datetime]
- materialization_intervals: List[Tuple[datetime.datetime, datetime.datetime]]
- property most_recent_end_time: Optional[datetime.datetime]
Retrieves the latest time up to which the feature view has been materialized.
- Returns
The latest time, or None if the feature view has not been materialized.
- property proto_class: Type[feast.core.FeatureView_pb2.FeatureView]
- schema: List[feast.field.Field]
- source: Optional[feast.data_source.DataSource]
- stream_source: Optional[feast.data_source.DataSource]
- to_proto() feast.core.FeatureView_pb2.FeatureView [source]
Converts a feature view object to its protobuf representation.
- Returns
A FeatureViewProto protobuf.
- ttl: Optional[datetime.timedelta]
- with_join_key_map(join_key_map: Dict[str, str])[source]
Returns a copy of this feature view with the join key map set to the given map. This join_key mapping operation is only used as part of query operations and will not modify the underlying FeatureView.
- Parameters
join_key_map – A map of join keys in which the left is the join_key that corresponds with the feature data and the right corresponds with the entity data.
Examples
Join a location feature data table to both the origin column and destination column of the entity data.
- temperatures_feature_service = FeatureService(
name=”temperatures”, features=[
- location_stats_feature_view
.with_name(“origin_stats”) .with_join_key_map(
{“location_id”: “origin_id”}
),
- location_stats_feature_view
.with_name(“destination_stats”) .with_join_key_map(
{“location_id”: “destination_id”}
),
],
)
- class feast.Field(*, name: str, dtype: Union[feast.types.ComplexFeastType, feast.types.PrimitiveFeastType], tags: Optional[Dict[str, str]] = None)[source]
Bases:
object
A Field represents a set of values with the same structure.
- dtype
The type of the field, such as string or float.
- dtype: Union[feast.types.ComplexFeastType, feast.types.PrimitiveFeastType]
- classmethod from_feature(feature: feast.feature.Feature)[source]
Creates a Field object from a Feature object.
- Parameters
feature – Feature object to convert.
- class feast.FileSource(*args, path: Optional[str] = None, event_timestamp_column: Optional[str] = '', file_format: Optional[feast.data_format.FileFormat] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', s3_endpoint_override: Optional[str] = None, name: Optional[str] = '', description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
- static create_filesystem_and_path(path: str, s3_endpoint_override: str) Tuple[Optional[pyarrow._fs.FileSystem], str] [source]
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- property path
Returns the path of this file data source.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.KafkaSource(*args, name: Optional[str] = None, event_timestamp_column: Optional[str] = '', bootstrap_servers: Optional[str] = None, message_format: Optional[feast.data_format.StreamFormat] = None, topic: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = '', batch_source: Optional[feast.data_source.DataSource] = None)[source]
Bases:
feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.KinesisSource(*args, name: Optional[str] = None, event_timestamp_column: Optional[str] = '', created_timestamp_column: Optional[str] = '', record_format: Optional[feast.data_format.StreamFormat] = None, region: Optional[str] = '', stream_name: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = '', description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = '', batch_source: Optional[feast.data_source.DataSource] = None)[source]
Bases:
feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.OnDemandFeatureView(*args, name: Optional[str] = None, features: Optional[List[feast.feature.Feature]] = None, sources: Optional[Dict[str, Union[feast.feature_view.FeatureView, feast.feature_view_projection.FeatureViewProjection, feast.data_source.RequestSource]]] = None, udf: Optional[method] = None, inputs: Optional[Dict[str, Union[feast.feature_view.FeatureView, feast.feature_view_projection.FeatureViewProjection, feast.data_source.RequestSource]]] = None, schema: Optional[List[feast.field.Field]] = None, description: str = '', tags: Optional[Dict[str, str]] = None, owner: str = '')[source]
Bases:
feast.base_feature_view.BaseFeatureView
[Experimental] An OnDemandFeatureView defines a logical group of features that are generated by applying a transformation on a set of input sources, such as feature views and request data sources.
- features
The list of features in the output of the on demand feature view.
- Type
List[feast.field.Field]
- source_feature_view_projections
A map from input source names to actual input sources with type FeatureViewProjection.
- source_request_sources
A map from input source names to the actual input sources with type RequestSource.
- Type
- udf
The user defined transformation function, which must take pandas dataframes as inputs.
- Type
method
- owner
The owner of the on demand feature view, typically the email of the primary maintainer.
- Type
- created_timestamp: Optional[datetime]
- features: List[feast.field.Field]
- classmethod from_proto(on_demand_feature_view_proto: feast.core.OnDemandFeatureView_pb2.OnDemandFeatureView)[source]
Creates an on demand feature view from a protobuf representation.
- Parameters
on_demand_feature_view_proto – A protobuf representation of an on-demand feature view.
- Returns
A OnDemandFeatureView object based on the on-demand feature view protobuf.
- get_request_data_schema() Dict[str, feast.value_type.ValueType] [source]
- get_transformed_features_df(df_with_features: pandas.core.frame.DataFrame, full_feature_names: bool = False) pandas.core.frame.DataFrame [source]
- infer_features()[source]
Infers the set of features associated to this feature view from the input source.
- Raises
RegistryInferenceFailure – The set of features could not be inferred.
- last_updated_timestamp: Optional[datetime]
- projection: FeatureViewProjection
- property proto_class: Type[feast.core.OnDemandFeatureView_pb2.OnDemandFeatureView]
- source_feature_view_projections: Dict[str, feast.feature_view_projection.FeatureViewProjection]
- source_request_sources: Dict[str, feast.data_source.RequestSource]
- to_proto() feast.core.OnDemandFeatureView_pb2.OnDemandFeatureView [source]
Converts an on demand feature view object to its protobuf representation.
- Returns
A OnDemandFeatureViewProto protobuf.
- udf: method
- class feast.PushSource(*args, name: Optional[str] = None, batch_source: Optional[feast.data_source.DataSource] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
A source that can be used to ingest features on request
- batch_source: feast.data_source.DataSource
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.RedshiftSource(*, event_timestamp_column: Optional[str] = '', table: Optional[str] = None, schema: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, date_partition_column: Optional[str] = None, query: Optional[str] = None, name: Optional[str] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', database: Optional[str] = '', timestamp_field: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
- property database
Returns the Redshift database of this Redshift source.
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Creates a RedshiftSource from a protobuf representation of a RedshiftSource.
- Parameters
data_source – A protobuf representation of a RedshiftSource
- Returns
A RedshiftSource object based on the data_source protobuf.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns a mapping of column names to types for this Redshift source.
- Parameters
config – A RepoConfig describing the feature repo
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- property query
Returns the Redshift query of this Redshift source.
- property schema
Returns the schema of this Redshift source.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- property table
Returns the table of this Redshift source.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a RedshiftSource object to its protobuf representation.
- Returns
A DataSourceProto object.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.RepoConfig(*, registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig] = 'data/registry.db', project: pydantic.types.StrictStr, provider: pydantic.types.StrictStr, online_store: Any = None, offline_store: Any = None, feature_server: Any = None, flags: Any = None, repo_path: pathlib.Path = None, go_feature_retrieval: bool = False, **data: Any)[source]
Bases:
feast.repo_config.FeastBaseModel
Repo config. Typically loaded from feature_store.yaml
- feature_server: Optional[Any]
Feature server configuration (optional depending on provider)
- Type
FeatureServerConfig
- flags: Any
Feature flags for experimental features (optional)
- Type
Flags
- offline_store: Any
Offline store configuration (optional depending on provider)
- Type
OfflineStoreConfig
- online_store: Any
Online store configuration (optional depending on provider)
- Type
OnlineStoreConfig
- project: pydantic.types.StrictStr
Feast project id. This can be any alphanumeric string up to 16 characters. You can have multiple independent feature repositories deployed to the same cloud provider account, as long as they have different project ids.
- Type
- registry: Union[pydantic.types.StrictStr, feast.repo_config.RegistryConfig]
Path to metadata store. Can be a local path, or remote object storage path, e.g. a GCS URI
- Type
- repo_path: Optional[pathlib.Path]
- write_to_path(repo_path: pathlib.Path)[source]
- class feast.RequestFeatureView(name: str, request_data_source: feast.data_source.RequestSource, description: str = '', tags: Optional[Dict[str, str]] = None, owner: str = '')[source]
Bases:
feast.base_feature_view.BaseFeatureView
[Experimental] A RequestFeatureView defines a logical group of features that should be available as an input to an on demand feature view at request time.
- request_source
The request source that specifies the schema and features of the request feature view.
- features
The list of features defined as part of this request feature view.
- Type
List[feast.field.Field]
- owner
The owner of the request feature view, typically the email of the primary maintainer.
- Type
- created_timestamp: Optional[datetime]
- features: List[feast.field.Field]
- classmethod from_proto(request_feature_view_proto: feast.core.RequestFeatureView_pb2.RequestFeatureView)[source]
Creates a request feature view from a protobuf representation.
- Parameters
request_feature_view_proto – A protobuf representation of an request feature view.
- Returns
A RequestFeatureView object based on the request feature view protobuf.
- last_updated_timestamp: Optional[datetime]
- projection: FeatureViewProjection
- property proto_class: Type[feast.core.RequestFeatureView_pb2.RequestFeatureView]
- request_source: feast.data_source.RequestSource
- class feast.RequestSource(*args, name: Optional[str] = None, schema: Optional[Union[Dict[str, feast.value_type.ValueType], List[feast.field.Field]]] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
RequestSource that can be used to provide input features for on demand transforms
- schema
Schema mapping from the input feature name to a ValueType
- Type
List[feast.field.Field]
- owner
The owner of the request data source, typically the email of the primary maintainer.
- Type
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Converts data source config in protobuf spec to a DataSource class object.
- Parameters
data_source – A protobuf representation of a DataSource.
- Returns
A DataSource class object.
- Raises
ValueError – The type of DataSource could not be identified.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns the list of column names and raw column types.
- Parameters
config – Configuration object used to configure a feature store.
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- schema: List[feast.field.Field]
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a DataSourceProto object to its protobuf representation.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- class feast.SnowflakeSource(*, database: Optional[str] = None, warehouse: Optional[str] = None, schema: Optional[str] = None, table: Optional[str] = None, query: Optional[str] = None, event_timestamp_column: Optional[str] = '', date_partition_column: Optional[str] = None, created_timestamp_column: Optional[str] = '', field_mapping: Optional[Dict[str, str]] = None, name: Optional[str] = None, description: Optional[str] = '', tags: Optional[Dict[str, str]] = None, owner: Optional[str] = '', timestamp_field: Optional[str] = '')[source]
Bases:
feast.data_source.DataSource
- property database
Returns the database of this snowflake source.
- static from_proto(data_source: feast.core.DataSource_pb2.DataSource)[source]
Creates a SnowflakeSource from a protobuf representation of a SnowflakeSource.
- Parameters
data_source – A protobuf representation of a SnowflakeSource
- Returns
A SnowflakeSource object based on the data_source protobuf.
- get_table_column_names_and_types(config: feast.repo_config.RepoConfig) Iterable[Tuple[str, str]] [source]
Returns a mapping of column names to types for this snowflake source.
- Parameters
config – A RepoConfig describing the feature repo
- get_table_query_string() str [source]
Returns a string that can directly be used to reference this table in SQL.
- property query
Returns the snowflake options of this snowflake source.
- property schema
Returns the schema of this snowflake source.
- static source_datatype_to_feast_value_type() Callable[[str], feast.value_type.ValueType] [source]
Returns the callable method that returns Feast type given the raw column type.
- property table
Returns the table of this snowflake source.
- to_proto() feast.core.DataSource_pb2.DataSource [source]
Converts a SnowflakeSource object to its protobuf representation.
- Returns
A DataSourceProto object.
- validate(config: feast.repo_config.RepoConfig)[source]
Validates the underlying data source.
- Parameters
config – Configuration object used to configure a feature store.
- property warehouse
Returns the warehouse of this snowflake source.
- class feast.SourceType(value)[source]
Bases:
enum.Enum
DataSource value type. Used to define source types in DataSource.
- BATCH_BIGQUERY = 2
- BATCH_FILE = 1
- BATCH_TRINO = 5
- STREAM_KAFKA = 3
- STREAM_KINESIS = 4
- UNKNOWN = 0
- class feast.ValueType(value)[source]
Bases:
enum.Enum
Feature value type. Used to define data types in Feature Tables.
- BOOL = 7
- BOOL_LIST = 17
- BYTES = 1
- BYTES_LIST = 11
- DOUBLE = 5
- DOUBLE_LIST = 15
- FLOAT = 6
- FLOAT_LIST = 16
- INT32 = 3
- INT32_LIST = 13
- INT64 = 4
- INT64_LIST = 14
- NULL = 19
- STRING = 2
- STRING_LIST = 12
- UNIX_TIMESTAMP = 8
- UNIX_TIMESTAMP_LIST = 18
- UNKNOWN = 0