Feast integration WIP #1163
Feast integration WIP #1163
Conversation
Making this available for review - I'll be back on the 22nd, and we can pick this up again; I have not started the tutorials, but this covers the majority of the other work identified in the doc plan
✅ Deploy Preview for hardcore-allen-f5257d ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
We can use the existing Feast ticket Fawaz created: https://hazelcast.atlassian.net/browse/AI-62 |
| :description: To use a Feast project, you must configure the feature store. The configuration is defined in a YAML configuration file. | ||
|
|
||
| {description} | ||
|
|
There was a problem hiding this comment.
It would also be great to introduce the "feature repository" concept.
It's actually a directory that stores configuration and feature definitions.
The user is able to change the feature repository (by default the current dir) using the -c DIRECTORY flag.
https://docs.feast.dev/reference/feature-repository
| cluster_name: <YOUR_CLUSTER_NAME> | ||
| cluster_members: [<YOUR_CLUSTER_MEMBERS>] | ||
| creds_username: <YOUR_CRED_USERNAME> | ||
| creds_password: <YOUR_CRED_PASSWORD> | ||
| ca_path: <YOUR_CA_PATH_HERE> | ||
| cert_path: <YOUR_CERT_PATH_HERE> | ||
| key_path: <YOUR_KEY_PATH_HERE> | ||
| ssl_password: <YOUR_SSL_PASSWORD_HERE> |
There was a problem hiding this comment.
The list above seems to be not accurate.
These are the actual configuration keys:
type: "Online store type selector"
cluster_name: default: "Name of the cluster you want to connect. The default cluster name is `dev`"
cluster_members: "List of member addresses which is connected to your cluster default: ['localhost:5701']"
discovery_token: "The discovery token of your Hazelcast Cloud cluster"
ssl_cafile_path: "Absolute path of CA certificates in PEM format"
ssl_certfile_path: "Absolute path of the client certificate in PEM format"
ssl_keyfile_path: "Absolute path of the private key file for the client certificate in the PEM format"
ssl_password: "Password for decrypting the keyfile if it is encrypted"
key_ttl_seconds: "Hazelcast key bin TTL (in seconds) for expiring entities"
|
|
||
| * One-to-one mapping of each feature view to a specific IMap data structure | ||
| + | ||
| This means that every feature view corresponds to an IMap in the Hazelcast cluster, and the entries in that IMap correspond to features of entitites. Each feature value is stored separately, and can be retrieved individually. |
There was a problem hiding this comment.
This paragraph sounds like the user is able to use their pre-existing IMaps and the entries in those with Feast. That's not the case. Feast creates a new IMap for each feature, and the user has no control over the entries in those IMaps.
| type: hazelcast | ||
| cluster_name: <YOUR_CLUSTER_NAME> | ||
| ca_path: <YOUR_CA_PATH_HERE> | ||
| cert_path: <YOUR_CERT_PATH_HERE> | ||
| key_path: <YOUR_KEY_PATH_HERE> | ||
| ssl_password: <YOUR_SSL_PASSWORD_HERE> |
There was a problem hiding this comment.
This section needs an update once the configuration keys in the following section are updated.
| Integrating Feast with Hazelcast allows you to do the following: | ||
|
|
||
| * Use a Hazelcast cluster for your online data store. An IMap in the cluster contains the feature values, identified by the project and feature view names. You can push data to, and retrieve data from, the IMap to provide a real-time online store | ||
| * Connect a gRPC server to stream data. This avoids the need for a temporary IMap sink, provides a real-time solution, and ensures that your data is formatted as expected |
There was a problem hiding this comment.
| * Connect a gRPC server to stream data. This avoids the need for a temporary IMap sink, provides a real-time solution, and ensures that your data is formatted as expected | |
| * Connect a Feast feature server to stream data. This provides a real-time solution, and ensures that your features are always fresh. |
Making this available for review - I'll be back on the 22nd, and we can pick this up again; I have not started the tutorials, but this covers the majority of the other work identified in the doc plan Sorry, @oliverhowell with no sprint active, I couldn't find a way to create a ticket on the doc board for this; we can jimmy rig it to connect the two later
Making this available for review - I'll be back on the 22nd, and we can pick this up again; I have not started the tutorials, but this covers the majority of the other work identified in the doc plan
Sorry, @oliverhowell with no sprint active, I couldn't find a way to create a ticket on the doc board for this; we can jimmy rig it to connect the two later