Implemented enhancements:
- Enable user to provide spark job template as input for jobservice deployment #1285 (khorshuheng)
- Add feature table name filter to jobs list api #1282 (terryyylim)
- Report observed value for aggregated checks in pre-ingestion feature validation #1278 (pyalex)
- Add docs page for Azure setup #1276 (jklegar)
- Azure example terraform #1274 (jklegar)
Fixed bugs:
- make EMR jar uploader work the same as k8s one #1284 (oavdeev)
- Don't error when azure vars not set #1277 (jklegar)
- Prevent ingestion job config parser from unwanted fieldMapping transformation #1261 (pyalex)
- Features are not being ingested due to max age overflow #1209 (pyalex)
- Feature Table is not being update when only max_age was changed #1208 (pyalex)
- Truncate staging timestamps in entities dataset to ms #1207 (pyalex)
- Bump terraform rds module version #1204 (oavdeev)
Merged pull requests:
- Use date partitioning column in FileSource #1293 (pyalex)
- Add EMR CI/CD entrypoint script #1290 (oavdeev)
- Online serving optimizations #1286 (pyalex)
- Make third party grpc packages recognizable as python module #1283 (khorshuheng)
- Report observed values in feature validation as Gauge #1280 (pyalex)
- Keep same amount of partitions after repartitioning in IngestionJob #1279 (pyalex)
- Add request feature counter metric #1272 (terryyylim)
- Use SEND_INTERRUPT to cancel EMR jobs #1271 (oavdeev)
- Fix historical test flakiness #1270 (jklegar)
- Allow https url for spark ingestion jar #1266 (jklegar)
- Add project name to feature validation metric #1264 (pyalex)
- Use dataproc console url instead of gcs for log uri #1263 (khorshuheng)
- Make nodes priority (for redis cluster) configurable in Serving #1260 (pyalex)
- Enhance job api to return associated feature table and start time #1259 (khorshuheng)
- Reporting metrics from validation UDF #1256 (pyalex)
- Allow use the same timestamp column for both created & even timestamp in Historical Retrieval #1255 (pyalex)
- Apply grpc tracing interceptor on Feast SDK #1243 (khorshuheng)
- Apply grpc tracing interceptor on online serving #1242 (khorshuheng)
- Python UDF in Ingestion being used for feature validation #1234 (pyalex)
- Add spark k8s operator launcher #1225 (oavdeev)
- Add deadletter/read-from-source metrics to batch and stream ingestion #1223 (terryyylim)
- Implement AbstractStagingClient for azure blob storage #1218 (jklegar)
- Configurable materialization destination for view in BigQuerySource #1201 (pyalex)
- Update Feast Core list features method #1176 (terryyylim)
- S3 endpoint configuration #1169 #1172 (mike0sv)
- Increase kafka consumer waiting time in e2e tests #1268 (pyalex)
- E2E tests support for jobservice's control loop #1267 (pyalex)
- Optimize memory footprint for Spark Ingestion Job #1265 (pyalex)
- Fix historical test for azure #1262 (jklegar)
- Change azure https to wasbs and add azure creds to spark #1258 (jklegar)
- Docs, fixes and scripts to run e2e tests in minikube #1254 (oavdeev)
- Fix azure blob storage access in e2e tests #1253 (jklegar)
- Update python version requirements to 3.7 for Dataproc launcher #1251 (pyalex)
- Fix build-ingestion-py-dependencies script #1250 (pyalex)
- Add datadog(statsd) client to python package for IngestionJob #1249 (pyalex)
- Add prow job for azure e2e test #1244 (jklegar)
- Azure e2e test #1241 (jklegar)
- Add Feast Serving histogram metrics #1240 (terryyylim)
- CI should work on python 3.6 #1237 (pyalex)
- Integration test for k8s spark operator support #1236 (oavdeev)
- Add prow config for spark k8s operator integration testing #1235 (oavdeev-tt)
- Upgrading spark to 3.0.1 #1227 (pyalex)
- Support TFRecord as one of the output formats for historical feature retrieval #1222 (khorshuheng)
- Remove stage_dataframe from the launcher interface #1220 (oavdeev)
- Refactor staging client uploader and use it in EMR launcher #1219 (oavdeev)
- Remove unused EMR code #1217 (oavdeev)
- Remove job id from ingested row counter metric #1216 (terryyylim)
- Quickstart link fixed #1213 (szczeles)
- Delete v1 concepts #1194 (terryyylim)
- Dont write defaults to config #1188 (mike0sv)
- Refactor tests which utilizes feature sets #1186 (terryyylim)
- Refactor configurable options and add sphinx docs #1174 (terryyylim)
- Remove unnecessary Google Auth dependency #1170 (woop)