kubeflow/pipelines 2.3.0 on GitHub

To deploy Kubeflow Pipelines in an existing cluster, follow the instruction in here or via UI here

Install python SDK (python 3.7 above) by running:

python3 -m pip install kfp kfp-server-api --upgrade

See the Change Log

What's Changed

chore: Update RELEASE.md by @chensun in #10767
fix(components): Add input param autorater_prompt_parameters to online_evaluation_pairwise component by @copybara-service in #10759
feat(components): create infer preprocessor component by @copybara-service in #10765
feat(components): create utility class for preprocessors and use it in rlhf preprocessor and infer preprocessor by @copybara-service in #10768
chore(sdk): drop Python 3.7 for kfp sdk by @rickyxie0929 in #10750
chore: Promote @rimolive as a backend approver by @chensun in #10780
feat(components): use preprocessor utility methods for the upload model graph by @copybara-service in #10774
chore: upgrade node to v18 by @HumairAK in #10794
feat(GH workflow): migrate periodic functional tests to GH actions by @shruti2522 in #10751
chore: Add Container Engine and Image Tag parameters to backend Makefiles by @gmfrasca in #10725
chore: add meaningful logs to client initialization. by @HumairAK in #10801
feat(kubernetes_platform): Update kubernetes_platform go package to include EnabledSharedMemory by @hsteude in #10703
feat(components): Retry on batch prediction internal errors in AutoSxS by @copybara-service in #10799
fix(components): Add staging and temp locations to prophet trainer component by @copybara-service in #10803
fix(components): Remove unused import function_based from infer pipeline by @copybara-service in #10802
docs(components): Bump the python version to 3.8 by @copybara-service in #10764
chore(components): GCPC 2.14.1 Release by @copybara-service in #10813
chore(components): Rollback GCPC 2.14.1 Release due to b/339911077 by @copybara-service in #10818
chore(components): Add TaskError proto by @copybara-service in #10823
chore(components): GCPC 2.14.1 Release by @copybara-service in #10821
feat(components): Create the write_user_defined_error function by @copybara-service in #10835
test: fix CI failure for periodic functional tests by @tmvfb in #10817
chore(docs): Fix link by @haoxins in #7533
chore: Update Periodic Functional Tests to run once a day by @DharmitD in #10842
test: fix version conflict for functional tests by @tmvfb in #10837
chore: server should skip sample load if path DNE by @DharmitD in #10833
fix(frontend): reduce list run latency by @droctothorpe in #10797
feat(components): Support parsing Gemini BP outputs in AutoSxS pipeline by @copybara-service in #10853
feat(components): Use GetModel integration test to manually test write_user_defined_error function by @copybara-service in #10843
docs: modified the ConfigMap as Env variable example in README.md with right key_name within use_config_map_as_env() block. by @vamsi-01 in #10855
fix(sdk): Add required auth scopes to RegistryClient for GCP service accounts credentials by @PChambino in #10819
chore(deps): bump requests from 2.31.0 to 2.32.2 in /backend/src/apiserver/visualization by @dependabot in #10858
chore: Change command to apply manifests in the docs by @rimolive in #10864
fix(manifests): Move metacontroller to the top in kustmization.yaml by @codablock in #10669
test: Migrate Backend tests to a GHA workflow by @DharmitD in #10871
feat(backend): mount EmptyDir volumes for launcher write locations by @HumairAK in #10857
docs(components): internal by @copybara-service in #10874
feat(components): Add role_field_name and model_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #10875
fix(components): add check and add log to call out the fallback to the default model checkpoint and remove the model checkpoint check condition in RLHF GCPC by @copybara-service in #10828
test: migrate kubeflow-pipeline-e2e-test to GitHub Actions by @hbelmiro in #10887
feat(components): Support dynamic machine type paramters in CustomTrainingJobOp by @KevinGrantLee in #10883
fix(kubernetes_platform): fix api-generator docker mount for SELinux by @gregsheremeta in #10890
feat(components): Add Starry Net forecasting pipeline to public preview by @copybara-service in #10888
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10906
feat(components): Update Starry Net image tags by @copybara-service in #10920
chore: Updated Python image to 3.8 in backend/Dockerfile by @hbelmiro in #10941
Expose starry_net yaml to GitHub by @chensun in #10943
chore(sdk): release KFP SDK 2.8.0 by @chensun in #10940
chore: Upgrade Python version to 3.8 in readthedocs config by @chensun in #10945
feat(frontend&backend): Add UI support for object store customization and prefixes by @HumairAK in #10787
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #10958
chore(components): GCPC 2.15.0 Release by @copybara-service in #10951
docs(backend): Remove deprecated v2 compatibility mode docs by @droctothorpe in #10956
Upgrade go version to 1.21 by @DharmitD in #10911
chore: Add ability to mount self-signed certs to kfp by @DharmitD in #10849
feat(components): add persistent_resource_id to v1 GCPC custom job components/utils by @copybara-service in #10954
chore: Added @rimolive, @hbelmiro and @DharmitD as approvers and reviewers for GitHub Actions by @hbelmiro in #10963
test: Migrate KFP SDK YAPF test to a GHA Workflow by @DharmitD in #10961
test: Created a new GitHub action to reuse the creation of a KFP cluster by @hbelmiro in #10946
test: kubeflow-pipeline-upgrade-test migrated to GitHub Actions by @hbelmiro in #10932
test: Migrate Frontend tests to a GHA Workflow by @DharmitD in #10972
test: install kfp on github periodic functional tests workflow by @tmvfb in #10859
chore: Add an option to re-run GHA workflows by @rimolive in #10930
chore(components): Update Starry Net description and add nan- and zero-threshold args by @copybara-service in #10986
test: Moved KFP backend visualization tests to GitHub Actions by @hbelmiro in #10984
test: Migrate KFP SDK Upgrade Test to GHA by @DharmitD in #10989
test: Migrate KFP SDK Runtime Code Test to GHA by @DharmitD in #10991
chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11000
chore(components): Fix bug due to protobuf library being upgraded by pinning protobuf version by @copybara-service in #11007
chore(components): Add target_field_name as input parameters to llm_evaluation_preprocessor component to support gemini model's input and output schema by @copybara-service in #11028
chore(components): Bump Starry Net images and enforce that TF Record generation always runs before test set generation to speed up pipelines runs by @copybara-service in #11026
chore(components): Add check that component in preview.custom_job.utils.create_custom_training_job_from_component doesn't have any parameters that share names with any custom job fields by @copybara-service in #11033
feat(internal): Adding proto field to support regional quota by @copybara-service in #11001
feat(kubernetes_platform): Add empty dir mount by @gregsheremeta in #10892
test: Migrate KFP SDK Execution Tests to GHA by @DharmitD in #10975
test: Removed mkp tests by @hbelmiro in #10924
fix(sdk): Throw 'exit_task cannot depend on any other tasks.' error when an ExitHandler has a parameter dependent on other task by @hbelmiro in #11005
fix(docs): add note about protoc dependency and version by @gregsheremeta in #10895
chore: Adding @gregsheremeta to SDK reviewers. by @chensun in #11042
chore: Adding @HumairAK to backend and frontend reviewers by @chensun in #11043
fix(docs): IR -> Pipeline Spec by @gregsheremeta in #11031
test: Migrate miscellaneous SDK tests to GHA Workflow by @DharmitD in #11032
chore: Adding @gregsheremeta to SDK reviewers (kubernetes_platform) by @gregsheremeta in #11044
test: Migrate KFP SDK Tests to GHA by @DharmitD in #11037
chore: adding @hbelmiro to kubernetes_platform/OWNERS by @hbelmiro in #11045
test: Moved kfp-kubernetes-execution-tests to GitHub Actions by @hbelmiro in #10926
add dev docs for kfp/sdk by @HumairAK in #11046
feat(components): Support dynamic machine parameters in preview.custom_job.utils.create_custom_training_job_from_component by @copybara-service in #10974
test: Moved kubeflow-pipelines-samples-v2 to GitHub Actions by @hbelmiro in #11048
feat(components): internal by @copybara-service in #11050
feat(component): internal by @copybara-service in #11049
test: Add script and GitHub Action for kfp-kubernetes-library by @diegolovison in #11006
chore(components): GCPC 2.16.0 Release by @copybara-service in #11065
fix: Incorrect typing in samples/cores/loop_parallism and fixing loop_parameter examples by @papagala in #11062
feat(components): internal by @copybara-service in #11070
chore: Fixed e2e tests by @hbelmiro in #11072
chore(components): Bump image version for Structured Data pipelines by @copybara-service in #11073
feat(components): internal by @copybara-service in #11074
feat(components): internal by @copybara-service in #11076
feat(components): internal by @copybara-service in #11081
feat(api): Add new ResourceSpec proto fields to support dynamic values. by @KevinGrantLee in #11075
feat(components): internal by @copybara-service in #11082
feat(components): internal by @copybara-service in #11083
docs: Fix environment variables names by @diegolovison in #11013
feat(backend): move comp logic to workflow params by @zazulam in #10979
feat(component): internal by @copybara-service in #11085
feat(components): internal by @copybara-service in #11087
fix(components): Fix to model batch explanation component for Structured Data pipelines by @copybara-service in #11088
feat(components): Support dynamic values for boot_disk_type, boot_disk_size in preview.custom_job.utils.create_custom_training_job_from_component by @copybara-service in #11055
chore: Upgrade Argo to v3.4.17 (fixes #10977) by @gmfrasca in #10978
test: Moved kubeflow-pipelines-manifests to GitHub Actions by @VaniHaripriya in #11066
fix: re-enable exit hanler test. by @liavweiss in #11100
fix(frontend): retrieve archived logs from correct location by @droctothorpe in #11010
feat(component): internal by @copybara-service in #11107
feat(component): internal by @copybara-service in #11110
chore(components): GCPC 2.16.1 Release by @copybara-service in #11111
test: Fail fast when image build fails on tests #11102 by @ElayAharoni in #11115
fix(components): Use instance.target_field_name format for text-bison models only, use target_field_name for gemini models by @copybara-service in #11118
chore: Renamed GitHub workflows from *.yaml to *.yml for consistency by @hbelmiro in #11126
Fix view edit cluster roles by @papagala in #11067
fix(components): Pass moddel name to eval_runner to process batch prediction's output as per the output schema of model used by @copybara-service in #11119
feat(components): release LLM Model Evaluation image version v0.7 by @copybara-service in #11128
chore: Adding @DharmitD to SDK reviewers by @DharmitD in #11131
test: Kubeflow Pipelines V2 integration Tests by @diegolovison in #11125
chore: Add make targets for building driver and launcher images by @gmfrasca in #11103
feat(Backend + SDK): Update kfp backend and kubernetes sdk to support EmptyDir by @gregsheremeta in #10913
docs:fixing broken links in readme by @Fiona-Waters in #11108
chore(deps): bump micromatch from 4.0.5 to 4.0.8 in /test/frontend-integration-test by @dependabot in #11132
Fix: Basic sample tests - sequential is flaky by @diegolovison in #11138
chore: Wrapped "Failed GetContextByTypeAndName" error for better troubleshooting by @hbelmiro in #11098
chore(components): Update AutoSxS and RLHF image tags by @copybara-service in #11116
test: Improvements to wait_for_pods function by @hbelmiro in #11162
fix(frontend): fixes filter pipeline text box shows error when typing anything in it. Fixes #10241 by @ElayAharoni in #11096
correct artifact preview behavior in UI by @HumairAK in #11059
chore: Added DCO link to PR template by @hbelmiro in #11176
chore(backend): Update driver and launcher licenses by @chensun in #11177
chore(backend): update driver and launcher default images by @chensun in #11178
chore: Add instructions for releasing driver and launcher images by @chensun in #11179
test: Fixed kfp-runtime-tests to run on master branch by @hbelmiro in #11158
(fix): reduce executor logs by @HumairAK in #11169
chore: add PaulinaPacyna and ouadakarim as reviewers by @chensun in #11180
test: Move run-all-gcpc-modules to GitHub Actions by @aman23bedi in #11157
fix(sdk): Kfp support for pip trusted host by @diegolovison in #11151
chore(sdk): Loosening kubernetes dependency constraint by @egeucak in #11079
chore: Remove unwanted Frontend test files by @DharmitD in #10973
fix(ui): fixes empty string value in pipeline parameters by @jan-stanek in #11175
chore(backend): update driver and launcher default images by @chensun in #11182

New Contributors

@shruti2522 made their first contribution in #10751
@tmvfb made their first contribution in #10817
@vamsi-01 made their first contribution in #10855
@PChambino made their first contribution in #10819
@codablock made their first contribution in #10669
@diegolovison made their first contribution in #11006
@papagala made their first contribution in #11062
@zazulam made their first contribution in #10979
@VaniHaripriya made their first contribution in #11066
@liavweiss made their first contribution in #11100
@ElayAharoni made their first contribution in #11115
@Fiona-Waters made their first contribution in #11108
@aman23bedi made their first contribution in #11157
@egeucak made their first contribution in #11079
@jan-stanek made their first contribution in #11175

Full Changelog: 2.2.0...2.3.0

kubeflow/pipelines 2.3.0 Version 2.3.0 on GitHub

What's Changed

New Contributors

kubeflow/pipelines 2.3.0
Version 2.3.0

on GitHub