apache/dolphinscheduler 3.1.0 on GitHub

Feature

Click to expand

[Feature-8252][doc] K8s and namespace manager docs and web page update (#9881)
[feat][task plugin] Add Jupyter task plugin (#9872)
[Feature-9474] [CI] Add cluster test script verify on shell script (#9997)
[Feature-10034][CI] Add postgresql cluster test in CI (#10035)
[feat] Add batch rerun and stop for process instance (#10013)
[Feat][UI] Add resource username for function and resource manage (#10086)
[Feature][plugin] Add k8s task in task plugin (#9425)
[Feature][Task Plugin] Add mlflow task plugin for MLops scenario. (#10036)
[Feature] [CI] Add alert server cluster test #10125
[python] Change distribute name avoid confusion with main pkg (#10134)
[Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker) (#10150)
[Feature][CI] Schedule Dead Link Checker (#10000)
[Feature][MLops] Add MLflow Projects (custom project) component (#10175)
[Feature][Permission] Project management module authority control refactoring, authority abstract interface setting. (#10261)
[Feature][Datasource]replace DataSourceController API with string JSON and cast to DTO (#10276)
[Feature][API] Modify the permissions of project management, security center, data source center and data quality module. (#10289)
[Feature][Permission] Reconstruction of permissions of resource center and monitoring center. (#10307)
[feat][task] Add OpenMLDB task plugin (#10198)
[Feature][Task Plugin] Add support for dynamic form for zeppelin task plugin (#9977) (#10269)
[Feature-10290] [API] Refactor org.apache.dolphinscheduler.api.controller.AccessTokenController#createToken api (#10301)
[Feature] Should not add refresh logic in the e2e case. (#10331)
[Feature][Monitoring] Added disk available capacity monitoring (#10287)
[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373)
[Feature][metrics] Add master, worker metrics (#10326)
[Feature] [MLOps] support mlflow deploy with docker compose (#10217)
[Feature][Metrics] Enable prometheus to collect metrics in standalone mode demo (#10398)
[Feature][UI] Add manual input for scheduling dates. (#10396)
Add mysql registry plugin (#10406)
[CI] Add OWASP Dependency Check (#10058)
[Feature][Improvement] Support multi cluster environments - k8s type (#10096)
[Feature-10411] Add tenant api test (#10442)
[Feature-10363][server] S3 Resource center supports bucket customization (#10433)
[feature][task] Enable zeppelin schedule a whole zeppelin note (#10434)
[Feature] Enable users to switch and install conda env in jupyter task (#10337)
[Feature][Task Plugin] Add DVC task plugin for MLops scenario (#10372) (#10407)
[Feature] [Permission]Data permission module added and Query class api remove verification. (#10504)
[Feature] TaskGroup remove query func permission check. #10557
[python] Support using full-name definitions for resources (#10551)
[feat] ui build no inherit from maven proxy (#10590)
[Feature] Time function analysis extension. (#10624)
[feature][task-flink] Support Flink application mode (#9577)
[Feature][Master] Failover workflow instance base on priority (#10754)
[Feature] Enable users to create python env from requirements.txt (#10658)
[Feature-10629][Improvement] Support multi cluster environments - namespace use cluster (#10650)
[Feature][dependencies] Bump spring-core from 5.3.12 to 5.3.19 (#10857)
[Feature-10219][EMR] EMR supports use to add steps to an existing cluster (#10657)
[Feature-10871] add workflow executing data query (#10875)
[Feat][UI] Uniform search button size. (#10880)
[Feature][Metrics] Add resource download related metrics for workers (#10749)
[Feat][UI] Add title display to the table in the security center. (#10969)
feat: Added global trim method (#10975)
[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#10925)
[Feature][Datax Task]Datax task plugin support custom parameters (#10950)
feat: Add trim method to all input (#10984)
[Feat][UI] Added LDAP function. (#11029)
[Feature][Task Plugin] Increase zeppelin task stability in production (#11010)
[Feature][Task Plugin] Add SageMaker Pipeline task plugin for MLOps scenario (#10826) (#10935)
[python] add import SageMaker task (#11121)
Add base loop task execotor and http template parser (#11137)
[Feature-10604] Task name in the copied and imported workflow without copy suffix (#10607)
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.ProjectController (#10832)
[Feature-10837][Task Plugin] Sqoop Task Add --split-by Parameter (#10960)
[Feature] Add CURD to the project/tenant/user section of the API and add its tests. (#10913)
[Feature] Add new task type chunjun (#10937)
Write alert result into db (#11221)
[Python] Migrate exists method call in PythonGateway to corresponding service (#11188)
[Feature][style] Add spotless maven plugin for automatic style fix. (#11272)
[Python] Combine gateway.entry_point call in python api side (#11330)
[Feature-9978][UI] add node dependency type feature (#11235)
[Feature][Api] Refactor org.apache.dolphinscheduler.api.controller.QueueController (#11154)
[python] Add local dev mod runing python integate tests (#11362)
[Feature-10117][UI] Hide node execution when start from the workflow definition list page (#11372)
[Feature-11223] support stream task (#11350)
[python] Add multiple version of document (#11391)
[python] Add support interpreter version 310 and 311-dev (#11170)
[Feature-7024] Add waiting strategy to support master/worker can recover from registry lost (#11368)
[Feature][Style] Enable spotless to manage imports (#11458)
Support reading application properties from kubernetes config map (#11402)
[Feature-10182][task-datax] datax access kerberos authenticated bigdata cluster (#11387)
[Feature][Style] Make sure there is one and only one empty line at the end of each java file (#11520)
[Feature] New task plugin Pytorch (#11498)
[feature-11408] add hive option (#11493)
feat: Locations Formats the layout if it does not exist (#11535)
[Feature][UI] Enable highlight and auto-complete for SQL scripts (#11659)
[python] Auto location when create process definition (#11681)
[Feature][Task Plugin] Support hive cli task plugin (#11651)
[feature][API]feature: add task type list and enable user to add task type to fav (#11727)
Add Kubernetes configmap reload to all components (#11730)
[Feature-11530] add state history for process instance (#11757)
[DSIP-11][python] create workflows from YAML configuration (#11988)
[Feature][Registry] Support etcd as registry (#10981) (#11997)
[Feat][python] Add parameter environment to task (#11763) (#11989)
[feat][python] Support Pytorch task in python api (#11975)
[feat][python] Add sagemaker yaml example (#11925)
[feat][python] Support OpenMLDB task in python api (#11944)
[feat][python] Support MLflow task in python api (#11962)
[Feature][PyDolphinScheduler] Support DVC task in pyds #11922 (#11941)
[Feature-10273][Datasource] Add dataSource for AWS Athena (#11157) (#12039)
[Feat][UI] Update package. (#12046)
[python] Support SageMaker task type (#11002)
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center (#11708)

Improvement

Click to expand

[refactor][perf] cache compiled regex pattern (#10099)
[improve] Change Mysql Driver (#10220)
[Style][UI] Split multilingual modules. (#10256)
Clean up unused dependencies and packaging issues (#9944)
[Style][UI] Format code. (#10314)
Add dolphinscheduler-task-all in api module (#10350)
Move datasource params to datasouce plugin module (#10353)
issues-10354: upgrade commons-io to fix CVE (#10355)
issue-10356: upgrade logback to fix cve (#10357)
Remove quartz at WorkerServer (#10358)
Add dolphinscheduler-scheduler module (#10360)
[improve] Using create or replace function in sql task (#10170)
Optimize MasterServer, add MasterRPCService (#10371)
change to common lang3 and fix checkstyle (#10380)
Upgrade log4j in E2E (#10359)
Add some warning log in master (#10383)
[Improve] Enhance complement function transformation (#10376)
Update chart.yaml dependencies version (#10420)
[imporve] Add index to table t_ds_relation_process_instance (#10461)
[Improvement][Api] Interface and entity optimization to improve code scalability (#10466). (#10468)
[Improve] Add LDAP user not exsitst action config (#10451)
Modify obsolete swagger parameters in AuditLogController (#10494)
Remove the schedule thread in LowerWeightHostManager (#10310)
[Improvement][Metrics] Apply micrometer naming convention to metrics (#10477)
change import method access to protected (#10536)
[Improve]Remove the code logger.isDebugEnabled() #5918 (#10542)
[Improvement][Metrics] Update some metrics names in grafana-demo dashboards (#10432) (#10552)
Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516)
add datasource health check to the healthcheck endpoint (#10588)
dependent node log output optimization and fix checkstyle (#10593)
[Improvement][Master] batch remove TaskInstaceId and workflowInstanceId (#10620)
[Improvement][Worker] fixed naming of rpc package (#10614)
Add JDK11 on unit-test (#10364)
Validate master/worker config (#10649)
[Improvement]Delete the getSession in the Registry interface (#10633)
[Refactor][UI] Refactor yarn queue manage using NSpace component. (#10680)
[Refactor][UI] Refactor token manage using NSpace component. (#10681)
[Refactor][UI] Refactor k8s namespace manage using NSpace component. (#10682)
[Improvement-10617] Add comment in slot check (#10618)
[Improvement] heartbeat info improvement (#10699)
Optimize sql script use insert ignore (#10700)
[Optimization] Calculate global parameter and local parameter at master. (#10704)
resource modules query fix. (#10687)
[Refactor][UI] Refactor environment manage using NSpace component. (#10702)
[Refactor][UI] Refactor cluster manage using NSpace component. (#10701)
[Enhancement][API] Support batch copy and export online workflow #10678
[Refactor][UI] Refactor worker group manage using NSpace component. (#10735)
[Task] Add Dinky task to better support the development and execution of FlinkSQL (#10640)
[Optimization]Optimize some details of MLFlow task plugin #10740 (#10739)
[Optimization] Config module resource relation fix. (#10718)
[Improvement] [Alert Server] Add default datasource url in alert server application.yaml for consistency (#10790)
[python] Correct some docs for python api (#10772)
[Task] Added Apache SeaTunnel 2.x task node (#10480)
[Refactor][UI] Refactor alarm instance manage using NSpace component. (#10799)
[Refactor][UI] Refactor alarm group manage using NSpace component. (#10803)
Remove quartz in service (#10748)
[Refactor][UI] Refactor user manage using NSpace component. (#10816)
[Improvement] Resource authority module database table unique index increased. (#10807)
[ui] Data Quality Result Display Optimize (#10835)
[Improvement] [CI] Add label robot to pull request (#10848)
[Refactor][UI] Refactor tenant manage using NSpace component. (#10879)
[Python] Supports creating or editing resources. (#10823)
[Refactor][UI] Refactor audio log using NSpace component. (#10991)
impv: not mark issue stale with security label (#10982)
[Refactor][UI] Refactor datasource using NSpace component. (#11022)
[Refactor][UI] Refactor task result using NSpace component. (#11054)
Add dolphinscheduler-bom to manage the dependency version (#11025)
[Improvement][Task Plugin]Refactor JupyterParameters.java with lombok (#11045)
[Improvement][Task]Improve seatunnel task command log (#11027)
[Improve] Hide Change Pwd Page When Login With LDAP (#11041)
[Refactor][UI] Refactor rule manage using NSpace component. (#11069)
[Refactor][UI] Refactor time display use datatable render function (#11078)
[Refactor][UI] Refactor file manage using NSpace component. (#11073)
[Improvement][Task Log] Task status log print description instead of code (#11009)
[Improvement-11013][dolphinscheduler-common] YarnHAAdminUtils#getActiveRMName function Add HTTPS Hadoop environment support (#11017)
[Refactor][UI] Refactor function manage using NSpace component. (#11124)
[Improvement 11072][common] Added HTTPS request support in kerberos environment (#11127)
[Refactor][UI] Refactor resource manage using NSpace component. (#11098)
[Improvement][Metrics] Switch to use tags to indicate task / workflow execution status for metrics (#11128)
[Enhancement][API] Copy workflow with scheduler info (#10677)
Refactor alert-api with lombok (#11133)
[Improvement][ExecutorServiceImpl] remove some unnecessary (#11122)
Add is success in loop task status (#11169)
[python] Refactor structure to avoid cycle import (#11167)
[Improvement][Metrics] Upgrade plugin versions in grafana demo dashboards (#11192) (#11193)
[Improvement][UI] Unified local parameters UI (#11190)
[Refactor][UI] Refactor task group option using NSpace component. (#11209)
[Improvement-10760][dolphinscheduler-api] Some variables are redundant and fix checkstyle (#11213)
[Improvement][UI] Improve monaco-editor component in ts. (#11274)
[Improvement][Metrics] Add metrics for alert server (#11240)
[Improvement-11293][dolphinscheduler-api] remove some variables that redundant in subclasses of AbstractAuthenticator (#11305)
[Improvement][UI] Scrolling will be applied to parent model on editor scrolled to top/bottom (#11335)
[Refactor][UI] Refactor task group queue using NSpace component. (#11341)
[Refactor][UI] Refactor project list using NSpace component. (#11369)
Split ExecutionStatus to WorkflowExecutionStatus and TaskExecutionStatus (#11340)
[Improvement-11394][Task Plugin] Remove Passed Parameter --queue in Seatunnel Task Plugin (#11395)
Add pause to task instance status (#11390)
[Refactor][UI] Refactor project workflow definition using NSpace component. (#11398)
[Improvement-11380][scp-host.sh] Set StrictHostKeyChecking=no option to ssh (#11382)
[Refactor][UI] Refactor workflow definition timing using NSpace component. (#11415)
[Refactor][UI] Refactor workflow definition version using NSpace component. (#11420)
[Improvement] Avoid using search in for and start using testSaveTaskDefine (#11383)
Clear thread name in BaseTaskProcessor (#11422)
[Refactor][UI] Refactor task instance batch using NSpace component. (#11447)
[Refactor][UI] Refactor workflow instance using NSpace component. (#11446)
[Improvement-11386][UI] Concise the logic available for task action buttons (#11419)
[Refactor][UI] Refactor task instance stream using NSpace component. (#11454)
optimize sorting priority for workflow define (#11462)
[Refactor][UI] Refactor task definition batch stream using NSpace component. (#11485)
[Refactor][UI] Refactor task definition version using NSpace component. (#11495)
[test-11432][HttpTask] Add unit tests to http task (#11453)
[Improvement] File deletion logic optimization (#11519)
[Improvement-11468] Support re running historical version workflow (#11489)
[Improvement]Add a stop function when you are ready to do pause operation (#11543)
[Improvement] [LDAP] Bug fix: Config value return "null" instead of real null currently. (#11325)
Remove logger header in task log file (#11555)
[Improvement-11550] [Doc]Document content update (#11577)
Upgrade spring boot and swagger (#11574)
Add dependent task instance log (#11541)
[improvement] Add two parameters in workergroup, and support the application of description display and other parameters (#11542)
[Improvement][TaskInstance] reduce database queries (#11522)
[Improvement-10942][UI]The workflow or task name is too long and results in incomplete display (#10957)
[Improvement][User] Query all users filter un enabled. (#11504)
[Improvement-#11613] Add spi priority factory (#11614)
[improvement-#11630]fix document about common.properties (#11653)
Refactor worker execute task process (#11540)
add unit test ci timeout (#11655)
[Improvement-11662][dao] fix foreach null items in mapper.xml
Refactor heart beat task, use json to serialize/deserialize (#11702)
optimize the process instance query, change the date time range (#11719)
reduce task dependencies (#11731)
[Improvement] Replace commons-lang 2 function invocations with commons-lang3 (#11810)
Bump up dependencies to fix cves (#11765)
Remove not use web service comment in alert server (#11819)
[Improvement-#11768][Monitor] Support monitor h2 database in monitor page (#11813)
[Improve] Upgrade Hadoop to 3.2.4 (#11832)
Remove htrace dependencies (#11881)
[Improvement-11834] Upgrade docker base image to support python3.9 (#11835)
Optimize date range condition (#11911)
Add debug logs for properties read from common.properties (#11910)
Add validations of possible malicious keys (#11966)
[Improvement] and task instance date range search (#11963)
[Improvement-11291][sql] Improvement the log output while executing multiple sql statements (#11292)
Script cannot contains ''' in params (#12068)
[improvement]Add Set cluster name (#12058)
[Improvement] Add Set cluster name (#12074)
add(dependent node's add unprivileged project list): dependent node's add unprivileged project list (#12073)
[improve] Optimize MLFlow task plugin for easy of use (#12071)
Remove dao in worker (#10994)
Move utils, builder under master (#11000)
[Improvement][Worker] Resource may store in hadoop/S3 (#11206)

Bugfix

Click to expand

[Fix][UI][V1.0.0-Beta] Fix the problem of packing failure. (#9962)
[Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (#9952)
[Fix-10002] Fix some bugs in datasource list (#10005)
[Fix][UI] Fix the task name validator error. (#10008)
[Fix-9975] The selected task instance was recreated when the Master service failed over (#9976)
[Fix][UI] Support only one file upload on the file manage page. (#10007)
[Fix][UI] Fix errorOutputPath column in dataquality page (#10015)
[fix-9991][worker]fix statement is closed before resultSet.getMetaData() (#10014)
[dataquality] Fix task commnd null bug (#9974)
[Fix-10039] Flink run command when perfecting Python jobs (#10042)
[Bug] fix run on docker and k8s (#10026)
[python] Fix pre_submit_check error when local params in tasks (#10032)
[Fix][UI]Modify error message (#10054)
Fix flaky filemanage_e2e_test (#10068)
[Fix][UI] Fix in the Chinese state, there is a problem with the multi-language support of the form. (#10066)
[Fix][UI] Fix bug where username is empty in file manage (#10070)
[Bug][Master] fix master task failover (#10065)
[Fix-10080]When the created tenant name is too long, an error message will be prompted (#10081)
Create process definition with task group and task group priority cannot be save into db (#10078)
[Fix][Docker] Support Multi-Architecture Docker Images (#10052)
[Fix-10082]Resource userName is null and udf function manager add a username (#10064)
[Fix-10049] Conditions Task branch flow failed (#10077)
[Fix][UI] fulfill missing fields in DataQuality (#10098)
[fix] Enhance name pre checker in resource center (#10094)
[BUG][TaskGroup] Task group does not take effect (#10093)
[Build][UI] Solve vite config throws terser warning. (#10105)
Fix alert server curator health check not work (#10111)
Fix unit test and remove unused import (#10113)
[Bug][doc] Fix the open-api docs request error (#10100)
[Fix][UI] Add the task status of pending. (#10129)
[Fix][UI] Limit task group resource capacity to only positive integers. (#10158)
[Fix]Change the HTTP or HTTPS regular expression (#10166)
[Fix][UI] Fix the problem that the failure strategy field in the time management form is multilingual. (#10169)
[Fix-10057][api] api parameter name misspelling (#10063)
[Fix][UI] Add seatunnel icon (#10178)
fix spcial week init bug (#10183)
[Fix-10076] queryUserByToken use server timezone (#10177)
[Fix][UI] Fix data-quality component multi-language support. (#10202)
[Fix-10199] [Workflow/Workflow Definition] After deleting messages in batches, the error message is repeated (#10201)
[Bug][Deploy]Fix worker-server path in script scp-hosts.sh (#10208) (#10209)
fix#10214 Delete online workflows in batches, prompt information is accurate (#10215)
[python] Fix process definition attr release state not work (#10151)
[Fix][UI] add assignment of totalPage (#10165)
[Fix-10103][k8s]Fix k8s Change DataSource Error (#10128)
[Fix][UI] Fix the problem of multi-language support of workflow instance host in Chinese state. (#10223)
[fix-10207] When the registered alarm plugin is empty, the error message is not clear (#10224)
[Fix][UI] Fix issue with treemap depth in workflow relationships. (#10229)
[fix] Task group queue query all list error (#10237)
[Fix#9923] The alarm instance management interface is always in the loading state (#9924)
[Fix][UI] Set task group name empty of default value (#10243)
[fix][deploy] File overwrite when specific value set to installPath (#10241)
[fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
[Bug] init docker compose DATABASE env (#10244)
[Fix-10181] Fix the logic of judging that the tenant does not exist (#10185)
[Fix][UI] Fix status statistics card height is inconsistent. (#10268)
[Fix] Fix the timeout alarm button is invalid (#10260)
[Fix] Fix the line.separator problem for alert-plugin' test case #10277 (#10278)
[fix] Fix the SERIAL_DISCARD execution_type not working (#10258)
[Fix][UI] Add mapping_columns filed for the data_quality ruleInputParamter. (#10286)
[Fix][UI] Display the result page when the DB page has no data. (#10298)
[Fix-8828] [Master] Assign tasks to worker optimization (#9919)
Fix database initialize late in standalone. (#10296)
[Fix-10317] [ui] Fix the tips of startTimeout in the jupyter form (#10319)
[Bug][Api-Server]Fix bug when user without tenant uploads file in resource center (#10303)
[Fix] Privilege Reconstruction Data Query Fix & Resource Creation Post Operations (#10313)
[Fix][UI] Fixed the issue that the status statistics on the project overview page were inconsistent. (#10328)
[Fix-10305][common] AWS config Constant definitions are repeated (#10311)
fix bug_10343 (#10344)
[Fix-10318][ui] Fix EMR Task UI custom parameters Is not correct (#10334)
[Fix-10338] Tenant does not belong to resource permission. (#10339)
[Fix] [Worker] Fix worker will hang if fails to start (#10342)
[fix][deploy]fix run on kubernetes bug (#10379)
[Fix-10367] Permission check related key adjustment. (#10368)
[Fix-10412] [Standalone] Failed to start standalone server according to the official document (#10414)
Fixed the problem of not having permission to modify the allocated resource data. (#10410)
[fix][Python] Support same task name in project defferent process definition (#10428)
[DolphinScheduler] Fix wrong class name in doc (#10438)
[Bug][Tools] Add datasource url for mysql profile in tools application.yaml (#10399)
[Fix-10425]Recovery LDAP Config (#10429)
[fix-10386]LDAP user create with status enable (#10426)
[Fix][UI] Add loading before file modification data is returned. (#10450)
[Bug-10394] [Update Possword] Change password shows error (#10437)
[fix][python] Correct variable type of SQL type (#10464)
fix rename form not init bug in file manange (#10471)
Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479)
[Fix] Fix the JSONUtils tool class time zone problem #10282 (#10284)
[fix][master]: fix condition npe (#10478)
Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check (#10491)
[Fix]Recovery Traffic Control Config (#10505)
[Fix-10413] Fix Master startup failure the server still hang (#10500)
[FIX] Could not find or load main class org.apache.zookeeper.ZooKeeperMain (#10535)
[Fix-10522] [API] Fix get executable time null pointer exception (#10523)
[Fix-10452] Serial wait for policy recovery (#10453)
[Fix-10274]Close InitialLDAPContext In LDAP (#10532)
fix: Current node settings number of failed retries may not be negative (#10530)
[fix][doc] Add DVC task plugin doc link (#10529)
[Fix] fix the HttpUtilsTest' test case , it is always time out. (#10294)
[fix] Change K8s postgresql password key name (#10545)
[Fix-10513] Fix the doc of pseudo-cluster mysql installation error #10564
[Fix-10547][data-quality] Fix Table or view not found #10549
[Fix-10514] The master frequently print the same info or error logs, leads to insufficient disk space #10568
fix standalone start error (#10577)
fix dependent task re-run params error (#10580)
[Fix-10400]Fix K8s deployment bitnami version error (#10567)
[Fix][dolphinscheduler-spi] json parse support LocalDateTime (#10440)
[Fix-10539] [Bug] [UI] Copy task a to B, change the user-defined parameter (#10611)
[Fix][UI] Fix bytesToSize function calculation error. (#10627)
[fix-10596] Workflow relationship cannot be displayed normally (#10625)
[fix-10632][ui] fix the workflow name with underline cannot be displayed correctly error. (#10636)
[Bug] [Master] Worker failover will cause task cannot be failover (#10631)
[Bug] [Api] The copied workflow create time and update time is still copied from the original workflow
[Fix-10641] [Common] Fix the hdfs NPE (#10642)
[Bug] [Task] SQL task no exception stack trace in task log (#10656)
[Fix-10652] fix docker-compose.yml (#10653)
[Fix][UI] Fix bug where the home page displays no data. (#10697)
fix bug where project overview page display no data (#10720)
[Bug] [Api-Service] Workflow and subworkflow instance does not change after clicking rerun。 (#10727)
[Fix-10666] Workflow submit failed will still in memory and never retry (#10667)
Fix language not matched (#10775)
[Bug][Script] Remove deprecated arg in remove-zk-node.sh (#10789)
[Fix][UI] Fix the environment name display back error. (#10797)
[API] Do not specify rollback for ensure in case of any exception or error. (#10793)
[Fix-10785] Fix state event handle error will not retry (#10786)
[CI] Fix checkstyle & review dog don't work (#10812)
[python] fix doc build error (#10810)
[python] Fix submit and run error (#10792)
[Fix-10829][UI] display real workflow relation in WorkFlow Relation page (#10836)
[Fix-10842] Fix master/worker failover will cause status incorrect (#10839)
Complement numbers will run in a loop under the serial strategy fixed. (#10862)
[Bug][Api] Complete Data workflow remove duplicates. (#10850)
[Fixed-10833] [Bug] [Quartz] timezone display doesn't match the next_fire_time in ds 3.0.0-beta1 version (#10865)
[Bug][Api] Complete Data workflow remove duplicates. (#10868)
[Fix-10863] Command execution result data statistics remove userId condition. (#10864)
[Fix-10854] Fix database restart may lost task instance status (#10866)
[fix] update queue and tenant with same code error (#10877)
[Fix][UI] Fix hive datasource doesn't show principal bug (#10876)
[Fix-10827] Fix network error cause worker cannot send message to master (#10886)
[bug][task-plugins]fix switch class name and add param check to the switch task plugin (#10894)
[fix] mlflow project success status (#10897)
[fix] delete workflow or task used by other tasks (#10873)
display timing record start time (#10915)
fix bug where task name display in gantt (#10900)
[Fix][UI] Add validator to the next node field of the switch task. (#10898)
Catch exception when check state in StateWheelExecuteThread (#10908)
Solve transaction problems under multi-threading (#10917)
[fix][plugin]: fix spark sql use custom parma. (#10821)
[FIX-10784][Bug] [ALERT-SERVER] FEISHU Plugin might block the whole alert process loop (#10888)
[fix] Can not query exists environments (#10884)
[Fix-10909] Returns executionStatus for each process instance status and access token module support. (#10922)
Fix taskInstance's host is not worker nettyServer address (#10926)
Fix kill yarn job error when failover caused by doesn't set ProcessDefinition (#10948)
[Fix][UI] Set the next node to not required. (#10932)
[fix-10954]Querying the task group list returns empty data (#10955)
fix: typo task definition. (#10958)
[Fix-10890] fix some bugs in the Resource page (#10907)
[Fix-10595][api] Fix Dependent Node Only View Projects Created By Self (#10834)
[python] Fix tasks with multiple upstream and workflow query error (#10941)
[fix] [MLflow Task plugin] Sometimes errors are reported using preset algorithms (#10986) (#10987)
Kill task when we do master failover to avoid task lost (#10997)
fix: typo executor service (#10980)
Fix worker cannot shutdown due to resource close failed or heart beat check failed (#10979)
[hotfix][Worker] Remove service dependency from worker module (#11008)
[Fix-11003]Task group queue is not updated to final state (#11004)
[Bug] fix task params pass (#11053)
[fix-10729]The request's url given by the test case looks different from the controller layer. Some usages and parameter format errors (#10734)
[Fix-11031] AccessToken can only be used by the creator. (#11032)
[E2E] [CI] Fix flaky e2e test (#11077)
Try to fix filemanage e2e flaky test (#11097)
add oracle dependency (#11095)
[Fix][UI] Widen the segment separator input box. (#11096)
fix view variable display bug (#11100)
[Fix][UI] Fix the problem of too large spacing in operation. (#11090)
fix space validate for create folder bug (#11106)
[fix] py sagemaker doc (#11109)
[Fix-10918] Close datasource when expire from guava cache (#11120)
[fix][Data Quality] Data Quality Report Display For Admin Role #10919 (#10977)
[Fix][UI] fix bug where The Gantt chart does not show the execution status (#11135)
[Bug] [API] If the single log length is long, it will causes the View Log page to hang (#10612)
[Fix][UI] Add duplicate of file suffix (#11136)
fix k8s master and worker start error (#11156)
[Bug-11101] fix task failover NPE (#11168)
[Fix-10665] [S3] Fix s3 download method (#10675)
[Fix-11140] Added resource parameters for function verification (#11141)
[Fix] Parameter values disappear when editing Zeppelin task (#11115)
[Bug-11071] fix local params (#11184)
fix nodeSize is not a function bug (#11200)
[BUG-11181] [Document] text check (#11183)
[Fix-11007] [Master] fix forced_success bug (#11088)
[Fix][UI] Fix datax task data instance replay error. (#11222)
fix work oom when task logs's size is very large (#11224)
Add alertPluginInstanceId in AlertInfo (#11231)
Fix recovery from failed task will dead loop (#11239)
fix bug where confirm disable error (#11251)
[Fix][UI] fix bug where log modal display object (#11271)
[fix-11233][ui] Remove custom template switch button in chunjun component module (#11234)
Fix TaskGroup cannot work and will cause master dead loop (#11254)
[fix] Process definition length too long when copy or import (#10621)
no-admin user query alert group fix (#11301)
[Fix][UI] Fix the taskPriority options not showing when hover. (#11294)
[Bug][Doc] Fix typos in docs which causes links to upgrade section to fail (#11333)
[Fix][UI] fix bug where icons are displayed in one line. (#11320)
[fix-11295] remove duplicate classes (#11355)
[Fix-11366][UI] Workflow instance should not support right-click running (#11367)
[Fix-11376][task plugin]Seatunnel Task cannot save successfully when select spark engine (#11377)
[fix-11378] fix API-Server release response message typo (#11400)
[Fix-10392][SqlTask]: Fix list param error when use sql task (#11285)
[Fix-11051][Task]Fix the process exitValue exception when the process times out and the process does not end. (#11099)
[Fix-1360]: Fix Description length check (#11371)
[Fix-11413] Cannot set task status to kill if the task is not in running(#11414)
[Fix][UI] fix bug where warn group name not display (#11429)
[fix-10961][alert server]Change the content of alert to an array (#11033)
Fix find last process instance in dependent task is not correct (#11424)
Fix workflow instance may failover many times due to doesn't check the restart time (#11445)
Set master's task running status in runTask to avoid the task group acquire failed, but the task status is in running (#11451)
If the task processor is not found need to throw error rather than exception (#11461)
[Fix-11465] Serial first, the last executing instance will be stopped (#11466)
[Fix][UI] Fix the table header in the workflow instance. (#11487)
[Fix][UI] Fix the preTask options are unavailable when creating a task in the task definition page. (#11501)
[fix] Database change not sync (#11258)
[Fix][UI] Remove unsupported datasource types in the datax task. (#11527)
Set process instance and task priority default value as 2 (#11539)
[fix-11404]: make the common.properties to configurable on values.yaml (#11441)
Fix some resource leak (#11576)
[Fix][Issue-11579] Cannot check the application status. (#11580)
Fix vulnerability in LDAP login (#11586)
Refresh server startup time when recovery from waiting (#11595)
Fix quartz threadPriority config name error (#11596)
Fix kill task failed will cause the taskGroup cannot release and add taskGroup log (#11469)
[Fix-9980] [Server] fix heartBeatTaskCount bug (#11232)
Fix homepage email subscription link (#11622)
[Fix][UI] Fix the port in the datasource edit. (#11624)
[Fix-11616] Workflow Definition Name Change Validation Optimization. (#11617)
[Fix][UI] Fix the problem that the pagination component is not centered in the environment management. (#11646)
[Fix][UI] Fix bug where crontab special of month and year (#11661)
[fix][python] Sql pattern add truncate. (#11666)
[bug][python] Fix task switch error when default branch not defined last (#11606)
[Fix-11633] The data type of the PostgreSQL database column is wrong. (#11633) (#11634)
[Bug] [Worker] Optimize the getAppId method to avoid worker OOM when kill task (#11701)
[Fix][UI] Add the task name entry when the workflow instance detail page goes to the task instance page. (#11761)
[fix][doc] Update the registry related configuration in values.yaml (#11444) (#11981)
[Bug][Workflow Definition] fix ordinary users can not create depend task (#11961) (#11977)
[Bug-11650][worker] #11650 fix SQL type task, stop task cause NPE (#11668) (#11958)
[Bug] [Master] WorkflowExecuteRunnable will face a infinite loop #11838 (#11864) (#11950)
[Fix][db] fix init&upgrade mysql-meta-schema bugs (#11887) (#11935)
[Bug] Fix procedure task param pass #11919 (#11926)
[Fix-11217] [Doc] add postgresql config in doc: datasource-setting (#11326)
[fix-#11689] change replaceAll to replace in method of tryExecuteSqlResolveColumnNames (#11696)
[fix-#11691] Add lz4-java in clickhouse datasource. (#11697)
Example Change the regular expression of the data source password (#11459) (#11990)
Fix insertOrUpdate plugin may failed due to concurrent operation (#11471) (#11996)
Update dolphinscheduler_ddl.sql fix alert t_ds_user add time_zone is not correct (#11795)
[python] Fix permission error when create related objects (#11783)
[Bug][Dependent]: Id also clone due to duplicate when use dependent mode. (#11929)
Fix listing processDefinition the schedulerReleaseState will never be null (#11888)
Add chunjun in dolphinscheduler-task-all/pom.xml (#11375)
[Bug] [dolphinscheduler-api] Data Analysis Service API responds with jackson deserialization exception (#11858)
[Fix][Monitor]Monitor UI not show DisakAvailable and MemoryUsage correctly (#11870)
fix workflow keep running when task fail (#11930)
Fix cannot save processDefinition (#11931)
Bug Dependent downstream trigger error when schedule cycle not day. (#11734)
Fix update TaskDefinition error (#12064)
[fix][python] Task switch branch not show in webui (#12120)
[Fix] Also auto formatter workflow instance if location is null (#12080)
fix python home conflict (#12112)
[Fix][UI] Fix workflow name overlaps bug (#11862) (#12125)
[BUG]Delete Setting ID (#12096)
Fix insert command error due to the id is not null (#12092) (#12098)
[BUG][BE-12113]fix htrace dependency conflicts (#12126)
[bug][k8s] Change the config file keys in K8s to be the same as those in common.properties (#11812)
[k8s] Delete unnecessary configurations. (#11808)
fix upgrade for 310 (#12137)
[fix#12000]Cannot remove the WorkerGroup from the master service (#12050)
fix env config space bug (#12147)

Doc

Click to expand

[Docs] Upgrade license. (#9967)
[Docs] Update README. (#9970)
[doc] Fix docs deadlink in readme (#9972)
[doc] Update the document picture to new UI (#9953)
[Doc] Add example and notice about task type Conditions (#9945)
[Docs] Fix docker link. (#9998)
[doc] Add the description about execute type in SQL task (#9987)
[doc] Add example and notice about task type Dependent (#10001)
[doc] Correct kubernetes (#9985)
[doc] Correct docs of development-environment-setup (#9995)
[Bug][Document] Fix task group management document link error (#10062)
[Fix-10083][Doc]Change Docker cmd && Change WebexTeams Chinese Translation (#10084)
[doc] Show all the functions of the home page through screenshots (#10118)
[doc] change document structure (#10124)
[doc] Remove flink call docs (#10122)
[doc] Update editorial workflow content and images (#10130)
[doc] Update website docs link (#10161)
[doc] Refactor global parameter docment (#10045)
[doc] Update project directory structure in docs (#10107)
[Doc] Add complie supporting system (#10188)
[doc] Update delete workflow description (#10203)
[doc] Update task DataX document (#10218)
[doc] Update kubernetes' FAQ (#10221)
[doc] Refactor local parameter docment (#10119)
[doc] Delete SPRING_DATASOURCE_DRIVER_CLASS_NAME in doc (#10228)
[doc] Add setValue syntax in local param (#10245)
[doc] Correct all setup documents (#10074)
[doc] Upgrade screenshot to latest version in dev document (#10242)
[doc] Refactor context parameter docment (#10227)
[docs] Added local file configuration guide for resource center (#10264)
[doc] Update the development environment to build the software version (#10246)
[doc] Fix api standard error (#10292)
[doc] Add how to use resource in shell task (#10308)
[doc] Use related path for img (#10325)
[Docs]Update Slack Invite Link (#10362)
[Docs] Upgrade license. (#10365)
[doc] Add switch task demo (#10247)
[doc] Correct resource in HDFS HA cluster (#10445)
[Doc] Add how to set the alarm mailbox in faq (#10401)
[Doc] Add api test doc (#10499)
[doc] Correct the complement add docs (#10521)
[doc] Add docs about complement dependent mode (#10587)
[doc] Add doc about how to use healthcheck endpoints (#10592)
[doc] Add run the task alone doc. (#10581)
[docs] Add links to metrics docs (#10583)
[doc] Correct installPath when using related path (#10597)
[doc] Modify Shell task file (#10634)
[doc] Change subprocess dependent (#10637)
[doc] Modified parameter doc files. (#10659)
[doc] Replace the dag screenshot of the README. (#10662)
[doc] Update metadata and design doc (#10600)
[doc] Add doc for pre-sql and post-sql (#10685)
[doc] Change datasource doc (#10660)
[doc] Change tasks doc (#10639)
[doc] Modified alert doc files (#10714)
[doc] Modify quick start. (#10712)
Merging general settings to faq (#10716)
[Docs] Replace the dag screenshot of the README. (#10741)
[doc] Modified project doc. (#10711)
[doc] Change tasks docs (#10751)
[doc] Unify data source switching of the docs during installation (#10688)
[doc] Replace the dag screenshot of the README. (#10798)
[doc] Modified dq, monitor, security, resources (#10715)
[doc] Modify features, hw, intro and homepage docs. (#10713)
[doc] Correct custom jdbc driver for K8s Doc (#10768)
[Docs] Updated license. (#10838)
[Doc][Upgrade] Fix tiny error in upgrade docs (#10847)
[doc] Add missing doc change by snehal (#10892)
[Doc]Chinese translation updated (#11146)
[doc] Fix bugs in doc installation, design and glossary (#11084)
[doc] Add new page incompatible change (#11153)
[doc] Add notice when environment can not select (#11276)
[Doc] update the configuration doc (#11113)
[doc] migrate community and development docs into main repo (#11313)
[Doc] Add tools/libs to the init database steps (#11345)
[doc] Fixed hadoop link to avoid dead link (#11363)
[doc] Improve the document introduction of complement (#11275)
[doc] Refine the deployment documentation (#11361)
[doc] Recover traffic configuration (#11288)
[doc] Add how obtain IP address from network card in FAQ (#11311)
[Doc][Resources] Instruct users to use local storage if they have remote storage mounted to local (#11435)
[Doc][DSIP] Move DSIP docs to the right place (#11442) (#11443)
[doc] Use Rainbond deployment DolphinScheduler Cluster (#10951)
[Doc][Security] Update instructions on worker groups (#11483)
[Doc] Update UI readme. (#11496)
[Doc][Bug] Fix resource center docs for standalone mode (#11600)
[DOC] should notice that need setting the full path when call the resource file #11620 (#11621)
[Docs] Update README. (#11641)
[doc] Correct E2E Doc, fix WorkerGroupPage typo (#11629)
[DOC] improve zk digest doc (#11703)
[Doc][Improvement] Add instructions for process execution type and data complement (#12121)
add Feishu alert doc and configure email alert doc to the docsdev.js (#11173)
[doc] Fix installation typo (#11220)
[Improvement-11481][docs] Add log specification document to contribution guidelines. (#11484)
Enable spotless to fix markdown checkstyle and refactor faq.md as an example (#11384)
Update doc according to recent update of Spring / Swagger (#11755)
[Doc] Put default task parameters in a new file #11776 (#11957)
[Fix-10830][Doc] Change the start or install script command sh in the document to bash (#10831)
[Doc][Style] Fix doc format once for all (#12006)

Chore

Click to expand

[ci] Correct CODEOWNERS and add new pom.xml owner (#9954)
[Build][UI] Upgrade dependencies. (#9956)
[chore] Combine version 206 and 210 sql upgrade to 300 (#9959)
[ci] Dead link check all markdown file (#10004)
[chore] update by release version (#10299)
[ci] Fix mergeable for ddl change and remove label unexpect (#7099)
[CI] bumping actions/cache to v3 to fix ci error (#10345)
[Build][UI] Upgrade dependencies. (#10346)
[CI] Add code owner of .github (#10347)
[CI] Add some codeowner (#10385)
[chore] Remove unnecessary badge total lines (#10405)
add codeowner (#10457)
add_some_code_owners (#10462)
Open checkstyle, change checkstyle severity from info to warning (#10492)
add codeowner (#10537)
[chore] update by release version (#10742)
[ci] mergable do not handle label (#10638)
Add improvement template (#10764)
Add doc issue template (#10804)
[API-Test] [Checkstyle] Reformat api-test code and enable checkstyle (#10805)
add reviewdog token to enable checkstyle (#10819)
[Build][UI] Updated dependencies. (#10813)
[CI] Try to fix label bot (#10852)
[ci] Fix mergeable will get mad add/remove label sometimes (#10856)
fix label bot again (#10859)
[python] Integrate test run strategy change (#10825)
Add EricGao888 to help review docs, tools, etc. (#11047)
[python] Add directly install without clone source code (#11179)
[ci] Make cluster test work on release branch (#11199)
[community] Remove stackoverflow ref link in website (#11342)
Modifying Slack Links (#11416)
update PyDolphinScheduler documentation link. (#11474)
update slack (#11578)
[CI] Fix cluster test abnormal (#11688)
add lombok config (#11212)