What's Changed
π Exciting New Features
π₯ Custom block & pipeline templates
With this release, Magers now have the option to create pipeline templates, then use those to populate new pipelines.
Additionally, you may now browse, create, and use custom block templates in your pipelines. π
by @tommydangerous in #3064, #3042 and #3065
π Save pipelines list filters and groups
Your pipeline filters and groups are now stickyβ that means setting filters/groups will persist through your Mage session.
by @tommydangerous in #3059
Horizontal scaling support for schedulers
You can now run the web server and scheduler in separate containers, allowing for horizontal scaling of the scheduler! Read more in our docs.
Run scheduler only:
/app/run_app.sh mage start project_name --instance-type schedulerRun web server only:
/app/run_app.sh mage start project_name --instance-type web_serverby @wangxiaoyou1993 in #3016
π Run parallel integration streams
Data integration streams may now be executed in parallel!
Update secrets management backend
The secrets management backend can now handle multiple environments!
π Include interval datetimes and intervals in block variables
Interval datetimes and durations are now returned in block variables. Check out our docs for more info!
by @tommydangerous in #3058 and #3068
π Bug Fixes
- Persist error message popup when selecting stream by @johnson-mage in #3028
- Add google oauth2 scopes to bigquery client by @mattppal in #3023
- enable backfills
deleteand add documentation by @mattppal in #3012 - Show warning if no streams displayed by @johnson-mage in #3034
- Fix SLA alerting as soon as trigger run starts by @dy46 in #3036
- Display error when fetching data providers by @johnson-mage in #3038
- Add
/filesroute to backend server and fix TypeError in FileEditor by @johnson-mage in #3041 - Create dynamic block runs in block executor by @wangxiaoyou1993 in #3035
- Fix block settings not updating after saving by @tommydangerous in #3021
- Catch redis error and fix logging by @wangxiaoyou1993 in #3047
- Fix roles query by entity bug by @dy46 in #3049
- API quickstart for rapid development by @mattppal in #3017
- Fix pipelines list filters and group bys not removing by @tommydangerous in #3060
- Fix deleting block by @wangxiaoyou1993 in #3011
- Temporary pymssql fix v2 by @dy46 in #3015
π Enhancements & Polish
- Add
Updated Atcolumn to Pipelines list page by @johnson-mage in #3050 - Default error log to Errors tab when opening log detail by @johnson-mage in #3052
- Add timeout for fetching files for git by @dy46 in #3051
- Encode URI component for file path on files page by @tommydangerous in #3067
- When clicking a file in file browser, select block if in pipeline by @tommydangerous in #3024
New Contributors
Full Changelog: 0.9.4...0.9.8




