Added
- Web: New Data Observability dashboard for stats on OpenLineage events (
24hrs
, past7.days
); views are also available for sources, datasets, and jobs; new job list view has also been introduced displaying the latestN
runs (and duration) for a given job#2913
@phixMe - Web:
404
page#2890
@phixMe - Web: Display parent job (if present) in job panel
#2868
@phixMe - Web: Allow override of
web.port
viaWEB_PORT
environment variable#2838
@bidlako - Web: Allow nullable columns for schema in dataset panel (use
N/A
)#2896
@phixMe - Web: Better feedback when lineage events are loading
#2916
@NisargChokshi45 - API:
Job
object will now returnJob.latestRuns
(latestN
runs) andJob.latestRun
(last run to execute)#2901
@phixMe - API: Use
io.openlineage.server.*
pkg and classMetadata
(utility class forOpenLineage.RunEvent
)#2853
@wslulciuc - API: Use
TIMESTAMPTZ
for timestamps in database; supports Data Observability dashboard with timezone of user#2924
@wslulciuc - API: Set
current_run_uuid
in tablejobs
optimizing query forJobDao.findAll()
#2929
@wslulciuc - API: New
GET
/api/v1/jobs
#2930
@wslulciuc - CLI: New cmd args for
cli.MetadataCommand
#2923
@wslulciuc--jobs
: limits OL jobs up to N (default: 5)--runs-per-job
: limits OL run executions per job up to N (default: 10)--runs-active
: limits OL run executions marked as active (='RUNNING') up to N--max-run-fails-per-job
: maximum OL run fails per job (default: 2)--min-run-duration
: minimum OL run duration (in seconds) per execution (default: 300)--run-start-time
: specifies the OL run start time in UTC ISO ('YYYY-MM-DDTHH:MM:SSZ'); used for the initial OL run, with subsequent runs starting relative to the initial start time. (default: 2024-10-15T01:00:11.080828Z)--run-end-time
: specifies the OL run end time in UTC ISO ('YYYY-MM-DDTHH:MM:SSZ'); used for the initial OL run, with subsequent runs ending relative to the initial end time. (default: 2024-10-15T01:07:25.080828Z)
Fixed
- Web: Better rendering of long text
#2942
@phixMe - Web: Display full
runID
and check icon when copied#2940
#2941
@wslulciuc @phixMe - Web: Use DatasetVersionAPI to display latest schema and remove extra job facets API call in dataset panel
#2938
@phixMe - Web: Use DatasetAPI for data quality assertions in dataset panel
#2937
@phixMe - Web: Fill-in job node in lineage graph with correct color for
JobEvent
s#2934
@phixMe - Web: Fill-in job node in lineage graph with correct color for run states
RUNNING
,COMPLETED
, etc#2897
@phixMe - API: Pagination for
DatasetVersion.findAll()
; not all dataset versions were returned forGET
/api/v1/namespaces/{namespace}/datasets/{dataset}/versions
#2944
@inanalper
- API:
null
namespace and dataset name in viewdataset_view
for old versions; use tabledataset_versions
instead in column lineage query #2881 @sophiely - API: Missing
DELETE CASCADE
on tablejob_facets
#2878
@mattwparas - API: Ensure
Job.latestRun
inJob
object is set for runs in aRUNNING
state; beforeJob.latestRun
was set only for a run in a done state (COMPLETED
/FAILED
)#2933
@phixMe - CLI: Repurpose cmd
db-migrate
to run all pending database migrations, no longer coupling migrations with HTTP server startup#2936
@davidjgoss - Chart: Missing common
labels
fordeployment.replicas
#2877
@alaturqua