Arize-ai/phoenix arize-phoenix-v4.6.0

arize-phoenix: v4.6.0

on GitHub

4.6.0 (2024-07-02)

Features

• create_evaluator decorators (#3642) (56acddd)
• ability to clear data older than X date, fix DB constraint errors for span.id from datasets to projects (#3670) (993ad5d)
• add annotations resolver on DatasetRun type (#3473) (c677091)
• Add basic evaluators for string experiment outputs (#3534) (85bec41)
• add dataset-related tables (#3169) (b164dfe)
• add experiment-related tables and migrations (#3381) (b08e8d4)
• add experiments resolver to DatasetExample gql type (#3446) (f526025)
• add graphql resolver for adding spans to datasets (#3205) (b80979e)
• Add LLM evaluators (#3571) (032672b)
• add patchDatasetExamples mutation (#3343) (9ffe198)
• add project resolver on span (#3406) (b64d78b)
• Add relevance evaluator (#3604) (da4a6b3)
• add runs resolver on Experiment type (#3465) (8140957)
• add span resolver on DatasetExample gql type (#3394) (6c46d50)
• auth: ability to set headers via environment variables (ff5b64d)
• compareExperiments resolver (#3481) (2becd18)
• dataset example slideover (#3325) (c64f99b)
• dataset: gql dataset versions connection (#3222) (de28b12)
• datasets: add reference as alias of expected for evaluator argument bindings (#3790) (fdd070a)
• datasets: add client method for appending to datasets (#3659) (9c444a8)
• datasets: add dataframe transformation to dataset (#3736) (fb5730a)
• datasets: add example modal (#3424) (e52867c)
• datasets: add graphql field from trace to project (#3606) (7a54241)
• datasets: add jsonl to download menu (#3495) (fcd6c27)
• datasets: add pagination to dataset examples table (#3299) (33d7a74)
• datasets: add sequence number for experiments of the same dataset (#3486) (1a692cf)
• datasets: add span to dataset from the trace page (#3230) (945af8c)
• datasets: add the ability to create a dataset dynamically (#3712) (81c0cae)
• datasets: allow unrecognized parameters in the evaluator function with default values (#3674) (8b97a5e)
• datasets: capture traces from experiments and their evaluations (#3579) (1917cd7)
• datasets: create dataset UI (#3217) (5183620)
• datasets: dataset upload endpoint (plus fixtures) (#3183) (626f18d)
• datasets: datasets graphql (#3192) (1697d96)
• datasets: datasets page (#3172) (89305fe)
• datasets: Delete dataset mutation (#3321) (053fa31)
• datasets: Delete dataset UI (#3336) (202e9f8)
• datasets: Delete examples (#3352) (42ab894)
• datasets: delete examples mutation (#3324) (febea33)
• datasets: deny v1 routes and gql mutations if readonly (#3501) (de376cf)
• datasets: Display latest version (#3373) (66cd6a8)
• datasets: download csv button (#3312) (e5b83a2)
• datasets: download dataset as CSV text file (#3250) (9629d39)
• datasets: download jsonl for openai (#3493) (e4412ef)
• datasets: example and experiment count on datasets table (#3447) (2e3413a)
• datasets: example experiment runs (#3476) (db592a8)
• datasets: expose the API playgrounds (#3204) (da1416b)
• datasets: get_dataset_by_name (726d97d)
• datasets: gql dataset create (#3203) (679a868)
• datasets: gql for adding examples (#3266) (4049228)
• datasets: gql resolver for dataset example count (#3437) (862bb1f)
• datasets: gql resolver for experiment count (#3443) (5b6bc5c)
• datasets: gql resolver returns examples in descending order (#3448) (624ba10)
• datasets: JSON endpoint to get dataset versions (#3323) (fec38ff)
• datasets: link to view source span (#3413) (faa925e)
• datasets: multi-select on span / traces tables (#3236) (160c4e6)
• datasets: navigate to examples if no experiments exist (cbbed30)
• datasets: post the result of each experiment/evaluation run immediately when it finishes (#3666) (4e21d2c)
• datasets: print experiment summaries (#3709) (7c70afa)
• datasets: print the URL to the dataset when uploaded (#3647) (76439cf)
• datasets: python instructions (#3569) (ee0788a)
• datasets: routing for examples and experiment pages (#3470) (141b90c)
• datasets: show example details in a slide-over (b1a1317)
• datasets: sort by name and createdAt (79f8c88)
• datasets: sort on version (#3370) (41348cf)
• datasets: spans as examples (#3279) (1d46c42)
• datasets: synchronously upload dataset examples returning dataset_id in JSON (#3347) (c32ac4d)
• datasets: UI to edit a dataset example (#3376) (3950256)
• datasets: upload JSON for dataset examples (#3658) (47ef311)
• datasets: usability enhancements (#3773) (912dc9b)
• datasets: version history modal (#3444) (86755a4)
• display average run latency in the experiments table (#3743) (cfaafd5)
• error rate resolver on Experiment type (#3588) (ceaea16)
• Experiments improvements (#3638) (bd85bea)
• experiments: add experiment name (#3512) (801ac29)
• experiments: add the ability to view an experiment's traces (#3603) (084a0c6)
• experiments: comparison details slideover (74d1bd0)
• experiments: delete experiments ui (623805c)
• experiments: delete experiments ui (b942b59)
• experiments: detail view for comparison (ebc4aa1)
• experiments: evaluator icon and ingestion (#3639) (70ba085)
• experiments: evaluator trace slide-over (#3680) (2df5b9d)
• experiments: experiment error rate column (#3657) (41d354f)
• experiments: experiment evaluation summaries in the table (#3575) (85c457a)
• experiments: experiments compare table (47af587)
• experiments: experiments table (#3454) (a9981da)
• experiments: full-text toggle for experiments table (537ed97)
• experiments: gql resolver for experiments (#3404) (6d70786)
• experiments: Implement run_experiment (#3471) (87a0501)
• experiments: navigation to experiments view (#3509) (a293f7e)
• experiments: run count resolver on experiments (#3679) (2444f42)
• experiments: show run count (#3690) (2c79a78)
• experiments: show trace slide-over on experiment page (#3640) (8457cb5)
• experments: ability to view evaluator traces (811290e)
• experments: add the ability to view experiment metadata in full (#3686) (3560e1d)
• experments: minimum viable dialog showing how to run an experiment (#3704) (4fb13b8)
• experments: Switch UI to use experiment name (#3523) (a953231)
• gql resolver for dataset examples (#3238) (fa0b4d2)
• Implement GET /datasets/id and GET /datasets (#3197) (36abede)
• Implement experiments REST API (#3411) (d369fb3)
• implement get_dataset method on phoenix.Client (#3490) (09fb3f0)
• implement initial experiment evals (#3526) (b6fabdf)
• implement patchDataset mutation (#3457) (a0240b3)
• Improve task argument binding and document run_experiment (#3789) (0b64cbe)
• List Dataset Examples (#3271) (d5f4391)
• resolvers for experiment annotation aggregations (#3549) (227e6e0)
• Support repetitions for experiment runs (#3532) (7942694)
• ui: display examples in dataset page (#3277) (829746a)
• Unify run_experiment and evaluate_experiment (#3585) (7e1ffb6)

Bug Fixes

• add tiebreak to versions resolver (#3488) (ac23ec7)
• Address relevance eval feedback (#3609) (b231169)
• datasets: allow duplicate keys for csv upload (#3464) (a0a5b25)
• datasets: api spec for upload endpoint (#3213) (b719267)
• datasets: bug with json upload (#3663) (d667b8f)
• datasets: colab usage of dataset.examples should no longer be list (#3781) (4f148ae)
• datasets: filter examples by dataset in gql (#3330) (e5606e7)
• datasets: free up the output keyword as attribute of experiment run objects (#3793) (6b4db71)
• datasets: get metadata as {} when its value is None in JSON (#3555) (6249ebe)
• datasets: json return payload for upload csv endpoint (#3364) (4a1d063)
• datasets: make tests pass with new client (5cfdc5b)
• datasets: missing annotation trace id (#3664) (d800e36)
• datasets: reconcile Dataset methods (#3508) (43db5bc)
• datasets: select nested rows on traces (#3489) (0bdb860)
• datasets: show full bar on evals of all 1s (#3733) (3faa051)
• datasets: squash experiment run output by "result" key for graphql query (#3672) (20dba43)
• datasets: typo on dict type for typed dict (#3684) (5e8e9a3)
• datasets: update span kind for evaluator with semantic conventions v0.1.9 (#3667) (ff2de45)
• ensure patches are sorted in numeric patch order (#3379) (70facf1)
• experiments: Improve the performance of the table (#3732) (8e33b77)
• experments: fix colab links (#3637) (841ac0d)
• fix annotation trace ts errors (8314aa5)
• json cell for experiment metadata (#3556) (f9e2b6d)
• openapi import error (#3619) (1f81c05)
• openapi yaml parsing for containers (#3788) (959abf7)
• order runs in descending order in runs resolver on Experiment type (#3480) (e1818b7)
• resolve sqlachemy warning regarding remote (#3522) (cd15d9b)
• style and type errors (#3540) (2cba662)
• switch to upload_dataset for examples (#3783) (bea7c2f)
• ui: right align numeric columns (#3587) (781ae7a)

Documentation

• Added more detail prepping and exporting eval data to the Bring Your Own Evaluator section (GITBOOK-704) (96a312b)
• api-ref: fix readthedocs build issues (#3706) (0827726)
• Cleanup datasets section (GITBOOK-694) (18a4d5b)
• Datasets documentaiton (GITBOOK-697) (8148f67)
• Datasets review - fixing typos, syntax, labels, links (GITBOOK-702) (fcb56ee)
• datasets tutorials and quickstart (#3734) (cfa641c)
• datasets: print useful URLs, disable repetitions (#3583) (14c7d9f)
• experiments: prompt template iteration for summarization task (#3669) (0842df4)
• experiments: txt2sql (#3626) (33cd194)
• experiments: txt2sql (#3714) (b083159)
• fix creating datasets (GITBOOK-701) (9b83b1d)
• fix typos (GITBOOK-698) (d413e54)
• GPT-4o first set (GITBOOK-695) (8dff0bf)
• No subject (GITBOOK-696) (88859e1)
• No subject (GITBOOK-699) (9beed78)
• No subject (GITBOOK-700) (5ac466c)
• No subject (GITBOOK-703) (f04e9c5)
• No subject (GITBOOK-707) (2237a88)
• notebook: datasets and experiments quickstart (#3703) (991df49)
• placeholders for experiments (GITBOOK-705) (1f7d183)
• readthedocs (71fceab)
• rest api guidance (#3314) (0309017)
• small fixes (GITBOOK-706) (297458e)
• small fixes (GITBOOK-708) (4990aa5)
• sphinx api-ref for readthedocs (0bcccbd)
• update dataset creation (GITBOOK-711) (51c5ea1)
• use kwargs with datasets (#3748) (530b2c6)
• use kwargs with datasets (#3748) (#3749) (599e340)