Bug fixes
- Fix version string in init.py by @qgallouedec in #8244
- fix conda build by @lhoestq in #8250
- Fix JSON loader schema inference for files starting with a UTF-8 BOM (#8241) by @archievi in #8243
- Support hermes traces by @lhoestq in #8255
- Fix batch(by_column=...) crashing after shard/shuffle/split by @pkooij in #8259
- fix traces streaming by @lhoestq in #8277
- support droid agent traces by @cfahlgren1 in #8263
- Fix CI: commit operation equality (hfh 1.20.0) and pytest parametrize collection error by @Wauplin in #8283
- Fix lance auth by @lhoestq in #8301
- Fix symlink-following arbitrary file write in archive extraction by @AAtomical in #8303
- Bug Fix: Resuming Twice Resets the Dataloader by @francesco-bertolotti in #8295
- Bump fsspec and simpler wds compr by @lhoestq in #8337
- fix: validate Arrow IPC record batches by @XciD in #8350
- Make the dataset fingerprint independent of Arrow chunking by @SuryanshSS1011 in #8339
- Fix casting a nullable LargeList to a different inner type by @vineethsaivs in #8346
- docs: replace AutoFeatureExtractor with AutoImageProcessor in image preprocessing docs by @gautamkishore in #8326
- Fix column drop in Arrow path of axis=1 concatenation by @ebarkhordar in #8342
- Raise on length mismatch in batched IterableDataset.map by @sohumt123 in #8332
- Fix require_storage_embed recursing into require_storage_cast by @vineethsaivs in #8349
- Fix path traversal via metadata file_name in folder-based builders by @Kaif10 in #8325
- Support batched=True in Dataset.to_dict by @vineethsaivs in #8333
- Fix hdf5 external files by @lhoestq in #8355
- remove bad require_storage test by @lhoestq in #8357
- ensure fiels are in repo by @lhoestq in #8356
- Keep flat numeric columns with nulls numeric in numpy format by @ebarkhordar in #8352
- Fix bucket dataset card handling and push metadata accounting by @pjh4993 in #8354
- Fix CSV loader dropping on_bad_lines/encoding_errors on pandas 2.0-2.2 by @ebarkhordar in #8358
- Keep integers on the python read path for fixed-shape ArrayXD columns with nulls by @ebarkhordar in #8363
- Rebatch arrow source before formatting in IterableDataset.filter to fix resume data loss by @ebarkhordar in #8360
- Decode Json() columns in Dataset.to_pandas() by @ebarkhordar in #8344
- fix buckets on windows by @lhoestq in #8369
- Fix DatasetDict.push_to_hub leaving removed splits in the dataset card by @pjh4993 in #8367
- Preserve nullable integer columns in to_json/to_csv/to_sql by @ebarkhordar in #8366
Docs
- docs: fix duplicate "to" in IterableDataset push-to-hub example by @DaoyuanLi2816 in #8252
- Clarify dataset creation vs loading workflows in create_dataset tutorial by @zanvari in #8235
New Contributors
- @DaoyuanLi2816 made their first contribution in #8252
- @zanvari made their first contribution in #8235
- @archievi made their first contribution in #8243
- @pkooij made their first contribution in #8259
- @AAtomical made their first contribution in #8303
- @francesco-bertolotti made their first contribution in #8295
- @XciD made their first contribution in #8350
- @SuryanshSS1011 made their first contribution in #8339
- @vineethsaivs made their first contribution in #8346
- @gautamkishore made their first contribution in #8326
- @ebarkhordar made their first contribution in #8342
- @sohumt123 made their first contribution in #8332
- @Kaif10 made their first contribution in #8325
- @pjh4993 made their first contribution in #8354
Full Changelog: 5.0.0...5.0.1