NVIDIA/ncore v20.0.0-beta.1

on GitHub

Highlights

First beta of NCore 20: APIs are expected to be stable for 20.0. Install with pip install --pre nvidia-ncore or nvidia-ncore==20.0.0b1. Compared to NCore 19.8 (details in the alpha.1 and alpha.2 release notes):

• zarr-python 3 support (zarr-python 2 remains supported), and component stores in zarr format v3 (zarr_format=3, opt-in; v2 stays the default).
• Faster reads: 1.7x to 5.5x faster frame reads of local .itar archives.
• Asynchronous APIs: *_async reads of all component readers and of the compat API (AsyncSequenceLoaderProtocol), and asynchronous opening of sequences (SequenceLoaderV4.open_async).
• Remote storage: .itar stores are opened with a single request, and exact range reads are available for synchronous reads as well (itar_remote_range_reads).
• Memory: opened sequences only keep the decoded indices of their .itar stores.
• Camera / lidar ray validity: ray generation returns valid flags (new in this beta, see below).

New since v20.0.0-alpha.2: valid flags of camera and lidar rays

• CameraModel.image_points_to_camera_rays / pixels_to_camera_rays return a CameraModel.CameraRaysReturn with the camera rays and a valid_flag: a ray is valid if the model has a well-defined ray for the image point within its calibrated field of view (fisheye / FTheta: within max_angle, OpenCV pinhole: the undistortion converged, ideal pinhole: always). Invalid rays are undefined and need to be masked.
• The world-ray unprojections forward the flag (WorldRaysReturn.valid_flag), and camera_rays_to_image_points / camera_rays_to_pixels accept a CameraRaysReturn and combine its flag with the projection's, e.g., the Rectificator marks pixels outside of the target's field of view invalid.
• Structured lidar models return their rays' flags as well (elements_to_sensor_rays, elements_to_sensor_points, LidarModel.WorldRaysReturn.valid_flag); all their rays are valid.

Upgrading from NCore 19.x

The public reader / writer APIs of components are unchanged. Code using the sensor models' ray generation, or NCore's APIs below the components (stores, zarr access), needs to be updated:

Sensor models (new in this beta)

• CameraModel.image_points_to_camera_rays / pixels_to_camera_rays return CameraModel.CameraRaysReturn instead of a tensor (use .camera_rays, and mask with .valid_flag). The camera_rays= argument of the image_points_to_world_rays_* / pixels_to_world_rays_* methods requires a CameraRaysReturn, and CameraModel.WorldRaysReturn has a new valid_flag field. Camera models implementing _image_points_to_camera_rays_impl need to return a CameraRaysReturn.
• StructuredLidarModel.elements_to_sensor_rays returns LidarModel.SensorRaysReturn (use .sensor_rays), and elements_to_sensor_points returns LidarModel.SensorPointsReturn (use .sensor_points). The sensor_rays= argument of elements_to_world_rays_shutter_pose requires a SensorRaysReturn, and LidarModel.WorldRaysReturn has a new valid_flag field.
• LidarModel.SensorRayReturn is renamed to LidarModel.SensorRaysReturn.
• OpenCVFisheyeCameraModel's inverse converges for calibrations where it previously didn't; newton_iterations is an upper bound (default 20 instead of 3).

Data APIs below the components (since alpha.1)

• IndexedTarStore is replaced by itar.IndexedTarArchive (records of an archive, independent of zarr-python) and stores.IndexedTarZarrStore (a zarr hierarchy in an archive). Other zarr-python stores are used via stores.ZarrStoreAdapter (e.g., ZarrStoreAdapter.directory(path)).
• stores.consolidate_compressed_metadata / open_compressed_consolidated are replaced by Store.consolidate() / Store.open_root_group().
• Component readers / writers access their data via nodes.Group / nodes.Array (self._group), the wrapped zarr-python objects via .zarr. Array reads return read-only arrays.
• JsonLike moved from ncore.impl.data.types to ncore.impl.data.json_types; documents to write are typed as JsonDocumentLike, values as read as Json / JsonDocument.
• BaseDataConverterConfig requires zarr_format.
• Node metadata records of .itar archives are appended when closing the archive, so archives are no longer byte-identical to archives written by NCore 19.8 (their records are; NCore 19.8 reads them).

Behavior changes (since alpha.2)

• SequenceLoaderProtocol.query_camera_labels returns a Sequence instead of a List.
• Readers release the tail read of each .itar store once its hierarchy is opened from its consolidated metadata, so records near the end of the archives are read from storage when accessed (itar_index_tail_retention="keep" restores the previous behavior).
• Readers no longer check .itar paths for existence before opening them; a missing archive still raises FileNotFoundError, from reading its index.

Changes since v20.0.0-alpha.2

➕ Added

• BREAKING (sensors) return validity flags from structured lidar ray generation (05e6e86) - @janickm
• BREAKING (sensors) return validity flags from camera ray generation (e1b44a8) - @janickm

🔄 Changed

• (sensors) construct sensor return types through their defining class (e7c26c3) - @janickm

🧪 Tests

• (sensors) check camera ray valid flags in the unprojection tests (22db77d) - @janickm

Changes since v20.0.0-alpha.2: v20.0.0-alpha.2...v20.0.0-beta.1
Full diff: v19.8.0...v20.0.0-beta.1