AcademySoftwareFoundation/OpenImageIO Release-2.2.4.0-beta1

OpenImageIO 2.2 beta1

on GitHub

We have now branched for a 2.2 release (for reference, 2.1 was released in December, 2019). Please be aware of the following:

• We have tagged "Release-2.2.4.0-beta" as the first beta release of the 2.2 family. (Note that the version numbers designated 2.2.0-2.2.3 were earlier developer snapshots.)

• The branch marker RB-2.2 is the tip of development/merging of future 2.2.x releases, which may add features destined for later 2.2.x patch releases, but (certainly after the beta period is over) will avoid any changes that break backwards API, ABI, or linkage compatibility within the 2.2 family.

• Henceforth, 'master' will be home to development for future releases (presumed 2.3?), and thus makes no guarantees at all about compatibility breaks.

• My goal is to have a "beta" period of about two weeks for last minute changes, then one or more "release candidates", culminating in a final official 2.2 release on or close to August 1.

• I strongly encourage you to test the beta while there is plenty of time to change things before a release. Please let us know if you find any bugs or build problems.

• After the official 2.2 release, further patches to the 2.1 family will start to tail off, becoming more conservative and only including the most critical or specifically requested fixes.

The preliminary release notes (subject to revision before final release) are as follows:

Release 2.2 (1 Sept 2020??) -- compared to 2.1

New minimum dependencies:

• pybind11 >= 2.4.2
• openjpeg >= 2.0 (if JPEG-2000 support is desired) #2555 (2.2.2)

New major features and public API changes:

• New IOProxy support: ImageInput and ImageOutput now have direct API level
  support for IOProxy in their open() and create() calls, as well as a
  new set_ioproxy() method in these classes. ImageBuf similarly can specify
  a proxy upon construction for reading, and for writing via a
  set_write_ioproxy() method that applies to subsequent write call.
  #2434 (2.2.0), #2477 (2.2.1).
• Python bindings:
  • Python bindings have been added for missing ParamValue constructors.
    We previously exposed the PV constructors from just a plain int, float,
    or string, but there wasn't an easy way to construct from arbitrary
    data like there is in C++. Now there is. #2417 (2.2.0)
  • ParamValueList.attribute(), when being passed attributes containing
    multiple values, now can have those values passed as Python lists and
    numpy arrays (previously they had to be tuples). #2437 (2.1.11/2.2.0)
  • ImageBufAlgo.color_range_check() is now available to the Python
    bindings (was previously only C++). #2602 (2.2.3)
  • Add a version of ImageBuf.write() that takes an open ImageOutput.
    This is the key to writing a multi-subimage file (such as a multi-part
    OpenEXR) using the Python ImageBuf interface. #2640 (2.2.4)
• ImageBuf:
  • Easier direct use of IOProxy with ImageBuf: constructor and reset()
    for file-reading ImageBuf now take an optional IProxy* parameter,
    and a new set_write_ioproxy() method can supply an IOProxy for
    subsequent write(). #2477 (2.2.1)
  • Add ImageBuf::setpixel() methods that use cspan instead of ptr/len.
    #2443 (2.1.10/2.2.0)
  • Add "missing" reset() varieties so that every IB constructor has a
    corresponding reset() with the same parameters and vice versa. #2460
• ImageBufAlgo:
  • New repremult() is like premult, but will not premult when alpha is
    zero. #2447 (2.2.0)
  • New max() and min() functions take the pixel-by-pixel maximum
    or minimum of two images. #2470 (2.2.1)
• ColorConfig: add OCIO "role" accessors. #2548
• oiiotool subimage support:
  • Nearly all operations now allow an optional :subimages=...
    modifier that restricts the operation to be performed on only a subset
    of named or indexed subimages. See docs for details. #2582
• Low-res I/O of images to terminals that support full color and Unicode
  characters. Just output to a file ending in ".term", and it will convert
  (on certain terminals) to an image displayed directly in the terminal.
  #2631 (2.2.4)
  Try:
  oiiotool myfile.exr -o out.term

Performance improvements:

• Greatly improved TextureSystem/ImageCache performance in highly threaded
  situations where access to the cache was a main bottlenecks. In renders of
  scenes with lots of texture access, with dozens of threads all contending
  for the cache, we are seeing some cases of 30-40% reduction in total
  render time. In scenes that are less texture-bottlenecked, or that don't
  use huge numbers of threads, the improvement is more modest. #2433 (2.2.0)

Fixes and feature enhancements:

• oiiotool:
  • Intelligible error messages (rather than crashes) if you attempt to
    create an image too big to fit in memory. #2414 (2.2.0)
  • --create and --proxy take an additional optional modifier:
    :type=name that specifies the type of buffer to be created (the
    default, as usual, is to create an internal float-based buffer). #2414
    (2.2.0)
  • -o optional argument :type=name is a new (and preferred) synonym
    for what used to be :datatype=. #2414 (2.2.0)
  • --autotrim now correctly trims to the union of the nonzero regions
    of all subimages, instead of incorrectly trimming all subimages to the
    nonzero region of the first subimage. #2497 (2.2.1.2)
  • --subimage now has an optional :delete=1 modifier that causes the
    operation to delete one named or indexed subimage (versus the default
    behavior of extracing one subimage and deleting the others). #2575
    (2.2.3)
  • The list of dependent libraries (part of oiiotool --help) now
    correctly reports the OpenEXR version. #2604 (2.2.3)
  • Fix: --eraseattrib did not correctly apply to all subimages when
    -a or :allsubimages=1 were used. #2632 (2.2.4)
• ImageBuf / ImageBufAlgo:
  • Huge ImageBuf allocation failures (more than available RAM) now are
    caught and treated as an ImageBuf error, rather than crashing with an
    uncaught exception. #2414 (2.2.0)
  • ImageBuf constructors that are passed an ImageSpec (for creating an
    allocated writable IB or "wrapping" a user buffer) now check that the
    spec passed has enough information to know the size of the buffer
    (i.e., it will be recognized as an error if the width, height, depth,
    channels, or data type have not been set validly). #2460
  • Fix: ImageBuf::getchannel() did not honor its wrap parameter.
    #2465 (2.2.1/2.1.12)
  • Fix: IBA::reorient() and IBA::computePixelHashSHA1() did not honor
    their nthreads parameter. #2465 (2.2.1/2.1.12)
  • resample() has been modified to more closely match resize by using
    clamp wrap mode to avoid a black fade at the outer edge of the
    resampled area. #2481
  • Fix: ImageBuf::get_pixels() did not honor the stride parameters.
    #2487. (2.1.12/2.2.1)
  • Fix resize() to avoid a crash / stack overflow in certain cases of
    very big images and very large filter kernels. #2643 (2.2.4)
• ImageCache / TextureSystem / maketx:
  • New IC/TS attribute "trust_file_extensions", if nonzero, is a promise
    that all files can be counted on for their formats to match their
    extensions, which eliminates some redundant opens and format checks
    in the IC/TS and can reduce needless network/filesystem work. Use with
    caution! #2421 (2.2.0)
  • texture3d() fixed some cases where derivative results were not
    correctly copied to the outputs. #2475 (2.2.1)
  • maketx/IBA::make_texture: better error detection and messages when
    using "overscan" textures with formats that can't support it properly.
    (Punchline: only OpenEXR textures can do it.) #2521 (2.2.0)
  • Fix possible redundant tile reads in multithread situations (harmless,
    but makes for redundant I/O). #2557 (2.2.2)
• Exif read: guard better against out of range offests, fixes crashes when
  reading jpeg files with malformed exif blocks. #2429 (2.1.10/2.2.0)
• Fix: ImageSpec::erase_attribute() did not honor its searchtype
  parameter. #2465 (2.2.1/2.1.12)
• Fix: Some ColorProcessor::apply() methods were not using their chanstride
  parameters correctly. #2475 (2.1.12)
• Fix: iinfo return code now properly indicates failures for files that
  can't be opened. #2511 (2.2.2/2.1.13)
• JPEG:
  • Fix resolution unit metadata that was not propery set in JPEG output.
    #2516 (2.2.2/2.1.13)
  • Fix loss of 'config' info upon close/reopen. #2549 (2.2.2)
• OpenEXR:
  • Add support for reading and writing float vector metadata. #2459 #2486
  • Fix bug in the channel sorting order when channels are "X" and
    "Y" (was reversing the order by confusing "Y" for "luminance"). #2595
    (2.1.16/2.2.3)
  • We no longer automatically rename the "worldToNDC" attribute to
    "worldtoscreen" and vice versa. #2609 (2.2.4)
• PNG:
  • Fix loss of 'config' info upon close/reopen. #2549 (2.2.2)
• Raw images:
  • Support for new Canon .cr3 file, but only if you build against
    libraw >= 0.20.0 developer snapshot. #2484 (2.2.1) #2613 (2.2.4)
  • RAW input: set the "raw:flip" attribute if the underlying libraw did a
    reorientation. #2572 (2.1.15/2.2.3)
  • Avoid errors (in libraw) that resulted from multiple threads opening
    raw files at the same time. #2633 (2.2.4)
• RLA:
  • Additional sanity checks and error checks/messages for detecting files
    that might be first mistaken for RLA files, but actually are not.
    #2600 (2.2.3)
• TIFF:
  • Internal improvements to handling metadata retrieval for certain
    unusual tags. #2504 (2.2.2/2.1.13)
  • Fix subtle bug when reading Exif directory in the header. #2540
    (2.2.2)
• Video files:
  • Fix posible infinite loop in the FFMpeg-based reader. #2576
    (2.1.15/2.2.3)

Developer goodies / internals:

• argparse.h:
  • Complete overhaul of ArgParse to make it more like Python argparse.
    Please read the extensive comments in argparse.h for documentation.
    For now, the old ArgParse interface still works, but is considered
    deprecated. #2531 (2.2.2) #2618 #2622 (2.2.4)
• attrdelegate.h:
  • New as_vec<> method returns the whole attribute as a std::vector.
    #2528 (2.2.2)
• filesystem.h:
  • Catch previously uncaught exceptions that could happen in certain
    Filesystem utility calls. #2522 (2.2.2/2.1.13)
  • New write_text_file() convenience function for opening, writing, and
    closing a text file all in one step. #2597 (2.2.3)
• fmath.h:
  • clamp() is 2x faster. #2491 (2.1.12/2.2.2)
  • Very minor fix to OIIO::clamp(), shouldn't affect normal use with
    floats at all, but fixed a subtle quasi-bug in OSL. #2594 (2.1.15/2.2.3)
  • madd() is improved especially on platforms without fma hardware
    #2492 (2.1.12/2.2.2)
  • Perf improvements to fast_sin, fast_cos #2495 (2.1.12/2.2.2)
  • New safe_fmod() is faster than std::fmod. #2495 (2.1.12/2.2.2)
  • New fast_neg is faster than simple negation in many cases, if you
    don't care that -(0.0) is 0.0 (rather than a true -0.0). #2495
    (2.1.12/2.2.2)
  • Add vint4, vint8, and vint16 versions of clamp(). #2617 (2.2.4)
• oiioversion.h:
  • Fix typo that left the OIIO_VERSION_RELEASE_TYPE symbol undefined.
    #2616 (2.2.4/2.1.16)
  • Add new OIIO_MAKE_VERSION(maj,min,patch) macro that constructs the
    proper single integer code for a release version. #2641 (2.2.4/2.1.17)
• paramlist.h:
  • New ParamValueList::find_pv() method that is similar to find() but
    returns a pointer rather than an iterator and nullptr if the attribute
    is not found. #2527 (2.2.2/2.1.13)
  • Add get_indexed() method to ParamValueList and AttrDelegate. #2526
    (2.2.2/2.1.13)
• platform.h:
  • OIIO_PRETTY_FUNCTION definition is more robust for weird compilers
    (will fall back to __FUNCTION__ if all else fails). #2413 (2.2.0)
  • OIIO_ALIGN definition is more robust, will fall back to C++11
    alignas when not a compiler with special declspecs (instead of being
    a compile time error!). #2412 (2.2.0)
  • A variety of OIIO_PRAGMA_... macros have been added to help deal
    with compiler-specific pragmas. #2467 (2.2.1)
• simd.h:
  • vfloat3 has added a normalize(), length(), and length2()
    methods, to more closely match the syntax of Imath::Vec3f. #2437
    (2.1.11/2.2.0)
  • fix errors in vbool == and !=. #2463 (2.1.11/2.2.1)
  • Add float3 versions of abs, sign, ceil, floor, and round (they already
    existed for float4, float8, float16, but not float4). #2612 (2.2.4)
  • Improved support for ARM NEON SIMD (caveat: this is still not well
    tested). #2614 (2.2.4)
  • Improve performance for many float8/int8 functions and operators when
    running on only 4-wide hardware, by using two 4-wide instructions
    instead of reverting to scalar. #2621
• span.h:
  • Allow the constructor from std::vector to allow vectors with custom
    allocators. #2533 (2.2.2)
• strutil.h / ustring.h:
  • New Strutil::concat() and ustring::concat() concatenate two
    strings, more efficiently than sprintf("%s%s") by avoiding any
    unnecessary copies or temporary heap allocations. #2478 (2.2.1)
  • Strutil::upper() and lower() return all-upper and all-lowercase
    versions of a string (like to_lower and to_upper, but not in-place
    modifications of the existing string). #2525 (2.2.2/2.1.13)
  • Strutil::repeat() has been internally rewritten to more efficient by
    avoiding any unnecessary copies or temporary heap allocations. #2478
    (2.2.1)
• typedesc.h:
  • TypeDesc has additional helpers of constexpr values TypeFloat2,
    TypeVector2, TypeVector4, TypeVector2i, TypePointer. #2592 (2.1.16/2.2.3)
• unordered_map_concurrent.h:
  • Fix missing decrement of size() after erase(). #2624 (2.2.4)
• More reshuffling of printf-style vs fmt-style string formatting. #2424
  (2.2.0) #2649 (2.2.4)
• Internals: changed a lot of assertions to only happen in debug build mode,
  and changed a lot that happen in release builds to only print the error
  but not force a termination. #2435 (2.1.11/2.2.0)
• Internals: Replaced most uses of boost::thread_specific_ptr with C++11
  thread_local. #2431 (2.2.0)
• Python: Fixed a bug that lost certain string arguments, especially when
  passing a TypeDesc as its string equivalent. #2587 (2.1.16/2.2.3)
• oiiotool: Big overhaul and simplification of internals. #2586 #2589 (2.2.3)

Build/test system improvements and platform ports:

• CMake build system and scripts:
  • New non-default CMake build flag EXTRA_WARNINGS, when turned on, will
    cause gcc and clang to compile with -Wextra. This identified many new
    warnings (mostly about unused parameters) and fixes were applied in
    #2464, #2465, #2471, #2475, #2476. (2.2.1)
  • FindOpenColorIO.cmake now correctly discerns the OCIO version (2.2.1),
    and now sets up a true imported target. #2529 (2.2.2)
  • FindOpenEXR.cmake has better detection of debug openexr libraries.
    #2505 (2.2.2/2.1.13)
  • Additional cmake controls to customize required vs optional
    dependencies: REQUIRED_DEPS (list of dependencies normally optional
    that should be treated as required) and OPTIONAL_DEPS (list of
    dependencies normally required that should be optional). The main use
    case is to force certain optional deps to be required for your studio,
    to be sure that missing deps are a full build break, and not a
    successful build that silently lacks features you need. #2507
    (2.2.2/2.1.13)
  • Fix exported cmake config file, it was not ensuring that the Imath
    headers properly ended up in the config include path. #2515
    (2.2.2/2.1.13)
  • Change all CMake references to PACKAGE_FOUND to Package_Found (or
    whatever capitalization matches the actual package name). #2569 (2.2.2)
  • The exported CMake config files now set cmake variable
    OpenImageIO_PLUGIN_SEARCH_PATH #2584 (2.1.16/2.2.3)
• Continuous integration (CI) systems:
  • Mostly retire TravisCI for ordinary Linux x64 and Mac builds, now we
    rely on GitHub Actions CI. Nightly test added. Use ASWF docker images
    to test exactly against VFX Platform 2019 and 2020 configurations.
    #2563 (2.2.2) #2579 (2.2.3)
  • Add Travis test for arm64 (aka aarch64) builds. This is still a work
    in progress, and not all testsuite tests pass. #2634 (2.2.4)
  • Our CI tests now have a "bleeding edge" matrix entry that tests against
    the current TOT master build of libtiff, openexr (#2549), and pybind11
    (#2556). (2.2.2)
  • GitHub CI tests, when they fail, leave behind an "artifact" tar file
    containing the output of the tests, so that they can be easily
    downloaded and inspected (or used to create new reference output).
    #2606 (2.2.4)
  • CI Mac tests switch to Python 3.8. (2.2.4)
• Dependency version support:
  • Pybind11 is no longer auto-downloaded. It is assumed to be
    pre-installed. A script src/build-scripts/build_pybind11.bash is
    provided for convenience if you lack a system install. #2503 (2.2.2)
    Bump the minimum pybind11 version that we accept, to 2.4.2 #2453,
    and add fixes to allow support of pybind11 2.5. #2637 (2.2.4)
  • fmt libray: Un-embed fmt headers. If they are not found on the system
    at build time, they will be auto-downloaded. #2439 (2.2.0)
  • Support for building against libraw 0.20. #2484 (2.2.1) #2580 (2.2.3)
  • Build properly against OpenColorIO's current master (which is the
    in-progress work on OCIO v2). #2530 (2.2.2)
  • Fix static boost to not overlink on Windows. #2537 (2.2.2)
  • Fix build breaks against TOT libtiff master, which had #define
    clashes with our GPSTag enum values. #2539 (2.2.2)
  • Ensure compatibility and clean builds with clang 10. #2518 (2.2.2/2.1.3)
  • Support verified for gcc 10, added to CI tests. #2590 (2.2.3)
  • Support for Qt 5.15. #2605 (2.2.3)
  • Fixes to support OpenColorIO 2.0. #2636 (2.2.4)
  • Build against more recent versions of fmtlib. #2639 (2.2.4)
• Progress on support for using Conan for dependency installation. This is
  experimental, it can't yet build all dependencies. Work in progress.
  #2461 (2.2.1)
• The version of gif.h that we embed for GIF output has been updated.
  #2466 (2.2.1)
• The farmhash functions have been cleaned up to be more careful that none
  of their internal symbols are left visible to the linker. #2473 (2.2.1)
• Clarification about .so name versioning: In supported releases, .so
  contains major.minor, but in master (where ABI is not guaranteed stable,
  we name major.minor.patch). #2488 (2.2.1)
• Protect against certain compiler preprocessor errors for user programs
  that include strutil.h but also inculde fmt on its own. #2498.
  (2.1.12/2.2.2)
• Testing of TGA now assumes the test images are in the oiio-images project,
  not separately downloaded (the download location disappered from the net).
  #2512 (2.2.2)
• Build: All the build_foo.bash helper scripts now use set -ex to ensure
  that if any individual commands in the script fails, the whole thing will
  exit with a failure. #2520 (2.2.2/2.1.3)
• Fix compiler warning about incorrect extra braces. #2554 (2.2.2)
• All build-scripts bash scripts now use /usr/bin/env to find bash. #2558
  (2.2.2)
• Avoid possible link errors by fully hiding IBA functions taking IplImage
  parameters, when no OpenCV headers are encountered. #2568 (2.2.2)
• In (obsolete) FindOpenImageIO.cmake, avoid CMake warnings by changing
  the name OPENIMAGEIO_FOUND -> OpenImageIO_FOUND. #2578 (2.2.3)
• Beef up OpenEXR compliance tests, many more examples from openexr-images,
  including many corrupted image failure cases. #2607 (2.2.4)
• "tinyformat" is no longer used, even optionally. We have switched entirely
  to fmtlib, which is more similar to the upcoming C++20 std::format.
  #2647 (2.2.4)
• Moved headers that are not part of OIIO's supported public API, but that
  still must be installed to be transitively included, do a "detail"
  subdirectory. #2648 (2.2.4)

Notable documentation changes:

• Many enhancements in the ImageBuf chapter. #2460 (2.1.11/2.2.0)
• The LICENSE-THIRD-PARTY.md file has been reorganized to be clearer,
  grouping parts with identical licenses. #2469
• Many fixes to the new readthedocs documentation, especially fixes to
  section cross-references and links.
• Improved INSTALL instructions. (2.2.2/2.1.13)
• Fix a variety of breaks on ReadTheDocs. #2581
• Improve the way we discuss optional modifiers.
• Document the PNG output controls for compression level. #2642 (2.2.4)