github anomalyco/opentui v0.8.1
Release v0.8.1
on GitHub

3 hours ago

OpenTUI 0.8.1 can play video. VideoRenderable, and <video> in React and Solid, plays an MP4 file with H.264 video and AAC audio. Native code does the decoding, the media clock, the audio, and the drawing. Frames draw like an ImageRenderable, with Kitty, Sixel, or Unicode blocks: 

const video = new VideoRenderable(renderer, {
  source: "./clip.mp4",
  width: 60,
  height: 20,
  autoplay: true,
  loop: true,
  onError: console.error,
})
renderer.root.add(video)

To show only part of a clip, you could make a node the size of the whole board and let its parent clip it. That gets expensive quickly. Nine such tiles of one 1080p clip used 2.3 cores and sent 58 MB of Kitty data per frame, which is far over the output limit. The new crop option selects a region in source pixels, and the decoder scales and copies only that region. The same nine tiles use 0.93 cores and 5.1 MB per frame. There is a second cost: each player of a clip with audio opens its own audio device, even when it is muted. Set audio: false on every tile except the one that plays sound: 

// The top-left quarter of a 1920x1080 clip, with no audio device.
const tile = new VideoRenderable(renderer, {
  source: "./clip.mp4",
  crop: { left: 0, top: 0, width: 960, height: 540 },
  audio: false,
  width: 30,
  height: 10,
})

ImageRenderable takes the same crop. To decode video, the native library now statically links a decode-only LGPL build of FFmpeg 9.0.2. This makes each stripped library 2.6 to 3.5 MB larger. On linux-x64 it grows from 5.20 MB to 8.40 MB.

Over a slow link such as ssh, the terminal reads frames more slowly than OpenTUI writes them. Once the pty and the link filled up, the JavaScript thread waited inside write(2), and keys and timers waited behind it. On a 48 KB/s link it spent 79% of its time there, and the median key wait was 123.7 ms. In 0.8.1, on Linux and macOS when stdout is a terminal, a native thread owned by the Session does the write. On the same link, the JavaScript thread spends no time in write(2), and the median key wait is 0.1 ms. The screen still lags by the size of the link's buffers. The app also uses more CPU per run, because it now does the updates that it used to skip while it waited. Set OTUI_STDOUT_THREAD=0 to write on the JavaScript thread as before.

A slow link also made exit unreliable. In 0.8.0, the close gave up after one second and cancelled the session before it wrote the restore sequence. A 40-row quit at 16 KB/s left the terminal in the alternate screen with mouse reporting on. Now the close waits as long as output keeps moving. If nothing is written for one second, or if the close has run for ten seconds, the renderer stops waiting. It lets the write in progress finish, drops the queued output, writes the restore, and rejects closed as late.

The Session writes at most 8 MiB in one frame. In 0.8.0, a larger frame failed, and the repaint after it was just as large. The screen froze, but input still worked. Three 1000x700 Kitty images were enough to cause it. Now a frame sends its cells and the images that fit, and later frames send the rest. If one image is too large for a frame on its own, OpenTUI sends a copy at the size it is shown. A 1272x2015 photo shown at 880x1400 pixels sends 6.6 MB instead of 13.7 MB. The image draws as blocks only if that copy is too large as well. If a frame's cells alone are too large, the new frame:dropped event reports it once for each content. In 0.8.0, console.error reported it on every frame: 

renderer.on(CliRenderEvents.FRAME_DROPPED, ({ bytes, limit }) => {
  console.warn(`dropped a frame of ${bytes} bytes (limit ${limit})`)
})

One layout change can show in existing apps. When Yoga gives a node zero width or height, OpenTUI used to paint and hit-test it as one cell, on top of its siblings. A common case is a one-row header above a ScrollBox with flexGrow. Once the list is taller than the screen, Yoga shrinks the header to zero rows. The header and the first list row then mixed into one line. Now the header is not drawn at all. To keep it, give the header flexShrink: 0, or give the ScrollBox flexBasis: 0: 

const column = new BoxRenderable(renderer, { flexDirection: "column", height: "100%" })
column.add(new TextRenderable(renderer, { content: "filter: (type)", flexShrink: 0 }))
column.add(new ScrollBoxRenderable(renderer, { flexGrow: 1, minHeight: 0 }))

Breaking changes

  • native: ot_session_renderer_state is 40 bytes instead of 32. Its reserved field at offset 28 is now image_uploads_pending, and dropped_frames and dropped_frame_bytes follow it, so allocate 40 bytes. See Host I/O and time. (#1890)
  • native: The native library and the Zig module need Zig 0.17.0. A Zig 0.16 build cache or FFmpeg prefix is rebuilt or rejected. See Zig. (#1586)

Added

  • core: VideoRenderable plays an MP4 file with H.264 video and AAC audio, with play(), pause(), seek(), loop, volume, maxFps, and callbacks such as onReady and onEnd. With audio: false, it opens no audio device and follows the wall clock. See Video. (#1800, #1884, #1883, #1885, #1887, #1889)
  • react: <video> renders a VideoRenderable, and VideoProps types its props. See Video. (#1800, #1884)
  • solid: <video> renders a VideoRenderable, and VideoProps types its props. See Video. (#1800, #1884)
  • core: ImageRenderable and VideoRenderable take crop: { left, top, width, height } in source pixels, and fit applies to that region. An invalid crop calls onError and keeps the previous crop. See Image and Video. (#1888)
  • core: The frame:dropped event (CliRenderEvents.FRAME_DROPPED) reports a frame that is too large for the output once for each content, with { bytes, limit }. It replaces the console.error call on each frame. See Renderer. (#1890)
  • core: A text draw in NativeBufferDraw takes maxWidth and ellipsis, which fit the text to a cell width and end wider text with "…". See Buffer API. (#1828)
  • native: OT_SCENE_VIDEO, ot_scene_set_video, and the ot_video_* functions add a video node to a scene. ot_scene_set_crop sets the source region of a video or image node, and the OT_VIDEO_OPEN_NO_AUDIO flag of ot_video_open opens no audio device. See C. (#1800, #1884, #1888)
  • native: ot_session_post_stdout gives queued output to the stdout writer thread of the Session and reports OT_STDOUT_IDLE or OT_STDOUT_IN_FLIGHT. ot_session_drop_output drops queued output but keeps the copy in flight and the restore. See Host I/O and time. (#1858, #1893)
  • native: The text draw record has a max_width field and the OT_BUFFER_DRAW_FIT_ELLIPSIS flag. Records of 44 bytes from older bindings still draw without a limit. See C. (#1828)

Changed

  • core: On Linux and macOS, when stdout is a terminal, a native thread writes the renderer output, so a slow link does not block keys and timers. Set OTUI_STDOUT_THREAD=0 to write on the JavaScript thread. See Environment variables. (#1858, #1881)
  • core: In split-footer mode, captured stdout has a limit of 64 MiB. From half the limit, stdout.write() returns false until 'drain', and a write past the limit replaces the oldest unwritten rows with one [N lines dropped] row. See Renderer. (#1856)
  • core: The console capture that console.dir fills keeps only its newest 1,000 writes. See Console overlay. (#1856)
  • core: A node with zero width or height from Yoga draws nothing and gets no mouse events, but its children still paint unless it clips them. To keep a header above a growing ScrollBox, give the header flexShrink: 0. See ScrollBox and Layout. (#1869)
  • core: setProgramStatus() accepts reports until the terminal close starts, so a destroy listener or a Solid onCleanup can clear a record at exit. On a failed session, it returns false instead of throwing. See Program status. (#1871)
  • core: TextTableRenderable, SelectRenderable, and TabSelectRenderable draw into the frame unless buffered is true, and they keep no offscreen buffer. Translucent colors blend once over the final background, so they can differ from 0.8.0. See TextTable, Select, and TabSelect. (#1834, #1831)
  • core: LineNumberRenderable draws its gutter into the frame, so a scroll frame makes no draw call. Translucent gutter colors under ancestor opacity blend for each primitive instead of as a group. See Line number gutter. (#1833)
  • core: The TimeToFirstDrawRenderable label draws with a transparent background and fades with the opacity of its ancestors. See TimeToFirstDraw. (#1828)
  • core: A z-index change or a reparent in an update, resize, or layout acknowledgement shows in the same frame, not one frame later. See Frames and output. (#1829)
  • core: A culling ScrollBox finds its visible children by binary search and prepares a culled subtree only when it shows. A one-row scroll of a 10,000-row list takes 1.28 M instructions per frame instead of 5.52 M. See ScrollBox. (#1837, #1867)
  • core: When a column that follows its content grows, text and editor nodes reuse their cached size. A grown or appended row costs one measure call instead of one call for each row. See Layout. (#1859)
  • core: NativeImage decode, crop, resize, and composite examine alpha 64 bytes at a time. In the crop benchmark, extract takes 10.6 M instructions instead of 56.2 M. See NativeImage. (#1880)
  • keymap: useBindings in the React and Solid adapters replaces its layer in one update, and the layer keeps its mount position instead of moving to the top. A plain-data layer that differs only in its functions stays registered and emits no state event. See Solid keymap integration and React keymap integration. (#1785)
  • native: OT_LAYOUT_PAINT of a node that the latest paint list did not paint returns its solved geometry with current translations and scroll offsets. Before, it returned the place where the node last painted. See Frames and output. (#1867)
  • native: The native library statically links decode-only LGPL FFmpeg 9.0.2 and zlib, so each stripped library is 2.6 to 3.5 MB larger. The packages ship the license, source, and relink notices. See Video. (#1800)

Fixed

  • core: A frame larger than the 8 MiB output limit no longer freezes the screen. The frame sends its cells and the images that fit, and an image too large for one frame goes at its placement size before it falls back to blocks. See Image. (#1890, #1896)
  • core: In split-footer mode, the renderer drops a scrollback commit that is larger than one frame and reports it through frame:dropped. A quit after such a commit restores the terminal. See Renderer. (#1890)
  • core: When an app quits over a slow link, the renderer restores the terminal instead of leaving the alternate screen and mouse reporting on. A late close drops the queued output and rejects closed. See Lifecycle and cleanup. (#1872, #1893)
  • core: A measured node with a margin, such as a text with margin: 1, wraps at the correct width after its parent grows by less than the margin. See Layout. (#1860)
  • core: A code block measures its text before the first highlight arrives, so a ScrollBox with Markdown that rebuilds each frame can scroll. A ScrollBox listens for selection only while it auto-scrolls, so rebuilds no longer leak listeners. See ScrollBox and Code. (#1863)
  • core: After a syntaxStyle or fg change, MarkdownRenderable recolors horizontal rules, keeps the blank lines under blocks, and shows a header-only table on every frame. In coalesced mode, code blocks in list items leave streaming mode after streaming = false. See Markdown. (#1857)
  • core: TabSelectRenderable cuts tab names and descriptions at a cell width, so a CJK name stays in its tab and a cut surrogate pair no longer draws U+FFFD. A TabSelect with a fixed width draws before it gets focus. See TabSelect. (#1828, #1831)
  • core: The line-number gutter measures signs with native glyph widths, so a number no longer covers part of a tab sign or a family emoji sign. See Line number gutter. (#1833)
  • core: In split-footer mode with captured stdout, Uint8Array writes decode as text, also when a multibyte character spans two writes. See Renderer. (#1735)
  • core: On WSLg, a clipboard read waits until its new helper has focus, so it no longer reads an offer without focus. See Clipboard. (#1862)

API changes and documentation: https://opentui.com/docs/releases/0.8.1

What's Changed

  • fix(renderer): decode Uint8Array writes in split-footer capture-stdout by @viralcodex in #1735
  • keymap: keep layers in place when useBindings re-registers them by @simonklee in #1785
  • docs: expose release history by @simonklee in #1861
  • core: fix markdown restyle leftovers after the list restyle by @simonklee in #1857
  • native: simulate output links deterministically by @simonklee in #1854
  • core: simulate the session driver on virtual time by @simonklee in #1855
  • native: sort z changes before paint and settle idle update timing by @simonklee in #1829
  • native: fit drawn text to a cell width by @simonklee in #1828
  • native: reuse a text size when a column's height cap grows by @simonklee in #1859
  • native: ignore a nil keyboard enter on the WSLg clipboard helper by @simonklee in #1862
  • core: fix ScrollBox Markdown collapse and selection listener leak by @simonklee in #1863
  • native: keep simulated input in arrival order by @simonklee in #1864
  • native: let a split key finish before the simulated stop by @simonklee in #1865
  • core: save a crash dump when the Yoga worker test crashes by @simonklee in #1868
  • core: bound captured stdout in split-footer mode by @simonklee in #1856
  • native: write terminal stdout from a native thread by @simonklee in #1858
  • native: compare like sizes in Yoga's stricter-size cache rule by @simonklee in #1860
  • native: add culled, arrow and remount kinds to the twin harness by @simonklee in #1836
  • native: paint a node Yoga sizes to zero as zero cells by @simonklee in #1869
  • core: paint Select and TabSelect into the frame by @simonklee in #1831
  • native: add view resizes and box details to the twin harness by @simonklee in #1842
  • core: paint the line-number gutter into the frame by @simonklee in #1833
  • core: paint TextTable into the frame by @simonklee in #1834
  • native: find viewport children with an exact index by @simonklee in #1837
  • native: add a video player node by @simonklee in #1800
  • native: prepare culled subtrees only where they show by @simonklee in #1867
  • core: hold the close deadline while a stdout copy is in flight by @simonklee in #1872
  • core: accept program status until the terminal closes by @simonklee in #1871
  • native: scan image alpha 16 pixels at a time by @simonklee in #1880
  • native: enable the stdout writer on macOS by @simonklee in #1881
  • native: add an audio option to play video without a device by @simonklee in #1884
  • native: decode the AAC priming frame on a seek near the start by @simonklee in #1883
  • native: pad video RGBA rows for swscale's SIMD stores by @simonklee in #1889
  • native: show a source region on video and image nodes by @simonklee in #1888
  • native: carry the overshoot when a video loops by @simonklee in #1885
  • native: open the AAC decoder anew on seek by @simonklee in #1887
  • build: migrate to Zig 0.17 by @simonklee in #1586
  • native: update two tests for the crop argument and Zig 0.17 by @simonklee in #1892
  • native: fix races in the stdout writer's abandon tests by @simonklee in #1891
  • native: draw frames that are larger than the output limit by @simonklee in #1890
  • core: make the TypeScript tests faster and fix two test-order flakes by @simonklee in #1895
  • core: bound the whole close and restore at the limit by @simonklee in #1893
  • examples: add a sliding puzzle of live video tiles by @simonklee in #1873
  • native: retry a too-large Kitty image at its placement size by @simonklee in #1896

Full Changelog: v0.8.0...v0.8.1

Check out latest releases or
releases around anomalyco/opentui v0.8.1

Don't miss a new opentui release

NewReleases is sending notifications on new releases.

Get notifications