observablehq/framework v1.4.0

on GitHub

Improved SQL and DuckDB 🦆

SQL code blocks and DuckDBClient now support querying remotely-hosted data. This is especially useful for live, rapidly-changing data that you do not want to “bake-in” to your built site. For example, to load recent earthquakes from the USGS real-time feed:

---
sql
  quakes: https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.csv
---

To get the maximum recent earthquake magnitude, you could then query:

SELECT MAX(mag) FROM quakes

Note: for faster page loads and to avoid a runtime dependency on external servers, we recommend using data loaders when possible to generate static snapshots of data at build time, with continuous deployment to re-build your site as needed.

Framework also now supports DuckDB’s database file format. These files conveniently store multiple tables and may offer better performance on complex or repeated queries. DuckDB database files can either be static or generated dynamically by data loaders. For example, to load a database file named chinook.db:

---
sql
  chinook: chinook.db
---

Then, to query the tracks table:

SELECT * FROM chinook.tracks

The new DuckDBClient.sql convenience method lets you redefine the sql tagged template literal used by SQL code blocks. This allows you to load or create data sources dynamically, say in response to user interaction. For example, to allow the user to choose the desired USGS real-time feed from a drop-down menu:

const feed = view(Inputs.select(new Map([["M4.5+", "4.5"], ["M2.5+", "2.5"], ["All", "all"]]), {label: "Earthquake feed"}));

const sql = DuckDBClient.sql({quakes: `https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/${feed}_day.csv`});

SELECT * FROM quakes ORDER BY updated DESC

We also fixed a bug where DuckDB was being loaded during preview even when it was not needed, and fixed the implementation of db.describeTables and db.describeColumns now that db.query returns an Arrow table instead of a materialized array-of-objects.

More local assets

You can now reference local assets such as images, stylesheets, and scripts from the head, header, and footer config options. For example, to set the favicon to a favicon.png stored in the source root:

export default {
  head: `<link rel="icon" type="image/png" href="/favicon.png" sizes="512x512">`
};

Like file attachments, referenced assets can be either static or generated dynamically by data loaders, are included in the built site, and have a content hash applied to the file name for cache breaking.

The new FileAttachment.lastModified property exposes the modification time of the given file, represented as the number of milliseconds since UNIX epoch (like File.lastModified). This is useful for showing how fresh data is, for example. For files generated by data loaders, this stores the time the data loader completed.

FileAttachment("results.csv").lastModified // 1710809036000

new Date(FileAttachment("results.csv").lastModified) // "Tue Mar 19 2024 00:43:56 UTC"

Framework also now supports script elements with the src attribute in Markdown. This is especially useful for loading older, non-module JavaScript — it “just works”. We also fixed a bug resolving file attachment paths with a leading slash.

More flexible deploys

The build and deploy commands are now decoupled, offering more flexibility: you can modify a built site prior to deploying (say to add additional assets), and you can re-deploy a site without having to re-build it (say if your deploy gets interrupted). To support this, the deploy command has two additional flags, --if-stale and --if-missing, that indicate what to do when the build is either stale (more than five minutes old) or missing. Both default to prompt in interactive terminals, meaning the deploy command will ask how you’d like to proceed, and cancel in non-interactive terminals.

Our new Deploying guide shows how to automate deploys from GitHub Actions.

Deploys to Observable are also now significantly faster thanks to concurrent file uploads. 🚀

And more…

Self-hosted npm: modules have been renamed from +esm.js to _esm.js to avoid a quirk with how Amazon S3 interprets the + character in paths. The pages config option now properly normalizes page paths, fixing a bug where the previous & next footer links wouldn’t appear. The EIA example dashboard timeline chart now has better tooltips (thanks, @pettiross!).

Full Changelog: v1.3.0...v1.4.0