github frappe/insights v3.50.0

one hour ago

Insights v3.50 rebuilds every chart and brings Insights into Frappe desk, with ready-made dashboards for ERPNext and Frappe HR. It runs on Frappe v15 and v16.

  • Charts, rebuilt. Every chart is redrawn on frappe-ui charts, with dark mode, a Heatmap, trend lines and Number cards that track a target.
  • Click a chart for its rows, or break a value down by another column.
  • App dashboards. ERPNext and Frappe HR v16 ship ready-made dashboards for Selling, Stock, Accounts, Payroll and more: Frappe HR from v16.54.0, ERPNext from its next v16 release.
  • Insights on desk. On Frappe v16.60.0 or later, put an Insights chart on a desk dashboard or workspace.
  • Share a dashboard with roles. Anyone with those roles can open it, even without an Insights role.

Some numbers change, because earlier releases computed them wrongly. Site data now follows desk permissions alone.

Important

Before you upgrade

  • Using Insights v2? Migrate first. A site that used v2 in the last 30 days stops at migrate until its v2 work is migrated. Migration guide
  • Take a backup, and check that the disk has free space equal to the data store.
  • Export Query Execution Logs older than 90 days if you need them. v3.50 deletes them.
  • Node v20.19 or later is required. A Frappe Cloud bench on Frappe v15 must raise Node first, or its build fails.
  • On Frappe v16, update Insights to v3.50 together with Frappe v16.60 and Frappe HR v16.54. With an older Insights, their dashboards stay classic and desk shows an "Install Insights" banner.

Upgrading

Insights v3.50 is on the version-3 branch.

  • Self-hosted: with Insights on version-3, run bench update --apps insights. It backs up each site, pulls Insights, builds it and migrates every site on the bench.
  • Frappe Cloud: benches deploy Insights from version-3. A site gets v3.50 when its bench group deploys an update and the site moves to the new bench.

Migrate first checks for v2 work, as above. Its patches then move saved charts, dashboards, shares and alerts to the new formats. After migrate, a background job reads the columns of every synced table. Until it finishes, column search finds fewer tables.

After upgrading, open Insights as an admin. A guide walks you through what's new and through anything the upgrade could not carry over, each item with a fix where one exists.

Migrating from v2

Insights v3.50 no longer contains v2 or the v2 migrator. Stay on your current Insights v3 release, migrate your v2 queries and dashboards, then update to v3.50.

  • On Insights v3.13 or later: as an admin, open Insights, then Settings > Migrate from v2.
  • On an older release: update to v3.15.1 first. Self-hosted, check out the v3.15.1 tag. On Frappe Cloud, raise a support ticket at https://frappe.io/support.
  • The migrator moves dashboards only. For a query outside any dashboard, add it to a v2 dashboard and migrate that dashboard, or rebuild it in v3.
  • If you don't need your v2 work, set "insights_allow_legacy_v2_upgrade": 1 in the site config. It skips the check, and v2 data you have not migrated is not carried over.
For everyone
  • A new filter picker, which remembers your last choice. A Company filter opens on your default company, as in desk.
  • Amounts print in each company's currency, with its decimal places and the site's number format, as in desk. Shortened amounts, such as ₹ 1.2L, are unchanged.
  • A chart, dashboard or workbook you can't read says so and names the access you need.
  • A missing number is blank, not "null". A table's row totals column is headed Total.
  • Numbers that change because they were wrong before: whole-day date filters, "next N months", period comparisons, date_diff in hours or minutes, and Number cards on a date column. A Number card on a datetime column with no period shows its latest day. A pivot keeps its largest columns, not the first ones alphabetically.
  • If desk does not let you read a doctype, Insights doesn't either.
For workbook authors
  • Desk charts (Frappe v16.60.0 or later): use Show in Desk, or pick the chart on a Dashboard Chart or Number Card form. Desk users with an Insights role see it when they may read the chart behind it.
  • Sharing: share a workbook, not its items. Sharing a chart or dashboard, or a team grant on one, gives read access only. Desk users without an Insights role get a Dashboards list of what they may open. Saving warns when a chosen role can't read a chart's tables.
  • Run as owner: a chart runs with each reader's permissions unless Run as owner is on. A public dashboard needs it on every chart. It holds only while the owner is enabled and may edit the chart.
  • Standard workbooks from ERPNext and Frappe HR are read-only. Duplicate one to change it. On Frappe v15 there is no template library; workbooks already made from templates stay.
  • Tighter sandbox: scripts, SQL expressions and stored procedure calls are admin-only, and existing ones keep running. Scripts lose sql(), frappe.qb, writes, jobs and mail. Expressions cannot call a table's methods (the upgrade rewrites t.source to t['source']). SQL cannot call functions that read files, reach other databases or run another query.
  • Alerts: an alert can post to a webhook. It needs write access to the workbook. An alert's cron must be one the scheduler can read. An alert set on a chart before v3.50 checks a saved copy of the chart's query.
  • Dates: a date axis on a Bar, Line or Row chart can group by Auto, the default for a new chart. It follows the range the filters set: a month by day, a quarter or a year by week, two years by month. Saved charts keep their grain. A Datetime dashboard filter reads minutes and hours, with presets from the last 15 minutes to the last 24 hours.
  • Charts: a statistic line reads only the first measure on its axis. A chart added to a dashboard takes half its width. A reading added to a Number chart shows on every dashboard that has the chart.
  • Smaller changes: the Dashboards page opens in list view, and a preview shows a placeholder until an editor refreshes it. A failing query says why in its results. ⌘Z in a text box edits the text, not the query.
For admins

The upgrade keeps every chart drawing as it did, shares each workbook with the users who worked in it or had its items shared with them, and keeps alerts, public links, and queries and tables that other workbooks read.

The in-app guide lists what needs you, such as:

  • Give desk roles to users who read site data through a team. Team access and Table Restrictions now apply only to other data sources.
  • Set Fiscal Year Start if it is blank. Blank now means ERPNext's current fiscal year.
  • Delete workbooks copied from old Insights templates. The standard workbooks replace them.
  • Enter the variables of script queries copied into another workbook. The copies don't have them, since they can hold passwords.

Workbook and dashboard lists now include colleagues' work for admins.

Users with the Insights User role become System Users, so they can sign in to desk and to MCP clients. After login they open desk, not Insights.

Data store queries now share the server's cores and memory, and stop at Max Execution Time. On a busy server a chart waits and retries instead of failing.

For developers
  • Guest embeds use insights.api.view.*.
  • data_query, is_public and permission_user are gone. Use visibility and run_as_owner.
  • 17 endpoints accept only POST. A share on a workbook member accepts only a named read share.
  • Apps ship workbooks as <module>/insights_workbook/<name>/<name>.json, imported on migrate.
  • A measure's currency can come from a linked document: currency_column takes Doctype:link_column:field.
  • On a Frappe without desk islands (frappe.utils.island), Insights adds nothing to desk.

Full changelog

Breaking Changes

  • Remove v2 code (#932)

Features

  • Weekly data store cleanup (#1255)
  • Reference lines, funnel-from-measures, percent measures (#1260)
  • Allow serving the app from a custom path (#1262)
  • Let string filters match an exact value (#1283)
  • Alerts via web hooks (#1293)
  • Acknowledge /iris review, report a failed run (#1309)
  • Make the effort level configurable (#1329)
  • Read the PR thread, answer questions, review the delta (#1330)
  • Add an end-to-end test suite (#1346)
  • Carry a SQL column expression through an ibis query (#1355)
  • Agent skill for authoring workbooks over frappectl (#1380)
  • Charts render through frappe-ui charts v2 (#1383)
  • Keep cards off the rows filters sit on (#1416)
  • Review a large change by question, in stages (#1430)
  • Configurable row limit for query exports (#1433)
  • List workbooks and dashboards by access, with quick filters (#1443)
  • Make the filter picker's key hints clickable (#1445)
  • Render Insights dashboards and charts inside desk pages (#1422)
  • Import each stored table on its own cron schedule (#1448)
  • Chart description, info and trend line (#1477)
  • What agent sessions taught the workbook skill (#1476)
  • Read a dashboard's span as the Period of a Number card that states none (#1480)
  • Clear old Insights Query Execution Logs (#1469)
  • Stack a line chart's areas (#1482)
  • Progress bar on number cards (#1517)
  • Add frappe-insights skill for the assistant, MCP and frappectl (#1531)
  • Pick a year, fiscal year, quarter or month in a date filter (#1534)
  • Profile where a warehouse query spends time and memory (#1547)
  • A webhook alert needs no token and sends its text as content (#1549)
  • Record and profile everything that loads the warehouse (#1553)
  • Expose method source in API discovery (#1554)
  • Keep failed, lock-refused and cached runs in the execution log (#1556)
  • Bound data store work by the server's cores and memory (#1560)
  • Insights v3.50 (#1541)
  • An Auto grain for a date x-axis that follows the span it is read over (#1570)
  • A Datetime filter with presets in minutes and hours (#1584)

Bug Fixes

  • Respect Week Starts On in timespan filters (#1252)
  • Incremental sync fails when a source table gains a column (#1259)
  • Restore PNG export for dashboards and charts (#1261)
  • Stop the weekly cleanup from dropping tables it cannot rebuild (#1296)
  • Remove the env probes that broke CI runs (#1306)
  • Add alt text to images missing it (#1297)
  • Stop an unrelated comment from cancelling a review (#1342)
  • Report failures, and rerun when the inputs change (#1354)
  • Read the condition off the whole expression (#1362)
  • Stack bars beside a right-axis line, and drop cleared axis bounds (#1396)
  • Blank a non-finite value before the result is cached (#1400)
  • Dashboard filter picker and number card title (#1406)
  • Store a chart config in one shape, instead of repairing it on load (#1407)
  • Three counts the first review round on the backport caught (#1411)
  • Number card retry placement and result row count copy (#1403)
  • Infer json_extract types from non-null values (#1418)
  • Defects found reviewing develop (#1437)
  • Map and number card regressions from the frappe-ui upgrade (#1444)
  • A division by zero returns null on PostgreSQL (#1447)
  • Reload once when a deploy removed a lazy chunk (#1453)
  • Fill the claims shipped files carry when the field is new (#1454)
  • New charts share a config, and Add series hides on an empty y-axis (#1455)
  • Fail when vue-tsc does not run, and run it in CI (#1462)
  • Print a second axis' data labels in its format (#1460)
  • Let a reader sort a table chart (#1473)
  • Read an expression's columns by item (#1475)
  • Open the folder of the open query or chart (#1483)
  • Keep a cleared filter cleared after a reload (#1484)
  • Resume an incremental import from the store's newest cursor (#1486)
  • Keep an incremental table's rows when its sync settings change (#1488)
  • Copy every row at a cursor value an import stops on (#1489)
  • Keep month labels on a ~120-day timeline (#1491)
  • Drop the date range beside each date option (#1492)
  • Cap Frappe version range to supported versions (#1493)
  • Pass the marketplace semgrep check (#1496)
  • Cap every DuckDB connection, never above DuckDB's default (#1499)
  • Drop leftover import tables a live table holds (#1498)
  • Connect the site database over the socket the site config names (#1502)
  • Validate json_value, keep the string null (#1511)
  • Keep the frontend lockfile frozen on install (#1515)
  • Pin a fiscal-year axis' ticks to its years (#1516)
  • Bind every table of a grouped join in native SQL (#1526)
  • Group digits by Frappe's number format, not the country (#1533)
  • Follow frappe to sqlglot 30.22 (#1543)
  • Keep a table import within its memory limit (#1546)
  • Wait out readers in the same process before a write (#1537)
  • An alert runs its query once per window and at most twice per send (#1551)
  • Keep small servers' dashboards within memory (#1562)
  • Write no Error Log row for a busy answer (#1563)
  • Answer busy, not a connection error, when a source's slot is full (#1565)
  • Prevent browser auto-translation of app (#1481)
  • Read and import a table whose name is a MariaDB uuid (#1567)
  • Run native SQL on a read-only PostgreSQL replica (#1568)
  • Read and import a table whose name is a MariaDB uuid (#1558)
  • Run native SQL on a read-only PostgreSQL replica (#1557)
  • Preserve anchored weeks in date filters (#1573)
  • A comparison reads a period, a target owns the fixed number (#1529)
  • No toast for a busy Load Count that a retry answers (#1576)
  • Filter date grain cards before summarize (#1577)
  • Show data store limits and their defaults by locale (#1579)
  • Give Insights User desk access, so a migrate keeps it (#1583)
  • Give a public table chart its find and card filter back (#1581)
  • Run develop on Frappe v15 and the column backfill in a stock worker (#1585)
  • v3.50 QA UI fixes: short axis ticks, Run-as-owner message, dark logos, wording (#1587)

Performance

  • Bring the desk islands under their size budget (#1451)
  • Compile each chart query once per request and cache the drill menu (#1559)
  • Run identical queries once, and stop a query its reader left (#1561)
  • Keep request cleanup from loading the data-source stack (#1572)

Thanks to @ankush, @Aradhya-Tripathi, @chdecultot, @cogk, @dhirajsingh-rgb, @icrc-fdeniger, @jay-glasswala-2110, @meganathanpc, @RaphaelFakhri, @s-aga-r, @shahzeelahmed and @swe-sanad for their contributions to this release.

Full Changelog: v3.15.1...v3.50.0

Don't miss a new insights release

NewReleases is sending notifications on new releases.