Harlequin CHANGELOG

All notable changes to this project will be documented in this file.

Unreleased

1.20.0 - 2024-04-29

Features

• For adapters that support it, Harlequin now provides a buttons to toggle the transaction mode of the database connection, and commit and roll back transactions (#334).
• Adapters can now implement HarlequinConnection.transaction_mode and HarlequinConnection.toggle_transaction_mode() to enable the new Transaction Mode UI for their adapter.

Changed

• SQLite adapter: The adapter no longer accepts an --isolation-level option on Python 3.12 or higher; instead, the adapter allows autocommit configuration via the Harlequin UI.

1.19.0 - 2024-04-25

Features

• SQLite adapter: Harlequin now executes an initialization script on start-up of the SQLite adapter. By default, it executes the script found at ~/.sqliterc. To execute a different script, start Harlequin with the --init-path option (aliases -i/-init):

  harlequin -a sqlite --init-path ./my-project-script

  To start Harlequin without executing an initialization script, use the --no-init flag:

  harlequin -a sqlite --no-init

  Note: SQLite initialization scripts can contain dot commands or SQL statements. If Harlequin encounters a dot command, it will attempt to rewrite it as a SQL statement, and then execute the rewritten statement. Otherwise, it will ignore the dot command. Currently, Harlequin can only rewrite .open and .load commands.

  (#325)

• SQLite adapter: Adds a new CLI option, --extension or -e, which will load a SQLite extension. Note: SQLite extensions are not supported by Python on most platforms by default. See here for more details (#533).

1.18.0 - 2024-04-19

Features

• The Query Editor's Open and Save dialogs now display the full computed file path that will be opened or saved (tconbeer/textual-textarea#232 - thank you @bjornasm!)
• The Query Editor adds a "Find" action with ctrl+f and "Find Next" action with F3.
• The Query Editor adds a "Go To Line" action with ctrl+g.
• The Query Editor adds bindings for ctrl+shift+home/end to select text while moving the cursor to the start/end of the document.

Bug Fixes

• Fixes a crash from initializing the Error Modal incorrectly from the Query Editor.
• Fixes a crash from saving to a path in a non-existent directory.

Changes

• The Query Editor uses a slightly different implementation of undo and redo, with improved performance and some subtly different behavior (#240.

1.17.0 - 2024-04-16

Features

• A new HarlequinConnection.close() method can be implemented by adapters to gracefully close database connections when the application exits.
• The ADBC adapter is now installable as an extra; use pip install harlequin[adbc].

Bug Fixes

• Fixes broken link on clipboard error message (#509)

1.16.2 - 2024-03-29

Bug Fixes

• If the cursor is after the final semicolon in the query editor, and there is only whitespace after the semicolon, Harlequin will now execute the last query before the semicolon, instead of doing nothing when clicking Run Query or pressing ctrl+j.

1.16.1 - 2024-03-27

Bug Fixes

• Pressing F8 on the history screen no longer causes a crash (#485)

1.16.0 - 2024-02-22

Changes

• The default search path and priority for config files has changed, to better align with the standard defined by each operating system. Harlequin now loads config files from the following locations (and merges them, with items listed first taking priority):
  1. The file located at the path provided by the --config-path CLI option.
  2. Files named harlequin.toml, .harlequin.toml, or pyproject.toml in the current working directory.
  3. Files named harlequin.toml, .harlequin.toml, or config.toml in the user's default config directory, in the harlequin subdirectory. For example:
     • Linux: $XDG_CONFIG_HOME/harlequin/config.toml or ~/.config/harlequin/config.toml
     • Mac: ~/Library/Application Support/harlequin/config.toml
     • Windows: ~\AppData\Local\harlequin\config.toml
  4. Files named harlequin.toml, .harlequin.toml, or pyproject.toml in the user's home directory (~).
     (#471)

Features

• harlequin --config option now accepts the --config-path CLI option (#466).
• harlequin --config now defaults to updating the nearest (highest priority) existing config file in the default search path, instead of ./.harlequin.toml.

Bug Fixes

• harlequin --config creates a new file (parent folder as well, if non-existent) instead of crashing with FileNotFoundError (#465)

1.15.0 - 2024-02-12

Features

• The Data Exporter has been refactored to work with any adapter.
• The Data Exporter now supports two additional formats: Feather and ORC (ORC is not supported on Windows).

Bug Fixes

• The Query Editor no longer loses focus after pressing escape (regression since 1.14.0).

1.14.0 - 2024-02-07

Features

• The Databricks adapter is now installable as an extra; use pip install harlequin[databricks]. Thank you @alexmalins!
• In the Results Viewer, values are now formatted based on their type. Numbers have separators based on the locale, and numbers, dates/times/etc., and bools are right-aligned. Null values are now shown as a dim ∅ null, instead of a blank cell.
• Adds a --locale option to override the system locale for number formatting.

Bug Fixes

• The result counts in the Query History view now contain thousands separators (#437 - thank you, @code-master-ajay!).
• Harlequin no longer crashes when executing SQLite queries that return multiple types in a single column (#453).

Performance

• Harlequin now starts much faster, especially when restoring multiple buffers from the cache.

1.13.0 - 2024-01-26

Features

• Adds a Query History Viewer: press F8 to view a list of up to 500 previously-executed queries (#259).

Bug Fixes

• The new --show-files and --show-s3 options are now correctly grouped under "Harlequin Options" in harlequin --help; installed adapters are now alphabetically sorted.

1.12.0 - 2024-01-22

Features

• Adds an option, --show-files (alias -f), which will display the passed directory in the Data Catalog, alongside the connected database schema, in a second tab. Like database catalog items, you can use ctrl+enter, ctrl+j, or double-click to insert the path into the query editor.
• Adds an option, --show-s3 (alias --s3), which will display objects from the passed URI in the Data Catalog (in another tab). Uses the credentials from the AWS CLI's default profile. Use --show-s3 all to show all objects in all buckets for the currently-authenticated user, or pass buckets and key prefixes to restrict the catalog. For example, these all work:

  harlequin --show-s3 my-bucket
  harlequin --show-s3 my-bucket/my-nested/key-prefix
  harlequin --show-s3 s3://my-bucket
  harlequin --show-s3 https://my-storage.com/my-bucket/my-prefix
  harlequin --show-s3 https://my-bucket.s3.amazonaws.com/my-prefix
  harlequin --show-s3 https://my-bucket.storage.googleapis.com/my-prefix

• Items in the Data Catalog can now be copied to the clipboard with ctrl+c.

1.11.0 - 2024-01-12

Features

• Harlequin now shows a more helpful error message when attempting to open a sqlite file with the duckdb adapter or vice versa (#401).
• ctrl+r forces a refresh of the Data Catalog (the catalog is automatically refreshed after DDL queries are executed in Harlequin) (#375).
• At startup, Harlequin attempts to load a cached version of the Data Catalog. The Data Catalog will be updated in the background. A loading indicator will be displayed if there is no cached catalog for the connection parameters (#397).

Bug Fixes

• The Data Catalog no longer shows the loading state after an error loading the catalog.
• Harlequin now exits if attempting to open an invalid file with the sqlite adapter.

1.10.0 - 2024-01-11

Features

• Harlequin now loads immediately and connects to your database in the background (#393).
• Harlequin shows a loading indicator before the Data Catalog is hydrated for the first time (#396).

Bug Fixes

• Fixes a bug where harlequin --config would crash if configuring an adapter that declared no options.

1.9.2 - 2024-01-10

Features

• The ODBC adapter is now installable as an extra; use pip install harlequin[odbc].

1.9.1 - 2024-01-09

Bug Fixes

• Improves compatibility for adapter return types to accept a sequence of any iterable (tconbeer/textual-fastdatatable#68).

1.9.0 - 2024-01-08

Features

• Improves keyboard navigation of the Results Viewer by adding key bindings, including ctrl+right/left/up/down/home/end, tab, and ctrl+a.
• The Trino adapter is now installable as an extra; use pip install harlequin[trino].
• Harlequin will automatically download a missing timezone database on Windows. Prevent this behavior with --no-download-tzdata.

Bug Fixes

• Fixes a crash when selecting data from a timestamptz field (#382) (or another field with an invalid Arrow data type).

1.8.0 - 2023-12-21

Features

• Select a range of cells in the Results Viewer by clicking and dragging or by holding shift while moving the cursor with the keyboard.
• Copy selected cells from the Results Viewer by pressing ctrl+c.
• Very long values in the Results Viewer are now truncated, with an elipsis (…). The full value is shown in a tooltip when hovering over a truncated value. (The full value will also be copied to the clipboard).
• The BigQuery adapter is now installable as an extra; use pip install harlequin[bigquery].

Bug Fixes

• Fixes an issue on Windows where pressing shift or ctrl would hide the member autocomplete menu.
• Fixes flaky query execution behavior on some platforms.

Testing

• tests/functional_tests/test_app.py has been refactored into many smaller files.
• Fixes an issue with cache tests where the user's main harlequin cache was used instead of a mocked cache location.

1.7.3 - 2023-12-15

Bug Fixes

• Fixes an issue where completions were truncated improperly in the autocomplete menu.

Testing

• Prevents limit input cursor blink when running tests in headless mode, for less flaky tests.

1.7.2 - 2023-12-14

Features

• The MySQL adapter is now installable as an extra; use pip install harlequin[mysql].

1.7.1 - 2023-12-14

Bug Fixes

• Fixes a crash when using harlequin-postgres and executing a select statement that returns zero records.

1.7.0 - 2023-12-13

Features

• AUTOCOMPLETE! Harlequin's query editor will now offer completions in a drop-down for SQL keywords, functions, and database objects (like tables, views, columns). With the autocomplete list open, use up, down, PgUp, PgDn, to select an option and enter or Tab to insert it into the editor.
• Harlequin now uses a new TextArea widget for its code editor. This improves performance for long queries, adds line numbers in a gutter, and changes the underlying engine for syntax highlighting from Pygments to Tree Sitter (tconbeer/textual-textarea#123).
• In the Query Editor: double-click to select a word, triple-click to select a line, and quadruple-click to select the entire query (tconbeer/textual-textarea#111, tconbeer/textual-textarea#112).

Changes

• Changes the default theme to harlequin.

Adapter API Changes

• Many key types are now exported from the main harlequin package: HarlequinAdapter, HarlequinConnection, HarlequinCursor, HarlequinAdapterOption, HarlequinCopyFormat, HarlequinCompletion.
• HarlequinConnections may now (optionally) define a get_completions() method, which should return a list of HarlequinCompletion instances; each returned completion will be available to users in the autocompletion list.

Bug Fixes

• Fixes a bug that was causing an empty line to appear at the bottom of the Query Editor pane.

1.6.0 - 2023-12-07

Features

• Harlequin can now be configured using a TOML file. The config file can both specify options for Harlequin (like the theme and row limit) and also for installed adapters (like the host, username, and password for a database connection). The config file can define multiple "profiles" (sets of configuration), and you can select the profile to use when starting Harlequin with the --profile option (alias -P). By default, Harlequin searches the current directory and home directories for files called either .harlequin.toml or pyproject.toml, and merges the config it finds in them. You can specify a different path using the --config-path option. Values loaded from config files can be overridden by passing CLI options (#206).
• Harlequin now ships with a wizard to make it easy to create or update config files. Simply run Harlequin with the --config option.
• Adds a harlequin theme. You can use it with harlequin -t harlequin.

1.5.0 - 2023-11-28

Breaking Changes

• The SQLite adapter no longer provides a check-same-thread option; the established connection sets this value to False to enable Harlequin features.

Features

• The Postgres adapter is now installable as an extra; use pip install harlequin[postgres].

Bug Fixes

• Harlequin no longer becomes unresponsive when loading a large data catalog or executing long-running queries (#236, #332, #331).
• Fixes a flaky test that was causing intermittent CI failures.

1.4.1 - 2023-11-20

Bug Fixes

• Adds a py.typed file to the harlequin package.

1.4.0 - 2023-11-18

Features

• Harlequin now ships with an experimental SQLite adapter and can be used to query any SQLite database (including an in-memory database). You can select the adapter by starting Harlequin with harlequin -a sqlite (for an in-memory session) or harlequin -a sqlite my.db.
• harlequin --help is all-new, with a glow-up provided by rich-click. Options for each adapter are separated into their own panels.
• harlequin --version now shows the versions of installed database adapters (#317).

Refactoring

• The code for the DuckDB adapter has been moved from /plugins/harlequin_duckdb to /src/harlequin_duckdb.
• The unused export_options.py module has been removed (#327).

1.3.1 - 2023-11-13

Bug Fixes

• When running multiple queries, Harlequin now activates the results tab for the last query, instead of the first one.
• Queries that return duplicate column names are now displayed correctly in the Results Viewer (tconbeer/textual-fastdatatable#26).
• List types returned by DuckDB no longer display as ?, but instead as [#], [s], etc. (#315).
• Map types returned by DuckDB now display as {m}, to differentiate them from structs ({}).
• The Results Viewer no longer displays "Query Returned No Records" before the first query is executed.
• The data returned by HarlequinCursor.fetchall() no longer needs to be a PyArrow Table (#281).

1.3.0 - 2023-11-06

Features

• Adds an --adapter CLI option (alias -a) for selecting an installed adapter plug-in.

Bug Fixes

• Fixes a crash that could happen when a query returned no records (tconbeer/textual-fastdatatable#19).

Adapter API Changes

• The function signature for HarlequinConnection.copy() has changed to add a format_name positional argument.
• The HarlequinAdapter.COPY_OPTIONS class variable has been renamed to HarlequinAdapter.COPY_FORMATS, and its
  type has changed.
• The function signature for HarlequinAdapter.connect() has changed to return only a HarlequinConnection; HarlequinConnection now accepts an init_message kwarg that will be displayed to the user as a notification.

Refactoring

• Harlequin's CLI now dynamically loads the available options from the installed adapters (#276).
• Harlequin now dynamically loads data export options from the selected adapter (#275).

1.2.0 - 2023-10-22

1.2.0-alpha.1 - 2023-10-22

Bug Fixes

• Harlequin's query notifications no longer count whitespace-only queries (#268).
• Harlequin's DataCatalog now displays "db" next to database names and "sch" next to schema names. Empty databases and schemas no longer have an arrow to expand them.
• If the cursor is after the final semicolon in the query editor, Harlequin will now execute the last query before the semicolon, instead of doing nothing when clicking Run Query or pressing ctrl+j.

Refactoring

• Harlequin's DuckDB integration has been refactored into a more general-purpose database adapter interface (#263).
• Harlequin's DuckDB adapter is now loaded as a plug-in (#279)

1.1.1 - 2023-10-09

Bug Fixes

• Harlequin no longer crashes if the data returned by DuckDB contains NoneType or complex (Struct, Map, List) columns (#265 - thank you @sjdurfey!).

Testing

• Harlequin now uses snapshot testing on screenshots to prevent regresssions (#252).
• Harlequin no longer installs extensions or connects to MotherDuck in CI, due to flaky failures around the time of DuckDB releases (#262).

1.1.0 - 2023-10-02

Features

• Harlequin now executes an initialization script on start-up. By default, it executes the script found at ~/.duckdbrc. To execute a different script, start Harlequin with the --init-path option:

  harlequin --init-path ./my-project-script.sql

  To start Harlequin without executing an initialization script, use the --no-init flag:

  harlequin --no-init

  Note: DuckDB initialization scripts can contain dot commands or SQL statements. If Harlequin encounters a dot command, it will attempt to rewrite it as a SQL statement, and then execute the rewritten statement. Otherwise, it will ignore the dot command. Currently, Harlequin can only rewrite .open commands.

  (#241 - thank you @pdpark!)

• Harlequin now displays notifications after completing successful queries (#235 - thank you @natir!), saving the contents of a buffer (#226), and receiving an error from the system clipboard.

• Harlequin now loads data from a completed query up to 1,000x faster by using a new DataTable widget (#181). By default, the Results Viewer is now limited to 100,000 records, instead of 10,000. This limit can be changed with the --limit option when starting Harlequin. This introduces a dependency on PyArrow >= 7.0.0.

1.0.1 - 2023-09-21

Bug Fixes

• Pasting text into Harlequin's text editor is now more performant and compatible with more terminals. (#120 - thank you @matsonj, #119).

1.0.0 - 2023-09-12

Features

• Double-click or press ctrl+enter on an item in the data catalog to insert the name in the query editor (#194).
• Harlequin now shows notifications when exporting data or executing DDL/DML.

Bug Fixes and Minor Updates

• Data table column headers are now bold on terminals and fonts that support it (#203).
• Bumped TextArea; cursor now better maintains x-position and other minor fixes.
• The query editor's cursor no longer blinks when a modal appears above it (#196).
• Harlequin now shows the results of successful queries in the Results Viewer if multiple queries are executed and one or more contain errors.
• Error and Help modals can now be dismissed with a click outside the modal (#218).

0.0.28 - 2023-09-07

• Buffers are now restored when Harlequin is restarted (#175).

[0.0.27] - 2023-08-23

New Features

• UI glow-up: Colors are more consistent, and themes set the styling for the entire app (#81). Try harlequin -t zenburn or harlequin -t one-dark for a new look.
• Harlequin's query editor now supports more key bindings: ctrl+z and ctrl+y to undo/redo, and shift+delete to delete an entire line.

Fixes

• It is now easier to focus on the current editor buffer, instead of the tabs above it.

0.0.26 - 2023-08-21

New Features

• Harlequin supports multiple buffers (for tabbed editing). Create a new tab with ctrl+n, close a tab with ctrl+w, and switch to the next tab with ctrl+k. Opening, saving, and running queries are operations on the current buffer and have no effect on the other buffers.

0.0.25 - 2023-08-13

New Features

• Harlequin now returns the result of multiple select queries to different tabs in the Results Viewer. To run multiple queries, type them into the Query Editor (separated by semicolons), then press ctrl+a to select all, and then ctrl+enter to run the selection (#34).
• If there are multiple results tabs, you can switch between them with j and k.
• ctrl+e exports the data from the current (visible) data table.

Bug Fixes

• Fixes issues with the loading state when loading large result sets.

0.0.24 - 2023-08-04

New Features

• Adds a new CLI option, --extension or -e, which will install and load a named DuckDB extension.
• Adds a new CLI option, --force-install-extensions, which will re-install the extensions provided
  with the -e option.
• Adds a new CLI option, --custom-extension-repo, which enables installing extensions other than
  the official DuckDB extensions.
• Taken together, Harlequin can now be loaded with the PRQL extension. Use PRQL with Harlequin:

  harlequin -u -e prql --custom-extension-repo welsch.lu/duckdb/prql/latest

  (#152 - thank you @dljsjr!)

0.0.23 - 2023-08-03

Features

• Changes the behavior of the "Run Query" button and ctrl+enter:
  • If text is selected, and that text does not contain parsing errors, the "Run Query" button will show "Run Selection", and ctrl+enter will run the selected text. If multiple queries are selected (separated by semicolons), they will all be run; if multiple select statements are selected, only data from the first selected select statement will be loaded into the Results Viewer (or exported).
  • If no text is selected, Harlequin will run the single query where the cursor is active. Other queries before and after semicolons will not be run.
  • To "Run All", first select all text with ctrl+a, and then run selection with ctrl+enter
• Adds path autocomplete and validation to the file save/open and export data inputs.

Other Changes

• Lowers the maximum number of records loaded into the results viewer to 10,000. (All records can be exported with ctrl+e)

0.0.22 - 2023-08-02

Features

• Export data from a query with ctrl+e (#149 - thank you, @Avsha-Chai).

0.0.21 - 2023-07-28

Features

• Add -u/-unsigned/--allow-unsigned-extensions CLI flag for allowing loading of unsigned extensions.
• File save and open dialog can now expand the user directory (~) (#61)

Bug Fixes

• Error modal no longer crashes.
• Text selection is now maintained when pressing more keys.

0.0.20 - 2023-07-17

Features

• F1 now displays a help screen that lists all keyboard bindings (#20).
• F2 focuses the keyboard on the query editor.
• F5 focuses the keyboard on the results viewer.
• F6 focuses the keyboard on the data catalog.

Bug Fixes

• ctrl+v for paste is now better-supported on all platforms.

0.0.19 - 2023-06-26

Features

• It's back: select text in the query editor using click and drag (#42).

Bug Fixes

• Fixes a bug where PgUp could cause a crash (#46).

0.0.18 - 2023-06-23

Bug Fixes

• Changes format action key binding from ctrl+` to F4. The original binding was causing compatibility
  issues with Windows Powershell and Command Prompt (#82).
• Adds key binding F9 as an alternative to ctrl+b to hide the left-hand panel.
• Fixed query editor scrollbar color to match other widgets (#109)
• Fixed compatibility with Textual v0.28.0 (#115)

0.0.17 - 2023-06-23

Features

• Supports MotherDuck! harlequin md: connects to your MotherDuck instance. Optionally pass token with --md_token <token> and set SaaS mode with --md_saas.

Bug Fixes

• Fixes issues with mouse input and focus by rolling back textual_textarea to v0.2.2

0.0.16 - 2023-06-20

• Press F10 with either the Query Editor or Results Viewer in focus to enter "full-screen" mode for those widgets (and hide the other widgets). (#100)
• Select text in the query editor using click and drag (textual-textarea/#8)

0.0.15 - 2023-06-17

• Adds checkbox for Limit with a configurable input (#35).
• Adds more obvious Run Query button (#76).
• Press ctrl+b to toggle (hide/show) the Data Catalog sidebar. (#29, #103)
• Removes the Header for more working space.

0.0.14 - 2023-06-15

Features

• The schema viewer (now called Data Catalog) now supports multiple databases.
  (#89 - thank you
  @ywelsch!)
• Harlequin can be opened with multiple databases by passing them as CLI args:
  harlequin f1.db iris.db. Databases can also be attached or detached using
  SQL executed in Harlequin.

Bug Fixes

• Reimplements ctrl+` to format files (regression from 0.0.13)
• Updates textual_textarea, which fixes two bugs when opening files
  and another bug related to scrolling the TextArea.

0.0.13 - 2023-06-15

Features

• Harlequin accepts a new argument, -t/--theme to set the Pygments theme for the query editor.
• Harlequin uses the system clipboard for copying and pasting queries.

Under the hood

• Refactors to use the new tconbeer/textual-textarea package.

0.0.12 - 2023-05-31

• improves documentation of ctrl+j as an alternative key binding for running a query (#71 - thank you @carteakey!)

0.0.11 - 2023-05-18

• adds a command-line option (-r, -readonly, or --read-only) for opening
  the database file in read-only mode.
• after a query is executed and the data is loaded, the focus shifts to the data table.

0.0.10 - 2023-05-17

• upgrades duckdb to v0.8.0, which includes some breaking changes around types. Harlequin can no longer support earlier versions of duckdb.

0.0.9 - 2023-05-16

• fixes an issue where a DuckDB Error could cause Harlequin to crash (#56 - thank you @Mause!)
• removes docker builds (app UX was poor in a container)

0.0.8 - 2023-05-15

• Cut, copy, paste in text editor with ctrl+x, ctrl+c, ctrl+u/ctrl+v
• Quit with ctrl+q, instead of ctrl+c
• tab indents selected text or inserts four-ish spaces in text editor; shift+tab dedents selected text
• scroll up and down with ctrl+up and ctrl+down
• fixes an issue where an extra space would be added to the end of lines when pressing enter in some situations.

0.0.7 - 2023-05-12

• Comment selected text with ctrl+/
• Smarter indentation after pressing enter

0.0.6 - 2023-05-09

• Select text in the query editor using shift and arrow keys, etc. Replace/delete/quote selection, etc.
• Improves behavior of inserting opening brackets in the query editor.
• Hopefully fixes Docker build

0.0.5 - 2023-05-08

• Adds column types to the column header in the results viewer.
• Text editor now handles page up/dn and ctrl+right/left keys.
• Fixes compatibility with all Pythons >= 3.8

0.0.4 - 2023-05-05

• All-new text area for query editing, with syntax highlighting, scrolling, and more.
• Loading states and progress bars for long-running queries. Better async use to maintain responsiveness.
• Fixed edge cases around empty and repeated queries.

0.0.3 - 2023-05-04

• Queries now run asynchronously.
• Errors from DuckDB are now handled and shown in a pop-up.
• View columns and data types in the schema viewer sidebar.
• Queries can be formatted using ctrl+`.
• Queries can be saved using ctrl+s and opened (loaded) using ctrl+o.

0.0.2 - 2023-05-02

• View the schema of a DuckDB database in the sidebar.
• Run queries and view the results.

0.0.1 - 2023-05-02

• Use the DuckDB CLI.