This release of Catala is dedicated to the Syndicat général des ingénieurs-conseils case of French administrative law, which confirms that the autonomous regulatory power of the State created by the Constitution of October 4th, 1958 is still subject to the General Principles of French administrative Law.
This release focuses on scaling up the build system and text editor tooling to handle large projects. As always, all the features are documented in the Catala book. See also the catala-language-server sister release for the details of the release on the side of the language server and VSCode plugin.
New features
⚠️ Breaking change: directory inclusion in Catala projects
The behavior of include_dirs in clerk.toml changes as of this release : you are no longer forced to list all directories containing Catala source files! From now on, include_dirs looks for source files recursively into the directories you list, and is even optional, since by default clerk starts from the project root. You can also exclude some directories to speed up clerk's scanning using exclude_dirs (useful if you have a large assets directory in your project).
Target separation and dependencies
Before 1.3.0, clerk's behavior and configuration files were built around the assumption that what users wanted out of a Catala project was a self-containing bundle of target source files that you could embed in a larger application as a trusted core. However, as Catala's industrialisation progresses, our users brought up the need for a single Catala project to generate multiple target packets that depend from one another.
Now, targets in clerk.toml can declare dependencies to other targets, making sure source files are bundled separately into each different targets, even if they depend on each other. This behavior comes with a major refactoring of clerk's internal dependency management, and now clerk is able to display meaningful error messages in case of cyclic dependencies between modules, etc. And it even spits out a graph to help debugging! Overall, you should also expect faster builds due to better caching.
Improved backend target generation
Along with our new dependency mechanism in place, we also worked toward improving the generated exportable backend code. Previously, building a target would copy over all Catala modules in the dependency chain which could cause some clashes when building multiple targets.
We now only copy over the necessary Catala modules and, if a target declares a dependency, we add a reference to it instead of duplicating the files.
Moreover, for each backend target generation, we also generate a standard build script (Makefile for C, maven for Java, etc.) that are also aware of the dependencies between targets and provide a simple way of building the generated libraries. The Catala runtime and standard library are provided alongside the user's targets in a single libcatala package.
Also, the Java backend, compatible with Java 17, now correctly sport their package declaration (and dependent target imports statements).
Finally, it is also possible to generate a .jar file per target when calling mvn package in the target directory (jar source files are also available in the build artifacts).
Improved tracing
Catala's trace generation mechanism is now accessible through clerk run --trace, and you can use --trace-format=json to output the trace in a handy JSON format that you can inspect automatically in a larger IT application. The JSON format of the trace, which has been revamped and improved to better describe function calls and exceptions handling, is now the cornerstone of auditability and explainability of Catala executions. We expect new visualisations tools to be built upon it in the coming months :)
The trace generation is also available in the generated code for Java, Python and OCaml (not C yet), and each of those backends can export a JSON from the trace recorded by the generated code. Traces recorded by the generated code should be identical to interpreter-generated traces, ensuring replication of results across development and production environments for Catala.
Fixes and small improvements
- Escape non-ascii characters in Java backend strings as unicode. All Java strings are now agnostic of encodings.
- Fixes a bug in the
--gen-externaltemplate generator where the generated OCaml code did not match theExternalTypefunctor signature: equal and compare were missing the_posparameter, andfrom_jsonhad incorrect signature_pos tinstead of_pos _s. - Fixes a bug in the JSON output format of enumerations yielding errors such as:
Invalid_argument("Json_encoding.construct: consequence of non exhaustive Json_encoding.string_enumand
Invalid_argument("Json_encoding.construct: consequence of bad union") - New lint warning for local variables (
let ... in) that are never used.
Contributor's log
- Release 1.2.0 by @vincent-botbol in #1036
- Debug: report CPU time rather than absolute time by @AltGr in #1053
- Detect duplicate declaration of enum cases and struct fields by @AltGr in #1051
- Clerk: improve reporting when the test locations specified have no valid tests by @AltGr in #1044
- Fix faulty string enum encoding json by @vincent-botbol in #1058
- Do not generate unit enum constructors methods in Java by @vincent-botbol in #1055
- Feature/ctor labels by @rprimet in #1063
- Switch CI to Woodpecker by @AltGr in #1060
- Optimisation pass: limit width of sub-expression exploration by @AltGr in #1059
- Fix, complete and unify the JSON outputs with the different backends by @AltGr in #1045
- Fix clerk run refusing to run non-test scopes by @vincent-botbol in #1064
- Improve clerk handling of the exception-tree building by @AltGr in #1050
- Restore java 17 compatibility by @vincent-botbol in #1066
- Test usage of modules from external modules by @AltGr in #1061
- Fix tests for OCaml 5.5.0 by @AltGr in #1068
- Error out on use of private types by @AltGr in #1067
- Clerk: strip trailing CR from scope-test output (fix 0 tests on native Windows) by @rprimet in #1070
- New trace mechanism by @vincent-botbol in #1069
- Fix too large generated java code by @vincent-botbol in #1075
- Bump version to 1.2.1 everywhere by @vincent-botbol in #1077
- Better description for List.find by @denismerigoux in #1079
- Fix trace argument invalidating ninja cache by @vincent-botbol in #1078
- Add traces support in python backend by @vincent-botbol in #1076
- Merge back adjacent LawText nodes together before calling pandoc by @denismerigoux in #1083
- Clerk refactoring, targets classification and dependency handling by @AltGr in #1072
- Fix coloring changes by @vincent-botbol in #1089
- Java fixes by @rprimet in #1087
- Fixes clerk related issues by @vincent-botbol in #1090
- Add lint for unused local variables by @rvaccone in #1082
- Fix --gen-external template to match ExternalType functor signature by @Sarztak in #1092
- Add -linkall to enforce dynamic backend registration by @vincent-botbol in #1097
- Patch java files target installations and generate maven project files by @vincent-botbol in #1093
- Further clerk fixes by @AltGr in #1098
- Propagate documentation as javadoc in the generated java code by @vincent-botbol in #1091
- clerk: keep catala's stderr out of the JSON output of exceptions by @rprimet in #1100
- Clerk: generate simplistic dependency Makefiles in the C backend by @AltGr in #1099
- Fix JSON input propagation from clerk to catala by @vincent-botbol in #1108
- Clerk: cleanup the handling of include dirs by @AltGr in #1110
- Fix escaping of module names in Clerk target handling by @AltGr in #1114
New Contributors
Full Changelog: 1.2.0...1.3.0