Note
This release includes a breaking change regarding cache'ing responses. If you are working on a current flow, export your ChainForge flow to acforgefile before installing the new version.
We're closer than ever to hosting ChainForge on chainforge.ai, so that no installation is required to try it out. Latest changes below.
The entire backend has been rewritten in TypeScript 🥷🧑💻️
Thousands of lines of Python code, comprising nearly the entire backend, has been rewritten in TypeScript. The mechanism for generating prompt permutations, querying LLMs and cache'ing responses is performed now in the front-end (entirely in the browser). Tests were added in jest to ensure the outputs of the TypeScript functions performed the same as their original Python versions. There are additional performance and maintainability benefits to adding static type checking. We've also added ample docstrings, which should help devs looking to get involved.
Functionally, you should not experience any difference (expect maybe a slight speed boost).
Javascript Evaluator Nodes 🧩
Because the application logic has moved to the browser, we added JavaScript evaluator nodes. These let you write evaluation functions in JavaScript, and function the same as Python evaluators.
Here is a side-by-side comparison of JavaScript and Python evaluator nodes, showing semantically equivalent code and the in-node support for displaying console.log and print output:
When you are running ChainForge on localhost, you can still use Python evaluator nodes, which will execute on your local Flask server (the Python backend) as before. JavaScript evaluators run entirely in the browser (specifically, eval sandboxed inside an iframe).
HuggingFace Models 🤗
We added support for querying text generation models hosted on the HuggingFace Inference API. For instance, here is falcon.7b.instruct, an open-source model:
For HF models, there is a 250 token limit. This can sometimes be rather limiting, so we've added a "number of continuations" setting to help with that. You can set it to > 0 to feed the response back into the API for text completions models, which will generate longer completions, for up to 1500 tokens.
We also support HF Inference Endpoints for text generation models. Simply put the API call URL in the custom_model field of the settings window.
Comment Nodes ✏️
You can write comments about your evaluation using a comment node:
'Browser unsupported' error 💢
If you load ChainForge on a mobile device or unsupported browser, it will now display an error message:
This helps for our public release. If you'd like ChainForge to support more browsers, open an Issue or (better yet) make a Pull Request.
Fun example
Finally, I wanted to share a fun practical example: an evaluation to check if the LLM reveals a secret key. This evaluation, including all API calls and JavaScript evaluation code, was run entirely in the browser:
Questions, concerns?
Open an Issue or start a Discussion!
This was a major, serious change to ChainForge. Although we've written tests, it's possible we have missed something, and there's a bug somewhere. Note that unfortunately, Azure OpenAI 🔷 support is again untested following the rewrite, as we don’t have access to it. Someone in the community, let me know if it works for you! (Also, if you work at Microsoft and can give us access, let us know!)