The New Computer Update Part II introduces our first local vision model, a suite of native Mac integrations, 5x launch speed, and dozens of other requested features.
Mac Control
The new calendar, contacts, browser, mail, and sms Computer API modules let Open Interpreter control your Mac's native applications.
For example, Open Interpreter can execute the following code to get a contact, then send a text and an email with an attachment.
LLM-first Web Browser
The browser lets Open Interpreter quickly browse the web by querying a web-enabled language model.
Point Model
We have developed a model called point which is capable of locating visual controls precisely. It was designed to run locally on consumer hardware.
We leverage existing open-source models to "semantically search the screen" for text and icons. Language models can then call on this composite model to 'point' at text or icons on the screen:
While this model is capable of understanding simple interfaces with discrete icons, we intend to explore more general solutions in the next few weeks.
And more:
- 5X launch speed
- Experimental Docker support
- Export conversation to a Jupyter notebook using %jupyter
- Experimental one-click installers for Mac, Linux, and Windows
- Profiles preview (a feature similar to custom GPTs)
…
- Added all docs and folders by @tyfiero in #873
- Add safety and docker support documentation by @tyfiero in #882
- Fix cli crash during screenshot generation by @double-thinker in #884
- Add documentation for local models by @tyfiero in #889
- Fix config edit mac os by @MikeBirdTech in #927
- Add reference to open-interpreter-termux in README by @MikeBirdTech in #906
- Fix feedback from input api key by @Notnaton in #932
- Fix azure openai coding error by @MonsterCoder in #924
- Fixed %tokens command by @CyanideByte in #896
- update documentation to include the parameters by @Notnaton in #908
- Fix: NameError: name 'computer' is not defined by @MikeBirdTech in #937
- Update ROADMAP.md by @eltociear in #861
- Fix small typo in README.md by @jameynakama in #918
- Update README.md by @codesmith-emmy in #929
- Update local model docs by @MikeBirdTech in #931
- Profiles~ by @Notnaton in #933
- Introduced a new documentation structure, added demos, and more by @tyfiero in #948
- create new cli entry point and update poetry script by @MikeBirdTech in #947
- Added settings by @tyfiero in #949
- Update All Settings page for Open Interpreter Docs by @tyfiero in #967
- Move the entry point to start_terminal_interface by @Notnaton in #960
- Try to get some error logs from Markdown() by @Notnaton in #969
- Add missing llm. by @Notnaton in #975
- Fix the conversation restoration problem in version 0.2.0 New Computer by @Steve235lab in #977
- Remove getpass add prompt_toolkit by @Notnaton in #991
- [WIP] Fix broken unit tests. by @Steve235lab in #995
- chore: update aifs by @shivenmian in #997
- added string to multiple python file generation for skill library by @akuten1298 in #996
- add docs to computer by @birbbit in #1007
- Option to pass paths to docs.search by @birbbit in #1008
- Load skills without overwriting them by @birbbit in #1009
- Litellm custom provider by @Notnaton in #955
- Jupyter export magic command by @tyfiero in #986
- Docs for magic commands updated by @CyanideByte in #1004
- Add doc strings to all computer API functions for the LLM to use in the skill library by @tyfiero in #1011
- Fix broken link in lm-studio.mdx by @jorgepvenegas in #1019
- Update README.md by @MikeBirdTech in #1017
- Boost performance by lazy loading modules by @tyfiero in #1020
- Quick fix to sync the computer api between processes by @tyfiero in #1022
- fix parsing of a function by @birbbit in #1024
- Make messages accessible to interpreter by @birbbit in #1025
- Correct punctuation in the document. by @Dragonchu in #1033
- ROADMAP.md by @matthewrygh in #1029
- Resolve #1034 Migrate from appdirs to platformdirs by @lombad in #1038
- Multi-line input support. by @Steve235lab in #1048
- Fixed local.py not working on Windows. Added GPU acceleration. by @CyanideByte in #1043
- Rename the app directory to open-intepreter, and migrate all files from older versions by @tyfiero in #1060
- Expand computer api with texting, mail, contacts, and calendar by @tyfiero in #1061
- Fix profile migration bug by @tyfiero in #1066
- fix some error when start command on macos by @982945902 in #1062
New Contributors
- @double-thinker made their first contribution in #884
- @MonsterCoder made their first contribution in #924
- @jameynakama made their first contribution in #918
- @codesmith-emmy made their first contribution in #929
- @Steve235lab made their first contribution in #977
- @shivenmian ❤️ made their first contribution in #997
- @akuten1298 made their first contribution in #996
- @birbbit made their first contribution in #1007
- @jorgepvenegas made their first contribution in #1019
- @Dragonchu made their first contribution in #1033
- @matthewrygh made their first contribution in #1029
- @lombad made their first contribution in #1038
- @982945902 made their first contribution in #1062
- Full Changelog*: v0.2.0...v0.2.1