Changelog
- Use original screenshot resolution instead of window resolution point except explore mode.
- New Workflow: Use Command+Control+Shift+W/V to set the target to a window/VOCursor and perform the OCR scan. After that, the features such as real-time OCR, explore, and ask will use the target.
- Reset shortcuts to default if there are different features after an update
- Bug fix: global shortcuts sometimes not active
- Customize shortcuts
- Token usage at the end of description
- Support system prompt for GPT
- Setting to toggle use last prompt without asking
- Save last screenshot
- Dismiss menu with command+Z instead of esc if realtime or navigation is active.
- You can just press return to ask GPT without editing.
- Changed diff algorithm for less verbose realtime OCR.
- Realtime OCR remains active at its initial location, allowing you to move the VOCursor during the process. To perform realtime OCR in a different location, stop the OCR, move the VOCursor, then restart realtime OCR.
- Realtime OCR of VOCursor: Command+Control+Shift+r
- Able to toggle obbject detection from the setings.
- OCR Window: Command+Control+Shift+w
- OCR VOCursor: Command+Control+Shift+v
- Ask GPT about VOCursor: Command+Control+Shift+a
- Settings: Command+Control+Shift+S
- Faster screenshot of VOCursor
- Open an image file in VOCR from finder to ask GPT
- Gpt response gets copied to the clipboard, so you can paste somewhere if you miss it.
- Object Detection through rectangles: Any boxes without text such as icons.
- Moved save OCR result to the menu.
- Moved target window to settings menu.
- auto Scan: Thanks @vick08
- Readme Improvement: Thanks @ssawczyn
The GPT features utilize GPT-4V, and they require your own OpenAI API key.
IMPORTANT: Location information from GPT-4V is extremely unreliable and inaccurate.
Experimental
These features may not make into the public release.
- Identify object when navigation is active: Command+Control+I
- Explore window with GPT: Command+Control+Shift+e
- an option to switch to using a local model such as Llava using llama.cpp instead of GPT.
Warning: It's very complex to set your own Llama.cpp server.