OpenC3 COSMOS Core 7.0.0 - The Next Generation
Welcome to OpenC3 COSMOS Core 7.0.0!
We are thrilled to announce the official release of OpenC3 COSMOS 7! This milestone represents months of intensive development, building on the foundation of v6 to deliver a faster, more scalable, and more integrated platform for command and control.
⚠️ Critical Update Note
If you are updating from a 7.0.0 Release Candidate, a full TSDB cleanup is required as table names now include the SCOPE.
- Action: Run
openc3.sh cleanup(Note: This drops all logged data!!!). - Alternative: Manually delete the TSDB volume via
docker volume rm xxx-tsdb-v.
🚀 Headline Feature: High-Performance TSDB (QuestDB)
The core of COSMOS 7 is a brand-new Time Series Database powered by QuestDB. This transition marks a significant leap forward:
- Speed: Data retrieval is now an order of magnitude faster.
- Simplicity: We have removed the reducer microservices and associated data overhead.
- Accessibility: Explore your data directly with SQL via the new TSDB Admin tab or the QuestDB console.
⚖️ License Update
We have transitioned from the AGPL to the COSMOS Builder’s License.
- Why? To ensure better compatibility with our upcoming App Store.
- Note: This license maintains your existing development rights but explicitly prohibits providing COSMOS as a managed or hosted service.
🛑 Breaking Changes
- S3 Backend Migration: Migrated from MINIO to versitygw.
- See the Migration Guide for details.
- Security: Support for plaintext passwords via the API has been removed.
- See the Password Security Guide for details.
🛠 New Behavior & Deprecations
- All containers have read-only file systems. This improves security across our containers.
- Unit Handling: The WITH_UNITS type has been removed. Existing with_units APIs (e.g., tlm_with_units) are now deprecated and will return the formatted value.
- Target Files: get_target_file now returns None/nil if a file is not found.
- Sparse Packets: Items outside buffer bounds in undersized packets now return nil/None instead of 0. This enables better support for sparsely populated packets.
- New keywords
CMD_DECOM_RETAIN_TIMEandTLM_DECOM_RETAIN_TIME- See the Configuration Changes for details.
🚧 Known Issues
- INT and UINTs larger than 64 bits are currently unsupported. This will be fixed in the next release. A workaround (and probably better design) is to declare any values greater than 64 bits as BLOCK types.
Enhancements
- Add docker containers now have read-only file systems
- Add CONVERTED_DATA keyword for conversions
- Add
check_boxScript Runner prmopt - Add open_bucket_dialog script API selecting bucket files
- Enable LATEST in Data Extractor
- Allow TlmViewer to clear transient errors
- Detailed logs around Scope/Settings changes
- Ability to hide Legend on Telemetry Grapher
- Allow manually entered values for states on Command Sender
- New OPENC3_ALLOW_HTTP env var to make it easier to allow http
- Filter Limits Monitor items by state
- UI theming support - check the Admin Settings to change COSMOS colors
- New TEMPLATE_BASE64 keyword for binary templates in the config files
- Queue interface packets by default with update at 0.1s
- Add hover tooltip, context menu, and stale overlay to LIMITSCOLOR widget
- App store API updates
- Optimize command and telemetry decom performance in Ruby and Python
- Add hint to admin console about setting k8s secrets
- New API for scripts and screens to open a browser tab
- Make Command Parameter list scrollable for Command Editor
- Validate SCREEN params and make options dialogs consistent throughout
- Bucket Explorer delete directory
- Telemetry Grapher select graph button
- Migrate Python package from Poetry to UV
- Workflow to restart microservices for a plugin
- Warn when packets are defined without ID_ITEMs
- Add UUID to Script Runner temp files to prevent name collision
- Update to Vite 7
- Update to Rails 8
- Switch from Redis to Valkey
- Switch from Vuex to Pinia
- Switch password hashing algorithm to argon2
Bug Fixes
- Fix Python Interface Microservice accessing attributes
- Fix Script Runner autocomplete when using LATEST
- Fix Command Sender Mode -> Disable Parameter Conversion checkbox
- Fix limits bar when displaying the value 0
- Fix FORMATVALUE widget for 64-bit integer values
- Fix Python Suite Runner to preserve add_script insertion order
- Fix plugin downloads
- Fix tools bucket routing in standalone EC2 deployment to S3 buckets
- Fix Python TcpipSocketStream write look lockup
- Fix interface disconnect loop when deleting packets from a plugin
- Fix xtce_converter CLI crash
- Fixes for STRUCTURE use
- Fix TSDB with Python conversions
- Fix Python custom microservices
- Fix Python script syntax check and CheckError display
- Fix screen overlay in Command Sender and clear transient screen errors in Telemetry Viewer
- Fix secret unpacking in Python
- Fix time picker to remove milliseconds
- Fix Python JSON telemetry performance (10x increase in speed)
- Fix Packet Viewer initial refresh
- Fix Python http client to close the socket on disconnect
- Fix recurring activities on Calendar (Enterprise)
- Fix variable_bit_size command corruption. If a command has more than one VARIABLE_BIT_SIZE declaration it would not be created correctly.
- Fix Script Runner mnemonic checker crashes and false positives
- Fix Telemetry Viewer Textfield input bug in float mode
- Fix Script Runner bug with no scrollbar in message dialogs
- Fix Telemetry Grapher performance with large historical data
- Fix zoom in Data Flow Diagrams
- Fix wait_check blocking on user input when sleep_time is 0
What's Changed
- Queue interface packets by default with update at 0.1s by @jmthomas in #2371
- Remove WITH_UNITS as type by @jmthomas in #2380
- Update to vite 7 by @ryan-pratt in #2518
- 2472 | get_target_file now returns None/nil when file is not found by @EmilyRagan in #2519
- Remove plaintext password support for api by @ryan-pratt in #2588
- Quest setup by @jmthomas in #2578
- Switch password hashing algorithm to argon2 by @ryan-pratt in #2608
- Rails 8 by @jmthomas in #2678
- Reinstall all plugins to support QuestDB by @jmthomas in #2636
- Disable dependabot PRs by @jmthomas in #2725
- Convert streaming api to the TSDB by @jmthomas in #2647
- Switch from Redis to Valkey by @ryanmelt in #2687
- Questdb migration by @jmthomas in #2683
- Switch from Minio to Versitygw by @jmthomas in #2656
- Fix Python script syntax check and CheckError display by @jmthomas in #2737
- Add hover tooltip, context menu, and stale overlay to LIMITSCOLOR widget by @jmthomas in #2734
- Clear transient screen errors and fix screen overlay in Command Sender by @jmthomas in #2733
- [Documentation] Embed feature highlight videos to docs by @clayandgen in #2735
- App store API updates by @ryan-pratt in #2497
- Optimize command and telemetry decom performance in Ruby and Python by @jmthomas in #2726
- Return nil for items outside buffer bounds in undersized packets by @jmthomas in #2685
- Fix secret unpacking by @ryan-pratt in #2736
- Clarify offline installation and project version by @jmthomas in #2745
- Sonarqube cleanup by @jmthomas in #2738
- Bump aquasecurity/setup-trivy from 0.2.4 to 0.2.5 by @dependabot[bot] in #2747
- Add hint to admin console about setting k8s secrets by @ryan-pratt in #2750
- Open tab api for scripts and screens by @jmthomas in #2744
- Update vulnerabilities and codeql config by @jmthomas in #2748
- Add JsonAccessor buffer type tests and improve code quality by @mcosgriff in #2754
- Validate SCREEN params and fix options dialogs by @jmthomas in #2752
- Move curl from devlopment to guides by @jmthomas in #2758
- Add TlmViewer playback test and fix time picker by @jmthomas in #2759
- Document keycloak tokens and update curl examples by @jmthomas in #2762
- Remove "verified" from plugins by @ryan-pratt in #2746
- TSDB decoding consistency by @jmthomas in #2757
- Add throughput testing infrastructure and fix Python telemetry performance by @jmthomas in #2742
- [Enhancement] Packet viewer refresh by @clayandgen in #2761
- Update documentation to include Windows 11 installation video by @clayandgen in #2765
- [Bug] Python Http Client Close by @clayandgen in #2767
- Update dependencies by @jmthomas in #2770
- [Bug] Recurring activities on Calendar by @clayandgen in #2756
- Add CMD/TLM to the TSDB table names by @jmthomas in #2766
- Fix VARIABLE_DESCRIPTION and VARIABLE_STATE docs by @jmthomas in #2778
- Update docs to mention TEMPLATE for tlm accessors by @jmthomas in #2781
- Remove unused packet_log_writer_pair from interface_thread.py by @jmthomas in #2789
- [Bug] Make Command Parameter list scrollable for Command Editor by @clayandgen in #2779
- Bump actions/checkout from 4 to 6 by @dependabot[bot] in #2791
- Bump actions/setup-python from 5 to 6 by @dependabot[bot] in #2790
- Rename bucket volume & fix received_time in TSDB by @jmthomas in #2784
- Fix variable_bit_size command corruption and add UI support by @jmthomas in #2777
- Fix mnemonic checker crashes and false positives by @jmthomas in #2788
- Better errors for missing required bucket param by @ryan-pratt in #2792
- Bucket Explorer delete directory by @jmthomas in #2794
- Update docs by @ryan-pratt in #2797
- Fix textfield input bug in float mode and improve drag UX by @mcosgriff in #2796
- [Enhancement] Telemetry Grapher select graph by @clayandgen in #2782
- [Bug] Scroll on MessageDialog by @clayandgen in #2793
- Update subscription docs by @jmthomas in #2801
- Add documentation for LATEST packet by @jmthomas in #2802
- Migrates the Python package from Poetry to UV for faster, more reliable dependency management. by @mcosgriff in #2755
- Bump the npm_and_yarn group across 2 directories with 2 updates by @dependabot[bot] in #2775
- Switch from Vuex to Pinia and Additional Notebook Support by @ryanmelt in #2783
- Fix Telemetry Grapher performance with large historical data by @jmthomas in #2800
- Remove REDUCER microservice and data by @jmthomas in #2785
- Workflow to restart microservices for a plugin by @markjmiller in #2667
- Remove the DECOM loggers and add new CMD/TLM_DECOM_RETAIN_TIME by @jmthomas in #2786
- New License for COSMOS 7 by @ryanmelt in #2716
- Maintain State in Data Flow Diagrams by @ryanmelt in #2817
- Update default plugins and cleanup AppNav by @jmthomas in #2812
- Add stored packet documentation to packet-types, interfaces, and protocols by @jmthomas in #2819
- Roadmap updates by @jmthomas in #2824
- Document CRC protocol ordering relative to delineation protocols by @jmthomas in #2820
- Remove deprecated items, update upgrade docs by @jmthomas in #2818
- Bump astral-sh/setup-uv from 7.0.0 to 7.3.0 by @dependabot[bot] in #2808
- Update documentation examples and URLs by @jmthomas in #2815
- Warn when packets are defined without ID_ITEMs by @jmthomas in #2816
- Optimize dockerfile for uv by @mcosgriff in #2811
- Add COSMOS_EXTRA to streaming data by @jmthomas in #2825
- Fix playwright tests and race condition in Bucket Explorer by @jmthomas in #2830
- Fix TSDB Streaming Api by @ryanmelt in #2833
- Add UUID to temp files, parallelize tests by @jmthomas in #2835
- Change uv sync from --locked to --frozen in Dockerfile by @jmthomas in #2836
- Fix wait_check blocking on user input when sleep_time is 0 by @jmthomas in #2832
- Fix xtce_converter CLI crash with gem_name nil and PythonProxy .class by @jmthomas in #2841
- TEMPLATE_BASE64 and Fixes for STRUCTURE use by @ryanmelt in #2846
- Prepend scope name to QuestDB table names by @jmthomas in #2848
- Upgrade updates by @jmthomas in #2838
- Add Linux video to documentation by @clayandgen in #2843
- Add new architecture page by @jmthomas in #2845
- Fix QuestDB ingress errors for Python conversions without converted_type by @jmthomas in #2849
- When looking for python binary to replace, need to handle single string and one word per parameter modes by @mcosgriff in #2851
- UI Theming support by @clayandgen in #2852
- Fix TlmViewer Playback missing scope argument by @ryanmelt in #2853
- Wrap videos row in docs by @ryan-pratt in #2864
- Fixed Python TcpipSocketStream write loop locking up on unhandled socket errors by @mcosgriff in #2865
- Update dependencies by @jmthomas in #2862
- Fix stale tlmcnt Redis keys causing interface disconnect loops by @mcosgriff in #2857
- [Accessibility] Add Astro compatibility to LimitsColor Widget by @clayandgen in #2866
- [Enhancement] Fixes Nav Bar clickability by @clayandgen in #2879
- [Enhancement] Limits Monitor Filtering by @clayandgen in #2881
- Resolved some SonarQube findings from uv conversion by @mcosgriff in #2807
- Allow traefik to route to the as-named tools bucket by @jmthomas in #2871
- Fix plugin downloads by @jmthomas in #2882
- Move CSP from HTML meta tag to Traefik header set by OPENC3_ALLOW_HTTP by @jmthomas in #2872
- Update Protocol Buffer documentation by @jmthomas in #2888
- Fix Python Suite Runner to preserve add_script insertion order by @jmthomas in #2885
- Fix FORMATVALUE widget for 64-bit integer values by @jmthomas in #2892
- [Bughancement] Limit Bar on Zero and Colors on DetailsDialog by @clayandgen in #2891
- Add note about command params with states to upgrade guide by @ryan-pratt in #2897
- App store plugin versions by @ryan-pratt in #2858
- [Enhancement] Highlight AppNav item on Refresh by @clayandgen in #2894
- Add validate and timeout info to queued commands by @jmthomas in #2893
- [Bug] Command Sender Disable Param Conversion checkbox stuck by @clayandgen in #2904
- Migrate plugin Python package management from pip3 to uv by @mcosgriff in #2902
- Add ability to pull to openc3_util by @jmthomas in #2898
- API rate limiting by @ryan-pratt in #2884
- Use parameterized queries to the TSDB by @jmthomas in #2908
- [Enhancement] Manually Entered on Command Sender by @clayandgen in #2906
- Disallow using session token to change password by @ryan-pratt in #2905
- Fix migration and better handle migration errors by @jmthomas in #2901
- Logout when changing password to clear sessions by @jmthomas in #2909
- Correcting some theming related color changes by @clayandgen in #2903
- Clear env vars from running scripts by @jmthomas in #2912
- Prevent path traversal in tool config names by @jmthomas in #2910
- [Enhancement] Ability to hide Legend on TlmGrapher by @clayandgen in #2895
- Follows Astral's recommended pattern for installing uv in Docker by copying the statically-compiled binary directly from the official image rather than installing via pip. by @mcosgriff in #2880
- Cancel previous CI jobs for current PR/branch when new commit(s) are pushed by @EmilyRagan in #2919
- Fix streaming from TSDB by @jmthomas in #2900
- The command
uv pip install --no-warn-script-locationno supported byuvby @mcosgriff in #2926 - One-time use tokens for AnyCable by @ryan-pratt in #2911
- [Enhancement] Detailed logs around Scope/Settings Changes by @clayandgen in #2916
- Bump docker/login-action from 3 to 4 by @dependabot[bot] in #2922
- Use redis to track bad password attempts by @ryan-pratt in #2921
- Add sslHeaders and remove OPENC3_ALLOW_HTTP to ssl and letsencrypt by @jmthomas in #2929
- Add migration to remove store_id from plugin by @ryan-pratt in #2928
- Bump dependencies by @jmthomas in #2920
- Fix issue for getting otp using keycloak auth by @ryan-pratt in #2937
- Increase max message size by @ryanmelt in #2938
- Fix latest autocomplete by @ryanmelt in #2941
- JSON.parse command sender array values by @jmthomas in #2935
- Add ALL wildcard support to streaming API packet subscriptions by @jmthomas in #2940
- Audit logging within Calendar activities by @clayandgen in #2943
- Finish release action by @jmthomas in #2944
- Allow TlmViewer to clear transient errors by @jmthomas in #2955
- Update dependencies by @jmthomas in #2957
- Enable LATEST in Data Extractor by @jmthomas in #2952
- Fix QuestDB playback hangs by using thread-local PG connections by @jmthomas in #2948
- Record the 'stored' flag as lowercase true/false by @jmthomas in #2949
- [Bug] Python Interface Microservice accessing attributes by @clayandgen in #2951
- Just print to docker logs for TSDB info messages. Increase ulimit and modify Dedup. by @ryanmelt in #2966
- Bump astral-sh/setup-uv from 7.3.0 to 7.4.0 by @dependabot[bot] in #2962
- Bump pnpm/action-setup from 4.2.0 to 4.3.0 by @dependabot[bot] in #2963
- Bump docker/setup-buildx-action from 3 to 4 by @dependabot[bot] in #2964
- Calendar docs and UI bug fix by @EmilyRagan in #2971
- Bump boto3 from 1.42.63 to 1.42.66 in /openc3/python by @dependabot[bot] in #2965
- Add open_bucket_dialog script API selecting bucket files by @jmthomas in #2954
- Fix UBI image builds and runtime on Apple Silicon (ARM64) by @mcosgriff in #2975
- Add check_box prompt, fix Cancel return, fix SR race condition by @jmthomas in #2977
- Add CONVERTED_DATA keyword for conversions by @jmthomas in #2953
- Calendar timeline scope matching by @EmilyRagan in #2976
- Readonly containers by @ryanmelt in #2978
- Move redis-ubi data volume to /data by @jmthomas in #2982
- Guard against OPENC3_LOCAL_MODE_PATH not existing by @jmthomas in #2985
- Add note about require in ERB by @jmthomas in #2992
- Turn issue templates into forms with required fields by @EmilyRagan in #2994
- Fix issue templates not populating in dialog by @EmilyRagan in #2999
- Change all controller status values to symbols by @jmthomas in #2995
- Fix issue templates by @EmilyRagan in #3000
- Add TSBD Admin Tab, and add tsdb error detection to scope cleanup by @ryanmelt in #2993
- Various code enhancements by @jmthomas in #3005
- Add CrawlChat integration to docs.openc3.com by @clayandgen in #3004
- Bump deps including traefik by @jmthomas in #3006
- [Enhancement] Grace Period added to Activity Creation by @clayandgen in #3009
- Remove ENV vars from scripts by @jmthomas in #3013
- Show unentitled App Store plugins with disabled install button by @ryan-pratt in #3015
Full Changelog: v6.10.4...v7.0.0