Dear community, we're delighted to announce the release of OpenCTI 5.12.0 🥳! This milestone marks a turning point for the platform, both in terms of the new features it brings and the bugs it fixes, as well as the improvements in system resource utilization and performance 🚀.
⚠️Breaking change in the list filters system in the API (and the Python library) ⚠️
First of all, in order to support more complex search and filtering use cases such as grouping, this version introduces a major breaking change in the way list filters are built 🔍. If you have specific integrations that use the Python library or the GraphQL API, please read the migration documentation carefully 👁️🗨️.
This filters enhancement will continue in future versions, but now allows you to switch logical operators (AND and OR) between two groups or within a group. Also, on several text fields it is now possible to use new modes such as "starts with / ends with" 🎉. The new filter syntax unlocks the most advanced uses of knowledge retrieval, including the ability, for example, to filter threats according to country AND sector (targeting both) 🧬.
OpenCTI 5.12 also introduces the import / export of dashboards and widgets within dashboards, as well as the export of audit logs in CSV format🗄️. Generally speaking, the data export experience has been greatly enhanced, with the introduction of several buttons to make it more fluid. You can easily convert a graph into an investigation and vice-versa, add entities to a report with their relationships, and so on 🗜️.
In OpenCTI Enterprise Edition, a new feature now enables direct indexing of raw documents (PDF, HTML, DOCX, etc.), whose content becomes immediately accessible in the global search 🚄. This is a long awaited feature which definitely solves multiple challenges if the data is not correctly extracted / modelized. It also paves the way for the platform's future integrated NLP system 🎊.
We also have started to introduce a new ACL system at the entity level (like in dashboards and investigations) for Feedback and will expand it in the future to all STIX objects. Finally, this version contains various user experience enhancement on colors usage, light theme and overall navigation. We will carry on the hard work to make the platform more accessible and user friendly all over the upcoming releases ✈️.
Last but not least many connectors have been developed and enhanced in 5.12, especially HarfangLab, Tanium, Microsoft Sentinel, Mandiant and Recorded Future but also a bunch of community additions. Thank you everyone for your help, your feedback and your great contributions 🙏.
Enhancements:
- #5029 Improve CPU usage of elastic/open.search results parsing
- #5005 Improve auditing to split file read and file download
- #4982 Move top menu of entities in tabs
- #4913 [Dashboard] The colours of the labels do not match
- #4902 Improve the "click to download file" in lists
- #4883 Add the possibility to download a file as an Encrypted archive in the Export list panel
- #4804 Avoid upserting an filled attribute with something null
- #4765 Display a "By Filigran" logo on login page and on all pages of the app
- #4636 Why is there a limit of 5000 entities in a csv feed?
- #4536 Modifying Schema for filter of feed/taxii/stream
- #4505 CSV column mapping when a column can have different representations
- #4494 Enrich activity logging capabilities to cover more use cases
- #4484 In the live stream, add NOT operator filter for “observable type”
- #4455 Align search panel in Investigation with others in the platform
- #4405 Searching in knowledge graph should not make nodes disappearing but just grey them
- #4201 Add a "Indicators propagation in reports" rule in the rule engine
- #4124 Implement checkboxes on "Observations / Infrastructure" list
- #3631 Be able to download user analytics (audit / history) in JSON or CSV
- #3532 Harmonize behavior of fields in the Settings => Activity => Configuration
- #3242 Export/Import Custom Dashboard (and Widget)
- #3181 Extend authorized_members ACL system to Feedbacks
- #3165 Refactor filters display everywhere.
- #2695 Add dst_ref and src_ref properties in networktraffic or networktrafficadd object.
- #2686 Full refactor of filtering and searching using grouping / operators / fuzzy filters
- #2515 Enhance export buttons
- #2510 In users overview, create an analytics table with KPIs and data stats / health
- #1680 Information on how the research field works needed
- #1483 Full text search for documents
Bug Fixes:
- #5062 When opening a filter for the first time, focus is not taken
- #5039 Can't remove search keyword from Global Search
- #5015 Error: Enabling photo on for image carousel on Threat Actor
- #5014 No more relationship redirection in Data>relationships
- #4992 Fail to remove element with unfinished works
- #4981 Rules engine use 100% CPU in some circumstances
- #4969 Activity audit update events for the same entity are mixed up
- #4965 In Knowledge views, clicking on the "Entities view" button or the "Relationships view" when active make the UI disappeared
- #4938 The list of available triggers does not appear when creating a regular digest
- #4933 Missing icon for users in Ingestion Creation
- #4930 Page "group" not updated when adding users
- #4926 In Knwoledge overview, when "free select" is enabled and the user wants to add an entity by search through it, the search box does not allow to enter text
- #4916 Investigations Entities Limit
- #4911 [Case to Invest to Case] Cyclic relationship bug
- #4899 Logo for relationship creation suggested stays green
- #4894 logout Page Not foud using APP__BASE_PATH
- #4891 Error when sending objects from an Investigation to a Container
- #4889 Error when export list
- #4880 Version 5.11.13 packages have not been generated correctly
- #4872 [Dashboard] Area and Line view don't seem to retrieve the data correctly
- #4869 [Dashboard] Adding a "related entity" filter results in an error
- #4868 [Dashboard] The donuts and radar view don't take dynamic filters into account
- #4863 Adding a tag to a dashboard is not working
- #4860 Cannot trigger a background task on Campaign => Knowledge => Indicators
- #4845 One Organisation on my platform can no longer be added as an "Author"
- #4832 Add threat actor location: can't create a location + bad design
- #4827 Investigation Entity Display
- #4798 Cannot modify the name of an entity if already present in the alias of the same entity
- #4735 Relationship popup window closes by itself when reverse the direction
- #4693 Table text does not change color after switching from dark theme to light theme
- #4501 A table too large is cut off in the pdf export
- #4476 Network Traffic refs fail to be updated / upserted by integration
- #4431 Correlation View not Identifying all correlations
Pull Requests:
- [frontend] Add quick export button in Report overview (#2515) by @lndrtrbn in #4831
- [frontend] Fix Empty TAXII Ingester User Reverts to Previous User instead of System (#4813) by @helene-nguyen in #4834
- [backend/frontend] Implement full text search for documents (#1483) by @SouadHadjiat in #4275
- [frontend] Fix tables wider than page width when exporting to pdf (#4501) by @marieflorescontact in #4632
- [Frontend/Backend] Skip line when the config give a specific char (#4505) by @jpkha in #4815
- [frontend] reset trigger creation form for instance triggers (#4865) by @Archidoit in #4866
- [frontend/backend] Export and import dashboard config (#issue/3242) by @Goumies in #4837
- [backend] new logic engine for resolving FilterGroups for event streams (#4536) by @labo-flg in #4847
- [frontend] Harmonize behavior of fields in the Settings => Activity => Configuration + fix bug #4874 (#issue/3532) by @SarahBocognano in #4890
- [frontend] Fix error in container knowledge views (#3242) by @SouadHadjiat in #4892
- Update dependency axios to v1.6.0 [SECURITY] by @renovate in #4888
- [frontend] Fix trigger background task on entity knowledge indicators (#4860) by @SouadHadjiat in #4877
- [backend] Fix cyclic relation (#issue/4891) by @Goumies in #4910
- Bump axios from 1.6.0 to 1.6.1 in /opencti-platform/opencti-graphql by @dependabot in #4904
- [Frontend] Put configurations options into dialog (#4505) by @jpkha in #4853
- [frontend] Keep html body attribute 'data-theme' in sync when changing theme (#4693) by @lndrtrbn in #4906
- [Frontend] Fix logo creation suggested stays green #4899 by @jpkha in #4925
- [frontend] Use base path to construct logout URL (#4894) by @lndrtrbn in #4922
- [frontend] Updated colors for Drawer titles to not match active text for clarity by @ParamConstructor in #4917
- [frontend] Changes to helper text for search widgets by @ParamConstructor in #4915
- [backend/frontend] Log and activity tracking / statistics refactoring by @SamuelHassine in #4928
- [backend] Add the rule to propagate indicators in reports by @SamuelHassine in #4929
- [backend/frontend] Improve identity search to be scored ordered (#4845) by @richard-julien in #4867
- [frontend/backend] Duplicate custom dashboard with new name (#issue/3… by @Goumies in #4886
- Fix investigations and workspaces entities limit (#4916) by @marieflorescontact in #4937
- [backend/frontend] Add EE identity support by @Kedae in #4841
- [frontend/backend] Custom named dashboard duplication by @Goumies in #4947
- [frontend] Fix add threat actor/intrusion set location: can't create a location + bad design (#4832) by @SarahBocognano in #4876
- [backend] Fix login page after EE identity support (#4765) by @SouadHadjiat in #4958
- [frontend] Relationship popup window closes by itself when reverse the direction (#4735) by @SarahBocognano in #4954
- [frontend] Fix changing view in Knowledge (#4965) by @lndrtrbn in #4971
- [frontend/backend] Filters refacto (#2686) by @Archidoit in #4953
- [frontend] Fix ItemIcon type (#4933) by @marieflorescontact in #4980
- Cloning performance improvements to prevent event loop locking by @richard-julien in #4983
- [frontend/backend] File indexing advanced configuration (#1483) by @marieflorescontact in #4882
- [frontend] Add includeAuthorities in triggersKnowledge query (#4938) by @marieflorescontact in #4988
- Bugfixes for the new filters by @labo-flg in #4984
- [backend] Fail to remove element with unfinished works (#4992) by @richard-julien in #4993
- [frontend] Minor fixes by @Kedae in #4986
- Update dependency fs-extra to v11.2.0 by @renovate in #4998
- Update dependency convert to v4.14.0 by @renovate in #5000
- Update dependency helmet to v7.1.0 by @renovate in #4999
- Update dependency eslint-plugin-jsx-a11y to v6.8.0 by @renovate in #4996
- Update dependency eslint to v8.54.0 by @renovate in #4994
- Update dependency file-type to v18.7.0 by @renovate in #4997
- [backend] Fix cyclic relation bug in tasks and notes by @Goumies in #4920
- [frontend] Fix Adding a tag to a dashboard is not working (#4863) by @SarahBocognano in #4875
- [frontend/backend] Filters bug fixes 2 (#2686) by @Archidoit in #5001
- [frontend] Fix group member edition (#4930) by @Kedae in #5009
- [backend/frontend] Improve global search in files (#1483) by @SouadHadjiat in #4989
- [frontend/backend] Filters bug fixes #3 (#2686-bug3) by @Archidoit in #5008
- [backend] entity_type filter with different operators and modes fix by @Archidoit in #5003
- [backend] OpenID Connect: Allow to optionally use userinfo for groups & organizations by @Augustin-FL in #4945
- [backend] Improve handling of refs relationships (#4476) by @richard-julien in #5004
- [frontend/backend] Authorized members for Feedback (#3181) by @lndrtrbn in #4918
- [frontend/backend] New filters bug fixes #4 (#2686) by @Archidoit in #5013
- Update opentelemetry-js monorepo by @renovate in #5023
- Update dependency ts-loader to v9.5.1 by @renovate in #5024
- Update dependency jsdom to v23 by @renovate in #5021
- Update dependency jose to v5 by @renovate in #5020
- Update redis Docker tag to v7.2.3 by @renovate in #5026
- Update dependency unified to v11.0.4 by @renovate in #5025
- [frontend/backend] Export and Import Widgets as are by @Goumies in #4991
- [backend] New filters bug fixes #5 (#2686) by @Archidoit in #5031
- [frontend] improve global search layout (#1483) by @SouadHadjiat in #5019
- [backend/frontend] delete investigation, dashboard and notification of a user when user is deleted (#3720) by @aHenryJard in #4976
- [backend] Fix image carousel update (#5015) by @marieflorescontact in #5036
- [frontend] Refactor Filters UI (#3165) by @jpkha in #4990
- [frontend] Fix search when lasso selection by @Kedae in #5037
- [frontend] Fix filters on entities page (#3165) by @Kedae in #5041
- [backend] Avoid upserting a filled attribute with something null (#4804) by @richard-julien in #5042
- [backend] Improve CPU usage of elastic/open.search results parsing (#5029) by @richard-julien in #5030
- [backend/frontend] Improve auditing to split file read and file download (#5005) by @richard-julien in #5006
- [frontend] Fix search on empty string by @Kedae in #5043
- [frontend/backend] Fix changing entity name to same as alias by @Megafredo in #4871
- Update dependency react-grid-layout to v1.4.4 by @renovate in #5053
- Update Yarn to v3.7.0 by @renovate in #5054
- [backend/frontend] Improve id conversion between platforms (#3242) by @richard-julien in #5044
- Update dependency nodemailer to v6.9.7 by @renovate in #5049
- Update dependency recharts to v2.10.3 by @renovate in #5056
- Update dependency react-intl to v6.5.5 by @renovate in #5058
- Update dependency prettier to v3.1.0 by @renovate in #5055
- Update dependency tap to v18.6.1 by @renovate in #5057
- Update dependency react-markdown to v9.0.1 by @renovate in #5059
- Update material-ui monorepo by @renovate in #5048
- Update graphql-tools monorepo by @renovate in #5047
- Update dependency postcss to v8.4.32 by @renovate in #5052
- Update dependency passport-auth0 to v1.4.4 by @renovate in #5050
- Update dependency relay-test-utils to v16 by @renovate in #4692
- Update dependency reactflow to v11.10.1 by @renovate in #5060
- Update dependency pdfmake to v0.2.8 by @renovate in #5051
- Update dependency relay-compiler to v16 by @renovate in #4690
- Update dependency migrate to v2.0.1 by @renovate in #5063
- Update dependency express-rate-limit to v7.1.5 by @renovate in #5066
- Update dependency fast-glob to v3.3.2 by @renovate in #5067
- Update dependency apexcharts to v3.44.1 by @renovate in #5075
- Update dependency nconf to v0.12.1 by @renovate in #5064
- Update dependency @types/uuid to v9.0.7 by @renovate in #5073
- Update dependency @types/xml2js to v0.4.14 by @renovate in #5074
- Update dependency @types/turndown to v5.0.4 by @renovate in #5072
- Update dependency axios to v1.6.2 by @renovate in #5076
- Update dependency eql to v0.9.19 by @renovate in #5077
- Update dependency opentelemetry-sdk to v1.21.0 by @renovate in #5070
- Update dependency html-to-pdfmake to v2.5.1 by @renovate in #5068
- Update dependency esbuild to v0.19.8 by @renovate in #5065
- Update dependency @types/tough-cookie to v4.0.5 by @renovate in #5071
- Update dependency opentelemetry-api to v1.21.0 by @renovate in #5069
- [backend/frontend] introduce text filters by @richard-julien in #5078
- [backend] More entities in a csv fee + noFiltersChecking refactor (#4636) by @richard-julien in #5061
- Update dependency @types/react-dom to v18.2.17 by @renovate in #4599
- [frontend] Improve file indexing layout (#1483) by @SouadHadjiat in #5083
- [frontend] Fix: When opening a filter for the first time, focus is not taken (#5062) by @jpkha in #5082
- [frontend/backend] New filters fixes #6 (#2686) by @Archidoit in #5079
New Contributors:
- @Augustin-FL made their first contribution in #4945
- @aHenryJard made their first contribution in #4976
Full Changelog: 5.11.14...5.12.0
What's Changed
- [frontend] Add quick export button in Report overview (#2515) by @lndrtrbn in #4831
- [frontend] Fix Empty TAXII Ingester User Reverts to Previous User instead of System (#4813) by @helene-nguyen in #4834
- [backend/frontend] Implement full text search for documents (#1483) by @SouadHadjiat in #4275
- [frontend] Fix tables wider than page width when exporting to pdf (#4501) by @marieflorescontact in #4632
- [Frontend/Backend] Skip line when the config give a specific char (#4505) by @jpkha in #4815
- [frontend] reset trigger creation form for instance triggers (#4865) by @Archidoit in #4866
- [frontend/backend] Export and import dashboard config (#issue/3242) by @Goumies in #4837
- [backend] new logic engine for resolving FilterGroups for event streams (#4536) by @labo-flg in #4847
- [frontend] Harmonize behavior of fields in the Settings => Activity => Configuration + fix bug #4874 (#issue/3532) by @SarahBocognano in #4890
- [frontend] Fix error in container knowledge views (#3242) by @SouadHadjiat in #4892
- Update dependency axios to v1.6.0 [SECURITY] by @renovate in #4888
- [frontend] Fix trigger background task on entity knowledge indicators (#4860) by @SouadHadjiat in #4877
- [backend] Fix cyclic relation (#issue/4891) by @Goumies in #4910
- Bump axios from 1.6.0 to 1.6.1 in /opencti-platform/opencti-graphql by @dependabot in #4904
- [Frontend] Put configurations options into dialog (#4505) by @jpkha in #4853
- [frontend] Keep html body attribute 'data-theme' in sync when changing theme (#4693) by @lndrtrbn in #4906
- [Frontend] Fix logo creation suggested stays green #4899 by @jpkha in #4925
- [frontend] Use base path to construct logout URL (#4894) by @lndrtrbn in #4922
- [frontend] Updated colors for Drawer titles to not match active text for clarity by @ParamConstructor in #4917
- [frontend] Changes to helper text for search widgets by @ParamConstructor in #4915
- [backend/frontend] Log and activity tracking / statistics refactoring by @SamuelHassine in #4928
- [backend] Add the rule to propagate indicators in reports by @SamuelHassine in #4929
- [backend/frontend] Improve identity search to be scored ordered (#4845) by @richard-julien in #4867
- [frontend/backend] Duplicate custom dashboard with new name (#issue/3… by @Goumies in #4886
- Fix investigations and workspaces entities limit (#4916) by @marieflorescontact in #4937
- [backend/frontend] Add EE identity support by @Kedae in #4841
- [frontend/backend] Custom named dashboard duplication by @Goumies in #4947
- [frontend] Fix add threat actor/intrusion set location: can't create a location + bad design (#4832) by @SarahBocognano in #4876
- [backend] Fix login page after EE identity support (#4765) by @SouadHadjiat in #4958
- [frontend] Relationship popup window closes by itself when reverse the direction (#4735) by @SarahBocognano in #4954
- [frontend] Fix changing view in Knowledge (#4965) by @lndrtrbn in #4971
- [frontend/backend] Filters refacto (#2686) by @Archidoit in #4953
- [frontend] Fix ItemIcon type (#4933) by @marieflorescontact in #4980
- Cloning performance improvements to prevent event loop locking by @richard-julien in #4983
- [frontend/backend] File indexing advanced configuration (#1483) by @marieflorescontact in #4882
- [frontend] Add includeAuthorities in triggersKnowledge query (#4938) by @marieflorescontact in #4988
- Bugfixes for the new filters by @labo-flg in #4984
- [backend] Fail to remove element with unfinished works (#4992) by @richard-julien in #4993
- [frontend] Minor fixes by @Kedae in #4986
- Update dependency fs-extra to v11.2.0 by @renovate in #4998
- Update dependency convert to v4.14.0 by @renovate in #5000
- Update dependency helmet to v7.1.0 by @renovate in #4999
- Update dependency eslint-plugin-jsx-a11y to v6.8.0 by @renovate in #4996
- Update dependency eslint to v8.54.0 by @renovate in #4994
- Update dependency file-type to v18.7.0 by @renovate in #4997
- [backend] Fix cyclic relation bug in tasks and notes by @Goumies in #4920
- [frontend] Fix Adding a tag to a dashboard is not working (#4863) by @SarahBocognano in #4875
- [frontend/backend] Filters bug fixes 2 (#2686) by @Archidoit in #5001
- [frontend] Fix group member edition (#4930) by @Kedae in #5009
- [backend/frontend] Improve global search in files (#1483) by @SouadHadjiat in #4989
- [frontend/backend] Filters bug fixes #3 (#2686-bug3) by @Archidoit in #5008
- [backend] entity_type filter with different operators and modes fix by @Archidoit in #5003
- [backend] OpenID Connect: Allow to optionally use userinfo for groups & organizations by @Augustin-FL in #4945
- [backend] Improve handling of refs relationships (#4476) by @richard-julien in #5004
- [frontend/backend] Authorized members for Feedback (#3181) by @lndrtrbn in #4918
- [frontend/backend] New filters bug fixes #4 (#2686) by @Archidoit in #5013
- Update opentelemetry-js monorepo by @renovate in #5023
- Update dependency ts-loader to v9.5.1 by @renovate in #5024
- Update dependency jsdom to v23 by @renovate in #5021
- Update dependency jose to v5 by @renovate in #5020
- Update redis Docker tag to v7.2.3 by @renovate in #5026
- Update dependency unified to v11.0.4 by @renovate in #5025
- [frontend/backend] Export and Import Widgets as are by @Goumies in #4991
- [backend] New filters bug fixes #5 (#2686) by @Archidoit in #5031
- [frontend] improve global search layout (#1483) by @SouadHadjiat in #5019
- [backend/frontend] delete investigation, dashboard and notification of a user when user is deleted (#3720) by @aHenryJard in #4976
- [backend] Fix image carousel update (#5015) by @marieflorescontact in #5036
- [frontend] Refactor Filters UI (#3165) by @jpkha in #4990
- [frontend] Fix search when lasso selection by @Kedae in #5037
- [frontend] Fix filters on entities page (#3165) by @Kedae in #5041
- [backend] Avoid upserting a filled attribute with something null (#4804) by @richard-julien in #5042
- [backend] Improve CPU usage of elastic/open.search results parsing (#5029) by @richard-julien in #5030
- [backend/frontend] Improve auditing to split file read and file download (#5005) by @richard-julien in #5006
- [frontend] Fix search on empty string by @Kedae in #5043
- [frontend/backend] Fix changing entity name to same as alias by @Megafredo in #4871
- Update dependency react-grid-layout to v1.4.4 by @renovate in #5053
- Update Yarn to v3.7.0 by @renovate in #5054
- [backend/frontend] Improve id conversion between platforms (#3242) by @richard-julien in #5044
- Update dependency nodemailer to v6.9.7 by @renovate in #5049
- Update dependency recharts to v2.10.3 by @renovate in #5056
- Update dependency react-intl to v6.5.5 by @renovate in #5058
- Update dependency prettier to v3.1.0 by @renovate in #5055
- Update dependency tap to v18.6.1 by @renovate in #5057
- Update dependency react-markdown to v9.0.1 by @renovate in #5059
- Update material-ui monorepo by @renovate in #5048
- Update graphql-tools monorepo by @renovate in #5047
- Update dependency postcss to v8.4.32 by @renovate in #5052
- Update dependency passport-auth0 to v1.4.4 by @renovate in #5050
- Update dependency relay-test-utils to v16 by @renovate in #4692
- Update dependency reactflow to v11.10.1 by @renovate in #5060
- Update dependency pdfmake to v0.2.8 by @renovate in #5051
- Update dependency relay-compiler to v16 by @renovate in #4690
- Update dependency migrate to v2.0.1 by @renovate in #5063
- Update dependency express-rate-limit to v7.1.5 by @renovate in #5066
- Update dependency fast-glob to v3.3.2 by @renovate in #5067
- Update dependency apexcharts to v3.44.1 by @renovate in #5075
- Update dependency nconf to v0.12.1 by @renovate in #5064
- Update dependency @types/uuid to v9.0.7 by @renovate in #5073
- Update dependency @types/xml2js to v0.4.14 by @renovate in #5074
- Update dependency @types/turndown to v5.0.4 by @renovate in #5072
- Update dependency axios to v1.6.2 by @renovate in #5076
- Update dependency eql to v0.9.19 by @renovate in #5077
- Update dependency opentelemetry-sdk to v1.21.0 by @renovate in #5070
- Update dependency html-to-pdfmake to v2.5.1 by @renovate in #5068
- Update dependency esbuild to v0.19.8 by @renovate in #5065
- Update dependency @types/tough-cookie to v4.0.5 by @renovate in #5071
- Update dependency opentelemetry-api to v1.21.0 by @renovate in #5069
- [backend/frontend] introduce text filters by @richard-julien in #5078
- [backend] More entities in a csv fee + noFiltersChecking refactor (#4636) by @richard-julien in #5061
- Update dependency @types/react-dom to v18.2.17 by @renovate in #4599
- [frontend] Improve file indexing layout (#1483) by @SouadHadjiat in #5083
- [frontend] Fix: When opening a filter for the first time, focus is not taken (#5062) by @jpkha in #5082
- [frontend/backend] New filters fixes #6 (#2686) by @Archidoit in #5079
New Contributors
- @Augustin-FL made their first contribution in #4945
- @aHenryJard made their first contribution in #4976
Full Changelog: 5.11.13...5.12.0