github SSC-ICT-Innovatie/nl-kat-coordination v1.23.0rc1
v1.23.0 rc1
on GitHub

pre-release5 hours ago

What's Changed

Debian 13 support has been added, and a whole bunch of performance enhancing fixes have been made. This was mainly possibly due to the existence of various long running and very large OpenKAT installs. On these large installs (20M scans+) we where given a chance to see where the system actually starts to misbehave due to missing indexes, timeouts etc.

  • Add octopoes_api_scanprofiles to release-example compose by @hasecon in #5147
  • Fix certificate expiration check to use UTC by @underdarknl in #5178
  • crisis_room: fix 500 after organization deletion (view guard + cascade delete) by @hasecon in #5191
  • Chore/add debian13 ubuntu24.04 build image by @underdarknl in #5195
  • Fix nuclei-cve detection and harden nuclei normalizers by @hasecon in #5209
  • various dependency locked to safe versions, and created requirement files by @underdarknl in #5212
  • Fix get_tree(search_types) dropping findings on descendant OOIs by @hasecon in #5213
  • Add unit tests for five high-impact octopoes bits by @hasecon in #5112
  • Refactor sed command to find and replace in services by @underdarknl in #5211
  • Add debian13 / Ubuntu 24.04 build targets with their respective python versions by @underdarknl in #5179
  • Add index on taskDB that allows quick filtering on org, type and sort on datetime, as this s a common pattern to search for. by @underdarknl in #5165
  • swap out Katalogusclient from ScanProfile propagation daemon for direct Xtdb client by @underdarknl in #5149
  • Remove duplicate mime-type append in Bytes create_raw by @hasecon in #5224
  • Allow for bounded/unpartial counts to be returned from the scheduler … by @underdarknl in #5166
  • Add integration tests for scheduler bounded/partial counts by @hasecon in #5225
  • Make the boefje crash when it sees an exception, instead of returning a non usefull mimetype by @underdarknl in #5256
  • Re-introduce the clearance level explanation table, as it was missing by @underdarknl in #5254
  • Update various deps due to upstream security fixes by @underdarknl in #5235
  • Also show the raw file when showing an ooi observation by @underdarknl in #5236
  • Fix incorrect indemnification present check in template by @underdarknl in #5226
  • Add normalizer unit tests for kat_shodan, kat_wpscan and kat_nuclei_cve by @hasecon in #5113
  • Add bytes API coverage for /normalizer_metas and auth edge cases by @hasecon in #5114
  • Add get_normalizer_metas to BytesAPIClient and use it in the integration tests by @hasecon in #5271
  • Point OCI_IMAGE defaults at the librekat registry by @hasecon in #5287
  • Fix kat_shodan vulns iteration passing dict to CVEFindingType.id by @hasecon in #5116
  • Fix nuclei boefjes: ship templates and pass -t by @hasecon in #5286
  • Make kat_cve_2023_34039 actually run its SSH-key check by @brennodewinter in #5300
  • Extract Software from Shodan banners by @brennodewinter in #5297
  • Enrich LeakIX leak findings with proof, source, scale and auth state by @brennodewinter in #5298
  • Emit InternetDB ports and stop the cdn tag from skipping the rest by @brennodewinter in #5296
  • Attach nuclei panel/takeover findings to the matched URL by @brennodewinter in #5295
  • Extract Software from nmap CPEs in the shared nmap normalizer by @brennodewinter in #5293
  • Extract Software from service banners and connect over IPv6 by @brennodewinter in #5294
  • Harden and enrich the Censys normalizer by @brennodewinter in #5288
  • Grant schema access to new PostgreSQL users by @underdarknl in #5277
  • Fix kat_external_db discarding one section when the other key is missing by @brennodewinter in #5290
  • Fix kat_kat_finding_types crash on unknown ids and missing risk by @brennodewinter in #5292
  • Fix kat_crt_sh crash on wildcard and email certificate identities by @brennodewinter in #5289
  • Update stale Rocky translation template by @brennodewinter in #5259
  • Report Git revision in development UI by @brennodewinter in #5310
  • Fix/octopoes missing systemd service by @underdarknl in #5268
  • Skip beyond the first response blocks, and avoid unparseable EDE options from resolvers by @underdarknl in #5315
  • Get more out of LeakIX: fix finding binding, dead stage mapping, and extract certificates/software/TLS by @hasecon in #5317
  • Add missing index, used for superuser task lists by @underdarknl in #5325
  • Fix kat_cve_finding_types crash on CVSS v4.0-only CVEs by @brennodewinter in #5291
  • Fix blank OOI graph by restoring the hydrated tree in the context by @hasecon in #5323
  • Harden the normalizer against documents that have text/html but no ac… by @underdarknl in #5320
  • Bundle the OOI graph's JS dependencies instead of relying on globals by @hasecon in #5324
  • Remove dead ooi-types json_script block from the graph template by @hasecon in #5329
  • Pin Playwright in the webpage-capture image so it stops breaking by @hasecon in #5333
  • Fix 500 on object list for ImageMetadata with a HostnameHTTPURL resource by @hasecon in #5341
  • Minor cleanup around the add objects form, automatically select the o… by @underdarknl in #5237
  • Bump playwright from v1.53.0-noble to v1.62.1-noble in /boefjes/boefjes/plugins/kat_webpage_capture by @dependabot[bot] in #5334
  • Fix scan level inheritance chain lookup, also pre-filter eligble neighbours based on known model/path inheritance by @underdarknl in #5284
  • Follow-up on #5284: rename, docstring, regression test by @brennodewinter in #5344
  • Introduce an update to the task DB migration that makes sure the uniq… by @underdarknl in #5342

dependency updates:

Full Changelog: v1.22.0...v1.23.0rc1

What's Changed

  • Bump actions/upload-artifact from 7.0.0 to 7.0.1 by @dependabot[bot] in #5145
  • Bump actions/upload-pages-artifact from 4.0.0 to 5.0.0 by @dependabot[bot] in #5144
  • Bump actions/cache from 5.0.4 to 5.0.5 by @dependabot[bot] in #5141
  • Bump docker/build-push-action from 7.0.0 to 7.1.0 by @dependabot[bot] in #5143
  • Bump SonarSource/sonarqube-scan-action from 7.0.0 to 8.0.0 by @dependabot[bot] in #5142
  • Add octopoes_api_scanprofiles to release-example compose by @hasecon in #5147
  • Bump urllib3 from 2.6.3 to 2.7.0 in /boefjes/boefjes/plugins/kat_security_txt_downloader by @dependabot[bot] in #5158
  • Bump urllib3 from 2.6.3 to 2.7.0 in /boefjes/boefjes/plugins/kat_webpage_analysis by @dependabot[bot] in #5161
  • Bump idna from 3.10 to 3.15 in /boefjes/images by @dependabot[bot] in #5168
  • upgrade various deps due upstream security fixes by @underdarknl in #5157
  • Fix certificate expiration check to use UTC by @underdarknl in #5178
  • crisis_room: fix 500 after organization deletion (view guard + cascade delete) by @hasecon in #5191
  • Bump github/codeql-action from 4.35.1 to 4.36.0 by @dependabot[bot] in #5181
  • Bump softprops/action-gh-release from 2.6.1 to 3.0.0 by @dependabot[bot] in #5183
  • Bump docker/build-push-action from 7.1.0 to 7.2.0 by @dependabot[bot] in #5184
  • Bump docker/login-action from 4.0.0 to 4.2.0 by @dependabot[bot] in #5182
  • Bump docker/setup-buildx-action from 4.0.0 to 4.1.0 by @dependabot[bot] in #5180
  • Chore/add debian13 ubuntu24.04 build image by @underdarknl in #5195
  • Fix nuclei-cve detection and harden nuclei normalizers by @hasecon in #5209
  • various dependency locked to safe versions, and created requirement files by @underdarknl in #5212
  • Fix get_tree(search_types) dropping findings on descendant OOIs by @hasecon in #5213
  • Add unit tests for five high-impact octopoes bits by @hasecon in #5112
  • Refactor sed command to find and replace in services by @underdarknl in #5211
  • Add debian13 / Ubuntu 24.04 build targets with their respective python versions by @underdarknl in #5179
  • Add index on taskDB that allows quick filtering on org, type and sort on datetime, as this s a common pattern to search for. by @underdarknl in #5165
  • Bump docker/metadata-action from 6.0.0 to 6.1.0 by @dependabot[bot] in #5223
  • Bump actions/cache from 5.0.5 to 6.1.0 by @dependabot[bot] in #5222
  • Bump actions/setup-python from 6.2.0 to 6.3.0 by @dependabot[bot] in #5220
  • Bump softprops/action-gh-release from 3.0.0 to 3.0.1 by @dependabot[bot] in #5219
  • Bump actions/checkout from 6.0.2 to 7.0.0 by @dependabot[bot] in #5221
  • Chore/dep updates 29 june by @underdarknl in #5218
  • swap out Katalogusclient from ScanProfile propagation daemon for direct Xtdb client by @underdarknl in #5149
  • Remove duplicate mime-type append in Bytes create_raw by @hasecon in #5224
  • Allow for bounded/unpartial counts to be returned from the scheduler … by @underdarknl in #5166
  • Add integration tests for scheduler bounded/partial counts by @hasecon in #5225
  • Bump svgo from 2.8.2 to 2.8.3 in /rocky by @dependabot[bot] in #5250
  • Bump immutable from 4.3.8 to 4.3.9 in /rocky by @dependabot[bot] in #5251
  • Make the boefje crash when it sees an exception, instead of returning a non usefull mimetype by @underdarknl in #5256
  • Re-introduce the clearance level explanation table, as it was missing by @underdarknl in #5254
  • Update various deps due to upstream security fixes by @underdarknl in #5235
  • Also show the raw file when showing an ooi observation by @underdarknl in #5236
  • Fix incorrect indemnification present check in template by @underdarknl in #5226
  • Add normalizer unit tests for kat_shodan, kat_wpscan and kat_nuclei_cve by @hasecon in #5113
  • Add bytes API coverage for /normalizer_metas and auth edge cases by @hasecon in #5114
  • Bump actions/setup-python from 6.3.0 to 7.0.0 by @dependabot[bot] in #5276
  • Bump actions/checkout from 7.0.0 to 7.0.1 by @dependabot[bot] in #5273
  • Bump docker/setup-buildx-action from 4.1.0 to 4.2.0 by @dependabot[bot] in #5274
  • Add get_normalizer_metas to BytesAPIClient and use it in the integration tests by @hasecon in #5271
  • Bump github/codeql-action/init from 4.36.0 to 4.37.4 by @dependabot[bot] in #5275
  • Bump SonarSource/sonarqube-scan-action from 8.0.0 to 8.2.1 by @dependabot[bot] in #5272
  • Fix CodeQL action version mismatch between init and analyze by @hasecon in #5281
  • Add @brennodewinter to CODEOWNERS by @hasecon in #5282
  • Point OCI_IMAGE defaults at the librekat registry by @hasecon in #5287
  • Fix kat_shodan vulns iteration passing dict to CVEFindingType.id by @hasecon in #5116
  • Fix nuclei boefjes: ship templates and pass -t by @hasecon in #5286
  • Make kat_cve_2023_34039 actually run its SSH-key check by @brennodewinter in #5300
  • Extract Software from Shodan banners by @brennodewinter in #5297
  • Enrich LeakIX leak findings with proof, source, scale and auth state by @brennodewinter in #5298
  • Emit InternetDB ports and stop the cdn tag from skipping the rest by @brennodewinter in #5296
  • Attach nuclei panel/takeover findings to the matched URL by @brennodewinter in #5295
  • Extract Software from nmap CPEs in the shared nmap normalizer by @brennodewinter in #5293
  • Extract Software from service banners and connect over IPv6 by @brennodewinter in #5294
  • Harden and enrich the Censys normalizer by @brennodewinter in #5288
  • Grant schema access to new PostgreSQL users by @underdarknl in #5277
  • Fix kat_external_db discarding one section when the other key is missing by @brennodewinter in #5290
  • Fix kat_kat_finding_types crash on unknown ids and missing risk by @brennodewinter in #5292
  • Fix kat_crt_sh crash on wildcard and email certificate identities by @brennodewinter in #5289
  • Update stale Rocky translation template by @brennodewinter in #5259
  • Report Git revision in development UI by @brennodewinter in #5310
  • Fix/octopoes missing systemd service by @underdarknl in #5268
  • Skip beyond the first response blocks, and avoid unparseable EDE options from resolvers by @underdarknl in #5315
  • Get more out of LeakIX: fix finding binding, dead stage mapping, and extract certificates/software/TLS by @hasecon in #5317
  • Add missing index, used for superuser task lists by @underdarknl in #5325
  • Fix kat_cve_finding_types crash on CVSS v4.0-only CVEs by @brennodewinter in #5291
  • Fix blank OOI graph by restoring the hydrated tree in the context by @hasecon in #5323
  • Harden the normalizer against documents that have text/html but no ac… by @underdarknl in #5320
  • Bundle the OOI graph's JS dependencies instead of relying on globals by @hasecon in #5324
  • Remove dead ooi-types json_script block from the graph template by @hasecon in #5329
  • Pin Playwright in the webpage-capture image so it stops breaking by @hasecon in #5333
  • Fix 500 on object list for ImageMetadata with a HostnameHTTPURL resource by @hasecon in #5341
  • Bump docker/metadata-action from 6.1.0 to 6.2.0 by @dependabot[bot] in #5336
  • Bump docker/build-push-action from 7.2.0 to 7.3.0 by @dependabot[bot] in #5337
  • Bump docker/login-action from 4.2.0 to 4.6.0 by @dependabot[bot] in #5338
  • Bump github/codeql-action/init from 4.37.4 to 4.37.7 by @dependabot[bot] in #5335
  • Minor cleanup around the add objects form, automatically select the o… by @underdarknl in #5237
  • Bump playwright from v1.53.0-noble to v1.62.1-noble in /boefjes/boefjes/plugins/kat_webpage_capture by @dependabot[bot] in #5334
  • Fix scan level inheritance chain lookup, also pre-filter eligble neighbours based on known model/path inheritance by @underdarknl in #5284
  • Follow-up on #5284: rename, docstring, regression test by @brennodewinter in #5344
  • Introduce an update to the task DB migration that makes sure the uniq… by @underdarknl in #5342

Full Changelog: v1.22.0...v1.23.0rc1

Check out latest releases or
releases around SSC-ICT-Innovatie/nl-kat-coordination v1.23.0rc1

Don't miss a new nl-kat-coordination release

NewReleases is sending notifications on new releases.

Get notifications