MSTICPy v3.0.0 Release Notes

MSTICPy v3.0.0 is a major release that brings Python 3.13 support, drops legacy
Python versions, replaces deprecated Azure SDK dependencies, adds a new
OpenObserve data provider, and includes significant improvements to the
Cybereason and Sentinel drivers.

⚡ Breaking Changes

Python Version Support

• Dropped: Python 3.8 and 3.9 are no longer supported.
• Added: Python 3.13 is now fully supported.
• Supported versions: Python 3.10, 3.11, 3.12, and 3.13.

Removed: msrestazure Dependency

The deprecated msrestazure package has been removed and replaced with
azure-mgmt-core>=1.6.0. All Azure authentication and management code
now uses azure-mgmt-core and azure.core.exceptions.

If you have code that catches msrestazure.azure_exceptions.CloudError,
update it to use azure.core.exceptions.HttpResponseError instead:

# Before (v2.x)
from msrestazure.azure_exceptions import CloudError
try:
    result = client.some_operation()
except CloudError as err:
    ...

# After (v3.0)
from azure.core.exceptions import HttpResponseError
try:
    result = client.some_operation()
except HttpResponseError as err:
    ...

Removed: kql Extra

The kql install extra (pip install msticpy[kql]) has been removed.
KqlmagicCustom is no longer a supported dependency.

Removed: aiagents Extra

The aiagents install extra (pip install msticpy[aiagents]) has been removed.
The autogen-agentchat integration is no longer included.

Updated: sql2kql Extra

The sql2kql extra now requires mo-sql-parsing>=11,<12 (previously >=8,<9).
If you use sql_to_kql functionality, ensure your environment is updated:

pip install msticpy[sql2kql]

Removed: pkg_resources

All usages of the deprecated pkg_resources module (from setuptools) have been
replaced with modern alternatives (importlib.resources / importlib.metadata).

Updated Azure SDK Compatibility

• azure-mgmt-resource v25 breaking changes have been addressed.
• azure-mgmt-core minimum version raised to >=1.6.0.

Linting Toolchain: Ruff Replaces Black/Flake8/isort

The project now uses Ruff as the sole linter and formatter, replacing
black, flake8, isort, pydocstyle, and pylint. If you contribute to
MSTICPy, update your development setup:

pip install pre-commit
pre-commit install
# Ruff is configured in pyproject.toml

🚀 New Features

New Data Provider: OpenObserve

A new data driver for OpenObserve has been added,
enabling querying of OpenObserve instances directly from MSTICPy.

Installation:

pip install msticpy[openobserve]

Configuration (in msticpyconfig.yaml):

DataProviders:
  OpenObserve:
    Args:
      connection_str: "https://your-openobserve-host:5080"
      user: "your-username"
      password:
        KeyVault:
      verify: true
      timeout: 300

Usage:

import msticpy as mp
mp.init_notebook()

qry_prov = mp.QueryProvider("OpenObserve")
qry_prov.connect(
    connection_str="https://localhost:5080",
    user="admin",
    password="[PLACEHOLDER]",
    verify=True,
)

# Query with time range
df = qry_prov.exec_query(
    'search "error"',
    days=1,
    limit=100,
    timeout=120,
)

The driver supports time-range parameters (days, start/end),
result limits, timezone configuration, and custom timeouts.

Cybereason Driver Improvements

The Cybereason data driver has received significant enhancements:

• More query parameters exposed — additional control over pagination,
  timeouts, and retry behavior:

  qry_prov = mp.QueryProvider("Cybereason")
  qry_prov.connect(
      instance="cr1",
      tenant_id="...",
      client_id="...",
      client_secret="[PLACEHOLDER]",
  )
  
  df = qry_prov.exec_query(
      "{}",
      page_size=100,
      timeout=120,
      retry_on_error=True,
  )

• HTTP 429 rate-limit handling — automatic retry with backoff when
  Cybereason returns rate-limit responses.

• Improved timeout handling — dedicated _handle_request_timeout method
  for graceful recovery from request timeouts.

• Better error reporting — MsticpyDataQueryError now includes exception
  name and args for clearer diagnostics.

• Improved data flattening — _flatten_simple_values and
  _flatten_element_values have been reworked for more reliable output,
  with extra fields now mapped to the result DataFrame.

• Structural pattern matching — query response parsing now uses Python
  match/case statements for cleaner, more maintainable code.

Sentinel Certificate Authentication

Microsoft Sentinel connections now support certificate-based authentication
as an alternative to client secrets, providing enhanced security for
automated pipelines and service accounts. See Sentinel Provider for details.

ThreatIntelIndicators: New Table Schema Support

The Azure Sentinel BYOTI (Bring Your Own Threat Intelligence) provider now
supports both the old and new ThreatIntelIndicators table schemas. This
enables seamless operation across Sentinel workspaces regardless of which
schema version they use.

• Configurable confidence threshold — the TI confidence threshold is now
  a class constant that can be overridden, rather than a hardcoded value.

Timeseries Anomaly Detection Fixes

• Fixed ts_anomalies_stl to correctly pass a Series (not raw values) to
  the STL decomposition, resolving errors with certain input shapes.
• Fixed handling of the seasonal == 0 edge case in timeseries anomaly
  detection.

KQL Timespan Conversion Utilities

New utility functions for converting KQL timespan strings to Python
timedelta objects. Uses azure-kusto-data's parse_timedelta for
accurate KQL-compatible timespan parsing.

AzureSearchDriver Fixes

Fixed the AzureSearchDriver to properly set the MSSentinelSearch
environment and correctly override the query method.

📦 Dependency Changes

Added

Removed

Updated

Removed Python-Version-Conditional Dependencies

• importlib-resources conditional on Python ≤3.8 has been removed.
• Separate ipython version pins for Python <3.8 / ≥3.8 consolidated to
  a single >=7.23.1 requirement.

🛠️ Developer & Tooling Changes

• Ruff is now the single linter and formatter (replaces black, flake8,
  isort, pydocstyle, pylint). Configuration is in pyproject.toml.
• Pre-commit hooks updated to use Ruff for both linting and formatting.
• CI now tests against Python 3.10, 3.11, 3.12, and 3.13 on Ubuntu.
• ReadTheDocs configuration updated to Python 3.12 / Ubuntu 24.02.
• Structural pattern matching (match/case) is used where appropriate
  (requires Python 3.10+).
• Modern type hint syntax throughout (e.g., list[str] instead of
  List[str], str | None instead of Optional[str]).

🐛 Bug Fixes

• Fixed driver connection error messages in Splunk driver and tests.
• Fixed sporadic WhoIs test failures.
• Fixed Self type import for compatibility across Python versions.
• Fixed check_cli_credentials to use up-to-date Azure authentication code.
• Fixed provider name handling in TI lookup objects.
• Fixed result_format type annotation in resource graph driver.
• Fixed linting issues across the codebase for Ruff compatibility.

📖 Documentation

• Added documentation for Sentinel certificate authentication.
• Updated documentation to use modern type hint syntax.
• Sphinx API docs updated for new and changed modules.
• Added GitHub Copilot instructions file for contributors.

🙏 Contributors

Thanks to all contributors who made this release possible:

• Florian Bracq (@FlorianBracq) — Cybereason driver improvements, query parameter enhancements, and structural pattern matching
• Ian Hellen (@ianhelle) — Python 3.13 support, Azure SDK modernization, tooling migration, and core maintenance
• Julien (@juju4) — OpenObserve data provider

Full Changelog: v2.17.2...v3.0.0

761 files changed, 45,270 insertions(+), 46,385 deletions(-)