microsoft/msticpy v2.4.0 on GitHub

Main changes for this release

There are no huge changes in this release but a good variety of important updates and fixes.
We're also delighted to welcome 3 new contributors to the MSTICPy family:

Thanks so much!

New Threat Intel provider for Pulsedive from @fr0gger #609

This includes a standard MSTICPy TI provider (so you can include it in you collection of providers used for
regular TI checks on IPs, URLs, etc. This provider also contain a few custom methods that let to query
some other facets of the Pulsedive data. For example, the explore function that allows you to use
the pulsedive query language

pddetail = pdlookup.explore(query="ioc=pulsedive.com or threat=AgentTesla")
pddetail

You can also request a can on a domain or URL

pdscan = pdlookup.scan(observable= "alvoportas.com.br")
pdscan

To use any of the Pulsedive features you'll need an account and API key from Pulsedive
See more details of the usage in the Pulsedive notebook

Process tree updates #637

@ZeArioch added Process Tree support for FireEye HX data so it should be automatically recognized and render correct
We also added the ability to export a process tree as a text object - which is useful if you want to copy and paste
a tree or part of it into a non-HTML document. See the Process Tree docs for more details

+--  Process: C:Program FilesMicrosoft Monitoring AgentAgentMonitoringHost.exe
   PID: 0x888
   Time: 1970-01-01 00:00:00+00:00
   Cmdline: nan
   Account: nan  LoginID: 0x3e7
   +--  Process: C:WindowsSystem32cscript.exe  PID: 0x364
      Time: 2019-01-15 04:15:26+00:00
      Cmdline: "C:Windowssystem32cscript.exe" /nologo
         "MonitorKnowledgeDiscovery.vbs"
      Account: WORKGROUPMSTICAlertsWin1$  LoginID: 0x3e7
   +--  Process: C:Program FilesMicrosoft Monitoring AgentAgentHealth Service
      StateCT_602681692NativeDSCDesiredStateConfigurationASMHost.exe  PID:
      0x1c4
      Time: 2019-01-15 04:16:24.007000+00:00
      Cmdline: "C:Program FilesMicrosoft Monitoring AgentAgentHealth
         Service
         StateCT_602681692NativeDSCDesiredStateConfigurationASMHost.exe"
         GetInventory "C:Program FilesMicrosoft Monitoring
         AgentAgentHealth Service
         StateCT_602681692workServiceStateServiceState.mof" "C:Program
         FilesMicrosoft Monitoring AgentAgentHealth Service
         StateCT_602681692workServiceState"
      Account: WORKGROUPMSTICAlertsWin1$  LoginID: 0x3e7

Miscellaneous fixes #644

This sounds like a small item but contain several important fixes:

Azure authentication (az_connect) now avoids throwing exceptions if you ask it to use authentication types (e.g. clientsecret) where parameters are not passed (or available in environment variables). It will now just ignore those credential types and only throw an exception if no usable credential types remain.
Updates to API documentation
A new IPython magic "%save_to_cell" - this lets you save a Python object (e.g. a DataFrame to a base64-encoded blob in a new cell. The cell contains code to restore the original data. This is subject to the usual caveats about pickle - including the security ones. Do Not run a cell that unpickles some arbitrary data in notebooks that you do not trust.
A bunch of changes/fixes to the Sentinel APIs
- Most of these are fixes related to the newly-supported Sentinel Dynamic Summaries feature
- Some minor fixes also to Sentinel core

Python Logging support #640

We should have had this from the beginning but it's never too late to start correcting your mistakes.
We've implemented a central logging module and started to instrument some of the code that is especially complex
and where people often get stuck with cryptic errors. E.g. the init_notebook function.
We also enabled in in the authentication modules (az_connect) in #644
Most of the time, this will be invisible. However, if you need it you can just do the following:

# import msticpy as mp   # if not already imported
mp.set_logging_level("INFO")

Then re-run the function that you are having trouble with again.
You can also use the MSTICPYLOGLEVEL variable to control this. And, if you want to log to a file, set the env variable MSTICPYLOGFILE to the path of your log file. (You'll need to restart the kernel/python session and reload MSTICPy for this to take effect).

Support for Bokeh 3.0 #630 #642 and #650

@ctoma73 did some awesome work to track down problems with compatibility with Bokeh 3.0 and fix all of them (a lot were tedious mypy/linting fixes due to some of the more dynamic nature of the Bokeh 3.0 object model).
You'll notice in #650 that we still have Bokeh 2.4.3 in the MSTICPy requirements. We're not going to change that just yet since we want compatibility with PyViz/HoloViz panel - you will likely see some panel-related features in the next minor release.
Despite this (and assuming you can ignore some pip warning about MSTICPy not being compatible with Bokeh 3.x) you can install Bokeh 3.0 after MSTICPy and enjoy the delights of the new release. All of our code should be compatible (tested with 3.0.0 and 3.1.0).

That's all for this release.
We'll likely be doing a follow-on 2.5.0 release that will include several contributions from our 2023 Hackmonth (which turned into a HackNMonths event).

What's Changed

Add support for FireEye HX acquisition packages in process_tree by @ZeArioch in #616
Adding Pulsedive as Threat Intel provider by @fr0gger in #609
Fix error when latest version 3.0.3 of bokeh is installed by @ctoma73 in #630
Adding logging and updating settings access by @ianhelle in #640
ProcTree and init_notebook fixes by @ianhelle in #637
Adding data query paths test for DEX support by @ianhelle in #638
Fixing RangeTool with bokeh 3.1.0 not a GestureTool by @ctoma73 in #642
Modified the upload_df method to split the data into batches of 10,00… by @jllangley in #633
Misc updates for 2.3.2 release: by @ianhelle in #644
Reverting to bokeh version 2.4.3 for default install by @ianhelle in #650

New Contributors

@ZeArioch made their first contribution in #616
@ctoma73 made their first contribution in #630
@jllangley made their first contribution in #633

Full Changelog: v2.3.1...v2.4.0