Highlights
Replaced dependency on IPWhois with local code #479
The ipwhois package seems to be abandoned and was causing conflicts with dnspython. We've
created equivalent functionality in msticpy removing build warnings and (minutely) speeding
up install time. We've also added a MSTICPy pandas accessor df.mp.whois() so that you can
do bulk queries from a dataframe.
Malware Bazaar TI Provider #459
Many thanks to @fr0gger for this.
Check out the notebook MBLookup to
see how you can use this new provider.
Documentation on how to build a Data Provider #465
This was previously a blog post but we've added it to the official docs - Writing and Contributing a Data Provider
Updates to Azure authentication to support more authentication types #484
We've switched from using DefaultAzureCredential to supporting the native credential types.
This lets us support additional credential types such as Client Secret and Certification authentication.
You can also create your own custom AzureCredential and pass this to az_connect.
Updates to SQL2Kql converter #488
This was really prompted by @tonybaloney in helping us get a build working on Python 3.8-3.11. This
module had a dependency on a now-deprecated moz_sql_parser. We've updated to use mo_sql_parsing - many thanks to @klahnakoski for work on keeping this alive and well.
Our module also contains some fixes and enhancements from the original.
Builds and tests now running on Python 3.8, 3.9, 3.10 #476
We were previously only building on Python 3.8. Huge thanks to @tonybaloney for working on this and bringing us
into the modern era. We still have some issues with Python 3.11dev - although this is due to SciPy breaking with
the last 3.11 version we tried. As soon as this is sorted we will add 3.11 back.
Added support for msticpy notebooks in Azure Synapse pipelines #493
This is mostly work done to support MS Sentinel running unattended notebooks in Synapse pipelines.
We've extended the mp.init_notebook() function so that it can correctly configure msticpy (looking for
msticpyconfig.yaml on a mounted blob storage container and persisting cached data there),
use the linked Key Vault to store secrets and supply service principal credentials to msticpy.
Important fixes
- Allow for missing columns in Folium map data frame #489
- Updated M365D/MDE driver to pass query request with JSON encoding #498 Defender have always supported this
but we were sending a JSON string, which they recently stopped supporting. This should be working again. - You can now see data query help before connecting to the data provider. It's also possible
to dry run the query to see the full query with parameters replaced without needing to connect. #482
What's Changed
- Azure ML init fix by @FlorianBracq in #471
- Sumologic-DataConnector.ipynb: fix s/url=/connection_str=/ by @juju4 in #436
- Fix typo in parameter name by @FlorianBracq in #470
- Update jinja2 requirement from <3.1.0 to <3.2.0 by @dependabot in #450
- Update docutils requirement from <0.18.0 to <0.20.0 by @dependabot in #464
- Ianhelle/writing data provs doc 2022 03 14 by @ianhelle in #465
- IP Whois and Azure Auth Fixes by @petebryan in #479
- Bump sphinx from 5.0.2 to 5.1.1 by @dependabot in #478
- Update UploadData.rst with fix for import by @pensivepaddle in #483
- [update] DNS resolver return dataframe with one IP per row by @ianhelle in #485
- Adding Malware Bazaar module as TI provider by @fr0gger in #459
- Allow query help when qry provider not connected by @ianhelle in #482
- Adding all supported auth types to config UI mpconfig_defaults.yaml by @ianhelle in #484
- Fixing build issues with requirements-dev and doc by @ianhelle in #490
- [fix] Updated SQL to KQL converter to use mo_sql_parsing by @ianhelle in #488
- [fix] Allow for missing columns in Folium GeoIP data. by @ianhelle in #489
- Add support for Python 3.10, 3.11 and test in CI by @tonybaloney in #476
- Revert "Add support for Python 3.10, 3.11 and test in CI" by @petebryan in #494
- Add support for Python 3.10, 3.11 and test in CI by @petebryan in #495
- Added azure_synapse_tools to support notebooks in Synapse by @ianhelle in #493
- Changing the MDE/M365D request content to json encoding. by @ianhelle in #498
- Fixes and updates to support notebooklet updates by @petebryan in #497
- Fix breaking issues in Auth and Browshot by @petebryan in #499
New Contributors
- @juju4 made their first contribution in #436
- @tonybaloney made their first contribution in #476
Full Changelog: v2.0.0...v2.1.0