Bugfixes
- Preserve the order of strings pulled from a file when deduplicating.
- Bring disabled classification strings back to their old behavior.
- Several bugs in service manifest parsing.
- Pin chardet library version for now.
- Solve zero handling errors in metrics logging.
Enhancements
- Monitor for OOM events in docker deployments.
- Surface 'severity' in error listings so warnings can be displayed as such.
If you're upgrading from previous version of 4.7 and experiencing issues, we recommend running the following script inside one of the containers on the cluster and restarting your deployment:
from assemblyline.common import forge
try:
ds = forge.get_datastore()
cls = forge.get_classification()
print('Patching service_delta classifications...')
for s in ds.service_delta.stream_search('*', fl='id,*', as_obj=False):
if s.get('classification') == '':
s['classification'] = cls.UNRESTRICTED
if s.get('default_result_classification') == '':
s['default_result_classification'] = cls.UNRESTRICTED
if s.get('update_config'):
for src in s['update_config'].get('sources', []):
if src.get('default_classification') == '':
src['default_classification'] = cls.UNRESTRICTED
if src.get('update_interval') == 1:
src['update_interval'] = None
ds.service_delta.save(s['id'], s)
ds.service_delta.commit()
print('Patching service classifications...')
for s in ds.service.stream_search('*', fl='id,*', as_obj=False):
if not s.get('classification'):
s['classification'] = cls.UNRESTRICTED
if not s.get('default_result_classification'):
s['default_result_classification'] = cls.UNRESTRICTED
if s.get('update_config'):
for src in s['update_config'].get('sources', []):
if not src.get('default_classification'):
src['default_classification'] = cls.UNRESTRICTED
if src.get('update_interval') == 1:
src['update_interval'] = None
ds.service.save(s['id'], s)
ds.service.commit()
print('Wiping heuristics...')
ds.heuristic.wipe()
print('Cleanup complete! The datastore has been successfully patched.')
except Exception as e:
print(f'Script failed with error: {e}')