phax/phoss-smp phoss-smp-parent-pom-8.6.0

v8.6.0

on GitHub

• Added the reconciliation of the local Service Groups with the SML, using the List() operation of chapter 3.1.2.7 of the SML specification, on the new page Administration / SML / Reconcile participants.
  It reports the participants that exist locally but are not registered at the SML - those cannot be resolved via DNS - and the participants the SML has registered for this SMP that do not exist locally - those resolve to this SMP, which then answers HTTP 404.
  Because the complete participant list has to be read from the SML, this runs as a long running job in the background and the result is stored on the server as a ZIP file containing summary.xml, missing-in-sml.txt and orphans-in-sml.txt.
  The new configuration property sml.sync.retention.days defines how long the reports are kept - the default is 30 days - and sml.sync.page.delay optionally adds a delay between two page requests, so that reading a large SMP does not hammer the SML.
  Note that the SML does not offer a way to read a single participant, so the report is verified against the local state only, by re-reading the local Service Groups after the SML list was read. That removes the false positives caused by Service Groups being created or deleted while the job was running.
• Added the bulk repair of the SML differences on the new page Administration / SML / Repair SML differences, based on a reconciliation report.
  It registers the missing participants via CreateList(), and resolves the orphaned ones either by removing them from the SML via DeleteList() or by creating them as local Service Groups - the latter is what is wanted when the participants are missing locally because local data was lost. The SML calls happen in chunks of sml.repair.chunk.size (default 100), as a long running job.
  Creating them locally deliberately performs no SML call at all, because these participants are already registered there - that is exactly why they show up as orphans.
  The SML rejects a CreateList() or DeleteList() as a whole with no per item result, so a failed chunk is retried one participant at a time - a single unusable participant therefore does not block the rest, and the log says which ones could not be handled.
  The page shows what would be sent before anything is sent, and the execution needs a separate confirmation.
• The Migrate outbound and Migrate inbound pages now show an SML state column, so that a participant migration can be verified at the SML.
  Outbound, a participant that disappeared from this SMPs list is the evidence that the receiving SMP completed Migrate(), so the migration can be finalized. Inbound, a migrated participant must be registered for this SMP - if it is not, the takeover did not take effect.
• The Unregister from SML page now shows how many participants would be deleted, read from the first page of the SML participant list.
  Unregistering an SMP deletes all of its participants at the SML, and every deleted participant becomes free for another SMP to claim, so this cannot reliably be undone.
• The Check DNS state action of the Service Groups page now runs as a long running job from smp.dnscheck.async.threshold Service Groups on - the default is 500, and 0 means always.
  Previously it performed one DNS lookup per participant while the page was rendered, gave up after 30 seconds and still rendered a full looking table in which the remaining rows carried no information.
  The lookups are now performed in parallel, with smp.dnscheck.threads (default 8) threads.
• The Tasks/Problems page now verifies the registration of this SMP at the SML, using the Read() operation of chapter 3.1.3.2 of the SML specification.
  It reports if this SMP is not registered at all, and if the logical address the SML resolves all participants to differs from the address this SMP is configured with - a mismatch that silently breaks every delivery and that was previously invisible from inside the SMP.
  The check only runs if the SML connection is enabled, an SML is selected and sml.smpid is configured.
  The SML is queried lazily and the result is cached for one hour, so that the page does not perform a remote call on every rendering.
  The new class SMLRegistrationCache offers a clearCache () method to drop the cached result.
• Fixed the Directory not being notified when a Process is deleted in the Endpoint Tree page.
  The method deleteSMPProcess of ISMPServiceInformationManager did not invoke the registered ISMPServiceInformationCallback instances, so the automatic Directory update did not happen.
  It now invokes onSMPServiceInformationUpdated, because the Service Information itself still exists - it just contains one Process less.
  This affected the XML, the SQL and the MongoDB backend.
• (XML, MongoDB) Fixed that the replacement of an existing Service Information via the REST API was handled as a deletion followed by a creation.
  Both backends used Java object identity to detect an update, which can never match for a REST API call, because the request payload is converted into a newly created object.
  They now replace the existing entry, so that a single MODIFY audit entry is written instead of a DELETE and a CREATE audit entry, and so that the behaviour matches the SQL backend.
  Note that the audit log therefore no longer contains the replaced content of the Service Information - that was previously part of the DELETE audit entry.
  The MongoDB backend performs the replacement in a single replaceOne operation, so that a concurrent read can no longer observe the short moment in which the Service Information did not exist at all.
  It also throws an exception if the server does not acknowledge the replacement - in mergeSMPServiceInformation as well as in deleteSMPProcess - as it already did for an insertion.
  Based on #545 - thx @vinit-thummar
• (XML) The lookup of a single Service Metadata is now a map lookup instead of a scan over all stored entries.
  getSMPRedirectOfServiceGroupAndDocumentType and getSMPServiceInformationOfServiceGroupAndDocumentType derive the deterministic object ID from the participant ID and the document type ID and resolve it directly, so that a GET of a Service Metadata no longer walks all Redirects and all Service Informations of the installation.
  The new methods SMPRedirect.createSMPRedirectID (...) and SMPServiceInformation.createSMPServiceInformationID (...) create these IDs and are used by the respective constructors as well.
  As a side effect the participant ID of the lookup is now unified the same way as the participant ID of the owning Service Group, so that a lookup with a differently cased participant ID finds the entry - as the Service Group lookup already did.
  Based on #569 - thx @vinit-thummar
• Fixed the QR code for the setup of the two-factor authentication not being displayed, if the SMP runs on a read-only file system.
  The PNG image was encoded via ImageIO, which by default buffers the encoded data in a temporary file below java.io.tmpdir, and the creation of that file failed.
  The encoding now happens entirely in memory, so that no temporary file is needed.
  The fix is contained in ph-totp 2.1.1.
  Based on #567 - thx @dv0gt

What's Changed

• Handle service information replacements as updates by @vinit-thummar in #545
• Feature/issue 570 sml reconciliation by @phax in #575

Full Changelog: phoss-smp-parent-pom-8.5.0...phoss-smp-parent-pom-8.6.0