phax/phoss-smp phoss-smp-parent-pom-8.5.0

v8.5.0

on GitHub

• Updated to ph-commons 12.5.0, ph-schedule 6.2.0, ph-web 11.4.6, ph-oton-bootstrap5 0.9.5, peppol-commons 13.0.0, peppol-shared-ui 0.11.0, phoss-directory 0.19.0, Flyway 13.7.0, the MongoDB driver 5.12.0, protobuf 4.36.2 and Jackson 3.2.3
• peppol-commons 13.0.0 moved the network neutral base types of the Peppol libraries into separate submodules, so that Peppol, OASIS BDXR, DBNAlliance and HR eDelivery share them instead of every network carrying its own copy.
  Nothing changes for a running SMP installation, and no configuration property and no REST API is affected, but the public API of phoss-SMP moved along with it.
  That is relevant if you maintain a custom backend or a fork, because the transport profile manager is one of the managers that a backend has to provide:
  • ISMPTransportProfile, SMPTransportProfile and ESMPTransportProfileState moved from com.helger.peppol.smp to com.helger.edelivery.smp, so ISMPTransportProfileManager, SMPTransportProfileManagerXML, SMPTransportProfileManagerJDBC, SMPTransportProfileManagerMongoDB, SMPTransportProfileCache and HCSMPTransportProfileSelect now use the new types in their signatures
  • ESMPAPIType moved from com.helger.peppol.sml to com.helger.edelivery.smp, so ESMPRESTType.getAPIType () returns the new type
  • EPeppolCodeListItemState was renamed to ECodeListItemState and moved from com.helger.peppolid.peppol to com.helger.peppolid.codelist - it is used by SMPNiceNameUI to render the state of a transport profile
  • The DNS based URL providers moved from com.helger.smpclient.url to com.helger.smpclient.url.dns, and IPeppolURLProvider is deprecated in favour of IBDXLURLProvider, so the Check DNS state action of the Service Groups page only distinguishes IBDXLURLProvider now. Both providers that can be active - PeppolNaptrURLProvider for smp.rest.type = peppol and BDXLURLProvider for the OASIS BDXR types - implement it, so the behaviour is unchanged
  • SMPExtensionList.getAsPeppolExtension () was removed, so the conversion of an SMP extension into the Peppol SMP v1 data model now goes through PeppolSMPExtensionHelper.getAsPeppolExtension (...). The OASIS BDXR SMP v1 and v2 conversions are unchanged
• The NAPTR lookups of the Check DNS state action of the Service Groups page and of the Remote Query API are faster for participants that are not registered in the SML:
  • peppol-commons 13.0.0 performs the lookup with an absolute DNS name, so that the search domains of the operating system resolver configuration are no longer appended to it. Previously an unresolvable participant caused one additional DNS query per search entry, each of which could run into the full DNS timeout
  • ph-web 11.4.6 only retries a lookup via TCP if the UDP lookup ended in a transient failure, instead of also retrying the definitive answers HOST_NOT_FOUND and TYPE_NOT_FOUND - that previously doubled the runtime of every unresolvable lookup
  • ph-web 11.4.6 additionally applies the configured DNS timeout to the single resolvers inside the ExtendedResolver, and gives each of them a smaller timeout than the overall lookup, so that a second configured DNS server is actually tried. Previously the overall timeout was already used up when the first server timed out
• The WAR files are roughly 2.6 MB smaller than in v8.4.3, mostly because BouncyCastle 1.86 - that comes with ph-commons 12.5.0 - is about 3 MB smaller than 1.85.2.
  The module split of peppol-commons 13.0.0 itself is size neutral for phoss-SMP: the Mozilla NSS root certificates that left peppol-commons save 160 KB, but the OASIS BDXR identifiers that phoss-SMP needs now arrive in the separate bdxr-id and edelivery-id JARs
• The configuration property webapp.security.login.errordetails now defaults to false, so that a failed login in the management GUI shows the same generic error message, no matter whether the login name is unknown, the password is wrong, or the user is deleted, disabled or lacks the required role.
  Previously it defaulted to true for backwards compatibility, which allowed an unauthenticated user to find out whether a specific user exists.
  Set the property to true explicitly to get the previous behaviour back on a development or a troubleshooting installation.
  The detailed reason is still written to the audit log, so it stays available under Administration / Audit.
  See #564 - thx @RetarusEDI.
• The session cookie (JSESSIONID) now uses SameSite=Strict by default, so that a browser does not send it on cross-site requests.
  The value is configurable with the new configuration property webapp.session.cookie.samesite, that accepts Strict, Lax and None - an empty value leaves the attribute untouched, so that only the web.xml or the application server determine it.
  Because of Strict, following a link from an external page into the management GUI now leads to the login page, as the session cookie is not sent with that request.
  If None is used, the cookie must be marked as Secure as well, otherwise browsers reject it - a warning is logged on startup in that case.
  See #565.
• The session cookie (JSESSIONID) can now be marked with the Secure flag, so that a browser only sends it via https, using the new configuration property webapp.session.cookie.secure.
  Possible values are true, false and auto, with auto being the default: it enables the flag, if smp.publicurl is an https URL, and leaves it untouched otherwise, so that installations that are only reachable via plain http - like a local development instance - keep working without any change.
  Only the explicit values true and false overrule an existing setting from the web.xml or from the application server.
  The flag is set on startup, before the ServletContext is initialized, because the session cookie configuration is immutable afterwards.
  See #563.
• (MongoDB) Added the indexes that were missing on the hot paths, as the counterpart of the SQL migration V38.
  They are created on startup, like all the other MongoDB indexes: dt on smp-audit, which grows unbounded and is only ever read sorted by date, so that reading it no longer needs a collection scan with an in-memory sort; sgid, doctypeid on smp-serviceinfo and on smp-redirect for the Service Metadata lookup of the REST API, which is the most frequent query of an SMP at all; ownerid on smp-servicegroup for the listing of the Service Groups of a single owner; email on users for the login; and tokens.token on user-tokens for the resolution of a Bearer token, which happens on every token authenticated REST request.
  On smp-redirect and smp-serviceinfo the Document Type is part of the index, so that the lookup by Service Group alone - used when a Service Group is deleted - can use it as well.
  The first startup after the upgrade can take noticeably longer if the audit collection is large, because the index is built then.
• The Service Group select boxes of the pages Endpoints, Redirects, Business Cards and Participant Migration (outbound) now load their entries on demand via Ajax, in chunks of 25 entries, instead of rendering every Service Group into the HTML.
  On an SMP with many Service Groups the affected pages took very long to load, or froze the browser altogether, because select2 had to ingest one <option> per Service Group on initialisation.
  The entries are now queried, searched, sorted and paged by the backend, using the new methods getAllSMPServiceGroups (ESMPServiceGroupFilter, IPagingSpec, String), getSMPServiceGroupCount (ESMPServiceGroupFilter, String) and containsAnySMPServiceGroup (ESMPServiceGroupFilter) of ISMPServiceGroupManager.
  The new ESMPServiceGroupFilter expresses the page specific filters - "has no Business Card" and "has no Participant Migration that prevents a new one" - as part of the backend query, with a NOT EXISTS sub select for SQL and a $lookup for MongoDB, so that they no longer need to read the whole table.
  (MongoDB) Two indexes are created for that on startup: sgid on the collection smp-businesscard, and pid as well as direction, state on the collection smp-participant-migration.
  The create forms now open on "(please select)" instead of silently preselecting the first Service Group.
  The class HCServiceGroupSelect is now a factory with the single method createAjax (...); the previous method create (...) and the eager select box were removed.
  Based on #548 - thx @vrbyjimmy.
  See #546.
• (SQL) Added the new database migration V38 that creates the indexes that were missing on the hot paths, for MySQL, PostgreSQL, Oracle, DB2 and SQL Server.
  Created in all dialects are IX_smp_endpoint_tprofile on smp_endpoint (transportProfile) for the transport profile usage check, IX_smp_pmigration_dir_state on smp_pmigration (direction, state) and IX_smp_pmigration_pid on smp_pmigration (pid) for the listing and the deletion of participant migrations, IX_smp_audit_dt on smp_audit (dt) and IX_smp_audit_userid on smp_audit (userid) for the audit list, which is ordered by date and filtered by user and which grows unbounded, and IX_smp_secusertoken_userid on smp_secusertoken (userid) for the resolution of a user token to its user.
  IX_smp_secuser_email on smp_secuser (email) is created for all dialects except DB2 and SQL Server, IX_smp_ownership_username on smp_ownership (username) for all except MySQL, IX_smp_secuser_loginname on smp_secuser (loginname) for MySQL, PostgreSQL and Oracle, IX_smp_smr_participant on smp_service_metadata_red (businessIdentifierScheme, businessIdentifier) for PostgreSQL, DB2 and SQL Server, and IX_smp_bce_pid on smp_bce (pid) for PostgreSQL and Oracle - the remaining dialects already have these indexes since V1 respectively V8.
  On MySQL the columns loginname and email are of type TEXT, so they are indexed by their first 191 characters.
  The indexes are created unconditionally - if you already created one of them manually, drop it before starting, because the migration fails otherwise.
  Based on #554 - thx @ismailbennani.
• (SQL) Fixed the counting of the Business Cards on MySQL, which failed with a syntax error because the statement used COUNT (DISTINCT pid), with a blank between the name of the function and the opening parenthesis.
  MySQL only accepts that blank if the SQL mode IGNORE_SPACE is enabled, which is not the default.
  The query therefore returned -1 on MySQL, so that the Business Cards page showed a wrong total number of entries, and the button Publish all Business Cards to the Directory was permanently disabled, because it is only enabled if the total count is positive.
  MySQL installations are affected since v8.4.0, because the server side pagination introduced there is what started to call the counting method.
  Based on #559 - thx @vrbyjimmy.
• (SQL) A new block of IDs is now reserved atomically, so that several SMP instances sharing one database can no longer hand out the same IDs.
  Previously the counter stored in the smp_settings entry latest-id was read with a plain SELECT and written back with an absolute UPDATE, without any locking, so that two instances reading the counter at the same time reserved the identical block of 20 IDs.
  The lock of the ID factory only serializes the threads of a single JVM and can not help across instances.
  Affected were all IDs taken from the global ID factory - among them the business card entities, contacts and identifiers, the participant migrations and the SML information - which either ran into a primary key violation or overwrote an existing row.
  The new counter value is now written with a conditional UPDATE that only matches if the stored value is still the one that was read, and the whole reservation is retried in a new transaction if another instance was faster.
  Additionally a failed reservation now leads to an exception, instead of silently handing out the IDs of a block that was never persisted.
• Added the new REST API PUT /{ServiceGroupId}/services/{DocumentTypeId}/{ProcessId} that adds a single Endpoint to the referenced Process, as the additive counterpart of PUT /{ServiceGroupId}/services/{DocumentTypeId}, which overwrites all existing Processes and Endpoints of a Document Type.
  The expected body is a single Endpoint element, in the flavour of the configured smp.rest.type. The Service Information and the Process are created on the fly if they do not exist yet, so a Document Type can be built up Endpoint by Endpoint without ever reading and re-PUTing the whole Service Metadata.
  An Endpoint whose validity period overlaps that of an already existing Endpoint of the same Transport Profile is rejected with 400, so the call can never silently replace an existing Endpoint. Redirects are not touched by this call.
  The new method is saveServiceRegistrationEndpoint (String, String, String, EndpointType, SMPAPICredentials) of SMPServerAPI, BDXR1ServerAPI and BDXR2ServerAPI.
  The body is read with the new marshallers SmpEndpointMarshaller, Bdxr1EndpointMarshaller and Bdxr2EndpointMarshaller of the new package com.helger.phoss.smp.jaxb. The Peppol SMP and the OASIS BDXR SMP v1 XSD declare no global Endpoint element, so for these two the body is read without XML Schema validation and the name of the root element is checked explicitly instead - the OASIS BDXR SMP v2 Aggregate Components XSD does declare one, so there the body is validated against the schema.

What's Changed

• Fixed mysql select count for smp_bce by @vrbyjimmy in #559
• Add the missing indexes on the hot paths, for all database systems by @ismailbennani in #554
• Ajax loading of service groups in secure section redirects, endpoints… by @vrbyjimmy in #548
• Include the cause message in SMPSMLException by @rgruendlervrtx in #568

New Contributors

• @ismailbennani made their first contribution in #554
• @rgruendlervrtx made their first contribution in #568

Full Changelog: phoss-smp-parent-pom-8.4.3...phoss-smp-parent-pom-8.5.0