metabrainz/musicbrainz-docker v-2024-05-13-mbdb29-pg16

on GitHub

Breaking changes

This release upgrades both the PostgreSQL version to 16 and the MusicBrainz database schema version to 29. It requires to run commands taking down all services during the upgrade.

Docker images for the services db and musicbrainz are now prebuilt too. It shorten the build time by half an hour to just a few minutes. Running any command that requires Perl in the service musicbrainz should now be prefixed with carton exec -- .

Disk space requirement has been slightly readjusted: 250 GB (or 100 GB without indexed search), system included.

If your musicbrainz-docker is earlier than v-2023-05-16-mbdb28, you must first update to it (see its release notes) and follow the upgrade instructions below, or remove everything using sudo docker-compose down --rmi local --volumes and reinstall.

Upgrade instructions

Assuming your checked out version of musicbrainz-docker is v-2023-05-16-mbdb28 or later, then:

1. Ensure you’ve replicated up to the most recent replication packet available with the old schema 28. If you’re not sure, run:

   sudo docker-compose exec musicbrainz bash -c ./admin/replication/LoadReplicationChanges

   and see what it tells you; if you’re ready to upgrade, it should say “This replication packet matches schema sequence #29, but the database is currently at #28.”

2. Turn off cron jobs:

   admin/configure rm replication-cron
   sudo docker-compose up -d musicbrainz

3. Switch to the new code with:

   git fetch --tags origin
   git checkout v-2024-05-13-mbdb29-pg16

4. Upgrade the PostgreSQL data format:

   ./admin/upgrade-to-postgres16

   This step is expected to take some time, especially the last vacuumdb commands.
   (It took ~20 minutes with 4GB of shared buffers in test.)
   It should say “Upgrade complete!” at the end.

5. Upgrade the MusicBrainz database schema:

   mkdir -p local/compose/ && cp -a admin/lib/upgrade-db-schema/musicbrainz-stopped.yml local/compose/
   admin/configure add local-compose-musicbrainz-stopped
   sudo docker-compose up -d musicbrainz
   sudo docker-compose exec musicbrainz upgrade-db-schema.sh

   This step is expected to take a very long time, especially the last “Vacuuming DB”.
   (It took more than 2h with 4GB of shared buffers in test.)
   At the end it should say “Done” followed with a message that you can safely ignore as handled already: “UPDATE THE DB_SCHEMA_SEQUENCE IN DBDefs.pm TO 29 !”

   Troubleshooting

   The following lines are expected and can be safely ignored:

   NOTICE:  constraint "allowed_collection_entity_type" of relation "editor_collection_type" does not exist, skipping
   NOTICE:  constraint "label_code_length" of relation "label" does not exist, skipping
   

   If the last command quickly stops with the following error, it means that previous step 4 failed for some reason:

   ERROR:  relation "edit_note_change" already exists
   

   Please report this issue to us with the full log of the previous step(s) if possible.
   Then run the following commands instead:

   sudo docker-compose stop
   sudo docker-compose rm db
   sudo docker-compose build
   sudo docker-compose up -d musicbrainz
   sudo docker-compose exec musicbrainz upgrade-db-schema.sh
   

6. Load the first replication packet available with the new schema 29:

   sudo docker-compose exec musicbrainz bash -c 'carton exec -- ./admin/replication/LoadReplicationChanges --limit=1'

   and see what it tells you; if the upgrade, it should say “This packet was produced (or begins) at 2024-05-13 21:29:12.739887+00”

   Troubleshooting

   The following lines are expected and can be safely ignored (for the first packet only):

   The current row in musicbrainz.replication_control with key id='1' contains a different value in column current_schema_sequence (29) than the replication packet suggests it should have as the old value (28). at /musicbrainz-server/admin/replication/ProcessReplicationChanges line 497.
   WARNING:  amqp could not commit tx mode on broker 1, reply_type=2, library_errno=4
   

7. Turn cron jobs back on:

   admin/configure rm local-compose-musicbrainz-stopped
   admin/configure add replication-cron

8. Restart all services:

   sudo docker-compose up -d musicbrainz

Changes

• MBS-13358: Upgrade Perl version to 5.38.2 in musicbrainz service
• Perl module dependencies are now installed using Carton in musicbrainz service (see 98732f4):
  • Their versions now match production and will no longer fail on unexpected upstream breaking changes.
  • Running any command that requires Perl should now be prefixed with carton exec --
    See the above step 6. for example.
• MBS-13361: Upgrade PostgreSQL to version 16 in db service
• Upgrade MusicBrainz Server to v-2024-05-13-schema-change (release notes)
• MBVM-42: Prebuild all docker images (including for db and musicbrainz services)
  It shorten installation time by half an hour to just a few minutes.