github inex/IXP-Manager v7.0.0

latest release: v7.0.1
5 months ago

IXP Manager v7 (v7.0.0)

image

We are pleased to announce the immediate availability of IXP Manager v7.0.0. This is a major upgrade of the underlying Laravel framework and introduces a new required PHP version of v8.4. We recognise that mandating a new PHP version is a significant change for our users and often requires an underlying operating system upgrade. However, the technologies on which v6.x is built have passed their end-of-support and security maintenance periods.

IXP Manager v7 is now built using:

  • PHP v8.4, which will be supported by the PHP core team for security updates until the end of 2028.
  • The latest version of the Laravel Framework, 12.x. In addition, its upcoming 13.x version will also be compatible with PHP 8.4 and provide support until at least Q1 2028.
  • All third-party PHP libraries and frontend JavaScript libraries have also been updated.

We've included detailed upgrade instructions below, including a video by @barryo.

Release Summary

git --no-pager diff --shortstat release-v6 release-v7
 696 files changed, 76639 insertions(+), 56310 deletions(-)

A summary of the most significant changes in IXP Manager v7 includes:

  • Underlying framework and third-party library updates as mentioned above.
  • The introduction of static code analysis into our CI pipeline.
  • A new Diagnostics Suite feature to significantly speed up troubleshooting member issues.
  • P2P Graph Sorting.
  • Various IRRDB Enhancements.
  • Database logging of IRRDB updates.
  • Configuration via a new frontend UI.
  • Versioned documentation.
  • Additional unit tests, including near-complete frontend coverage.
  • A Vagrant development environment.

Acknowledgements

László Kiss (@griphons) joined the IXP Manager core team from March 2024 to April 2025, making a significant contribution to IXP Manager v7. Moreover, some additional features he was working on are largely complete but not production-ready and will be held back for a future release. We thank László for his contributions.

Bringing on developers such as László would not be possible without our sponsors. In addition to INEX, several IXPs sponsored the development of v7, including NIX, BCIX, INX-ZA, LONAP, INTERLAN, NAMEX and GRIX. We also had two patrons, ISOC and APNIC Foundation. The APNIC Foundation deserves special mention as our only diamond patron.

See https://www.ixpmanager.org/sponsors for more information on how to sponsor the project.

Talks About IXP Manager v7

We have already provided updates on IXP Manager v7 at several conferences, including RIPE 89 and the 41st Euro-IX Forum. You can find the presentations and, where available, links to videos at https://www.ixpmanager.org/support/talks.

NB: In these pre-release talks, we were targeting PHP 8.3/Laravel 11 as our release target. These have since been updated to PHP 8.4/Laravel 12.

New Features

IRRDB Enhancements

The possibility of IRRDB updates failing silently was raised in #877, and this has been addressed in several ways; all documented in the updated page at https://docs.ixpmanager.org/7.0/features/irrdb/.

The primary change to alert administrators of an issue is that an email is now sent to the configured IDENTITY_ALERTS_EMAIL recipient. This defaults to IDENTITY_SUPPORT_EMAIL, and so alerts should work once v7.0.0 is installed.

Several additional features have also been added relating to the IRRDB database management:

  1. The new IRRDB Summary (left-hand side menu) provides a summary of all members' IRRDB database entries/status. This menu highlights members for whom there has never been an IRRDB update (usually new members or members for whom the process has failed) and highlights as stale any member whose entries have not been updated in the last 24 hours.
  2. The last time a member's IRRDB entries were updated is also shown on their overview page.
  3. A member's IRRDB entry status can be checked using the new diagnostics functionality.
  4. You can click through from many places to view and update a member's IRRDB entries:
    • Via the summary page above in (1);
    • Via the IRRDB information on the member's overview page in (2); and
    • Via a menu option on the member's overview page.

Peer-to-Peer Graph Sorting

When IXP Manager introduced peer-to-peer (p2p) graphs, they quickly became the most popular feature for end users and the typical driver for them to log into the portal routinely. However, scrolling through tens or even hundreds of graphs that are neither sorted nor y-axis aligned is not an efficient or helpful way to identify a network's "top ten peers" or traffic anomalies. While the ultimate goal is to use a time-series database, as a stop-gap measure, we are introducing a new feature in v7 to order a member's p2p graphs by volume.

This will be enabled by default if your GRAPHER_BACKEND_SFLOW_ENABLED setting is on. I.e. if you are already using IXP Manager's p2p functionality, then this will "just work" when you upgrade to v7 via the task scheduler.

If you want to give it a kick start, run the following with yesterday's date:

./artisan grapher:upload-daily-p2p -v YYYY-MM-DD

More information can be found here in the documentation.

Frontend Configuration

Traditionally, applications using the Laravel framework are configured via a .env file in the root directory of the web application. This requires server access to make even minor changes, and even then, it is a large and complex file. Moreover, as IXPs upgrade IXP Manager in situ, they may often forget to add new configuration options in newer releases, even if only using the defaults for documentation purposes.

In IXP Manager v7, we introduce a new Settings component, accessible from the bottom left menu under IXP UTILITIES. Most common settings can now be viewed and changed here.

When you access this new feature for the first time, it will run several checks to ensure that the .env file is writable by the web server process, and that your existing .env file is compatible. More details can be found in the documentation.

Versioned Documentation

As each release adds features, we recognise that the documentation can become confusing when it is not clear when a feature was introduced.

To resolve this, we have now started versioning the documentation. When you browse to https://docs.ixpmanager.org/, you will be redirected to https://docs.ixpmanager.org/latest/ by default. 'latest' will always reference the last released major.minor version. As of now, v7.0 is the latest release, and so https://docs.ixpmanager.org/latest/ will provide the 7.0 documentation.

All new documentation will be added to the 'dev' identifier - https://docs.ixpmanager.org/dev/ - which currently points to 7.1 in preparation for that release.

All links within IXP Manager have been updated to reference the /latest documentation now.

Member Diagnostics Suite

This release includes an early proof-of-concept implementation of a work in progress to ease the support burden of supporting member connections. In the tools (gear icon) menu on customer overview pages, there is a new Run diagnostics... command. This will run a suite of tests, including:

  • Customer/member set-up validation;
  • IRRDB tests (database populated, entries fresh, etc.) for customers with IRRDB filtering;
  • Virtual interface tests;
  • Physical interface tests (including snmp tests to see if the port is up or down, to check MTU, speed, etc.);
  • Route BGP tests for route server, route collector, and AS112 session status where the Birdseye looking glass is available; and
  • Transceiver rx/tx power tests, where supported.

Future iterations on this feature will include UI/UX review and connectivity tests, such as ARP and ping replies.

Security Information and Updates

Secure Application Development Policy

As cybersecurity becomes more regulated through legislation such as the CRA and NIS2, users of IXP Manager must review, risk assess, and document their use of third-party software and supply-chain security. We recognise that IXP Manager is a key element of that for many IXPs.

INEX has been ISO 27001:2022 Information Security Management System (ISMS) certified since 2023, and the further development of IXP Manager is within the scope of INEX's ISMS Secure Application Development Policy. This includes the review and merging of contributions from third parties.

The IXP Manager project has now published its secure application development policy and our long-standing security policy.

2FA Now Enforced for All Users by Default

Continuing to consider the current cybersecurity environment and best practices, new installations of IXP Manager will enforce 2FA for all users by default.

In previous versions, it was an optional measure that users could elect to use. To restore this behaviour, you can set the following .env variable (or use the new Settings UI):

2FA_IXPM_ENFORCE_FOR_USERS="4"

At a minimum, we strongly suggest and recommend requiring 2FA for admin and customer admin users via:

2FA_IXPM_ENFORCE_FOR_USERS="2"

Continuous Integration (CI)

IXP Manager grew out of a code base and schema that started in the early '90s. Long before test-driven development or behaviour-driven development was standard for PHP. However, as the IXP Manager assumes more critical configuration tasks, we continue to backfill automated testing with continuous integration using GitHub actions.

This release introduces static code analysis, a method of reviewing source code to find potential errors, bugs, security vulnerabilities, and deviations from coding standards without executing the code. We use psalm for this, and we have integrated it into our CI pipeline.

Introducing static code analysis into existing projects, especially ones as old as IXP Manager, is notoriously tricky. We took the following approach:

  1. Psalm can be configured with different 'difficulty' levels. We chose level 4 in our CI as a good trade-off between catching actual issues while ignoring false positives.
  2. When initially deploying it, we chose a harder level first and identified all real issues and corrected them.
  3. We then set a baseline, per Psalm's documentation, to grandfather-in warnings in existing code.
  4. With the baseline in place, all new commits are then checked, reviewed and corrected if they generate Psalm errors.

In total, we corrected several hundred issues - a particular nod to László for this dedicated effort. There were no security issues found; these were more related to modern best-practice programming and changes in PHP over the years.

Frontend Unit Testing

This release improves all existing frontend unit testing and adds several more. The IXP Manager admin interface now has close to full coverage for IXP ADMIN ACTIONS.

Upgrade Instructions

If you are upgrading to IXP Manager v7, this is where to start. We assume that you are upgrading from v6.4.x. No other upgrade paths are supported or recommended.

To help with upgrades, we've cloned INEX's installation of IXP Manager v6.4.2 running on Ubuntu 20.04 LTS with PHP 8.0, and we've recorded a video of the upgrade process for both IXP Manager to v7.0.0 and to replace PHP 8.0 with PHP 8.4. You can find that video here, and these instructions were created when doing test runs for that same video. The video also demonstrates the upgrade from Ubuntu 20.04 through 22.04 to 24.04 and how to overcome a couple of issues.

Prerequisite: PHP v8.4 is required for IXP Manager v7.0.0. If you are using Ubuntu, we recommend this PPA, and the video above shows the 8.0 -> 8.4 upgrade process. We recommend that you upgrade to Ubuntu 24.04 LTS as part of this process if you are using an older version of Ubuntu (however, it will work fine on 20.04 with PHP 8.4).

The process we describe here is based on a standard installation of IXP Manager using the automated installation scripts. If you have installed IXP Manager in a custom location / using custom settings, then just be sure to adjust the following to match your own environment.

We will assume your installation of IXP Manager is installed at a location identified by the shell variable IXPROOT. This is set in step (1) below. Remember to reset this after reboots.

Remember: you can follow along with the video for a more detailed discussion on these steps, and we've included timestamped links below via the 🎬 icon.

  1. Set up some variables and ensure directory permissions are okay:

    # set this to your IXP Manager installation directory
    IXPROOT=/srv/ixpmanager
    
    # fix as appropriate to your operating system. ubuntu/debian is fine with this:
    MY_WWW_USER=www-data
  2. 🎬 Let's take some backups of both the IXP Manager installation and the database:

    # Start with the IXP Manager installation
    tar -jcf /srv/ixpmanager-v6.tar.bz2 $IXPROOT
    
    # And now the database:
    source $IXPROOT/.env
    mysqldump --no-tablespaces -h $DB_HOST -u $DB_USERNAME -p$DB_PASSWORD $DB_DATABASE | \
        bzip2 -9 >/srv/ixpmanager-database-v6.sql.bz2

    You should copy both of these files off the server before proceeding.

    If possible with your hypervisor, take a snapshot of your virtual machine so that you can rollback to this point should you need to.

    Before continuing, you should also ensure your IXP Manager MySQL user has SUPER privileges. This needs to be done as the MySQL root user. The command you need to execute with MySQL can be determined as follows:

    source $IXPROOT/.env
    echo "GRANT SUPER ON *.* TO \`$DB_USERNAME\`@\`$DB_HOST\`;"

    This command must be run on MySQL as the root user.

  3. 🎬 Enable maintenance mode to let your users know what's happening:

    cd $IXPROOT
    php artisan down

    Contrary to the above, I would recommend stopping Apache for this upgrade as we will be deinstalling PHP. During the time, PHP files could be viewed as text via Apache. This is not important for IXP Manager, we we only have a single file in /public, but you may be running other PHP applications on the same server.

    systemctl stop apache2.service
    systemctl disable apache2.service

    Added after the video recording: In a production environment, you should also pause your route server, route collector and as112 update cron jobs. For example, comment out the lines on those servers that cause them to pull new configurations from IXP Manager. While maintenance mode will prevent them from updating, when you disable maintenance mode, you may want to separately restart the router updates at a time of your choosing in your maintenance plan, and in an order you decide.

  4. 🎬 Ensure our operating system is fully up to date:

    apt update
    apt upgrade
    apt dist-upgrade
    apt autoremove
  5. 🎬 (Optional) Upgrade Ubuntu 20.04 to 22.04:

    # run the upgrade command:
    do-release-upgrade
    # see the video for how this works if you are performing this upgrade.
    
    # complete the upgrade and reboot the server
  6. 🎬 (Optional) Upgrade Ubuntu 22.04 to 24.04:

    # run the upgrade command:
    do-release-upgrade
    # see the video for how this works if you are performing this upgrade.
    
    # complete the upgrade and reboot the server
    # NB: interface names change from ethY to enXY - see the video!
    # We also solve a DNS resolution issue with `/etc/resolv.conf`.

    Remember to reset your environment variables, as appropriate for your system, e.g.:

    # set this to your IXP Manager installation directory
    IXPROOT=/srv/ixpmanager
    
    # fix as appropriate to your operating system. ubuntu/debian is fine with this:
    MY_WWW_USER=www-data
    
  7. 🎬 We now need to upgrade PHP to 8.4 and install any other required packages:

    # We use a Ubuntu PPA for the 8.4 packages. You may already be using this but it can be set-up as follows:
    apt-get install -yq software-properties-common
    add-apt-repository -y ppa:ondrej/php
    apt update
    
    # before we install anything, let's get back to no php for a clean system
    #
    # you may need to run these multiple times with various iterations - see the video:
    dpkg -r composer php-common `dpkg -l | grep 'ii  php' | awk '{print $2'}`
    dpkg --purge composer php-common `dpkg -l | grep 'rc  php' | awk '{print $2'}`
    
    # you'll know it worked when "dpkg -l | grep php' returns nothing
    
    # With that set-up, we install all the php packages required by IXP Manager v7.0.0.
    apt install -qy php8.4 php8.4-intl php8.4-rrd php8.4-cgi php8.4-cli     \
        php8.4-snmp php8.4-curl  php8.4-memcached libapache2-mod-php8.4     \
        php8.4-mysql php8.4-mbstring php8.4-xml php8.4-gd                   \
        php8.4-bcmath php8.4-memcache unzip php8.4-zip php8.4-yaml          \
        php8.4-ds 
    
    # do a package upgrade as the ppa has other updates also:
    apt upgrade 
    apt autoremove
  8. 🎬 Check your php version now and it should be 8.4.x:

    php -v

    If you have more than ~20 members or members with huge route server prefix lists, then you should edit /etc/php/8.4/apache2/php.ini and set memory_limit to >= 1024 and max_execution_time to 300.

    Finally, ensure Apache is setup to run PHP 8.4 after the above. This may be a no-op:

    apt install libapache2-mod-php8.4
    a2enmod php8.4
    
    # and ensure it is stopped again until we are ready:
    systemctl stop apache2.service
  9. 🎬 Upgrade IXP Manager:

    Updated: updated from the video, as we can jump straight to v7.0.1 here:

    cd $IXPROOT
    
    # pull the latest code
    git fetch --all
    
    # check out the version you are upgrading to
    git checkout v7.0.1
  10. 🎬 Install Composer locally. This is important because Ubuntu's stock version can go out of date. Get the instructions from getcomposer.org as the hash in this example may be out of date:

    php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"
    php -r "if (hash_file('sha384', 'composer-setup.php') === 'dac665fdc30fdd8ec78b38b9800061b4150413ff2e3b6f88543c636f7cd84f6db9189d43a81e5503cda447da73c7e5b6') { echo 'Installer verified'.PHP_EOL; } else { echo 'Installer corrupt'.PHP_EOL; unlink('composer-setup.php'); exit(1); }"
    php composer-setup.php
    php -r "unlink('composer-setup.php');"
  11. 🎬 Install the required libraries via composer:

    # delete stale/old bootstrapped caches:
    rm bootstrap/cache/*
    
    # this assumes composer.phar is in the IXP Manager install directory. YMMV - see notes on the upgrade instructions.
    chown -R $MY_WWW_USER: vendor/ storage/ 
    sudo -u $MY_WWW_USER bash -c "HOME=${IXPROOT}/storage && cd ${IXPROOT} && php ./composer.phar install --no-dev --prefer-dist"
  12. 🎬 Restart Memcached and clear the cache. Do not forget / skip this step!

    # (assuming we're still in $IXPROOT)
    systemctl restart memcached.service
    php artisan cache:clear
  13. 🎬 Update the database schema:

    # (assuming we're still in $IXPROOT)
    # (you really should take a mysqldump of your database first if you haven't already)
    # see what will change:
    php artisan migrate:status
    
    # migrate:
    php artisan migrate
  14. 🎬 Ensure file permissions are still correct.

    chown -R $MY_WWW_USER: ${IXPROOT}/storage $IXPROOT/vendor $IXPROOT/bootstrap/cache .env
    chmod -R u+rwX ${IXPROOT}/storage $IXPROOT/vendor $IXPROOT/bootstrap/cache
  15. 🎬 Clear out all caches:

    # (assuming we're still in $IXPROOT)
    php artisan cache:clear
    php artisan config:clear
    php artisan route:clear
    php artisan view:clear
  16. 🎬 Now work through the rest of the release notes before proceeding and disabling maintenance mode.

  17. 🎬 Recreate SQL views

    Some older scripts, including the sflow modules, rely on MySQL view tables that may be affected by SQL updates. You can safely run this to recreate them:

    # (assuming we're still in $IXPROOT)
    php artisan update:reset-mysql-views
  18. 🎬 Disable maintenance mode:

    # (assuming we're still in $IXPROOT)
    ./artisan up
    
    # and enabled and start apache
    systemctl enable apache2.service
    systemctl start apache2.service

    Added after the video recording: if you disabled the cron jobs for updating your routes in step (3) above, you can now start to re-enable those cron jobs.

Smaller Improvements, Fixes and Security

Full Changelog: v6.4.2...v7.0.0

Don't miss a new IXP-Manager release

NewReleases is sending notifications on new releases.