mikebrady/shairport-sync 3.3

Version 3.3. Audio Improvements, MQTT, Jack Audio, Stability Improvements, Bug Fixes and Enhancements.

on GitHub

New Feature Highlights:

• Automatic Format and Speed Selection for alsa devices. Using the alsa back end, when you specify the output device, Shairport Sync will now, by default, select the best output format and speed. For instance, if your output device is capable of 32-bit operation, it will be selected. In some situations, this will give a noticeable improvement in sound quality – see the discussion of audio improvements below. The greatest bit depth will always be selected, and the lowest speed that is a multiple of 44,100.
• Audio Transient Reduction: A Disable Standby feature to eliminate certain faint-but-annoying audible pops and clicks has been added for alsa devices. The setting prevents an output device entering standby mode and thus minimises standby/busy transitions, which can sometimes be heard. It is pretty impressive, especially combined with fixes to the dithering code described below. If you can enable 24- or 32-bit audio output to your output device, you can get even better results. This feature is available in the ALSA back end (standard in Linux) when it is driving output devices that can provide precision delay information, i.e. most hardware output devices and possibly some virtual devices.
• New active/inactive program hooks. Basically, these are "deglitched" replacements for play start/play end program hooks. When a play session starts, the system goes active. When the play session ends, the system remains active for a period determined by the active_state_timeout setting, 10 seconds by default. If another play session starts before the period elapses, the system stays active; otherwise the system goes inactive. The hooks are performed when the system goes active or inactive.
• A new MQTT client interface, with support for metadata and for some remote control commands.
• A new Jack Audio back end.
• PulseAudio: A new option for the pa backend to allow the specification of the audio sink.
• CYGWIN: An updated and enhanced installation procedure to install Shairport Sync as a Cygwin service.
• Opt-in default for certain added features: If support for soxr, metadata or the Apple ALAC decoder is included during building (i.e. at the ./configure stage), then the features are enabled by default at run time. The features are (1) soxr interpolation – chosen automatically if the device is fast enough, (2) the Apple ALAC decoder and (3) metadata, which is implied when you include support for metadata, D-Bus, MPRIS or MQTT features.
• Shairport Sync can now receive AirPlay streams consisting of raw PCM – 16 bit stereo frames at 44,100 samples per second. Packets must be 352 frames exactly.
• Improved support for big-endian CPUs by adding support for explicit -endian formats, i.e. S16_LE, S16_BE, S24_LE, S24_BE, S32_LE, S32_BE.
• Add a list of alsa hardware devices found to the alsa backend help section in the shairport-sync -h text. (Linux only – doesn't work on FreeBSD/OpenBSD.)

Stability Improvement Highlights:

• Greatly improved stability when playing is interrupted by another play request, by the abrupt disappearance of the audio source or by severe network problems.
• Enhanced detection of unrecoverable device or player errors. A hook is provided to call an external program when there errors are detected.
• Improved robustness when severe transient network problems occur.

More Details

New Features

• A new Jack Audio back end. The rough back end for Jack Audio present in earlier development builds has been extensively rewritten by Jörn Nettingsmeier in a way that is more in keeping with the Jack Audio style. It uses native Jack Audio lockless buffers and offers autoconnect facilities that the previous version didn't have. Many thanks to him.
• A new MQTT client interface, with support for metadata and for some remote control commands, is a new configuration option, thanks to the work of Till Zimmermann. MQTT is an often-used protocol in FOSS home-automation projects (as well as in commercial ones), and as Shairport Sync is often used in these setups, this adds client support for this protocol.
• The volume-control software has been completely rewritten. From a user's point of view, the result should be a much smoother response to volume control changes, free from artefacts. It is now also possible to combine the hardware mixer and the software attenuator in two ways to extend the attenuation range -- giving priority to the software mixer or giving priority to the hardware mixer. see the new volume_range_combined_hardware_priority setting in the general section of the configuration file.
• The muting/unmuting code has been rewritten to be simpler and more consistent.
• Two new external program/script hooks – run_this_before_entering_active_state and run_this_after_exiting_active_state are provided for when the system goes active or inactive. Background: Many users use the run_this_before_play_begins program hook to turn on an amplifier and the run_this_after_play_ends hook to turn it off. A big problem is when another play session starts immediately after a play session ends, causing the amplifier to be switched off and then on again very quickly. This happens, for example, when a YouTube clip ends and the next one begins. To get around this, the concept of an active state covering a sequence of play sessions is introduced. When a play session starts, the system goes active -- it enters the active state. When the play session ends, the system remains active for a period determined by the active_state_timeout setting, 10 seconds by default. If another play session starts before the period elapses, the system stays active; otherwise the system goes inactive -- it leaves the active state. The two new hooks mentioned above can be used to execute programs/scripts when the system goes active or inactive. They are to be found the sessioncontrol group of settings.
• A new command-line option, -u, directs logging to STDERR rather than the system log. Useful when you compile Shairport Sync without libdaemon using the --without-libdaemon configuration option.
• A new feature called Disable Standby keeps the output DAC in the play state all the time and helps to remove some annoying popping or clicking noises. This is an attempt to remove the annoying low-level clicking sounds that some output devices make just when they start processing audio and sometimes when they stop. Typically a faint click might be heard just before a play session starts or just before audio resumes after a pause. Similarly, a faint click can sometimes be heard just after a play session ends. It is extremely difficult to remove these clicks completely from the hardware, so this new feature ensures that the output device avoids situations where these clicks might be generated by always playing audio. To accomplish this, if Shairport Sync isn't actually playing anything, audio frames consisting of silence are sent to the output device, keeping it playing. Apart from the initial startup transition, the output device never stops playing and thus never transitions to and from standby mode, avoiding the possibility of generating associated audio disturbances. To enable this feature, a new alsa group setting with the name disable_standby_mode is available. (If you do a full $ sudo make install, a new sample configuration file with this setting in it is installed at /etc/shairport-sync.conf.sample or /usr/local/etc/shairport-sync.conf.sample.) To control this feature, a new alsa-only disable_standby_mode setting can be set to always, auto or never. The always setting is recommended for systems where the output device is dedicated to Shairport Sync. The "Disable Standby" state itself can be set or cleared via the D-Bus interface DisableStandby property.
• A new run_this_if_an_unfixable_error_is_detected (in the sessioncontrol group of settings) program hook is provided. At the moment, two conditions can trigger this. The first is if the watchdog is unable to terminate a play session. The second is if the output device stalls for a long period. Both conditions can be caused by malfunctioning DACs. The external program could, for example, reboot the device.
• Add a configuration option to specify the PulseAudio output sink. An extra option for the pa backend to allow the specification of the audio sink. Thanks to Maciej Wilczyński for his work on this.

Enhancements

• In the alsa backend, new play() and delay() functions minimise the use of snd_pcm_recover() to prevent unnecessary resets of the output DACs.
• In the alsa backend driver, hardware isn't accessed until the first time it is needed. That is, when Shairport Sync starts up, it no longer needs to access the device momentarily. Instead, it waits for the first use.
• The libdaemon library is now an optional build. It is not necessary for systemd systems and can be omitted. Use the --without-libdaemon configuration option to leave it out. Note that when libdaemon is omitted, log messages will be sent to the system log by default. Use the -u command line command to direct log messages to STDERR, which is typically the command line console.
• At present, Shairport Sync adds dither to the audio (a) if the built-in software-based volume control is used, (b) if the audio is mixed to mono or (3) if there is a reduction in sample size, say from 16- to 8-bit. The code for generating dither has been much improved. Due to a poor pseudo-random number arrangement, the dither noise didn't sound like white noise -- now it does. In addition, if dither is to be added, it is also added to the silence inserted just prior to the start of play, and is also added to the silent frames used to prevent the output device from going into standby mode, if selected.
• Use /dev/urandom rather than /dev/random as a source of some kind of randomness for the cryptographic "nonce" used in AirPlay password exchange, as /dev/random blocks occasionally. Please see here for a discussion of the merits of both. The effect of /dev/random's blocking on Shairport Sync was to make the source of randomness somewhat less random. (By the way, you should never use an important password as an AirPlay password for Shairport Sync -- it is stored in Shairport Sync's configuration file in plain text.)
• The code that deals with packet loss resulting from network problems has been completely rewritten. The reason is that, while the old code worked quite well, it could be overwhelmed if the network problems were very serious. The new code is simpler and more robust in testing so far. This code interacts with the code for flushing audio before and after a play session, so it may have introduced regressions.
• The build instructions have been changed to avoid compiler warnings coming from automatically-generated code produced for the D-Bus-based interfaces.
• Warnings are now logged if the ALSA subsystem fails to recover properly after an error has been cleared.
• Extend the range of audio_backend_latency_offset_in_seconds to ± 1.75 seconds. Note that no sanity checking of any kind is done on this – if it is too large the program may simply crash.
• Formatting of the settings file shairport-sync.conf has been fixed, thanks to the work of roblan.
• Adjust the latency calculation to accommodate changes in iOS 12 and AirPlay connections from macOS Mojave. Thanks to artenverho who first reported the issue.
• Many changes to compilation and linking flags. Stop using HAVE_* flags except where necessary, use CONFIG_* for optional stuff, use HAS_* for immediate definitions to be used during configuration, use USE_* for Automake definitions. Probably introduced bugs, sigh.
• Sometimes libsoxr is built to rely on libavutil. With the present changes to configure.ac, if libavutil is present, it will be linked to when linking libsoxr, but if it's not present, no attempt will be made to link to it when linking libsoxr. In addition, add pgk-config support for libsoxr selection. Thanks to Jörg Krause for identifying these issues and proposing fixes.
• Modify the code that synchronises the Shairport Sync system's clock with the source clock to try to take account of the sources's nominal rate, which (oddly, e.g. iTunes on a Mac) might not be exactly 44,100 fps.
• Try to interpolate for the measured drift between the standard three-second timing snapshots. These changes make a very slight difference from time to time, of the order of microseconds, and it's not clear yet how reliable the drift interpolation is.
• Add some extra fields to the statistics output, including nominal source rate, actual input rate, actual output rate, source clock drift, calculated correction needed in ppm. All these numbers agree to a high degree, but the generation of them is fragile and susceptible to errors when there are problems like underrun, and they are not actually running averages, which would be genuinely useful. But they show promise!
• Internal changes are being made for version 3.3 to avoid using SIGUSR1 and pthread_kill to stop threads; the standard pthread_cancel and friends are being used instead. This should lead to more reliable and orderly cancellation of sessions and threads. However, it is quite a complex change, so bugs may have been introduced or reactivated. Lots of testing needed.
• Frame rates! Exact input and output frame rates are now included in the statistics output. The figures generated are averaged over the entire play session since the last pause/resume, if any, so they should settle down and become more accurate over a long play session, say a couple of hours. Timing is relative to CLOCK_MONOTONIC. When your system is connected for an appreciable period to network time, e.g. using an NTP client, CLOCK_MONOTONIC is adjusted ("conditioned") to keep time extremely accurately. The calculated output frame rate should be very accurate. The calculated input frame rate will vary considerably over short intervals due to network conditions, but over a long play session it should also become very accurate. Timing is done from the start of the play session, or from the resumption of play following a pause. Note that while some audio sources pause and resume between tracks, both iTunes on the Mac and the Music app on iOS play all the tracks on a playlist without pause (so long as the tracks are downloaded and present in the device in time).
• Add the command quit to the MPRIS and the D-Bus interfaces. The main motivation for this is that it makes it easier to search for memory leaks.

Bug Fixes

• The mdns-external method used to advertise the Shairport Sync AirPlay service on ZeroConf is now an optional build and is omitted by default. Previously it was included with --with-avahi and could not be included on its own.
• A number of memory leaks associated with the use of OpenSSL have been fixed.
• Use CLOCK_RUNTIME in place of CLOCK_MONOTONIC when doing synchronisation timing using older versions of the alsa library. Necessary for compabibility with OpenWRT Barrier Breaker.
• Always place D-Bus access policy documents in /etc/dbus-1/system.d in Linux, but adhere to the standard for FreeBSD/OpenBSD.

With great help from gibman — see #772 for the gory details — a myriad of issues have been identified and fixed. In particular, gibman shared an automated way of stress-testing Shairport Sync, and this has resulted in the detection of many bugs. Here is a flavour of some of the issues addressed:

• Replace the existing watchdog, which only offered partial coverage, with a much more robust thread-based watchdog.
• Allow a reported remote processing time of zero.
• Fix a logical error that could cause an endless loop during a flush.
• Ensure a player thread is correctly initialised before allowing it to be cancelled cleanly.
• Ensure the player thread always has a cancellation point so that it should always be possible to ask it to terminate.
• If a play session is interrupted, wait for it to terminate for up to three seconds.
• If a play session is interrupted, ensure the new session uses a different set of UDP ports. This is to ensure that data from the interrupted session – which might be still in transit – doesn't enter the new session.
• Make all ALSA command sequences un-cancellable. This is to ensure that ALSA subsystem will not be left in a partially-initialised state if Shairport Sync terminates abruptly.
• When a connection is terminated abruptly, ensure that all the UDP ports is use are closed properly.
• Impose timeouts on both reading and writing to the supervisory RTSP connection governing a session.
• When closing the RTSP connection due to an error, close it immediately, without waiting for a full TCP handshake, because, if the other end has erred, the handshake may never come.
• Fix a parameter initialisation error in a situation where there is no hardware mixer.
• Fix an MQTT-related crash by ignoring unrecognised commands.
• Fix a compilation error and a warning when using the --with-convolution configuration option.
• A compilation problem in OpenBSD has been fixed by changing the order of some include files.
• Fix a problem when calling the program to be run when the volume control is changed. Thanks to shaven for the report.
• Remove unrecognised options from FreeBSD and OpenBSD compilation instructions.
• Ensure the compiler flag HAVE_LIBSOXR is defined if the --with-soxr configuration flag is used and pkg-config is in use.
• Fix a bug in the dbus native interface which would silently switch soxr interpolation to basic.
• Fix a mutex lock bug in the metadata hub. No known effects.
• Fix an arithmetic overflow in frame rate calculations that could occur after 2^32 frames – approximately 27 hours at 44,100 frames per second.
• A number of memory leaks have been identified and removed.
• An incorrect warning about using the deprecated general statistics options has been fixed.