InterNetNews/inn 2.5.3

INN 2.5.3

on GitHub

2012-06-15
Full changes and diff from previous release

Highlights

• Please note that the HTML_STATUS compile-time option has been replaced with the htmlstatus parameter in inn.conf. If you used HTML_STATUS, you should set htmlstatus accordingly.

• A confusion in the name of a key in innfeed.conf existed in the source code. Make sure that the misspelled, undocumented backlog-limit-high key is not used in your innfeed.conf file; its real name is backlog-limit-highwater. You should rename the key in case it is present in your configuration file. Otherwise, it will not be taken into account. You can run inncheck to verify that the syntax of this file is correct.
  It is generally recommended to run inncheck after any changes done to configuration files, especially with the new improved version of this script shipped with INN 2.5.3, thanks to the hard work of Florian Schlichting who added support for the syntax of incoming.conf, innfeed.conf, readers.conf and storage.conf.

• An up-to-date control.ctl file is provided with this release. You should manually update your control.ctl file with the new information recorded about Usenet hierarchies.

Bug Fixes

• When HDR/XHDR/XPAT were used on a new article coming into a newsgroup, requesting a header field not present in the overview database, the first subsequent OVER/XOVER command did not show that article. A remap of the overview data file was missing in nnrpd. Thanks to Sam Varshavchik for the bug report.

• When a header field appeared more than once in an article, it was missing from the overview data. OVER/XOVER, as well as HDR/XHDR/XPAT using the overview, were therefore returning an empty field. The content of the first occurrence is now returned, in accordance with RFC 3977.
  Perl and Python filters for innd now also properly initialize their header field variables with the first occurrence of header fields. (It is still the last occurrence for the Perl filter for nnrpd.)

• Fixed a possible plaintext command injection during the negotiation of a TLS layer. The vulnerability detailed in CVE-2011-0411 (and CVE-2012-3523, specifically for INN) affects the STARTTLS and AUTHINFO SASL commands. nnrpd now resets its read buffer upon a successful negotiation of a TLS layer. It prevents malicious commands, sent unencrypted, from being executed in the new encrypted state of the session.

• Fixed a regression that occurred in INN 2.5.0 when leading whitespace characters have been made significant in header field bodies. It could lead INN to drop articles and throttle itself when running as a slave because Xref header fields generated by other news servers, or even INN 2.4.6, could contain (valid) leading whitespace. Thanks to Matija Nalis for having caught this bug.

• Fixed an invalid 431 response to CHECK commands when innd is paused: the Message-ID of the article to defer was missing. Also fixed another issue in the messages innd replied; when an error occurred during a write on a channel, a trailing extra junk byte was added to the reply. Thanks to River Tarnell for these bug reports.

• cnfsheadconf now properly recognizes continuation lines in cycbuff.conf, that is to say lines ending with a backslash (\). Thanks to John F. Morse for the bug report.

• The order of CNFS buffers in a metacycbuff is now properly read and written by cnfsheadconf. There previously was a confusion between hexadecimal and decimal values. Thanks again to John F. Morse.

• When the -l flag is given to cnfsstat, the cycbuff.conf and storage.conf files are now reloaded if they have been modified since the previous output of cnfsstat.

• Fixed an issue at configure time that made INN wrongly assume that OpenBSD (4.6) didn't support Unix-domain sockets. Thanks to Wim Lewis for the patch.

• Fixed an issue on systems which do not have a working flock(2) function (Solaris, for instance). mailpost and pullnews are reported not to be usable on such systems. Many thanks to Dennis Davis for the bug report.
  A wrapper around shlock is now called in Perl scripts. The INN::Utils::Shlock module has been added for that use.

• Fixed an issue in the Python access hook for nnrpd: it has not been working since Python 2.5 on 64-bit platforms, owing to a change to Python's C API, using a new Py_ssize_t type definition instead of int. Thanks to Raphael Barrois for the patch.

• Improve the stability of the Perl filters for innd and nnrpd: properly save and restore the stack pointer when needed.

• controlchan now imposes a date cutoff on processing control articles. The artcutoff parameter set in inn.conf is used. Otherwise, without that cutoff, old control articles could be maliciously reinjected into Usenet, and replayed. (An unsigned Injection-Date header field could be added to an article that only had a Date header field.) A new -c flag has been added to controlchan to disable the cutoff check, if needed (usually when manually invoking the program).

• nnrpd no longer adds or updates the Path header field when an article is forwarded to a moderator. It could otherwise lead to rejects at injection time when the article was approved by the moderator.

• The X-Trace header field was not properly generated when an article was locally posted. The field mentioning the IP address was skipped, resulting in a wrong syntax for this header field. The local 127.0.0.1 IP address is now used. Besides, localhost is now mentioned instead of an obscure stdin in injection header fields.

• Fixed a bug in the frequency innfeed logs its status: too many useless lines were written to news.notice. Thanks to Florian Schlichting for the fix.

• When unset in innfeed.conf, the dynamic-method parameter now properly defaults to 3 (instead of 0) and use-mmap to false (instead of true). These two values were already the recommended ones in the documentation and the sample file. Note that use-mmap is only used when innfeed is given file names to send instead of storage API tokens, which is a fairly rare use case.

• innfeed no longer generates an error message (logged in news.err) when a parameter is not defined in innfeed.conf. All the parameters have a default value, so there is no need to warn the user if they are not present in innfeed.conf. Thanks to Dieter Stussy for having reported this problem.

• Implement an upper limit to the number of file descriptors innd can handle. At most (FD_SETSIZE-1) file descriptors can be used. This upper limit now overrides any superior number set with rlimitnofile in inn.conf. Thanks to Steve Crook for the bug report.

• A default timeout on outgoing sockets (using NNTPconnect) has been added by Florian Schlichting. For a long time, there have been occasional problems with actsync (and probably other programs) that would hang until manually killed or restarted.

• inncheck now properly finds the boundaries of substituted variables in newsfeeds thanks to Alexander Bartolich.

• docheckgroups no longer uses AWK. On a few systems, the script was failing because of the presence of an old version of AWK that has a limit in the size of the input it can handle. Processing large newsgroups files was consequently impossible. docheckgroups now uses Perl instead of AWK, which solves the issue reported by John F. Morse.

• pullnews no longer stops processing newsgroups when an error occurs during its run (for instance when a newsgroup mentioned in the configuration file is removed from an upstream server). Besides, it can now use authentication when posting to the downstream server.
  A few other minor bugs have been fixed as for the way pullnews counts the articles.

• Fixed the way innreport handles leap years. It now properly generates HTML reports; dates were assumed to be relative to the current year, which may break their computation during for instance the whole 2012 leap year. Please note that no HTML reports have been lost, and that they will appear when INN is updated to this new version.

• Other minor bug fixes and documentation improvements. In particular, the debug-shrinking, fast-exit and initial-sleep keys in innfeed.conf are now documented. The function filter_end(), called when Perl filtering is turned off, is also documented for the innd and nnrpd Perl filters.

New Features

• It is now possible to properly generate daily statistics with sendinpaths thanks to the new -k and -r flags that permit controlling the interval of days for processing dump files. The new -c flag permits sending a copy of the generated e-mail to the newsmaster.
  Also fixed an issue with statistics that could be missing or duplicated for a couple of days when monthly sent.
  The documentation has been updated and mentions a preferred daily run of sendinpaths. This script is a complete rewrite in Perl, and is based on Mohan Kokal's initial work.

• A single header field line is limited to 998 bytes, per RFC 5536. innd was previously accepting, and also generating Xref header field lines, up to 1022 bytes. Now, nnrpd (acting as an injecting agent) rejects articles which contain header field lines whose length exceeds 998 bytes. And innd (acting as a relaying or serving agent) no longer checks that.

• nnrpd advertises the COUNTS, DISTRIBUTIONS, MODERATORS, MOTD and SUBSCRIPTIONS variants of the LIST command in response to CAPABILITIES. These commands already existed in nnrpd but RFC 6048 had not yet been published.

• Add support for LIST MOTD in innd. Consequently, the motd.news configuration file which was previously used only by nnrpd is renamed to motd.nnrpd (innupgrade takes care of the rename). innd uses the new motd.innd file in pathetc for its message of the day.

• The Injection-Date header field, when present, is now used by innd and makehistory to determine the posting date of an article. Otherwise, the Date header field is used.

• The flag -S has been added to innd by Florian Schlichting. When used, innd reports the errors found in incoming.conf and exits.

• A new parameter has been added to inn.conf to determine whether the status file that innd can write out (depending on the value of the status parameter) is plain text or wrapped in HTML. It previously only was a compile-time option, set to true by default. Florian Schlichting added the htmlstatus parameter to provide a configurable behaviour.

• It is now possible to run a script at the end of the execution of innshellvars scripts. If a file named innshellvars.local, innshellvars.pl.local or innshellvars.tcl.local is present and executable in pathetc, then it will be executed by the corresponding innshellvars script (respectively shell, INN::Config Perl module, and Tcl). A typical use is to add or override variables.

• Add support for wire-formatted articles in scanspool.

• A lot of work on cleaning old perl4-style code has been done by Florian Schlichting.

• inncheck now generates a proper non-zero exit value when errors are found, and allows quiet mode with the -q flag. Florian Schlichting has greatly improved this script in many regards, especially with a config-syntax parser for incoming.conf, innfeed.conf, readers.conf and storage.conf.