InterNetNews/inn 2.5.2

INN 2.5.2

on GitHub

2010-03-24
Full changes and diff from previous release

Highlights

• The way checkpoints are handled by innreport for innd and innfeed has totally changed to provide more accurate daily statistics. The first Usenet report after an upgrade to INN 2.5.2 will probably contain incorrect statistics for incoming and outgoing articles because the beginning of the log files that will be used was generated by a previous version of INN.

• A new version of innreport.conf is shipped with INN 2.5.2 but, in order to preserve any local changes, will not be automatically installed with make update. The changes are minor and not mandatory for the upgrade.

• Julien Elie has implemented in innd the new version of the NNTP protocol described in RFC 3977, RFC 4643 and RFC 4644, and innd now recognizes the CAPABILITIES command. Despite these standards, three commands (IHAVE, CHECK and TAKETHIS) will continue, for interoperability reasons, to return a reject code (respectively 435, 438, and 439) when the command contains a syntax error instead of 501. The mandatory username argument for authenticated peers is not enforced in INN 2.5.2 but will be enforced by INN 2.6.0 when it is released.

Bug Fixes

• The keyword generation code now generates a Keywords header field only if the original article does not already have one. The generated Keywords header field no longer begins with a comma. If keyword generation is set to true in inn.conf but the Keywords header field is not stored in the overview, the news administrator is warned and keyword generation deactivated, since it exists only to populate the overview data.

• Two segfaults in keyword generation were fixed. The first occurred when an article already had a Keywords header field longer than the keylimit parameter. The second was caused by a possible invalid pointer beyond the newly allocated Keywords header field.

• Fixed innd handling of empty lines. innd was not properly discarding an empty command and was closing the connection when it received only whitespace in a command.

• Fixed a bug in how innd responded to reader commands when readers were not allowed. A superfluous blank line was sent in its response.

• Fixed a bug in innd's response to TAKETHIS when authentication is required. Previously, 480 code was returned immediately without accepting the multi-line data block first, which broke synchronization in the NNTP protocol.

• Fixed a bug in recognizing the article terminator when empty articles were fed to innd via IHAVE or TAKETHIS, leading to treating subsequent NNTP commands as part of the article.

• When innd could not provide information for LIST ACTIVE.TIMES and LIST NEWSGROUPS, it was returning an invalid error message without a response code. The proper 503 answer code is now returned.

• When an unauthenticated user tried to post an article, nnrpd replied 440 (posting not allowed) instead of the correct 480 (authentication required) response if the user might be able to post after authentication. Thanks to Daniel Weber for the bug report.

• Fixed a bug in both innd and nnrpd answers to LIST commands where the output was not checked for valid dot stuffing.

• Fixed a bug leading to junked non-control articles being sent to control-only feeds, and also fixed handling of poisoned control groups. Thanks to Andrew Gierth for the patch.

• Fixed a bug in innreport leading to incorrect summing of innd stats when hostname was set to an IPv6 address instead of a fully qualified domain name. Thanks to Petr Novopashenniy for the bug report.

• Changed how innreport uses innd and innfeed checkpoint messages. Previously, connections held open for multiple days led to skewed and incorrect statistics on how many articles had been received or sent. The count is now more accurate and, for each connection of a feed, only depends on incominglogfrequency in inn.conf and stats-period in innfeed.conf.

• Fixed a bug in nnrpd Perl filter: a header field whose name begins with the name of a standardized header field was not properly handled.

• Fixed a bug in how innd was parsing Message-ID and Supersedes header field bodies which contained trailing whitespace. The article was corrupted by an unexpected \r in the middle of the header field body. nnrpd now checks the syntax of the Message-ID header field, if present.

• Fixed various bugs in how leading whitespace was treated in header fields. The HDR, XHDR and XPAT commands were not properly showing leading whitespace in header field bodies. Lone \n and \r characters are now changed into spaces and \r\n is just removed. archive, makehistory, and tdx-util now keep leading whitespace in header field bodies when generating overview data, and archive now changes \n (when not preceded by \r) into a space when generating overview data.

• Fixed a bug in the generation of overview data which may corrupt previously generated overview data when a pseudo Xref header field is injected in an extra overview field.

• Fixed a bug in the parsing of the ovgrouppat wildmat in inn.conf that prevented overview data from being generated when poisoned groups were specified but a latter sub-pattern matched the group. A uwildmat expression is now correctly handled, and a potential segfault has been fixed. Thanks to Dieter Stussy for the bug report.

• Fixed a bug when HDR, XHDR and XPAT were used when virtualhost was set to true in readers.conf. The Xref header field of articles posted to only one newsgroup appeared empty.

• HDR and OVER commands now return the correct 423 code (instead of 420) when the current article number is used but the article no longer exists.

• Fixed a bug in tdx-util in parsing empty overview fields when called with -A or -F.

• Fixed a bug in cvtbatch, which was returning only the size of the headers of an article when the b parameter was used with the -w flag. It now correctly returns the size of the whole article, which is what b was documented to do. cvtbatch also has a new t parameter, which can be used with the -w flag to retrieve the arrival time of an article.

• Fixed a bug in how mailpost handles cross-posting feature. It was not properly detaching from sendmail. Thanks to Harald Dunkel for the patch.

• Fixed a bug in the newsfeeds C flag: the count of followup groups was one less than the real number. When the value of the Followup-To header field is poster, it is no longer considered to be a followup. Thanks to Dieter Stussy for the patch.

• When using tradindexed, the overview data for a cancelled article is now immediately removed from the overview. Thanks to Lars Magne Ingebrigtsen for the patch.

• batcher has not supported the retrieval of an article with its file name for a long time. The -S flag has therefore been removed.

• inews no longer rejects articles that contain more than 50 header fields. Thanks to Torsten Jerzembeck for the bug report.

• news.daily no longer sends superfluous mails when the nomail keyword is given. Mail is only sent when there is real output. Previously, there would always be headings and empty lines left over from the structuring of the full report, which are now omitted. Also, the output of programs executed with postexec is now included in the regular mail. Thanks to Florian Schlichting for the patch.

• innconfval no longer maps NULL string or list values to an empty string or list and instead maps them to undefined values. This fixes an issue reported by Kamil Jonca: nnrpd was inserting an empty Organization header field when the organization parameter in inn.conf was unset.

• Other minor bug fixes and documentation improvements.

New Features

• innd now has a decent parser for NNTP commands. The parser is more correct (commands like IHAVE<mid>, without a space between the command and its argument, are no longer valid) and allows leading and trailing whitespaces in commands. innd also now checks the length of the NNTP command sent by the client. If the command contains more than 512 bytes (or 497 bytes for an argument), an error is returned and the command is discarded. After ten unrecognized commands, innd closes the connection with the appropriate code (400 instead of 500).

• The output of the HELP command specifies the arguments expected by NNTP commands, similar to nnrpd's HELP command.

• LIST ACTIVE, LIST ACTIVE.TIMES and LIST NEWSGROUPS now allow an optional wildmat argument to restrict the results of those commands to specific newsgroups.

• LIST SUBSCRIPTIONS now accepts an optional wildmat argument to restrict the results of this command to specific newsgroups.

• When using HEAD or STAT with an article number or a range, 412 (no group selected) is now returned instead of 501 (syntax error).

• Jeffrey M. Vinocur has implemented support in both innd and nnrpd for whitespace in usernames/passwords provided with AUTHINFO USER/PASS. They were previously treated as invalid arguments or incorrectly parsed. innd and nnrpd now treat everything after the first whitespace character following AUTHINFO USER/PASS, up to, but not including, the final CRLF, as the username/password, in conformity with RFC 4643.

• The syntax of Message-IDs is now based on RFC 5536 (USEFOR) instead of RFC 1036. The major change is that quoted-pairs have been removed from the syntax.

• The Perl and Python filters for innd now check the Message-ID of articles arriving through TAKETHIS. Only CHECK and IHAVE commands previously used them.

• Case-insensitive matches are now used for distributions, path identities, IMAP commands, header field names, and control commands. (Newsgroups are still matched case-sensitively.) Message-IDs are case-sensitively matched, except for history hashes.

• The new Archive, Archive-At, Comments, and Summary header fields defined in RFC 5064 and RFC 5536 can be used in innd filters. nnrpd now checks at injection time that an article does not contain an Injection-Info header field, that an Injection-Date header field (if provided) is valid, and that the Path header field body does not contain .POSTED. Note that INN does not yet generate these two injection fields or include the new .POSTED keyword in Path header fields. These new features will be in the next major release of INN.

• nnrpd now supports a new LIST variant named COUNTS. LIST COUNTS is a combination of LIST ACTIVE and GROUP. It returns the same result as LIST ACTIVE except that the number of articles in a newsgroup is inserted before its status.

• A new flag has been added to newsfeeds entries: Aj, when present, says to feed articles accepted and filed in junk (due to wanttrash) to peers based on their newsfeeds feed patterns applied to the Newsgroups header field as though the article were accepted and all those groups were locally carried. This is useful if you want to run INN with a minimal active file and propagate all posts. Thanks to Andrew Gierth for the patch.

• A new parameter has been added to inn.conf: logtrash defines whether a line for articles posted to groups not locally carried by the news server should be added in the news log file to report unwanted newsgroups. The default is true but it can be useful to set it to false (especially when wanttrash is also used).

• The procbatchdir keyword has been added to news.daily to specify the backlog directory of innfeed. This is useful when several instances of innfeed are running or when its configuration file is not the default one.

• sm now supports a new flag, -c, which shows a decoded form of the storage API token. This was previously done by the contrib showtoken script developed by Olaf Titz and Marco d'Itri.

• The O flag in newsfeeds now relies on the contents of the Injection-Info header field if it is present to determine the origin of an article. It falls back on X-Trace if there is no Injection-Info header field.

• A new "unsigned long" type bas been added to the configuration parser. It will properly warn the news administrator when a variable supposed to be positive contains a negative integer. It will prevent INN from crashing due to misconfiguration at several places where it did not expect negative values.

• innxbatch and innxmit now recognize the new 403 code introduced by RFC 3977 for a problem preventing the requested action from being taken.

• actsync, inews, innxbatch, innxmit, nntpget and rnews can now authenticate to news servers which only expect a username, without password, conforming to RFC 4643.