Log files you can actually poll, and settings urls that carry their query

0.16.1 is a follow-up release to 0.16.0. It makes check_logfile usable as a polled
check — it can now report only what is new, or only the newest lines — lets http
settings urls carry query parameters and host name placeholders so one boot.ini
can configure a whole fleet, gets TLS working out of the box on RHEL-family and
SUSE, and lets a scheduled check report immediately after a restart.

Highlights

• check_logfile bookmarks. A bookmark option remembers how far the previous
  check read and resumes from there, so a single ERROR line is reported once
  instead of failing the check for as long as it stays in the file. Rotation and
  truncation are detected by size and content fingerprint. #561
• check_logfile tailing. max-lines=N examines only the newest N lines, and
  newest=last|first says which end of the file those are at. Without a bookmark the
  check seeks to the last N records rather than reading the file, so tailing a
  multi-gigabyte log costs a few kilobytes. #583
• Settings urls send their query. A query string on an http(s):// settings url
  was silently dropped, so a script generating per-host configuration never saw the
  parameters. It is now sent, percent-encoded where the request line requires it, and
  each distinct query caches separately. #460
• Host name placeholders in settings urls. ${hostname}, ${host}, ${domain}
  (and their _lc/_uc variants) expand anywhere in a settings url, which turns one
  boot.ini into fleet-wide configuration.
• TLS works on RHEL and SUSE. ${ca-path} was hardcoded to the Debian bundle on
  every non-Windows platform, so on RHEL-family every TLS check — and nscp enroll,
  which defaults --ca to this token since 0.16.0 — failed before opening a socket.
  The path is now detected per platform at build time.
• run on startup for scheduled checks. A schedule can run its command once as
  soon as the agent is up instead of leaving a stale result behind for a whole
  interval after a reboot. #392
• Settings urls stay out of the log. A token in a settings url was written to the
  log verbatim on every boot, and printed by nscp settings --show. Settings urls are
  now rendered as scheme, host and path everywhere.
• Scheduler reload fixes. Reloading the Scheduler kept the pre-reload tasks
  alongside the new ones, so every schedule ran twice after a reload.

Detailed changes

CheckLogFile — report only what is new (bookmark)

check_logfile read the entire file on every run, so the only way to have a line
reported once was real-time monitoring, which has to be configured on the agent; the
polled path had no equivalent of the bookmarks check_eventlog already had.

check_logfile file=/var/log/app.log "filter=column1 like 'ERROR'" "warning=count > 0" bookmark=app-errors

The value is optional: bookmark, bookmark= and bookmark=auto all derive the name
from the file plus a hash of the filter, warning and critical expressions, so two
checks over one file do not consume each other's lines. Positions live in the module and
are persisted to ${data-path}/nsclient.db on shutdown.

Two costs are documented rather than hidden: a line is consumed when the check runs,
not when its result is submitted (so a failed passive submission does not get a second
chance), and positions are saved on a clean shutdown, so a crash re-reports the backlog
rather than losing it. ${total} counts the lines examined, not the lines in the file.

CheckLogFile — look at only the newest lines (max-lines, newest)

check_logfile file=/var/log/app.log "filter=column1 like 'ERROR'" "warning=count > 0" max-lines=100

Selected lines are always matched in file order, so %(list) reads the way the file
does. The limit bounds the reading, not just the matching. A line-split value which
can overlap itself (aaa, --) cannot be located from the end, so those files fall
back to being read in full with the surplus dropped — same result, more I/O.
max-lines combines with bookmark to cap how much of a burst is reported (the
dropped lines are consumed, not deferred); newest=first is rejected together with a
bookmark, since a file rewritten from the top changes its fingerprint on every write.

Settings — query parameters on http(s) urls

[settings]
1 = http://nsclient.mydom.local/nsclient.php?RootFolder=myhost/&Filename=nsclient.ini

net::parse split the query off into url.query, but settings_http only handed
url.path to the downloader, so the request went out as a bare GET /nsclient.php.
The new net::url::get_request_path() reassembles path and query for the request line,
and proxied requests build their absolute URI from the same path.

Caching followed: the cache file name derived from the path alone, so two entries
pointing at the same script with different parameters overwrote each other. The query
now contributes a short digest (? and & are not legal in a Windows file name), and
a url with no file name (http://host/?file=x) falls back to cached.ini instead of
collapsing onto the cache directory. Since cache_remote_file falls back to the cached
copy when the settings server is unreachable, an existing cache file is migrated to the
new name once on first start — otherwise an agent upgrading while its server was down
would have booted with an empty configuration.

The query is percent-encoded per RFC 3986 before it reaches the wire (a space produced
a malformed three-token request line; a CR or LF split one request into two). An
existing %XX is passed through untouched, and a % introducing no valid pair is
escaped.

Settings — host name placeholders

[settings]
1 = http://cfgsrv/nsclient.php?host=${hostname}

Settings urls now run through socket_helpers::expand_hostname, the same helper the
submit clients (NRDP, Graphite, Syslog, Icinga, …) use for their hostname setting, so
the placeholders mean the same thing wherever they appear.

Each has a _lc and _uc variant. ${hostname} / ${hostname_lc} / ${hostname_uc}
are new — expand_hostname only had ${host} and ${domain}, so a template had no way
to ask for the name as reported. This is additive (the token was previously left in place
as literal text) and every module using expand_hostname picks it up, which is the
intent. Expansion happens before parsing, so a placeholder may sit in the query, the path
or the host, and before percent-encoding, so a host name needing an escape gets one.
Attachment urls and the cache file name use the expanded url, so each host caches its own
configuration.

Settings — keep the url out of the log

The documentation promised that NSClient++ logs settings urls as scheme, host and path
only, but only the two TLS warnings in settings_http went through
to_log_safe_string(). boot() echoed the raw boot.ini entry three times — once at
info level — and get_info(), which nscp settings --show prints, embedded the raw
context. An operator who put a token in a settings url on the strength of that paragraph
got it in the log file on every boot.

url::get_baseurl(), url::get_path() and url::to_log_safe_string() are now used for
the "Activating" / "Failed to activate" / "using that" messages, the boot order list, the
"Undefined settings protocol" exception and get_info(). to_string() keeps the query
and remains the faithful rendering. The docs now also note that only the agent's own
output is covered — not a proxy, and not the settings server's access log.

Networking — detect the platform CA bundle

${ca-path} was one hardcoded path for every non-Windows platform,
/etc/ssl/certs/ca-certificates.crt. On RHEL-family the bundle is
/etc/pki/tls/certs/ca-bundle.crt and on SUSE /etc/ssl/ca-bundle.pem, so the token
named a file that is not there:

Failed to load CA /etc/ssl/certs/ca-certificates.crt: No such file

This was not limited to public hosts — make_context loads the CA whenever ca is
non-empty, before the verify mode is considered, so a check against a local self-signed
server with verify=none failed too. It also took nscp enroll with it, since 0.16.0
defaults --ca to this token, making fleet enrollment impossible on RHEL without naming
a bundle by hand.

The path is now detected at configure time (a package is built in a container of the
distribution it targets, so the build host has the right answer) and reaches
path_manager through config.h like the other path defaults. A packager who knows
better can override it:

cmake -DCONFIG_CA_PATH=/etc/pki/tls/certs/ca-bundle.crt ...

Verified in the CI images: detection resolves to /etc/pki/tls/certs/ca-bundle.crt on
rockylinux:10 and /etc/ssl/certs/ca-certificates.crt on ubuntu:24.04. Pointing
ca-path at the real bundle takes the CheckNet suite on Rocky from 5 failed / 48 passed
to 53 passed.

Scheduler — run a check at startup

A schedule only reported for the first time once its interval (or the next matching cron
time) had elapsed, so after a reboot or a configuration change the monitoring server kept
the old result for as long as the interval — hours for the checks whose status is most
likely to have changed.

[/settings/scheduler/schedules/uptime]
interval = 1h
channel = NSCA
command = check_uptime
run on startup = true

[/settings/scheduler]
startup window = 30s

The startup runs fire from the plugin start hook, which the core calls once every plugin
is loaded — running them during module load would query commands that later modules have
not registered yet. A reload gets no second start hook and is not exposed to that
ordering problem, so it fires them directly. The scheduler learned to register a task
without queueing its first run, so a startup schedule does not end up with two
independent chains of queued instances (which would make it run twice per interval
forever).

Bug fixes

• Scheduler reload left the old tasks behind. The tasks and queued instances from
  before a reload stayed in place alongside the newly added ones, so every schedule ran
  twice after a reload.
• A plugin loaded into a running agent never got its start hook. This also affected
  LUAScript's on-start scripts.
• The schedule copy constructor did not copy its id, so the "Adding scheduled item"
  log line printed a random number.
• bookmark= (explicitly empty) silently disabled bookmarking even though an empty
  value is documented to mean auto. The REST API renders a valueless parameter as a
  bare token so it never hit this, but the client-query path passes k=v verbatim, and
  a check written that way quietly went back to reporting the whole file every run. The
  option now uses a notifier, which tells "given without a value" apart from "not given
  at all".
• Typos cleared across comments, cache doc strings and one log message — including
  the contect parameter of switch_context, renamed in both declaration and
  definition. No behaviour change.

Build and CI

• The HTML docs are no longer in ALL. Only the Windows installer ships the mkdocs
  site, but every build on every platform rendered it, which also made a docs failure
  abort the whole build before the binaries were done. New NSCP_BUILD_DOCS_HTML
  defaults to ON on Windows and OFF elsewhere; the target still exists everywhere
  (cmake --build . --target build_docs_html).
• Linux packages build in parallel. The Build nsclient step was 96–97% of every
  Linux package job (43 minutes on x64, 64 on arm64) and ran a bare make, leaving
  three of four cores idle. The graph is wide — 6600 CPU-seconds over 819 translation
  units, sustaining 1433% CPU at -j16.
• ccache for the Linux jobs. Nothing was cached between runs, so every push
  recompiled all ~820 translation units in each of three Linux jobs. ccache is wired in
  as a CMake compiler launcher, only when it is on PATH, capped at 500M per job, with
  ccache --show-stats printed after every build.
• The integration suite runs on RHEL builds too. The jest harness had been gated to
  deb since it needed a docker daemon; NSCP_SKIP_DOCKER=1 removed that need but the
  condition stayed, so RPM packages were covered by acceptance-tests.sh alone. Turning
  it on immediately found the ${ca-path} bug above.
• New tests cover url parsing and get_request_path round-tripping, the request
  line the loopback settings server actually receives, cache file separation, the
  logfile bookmark state and file reader, the scheduler's register-without-queue path,
  run on startup end to end, and a trust-store assertion in acceptance-tests.sh
  (which runs for both deb and rpm). check_ping against the public internet is opt-in
  behind NSCP_EXTERNAL_ICMP=1, since GitHub-hosted runners drop outbound ICMP.

Upgrade notes

• RHEL/SUSE users can drop workaround ca= arguments. If you worked around the
  hardcoded Debian CA path by naming a bundle explicitly, that still works; ${ca-path}
  now resolves correctly on its own. Packagers cross-building for another distribution
  should set -DCONFIG_CA_PATH=.
• check_logfile behaviour is unchanged unless you opt in. A check without
  bookmark or max-lines reads the whole file exactly as before, and neither reads
  nor advances any stored position.
• Adopting bookmark is a trade. A line is consumed when the check runs, not when
  its result is submitted, and positions are saved on a clean shutdown — so a crash
  re-reports the backlog. Prefer an explicit bookmark name for a check whose filter
  changes often: an automatic name covers the expressions, so editing the filter starts
  a new position.
• Settings urls with a query string now actually send it. If a server was relying on
  receiving the bare path from NSClient++ while the url carried parameters, it will now
  see them. The old cache file is migrated to the new query-aware name once on first
  start, so the offline-boot fallback survives the upgrade.
• ${hostname} in an existing configuration changes meaning. It used to be left in
  place as literal text by expand_hostname and is now expanded — everywhere that
  helper is used, including the submit clients' hostname setting.
• Building the docs on non-Windows now needs an explicit flag:
  -DNSCP_BUILD_DOCS_HTML=ON, or cmake --build . --target build_docs_html.
• run on startup is off by default, so the default install is unaffected. Turning
  it on for the default schedule enables it fleet-wide; use startup window to avoid
  a thundering herd of startup submissions.

Full Changelog: 0.16.0...0.16.1