RTPProxy v2.1.0 Release Notes
June 2019
Version 2.1.0 of the RTPProxy brings 3+ years of improvements and fixes into our 2.0 version. The goal of this release has been to stabilize the new code and subsystems added in 2.0.
We heavily focused on developing test framework that is based on high-level functional testing and fixed the issues as we go. Basic testing is done in the RTPProxy repository itself.
For more advanced test scenarios that use external components, an additional repository has been created, Sippy/VoIPTests. The code in that repository is building on our python SIP stack and brings together many-many different versions of Kamailio, OpenSIPS, RTPProxy, Sipppy B2BUA (Python) and our recent addition Sippy B2BUA (Go) to tests that they can perform basic functions that an user might want them to perform. So if we (god forbid!) break our ability to work with any of those software by making any changes to our code we are instantly notified. And in reverse, if any of those nice folks break (never happens!:) whatever we think of a basic level of the support for the RTPProxy, we are aware of it and do our best to bring the issue to the light.
Notable Changes
Improved session latching
Make session latching more intelligent. Instead of disallowing
another latch within certain interval after previous one
(UPDATE_WINDOW), allow another update any time if new packet
meets criteria:
- It's valid RTP packet (as far as
rtp_packet_parse()can tell); - It has the same SSRC value that has been saved during the
previous latching; - Its sequence number is greater than the sequence number that
we saved while doing the previous latching.
If any of (1)-(3) fails, then we revert to the old way with
disallowing new update within UPDATE_WINDOW after the previous
one.
This fixes the issue when endpoint generates 3 packets from
the proper IP, but then switches to completely different port
and sticks with that for the rest of the session. Right now,
the rtpproxy would latch to the first packet out of those 3 and
discard the rest considering those a garbage. We are not 100%
sure if it's endpoint's fault or just some NAT stupidity.
Printing of SSRC and initial sequence number is done while latching. It
is helpful to identify session in the log and could also be useful in some
other cases.
Count ignored packets
rtpproxy now counts the number of ignored packets per session. This counter
represents the number of packets that were coming from unrecognized IP/Ports.
New U/L command modifier n to allocate in RTP/RTCP ports
The new n option to the RTPP U/L commands that force a new pair of RTP/RTCP
ports to be allocated and the old ones released. This is useful when a call
route-advances or fails over to a new destination, where the failed route may
still be sending superfluous traffic to the previous ports.
RTPP Q command takes arguments of what counters to return
Extend the Q (session stats query) command to take optional list of specific
counters to pull. This allows fetching only those stats that are relevant and
makes Q quite usable for the test suite runs.
Example: Q call-id from_tag to_tag rtpa_nsent rtpa_nrcvd rtpa_ndups rtpa_nlost
New stats counters to track overall packet loss
rtpa_nsent rtpa_nrcvd and rtpa_ndups counters allow to monitor overall
packet loss in all streams going to the rtpproxy as function of time.
SSRC packet tracking
See commit for WIP: b1ac79e
Python rtpp client
python/rtpp_query.py is a simple client that allows the user interactive rtpp
session. Also useful for injecting commands via stdin or by files.
Notification socket wildcard
A notification wild card %%CC_SELF%% can be used to automatically register
notifications sockets for callers of the U and L commands.
The SIGHUP signal causes rtpproxy to shutdown gracefully
Use SIGHUP instead of SIGTERM to shut down the proxy. The difference is that
SIGTERM tries to tear down everything from the signal handler, while SIGHUP
actually makes main() to exit gracefully so that there is no racing condition
between main() posting event into the rtpp_proc queue and the rtpp_proc
actually exiting.
Improved test coverage
The rtpproxy test suite has grown, and covers all common use cases.
- repacketization
New pertools/udp_contention tool to simulate UDP traffic.
U and L command option z to set ptime
Save ptime value if provided by the z options for the U/L command into
session parameters and use it to request specific packet size when P (playback)
command is received with "session" as a codec type. Extend playback
(rtp_server) API to be able generating frames with ptime other than absolute
minimum supported by a codec.
rtpp_streamdb module
Add rtpp_streamdb, a new module to serve as a central in-memory DB for the
RTP/RTCP streams. Instead of using call-id as an indexing/lookup parameter or
passing around a pointer to the structure, assign unique (within
rtpp_streamdb instance) 64-bit integer for each session and provide a way to
use that integer to quickly pull the session in question. Also integrate
ref-counting and require it in order to protect a session from being released
while it's being used by some part of the code. This allows us to untangle some
global locking between command and RTP forwarding threads.
The 64-bit id will be used as a "weak reference" in critical path of the code,
i.e. mapping from the file descriptor into the session structure while doing
I/O, so that some performance assessment has been done to make sure lookup
performance is adequate for the task. Quick assessment shows that we can do
some 6,000,000 lookups/sec on a single thread on E3-1220 V2 @ 3.10GHz on set of
4,000 sessions with initial code. This should be enough for the current
architecture and there are several relatively easy ways to reduce the overhead
at least 2-3 times further by increasing the size of the hash table and/or
in-lining/optimizing hashing code. We expect current code doing some 70-100k
lookups/second at 1.5-2k sessions top.
Re-factor session handling (internal changes)
Refactor mostly related with the fact that struct rtpp_session is now struct
rtpp_session_obj and that the sp->stream[N] is a pointer, not
a sub-structure. Also deals with some API changes in the weakref code and
elsewhere.
Jitter Tracking
RTPProxy now tracks jitter metrics as per Appendix A.8 of rfc3550
Dynamic Loadable Modules
Infrastructure to support dynamic loadable modules has been implemented, and it
is used for the new Accounting CSV module.
Accounting CSV Module
An accounting module that will output stats for each ended call to a csv file.
The Accounting CSV module will output a CSV string of user specified (at start
time) metrics. At the end of each session, this module will append a new live
to the CSV file. Operators typically will configure a log collection agent to
further process the metrics.
TODO: Provide flag examples
IPv6 Improvements and fixes
Fixes and improvements plus integration tests added to our test suite.
TODO: MAX, can you flesh this paragraph out a bit more?
MAX: Yes, sure, fixed lot of bugs and made overall improvement and code unification.
Thank you to our contributors!