python-caldav/caldav v3.5.0a0

v3.5.0a0 (without niquests)

on GitHub

caldav v3.5.0a0 - v3.5.0 without the niquests dependency

Yet another release primarily focusing on bugfixes and compatibility. iCloud has now been tested and got its compatibility profile in compatibility_hints.py. Test runs towards GMX, Posteo and Purelymail have been skipped for the last few releases; with 3.5.0 the tests pass towards those. More work on getting synchronization right, and on avoiding overwriting changes when multiple clients update the same object.

Also released without the niquests dependency as 3.5.0a0.

Security

• A small DoS-risk found. An attacker controlling the icalendar data (a hostile server, a shared calendar, calendar invites, etc.) flowing through the library can aggressively burn CPU-cycles. vcal.fix(), run on every iCalendar object fetched from a server, took quadratic time on a long run of blanks or backslashes - a 100 kB icalendar fragment was measured to take 20 s. The regexp has been fixed up and now takes linear time to run; the output is unchanged. Reported by @BrianWillows. GHSA-5h4q-xm57-3mh5 Also fixed in 3.4.1 and 3.3.2.

Potentially breaking changes

These are bug fixes, but code relying on the old behaviour may notice them.

• get_calendars() raises NotFoundError or PropfindError when the server answers the PROPFIND for the calendar home or the calendar list with an error. Earlier it would return an empty list. Transient errors may happen with any server. The empty list could not be told apart from an account without calendars, and a Home Assistant user was reporting that their calendars would sometimes "disappear". Code that relied on getting an empty list on such errors must now catch the exception; 2.x raised as well. NotFoundError is raised for a 404, PropfindError otherwise, and the same goes for CalendarSet.get_calendars(). #741 #742
• A time-range search (also date_search()) filters the result client-side by default and drops objects the server returned outside the range. This affects every server, for users not in UTC: CalDAV servers compare all-day events and floating times as UTC, so a search in another time zone missed all-day events at the edges of the range and returned some from the neighbour days (reported on Baïkal). The server is now asked for one extra day on each side (also for alarm searches and server-side expansion), and the exact cut is made client-side: with a timezone-aware start/end, all-day events and floating times are matched in the time zone of the search, not in that of the Python process. A search may therefore return fewer objects than before, and a plain date as start/end now means a day in UTC, also for matching all-day events (previously, client-side, a day in the time zone of the Python process). Pass post_filter=False or compatibility_workarounds=False to send the range as given and get what the server returned. #545 #758
• A save() on an object loaded through search() or multiget() sends If-Match (or If-Schedule-Tag-Match) and raises ETagMismatchError (or ScheduleTagMismatchError) if someone else changed the object since it was loaded. This affects every server that returns ETags, i.e. nearly all. Previously no ETag was requested in searches, so such a save silently overwrote the other client's change. Code that saves search results must now be prepared to reload the object and retry. (Servers sending only weak ETags, like GMX, still get an unconditional save.) #751
• Sync: a 401/403 on the sync-collection REPORT naming a precondition other than DAV:valid-sync-token - or naming none, on a server configured with full sync-token support (not with no features configured, or fragile) - is raised instead of silently falling back to a full fetch. An expired sync token (RFC 6578 section 3.2) could not be told apart from a permission error, so a real permission problem was hidden behind an expensive full fetch. An expired token (also Xandikos' 412) and an unsupported sync-collection report (Zimbra) still lead to the fallback. #738 #756 #748 #759
• FeatureSet.is_supported() gave inverted answers for server observations (e.g. get-current-user-principal.has-calendar) and client features (e.g. rate-limit) when asked for a bool. This concerns code querying the compatibility hints directly, not any particular server; code that compensated for the inverted answers will now be wrong. Asking for a server-peculiarity, tests-behaviour or client-hints feature as a bool now raises TypeError, as no meaningful bool exists for those. #732
• icalendar-searcher 1.1.0 or later is required, for the negated searches and the time-zone-correct cut of time-range searches. This matters to packagers and to environments pinning an older version. #750

Added

Library API

• SynchronizableCalendarObjectCollection.deleted_urls: the URLs of the members a sync-collection report listed as removed (RFC 6578 section 3.2). They are still among the returned objects, as before, but a caller no longer has to guess from a missing ETag, and sync() no longer fetches each removed object to find out. The set is empty after the full-retrieval fallback, which reports no removals. PyCalendar/calendaring-sync#19 (comment)
• delete(if_match=...) on events, todos and journals: with if_match=True the DELETE carries the same precondition save() sends (If-Schedule-Tag-Match if a Schedule-Tag is cached, else If-Match with the cached ETag); a string is sent as If-Match. A 412 raises ScheduleTagMismatchError / ETagMismatchError, and a 404 raises NotFoundError instead of counting as success. Plain delete() is unchanged. DAVClient.delete() takes a headers argument, like AsyncDAVClient.delete() already did. #740 #747
• save(precondition="etag") and delete(if_match=True, precondition="etag") prefer If-Match with the cached ETag over a cached Schedule-Tag, so a 412 raises ETagMismatchError when anything changed, also an attendee's reply. Under If-Schedule-Tag-Match, which the default "auto" still prefers, the server merges such a reply into the write (RFC 6638 section 3.2.10), and a sync client was never told its copy was stale. With no strong ETag cached, "etag" falls back to the Schedule-Tag. #762 #764
• Negated searches: add_property_filter() passes the != (alias <>), !contains and def operators of icalendar-searcher 1.1.0 on to the search. They are decided client-side (unless post_filter=False): the server query leaves them out, as no server has been probed for negate-condition support yet. build_search_xml_query() renders !contains as a negated text-match, and != and def as a bare prop-filter - a negated text-match is a negated substring match, so it cannot express !=. #568 #750

Server compatibility

• A features="icloud" profile, from a caldav-server-tester run by a user - the documentation already referred to it, but it did not exist. #730 #733
• Compatibility feature sync-token.invalid-token: whether a sync token the server never issued is refused with DAV:valid-sync-token as RFC 6578 section 3.2 says. Xandikos and CCS answer some such tokens with a 500, and Stalwart, Ox and Purelymail with a report and no error. jelmer/xandikos#743 #752
• Compatibility features sync-token.limit and sync-token.limit.initial-sync, for DAV:limit and the checkpoint tokens of a truncated sync-collection report (RFC 6578 sections 3.6 and 3.7). Hints for Stalwart, Xandikos, Radicale, Baïkal, Davis, Nextcloud, CCS and Infomaniak. #754 #770
• Compatibility feature propfind.principal-depth-1: whether a Depth: 1 PROPFIND on the principal URL works. GMX answers it with a 400. #755
• Compatibility feature save-load.event.no-dtend: whether a VEVENT with neither DTEND nor DURATION is stored, as RFC 5545 section 3.6.1 allows. iCloud refuses one with a 404. python-caldav/caldav-server-tester#24 #772
• Compatibility feature search-cache takes a filtered-only key, saying only searches with a time range or a property filter come from the lagging cache while a plain listing is up to date at once. The library itself ignores it; the integration tests and the caldav-server-tester wait less where it is set. Set for Purelymail. #775

Fixed

Calendar discovery

• On a server that advertises no calendar-home-set, Principal.calendar(cal_id=...), Principal.make_calendar() and their async twins resolved against the client URL, while get_calendars() looks below the principal URL. They now use the principal URL as the calendar home too. #749 #755
• get_calendars() returned an empty list when the client URL points directly at a calendar on a server that advertises no calendar-home-set and keeps no calendars below the principal, such as a Nextcloud public calendar share (/remote.php/dav/public-calendars/<token>, principal /principals/system/public/). Without a calendar-home-set, the principal URL is still tried first (GMX); if it holds no calendars, the client URL is tried as well, as in 2.x. An error or a 401/403 on the client URL then means no calendars rather than an exception. Regression since 3.0. #744
• iCloud: searches could fail with ValueError: ... can't be joined with ... once the client had moved to the account's pNN-caldav.icloud.com host, on calendars found before the move. Also, a configured url.encode-at.* feature was lost at the move. #730 #733

Searching and time ranges

• A naive datetime.min or datetime.max (e.g. as the open end of a search) raised OverflowError in time zones behind UTC (e.g. the Americas). It now converts, and a time that UTC cannot hold is clamped to the earliest or latest UTC time. A timezone-aware datetime within a day of datetime.min or datetime.max is now converted from its own time zone rather than from the local one. #758
• On Linux with Python 3.13 and older, a time-range search starting before the year 1000 (e.g. at datetime.min) sent an invalid date, such as 10101T000000Z for year 1: the year was not zero-padded to four digits. #758

Synchronization

• SynchronizableCalendarObjectCollection.sync() no longer returns objects the collection never held as deleted. Nextcloud reports a trashed object as removed when it purges it, e.g. because an object with the same UID is added again. #767
• Calendar.get_objects_by_sync_token() (and so objects() and SynchronizableCalendarObjectCollection.sync()) threw away a truncated sync-collection reply - RFC 6578 section 3.6, a 507 response for the collection itself - and silently fell back to fetching the whole calendar. It now keeps the members and the new token, and repeats the REPORT with that token until the reply is complete, at most Calendar.sync_max_pages (100) times. If the server stops early or keeps truncating, the returned collection has truncated set. So does a truncated initial sync on a server whose checkpoint token would skip the remaining objects (sync-token.limit.initial-sync broken, Stalwart: it truncates every report at its maxResults setting, default 2000). SyncCollectionResult.truncated and DAVResponse.sync_truncated report a truncated page to callers of the lower-level API. #737 #745
• AsyncDAVClient.sync_collection() raised AttributeError on every reply with a body, as it assigned to the read-only DAVResponse.sync_token. #745

ETags and concurrent updates

• A cached ETag or Schedule-Tag was kept after a save() whose response carried none - common, since RFC 4791 section 5.3.4 forbids a strong ETag when the server altered the data - so the next save() sent a stale If-Match and raised ETagMismatchError. The stale tag is now dropped. #751
• A weak ETag (W/"...") was sent as If-Match, which can never match since If-Match compares strongly (RFC 9110 section 13.1.1); every save() after an add_event() raised ETagMismatchError on servers answering a PUT with a weak ETag (seen on GMX). A weak ETag is no longer sent, so such an update goes out unconditionally, and delete(if_match=True) with only a weak ETag cached raises ValueError. #751
• An ETag the server sent unasked in a search REPORT reply (seen on GMX) was cached as an lxml element instead of a string, so obj.etag returned an element, and lxml raised a FutureWarning when it was checked. #751
• A save() refused because another object in the calendar holds the UID (CALDAV:no-uid-conflict, RFC 4791 section 5.3.2.1, or CALDAV:unique-scheduling-object-resource, RFC 6638 section 3.2.4.1) now raises the new UidConflictError, a PutError. Previously a 412 (Xandikos, Stalwart) on a save carrying If-Match raised ETagMismatchError, so "the UID is taken" looked like "someone changed the object"; and a 409 (Radicale) was retried once. Where earlier versions raised ETagMismatchError, ScheduleTagMismatchError or AuthorizationError (a 403 from Cyrus), the UidConflictError is a subclass of that as well, so existing handlers keep working. #756 #759

Server responses and error reporting

• An error response lost its body, so a 403 meaning "your sync token has expired" (RFC 6578 section 3.2) could not be told apart from a permission error, nor a 412 meaning the same (Xandikos) from any other failed REPORT. Every DAVError (AuthorizationError, ReportError, PutError, ETagMismatchError, ...) raised for a failed response now carries body and preconditions, the latter listing the elements of a DAV:error body, e.g. ["{DAV:}valid-sync-token"], whatever the status. For the effect on sync, see "Potentially breaking changes" above. #738 #756 #748 #759
• The async client raised an XML parse error instead of AuthorizationError for a 401/403 whose body claimed to be XML but was not (e.g. an HTML error page sent as application/xml). It now handles 401/403 like the sync client. #748

Calendar objects, tasks and invites

• SEQUENCE handling on the save paths - #739 #746:
  • add_object(), add_event(), add_todo(), add_journal() (and the deprecated save_*() aliases) took increase_seqno as an iCalendar property: it did not reach save(), so SEQUENCE was bumped anyway, and the bogus property was written to the object. It is now passed through, and save_with_invites() accepts it too.
  • Saving a single modified recurrence bumped the master's SEQUENCE rather than the override's. The override is now bumped (an override without SEQUENCE gets the master's plus one) and the master left alone (with all_recurrences=True the master is still bumped).
  • Accepting, declining or tentatively accepting an invite bumped the SEQUENCE on the attendee's copy. Only the organizer revises an event; replies no longer bump it.
• Completing a recurring task with Todo.complete(handle_rrule=True, rrule_mode="this_and_future") raised AttributeError; only the undeclared "thisandfuture" worked. Both spellings are now accepted. Fix by @muratatar06 in #736 #735
• Todo.complete(handle_rrule=True, rrule_mode="safe") on a recurring task with no further occurrences (e.g. COUNT=1) ignored the given completion_timestamp and stamped COMPLETED with the current time. The sync path now passes it on, as the async one already did. #746
• The same fallback, reached through an RRULE with both COUNT and UNTIL (invalid, but seen in the wild), saved the task with COUNT lowered by one. COUNT is now only touched once a next occurrence is found. #746
• change_attendee_status() now awaits async principal and address discovery when the attendee is omitted or a Principal is passed, filters empty addresses, and shares the principal URL and username fallbacks with invite replies. Explicit attendee strings remain synchronous. #678 #734
• Empty CATEGORIES: lines are dropped when calendar data is loaded. The icalendar library inserts one when the categories of a component without categories is read, so such tasks could end up with both CATEGORIES: and the real CATEGORIES line. Data saved from such a component is not cleaned until it is loaded again. Temporary workaround for collective/icalendar#1722 #773
• The warning logged when vcal.fix() modifies icalendar data no longer says the calendar server broke the standard - the server, the client or a library in between may be the cause. #773

Configuration

• compatibility_hints: the inverted FeatureSet.is_supported() answers (see "Potentially breaking changes" above) came from how the features were stored. An observation is now stored as {"value": <bool>} - set_feature(observation, False) writes {"value": False} rather than {"support": "unsupported"} - and read back as that; the legacy observed key and support levels are still understood. A client-feature reads as its enable flag, and a bool given for one is stored as {"enable": <bool>}. Asked for as a str, both spell the bool out ("True"/"False"). The Radicale, DAViCal and Davis profiles declare that a fresh account has no calendar. #732