dateparser 1.5.0

on Python PyPI

Breaking changes:

• DateData.period can take the new values "quarter" and
  "part_of_day", and with RETURN_TIME_AS_PERIOD enabled a relative
  date with hours, minutes or seconds, such as "2 hours ago", now has the
  "time" period instead of "day" (#1417, #1437, #1473)
• Stop caching and sharing Settings instances: creating one with the
  same settings twice now gives two different objects, and the
  Settings.registry_key attribute no longer exists (#1442)
• Remove the leftovers of fastText support from dateparser_cli: the
  fasttext_manager and exceptions modules and
  utils.create_data_model_home(); utils.dateparser_model_home and
  utils.DEFAULT_WINDOWS_CACHE_DIR are now pathlib.Path objects
  (#1406)
• Remove Locale.to_parserinfo() and HijriDate.weekday(), which
  always raised an exception (#1406, #1457)

Deprecations:

• Deprecate dateparser.timezone_parser.convert_to_local_tz() (#1481)
  and dateparser.timezone_parser.build_tz_offsets() (#1405)
• Emit a FutureWarning naming the TZ value from parse(),
  DateDataParser and search_dates() when the local timezone cannot
  be determined; a future version will raise RuntimeError instead of
  failing to parse the date (#1403, #1556)
• Deprecate dateparser.utils.registry() and Settings.get_key(),
  which now emit a FutureWarning (#1442, #1556)
• Deprecate get_week_start() and get_week_end() in
  dateparser.utils.time_spans, and calling generate_time_span()
  without base_date or settings; all of them now emit a
  FutureWarning (#1393, #1556)
• Deprecate dateparser.date.RE_TRIM_COLONS (GHSA-fmxc-97cw-ph6p <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-fmxc-97cw-ph6p>_)

Security fixes:

• Make sentence splitting in search_dates() linear on long punctuation
  runs, to prevent quadratic backtracking (ReDoS) for Arabic, Bengali,
  Cantonese, Chinese, Hindi, Japanese and Persian (GHSA-rx9v-94ff-9cc2 <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-rx9v-94ff-9cc2>_)
• Prevent quadratic regex matching (ReDoS) on long digit runs in Cantonese,
  Chinese and Japanese, and a ValueError on runs of more than 4,300
  digits on Python 3.10.7 and later (GHSA-g8fh-jxm9-6qfx <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-g8fh-jxm9-6qfx>_)
• Bound the lookahead of the Chinese month simplification, to prevent quadratic
  parsing time on long runs of numbers followed by 月 (GHSA-q7fp-xwcm-6x9f <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-q7fp-xwcm-6x9f>_)
• Prevent quadratic regex matching (ReDoS) on long runs of digits or whitespace
  (GHSA-gqj4-wf6c-wmcg <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-gqj4-wf6c-wmcg>_)
• Strip trailing colons in linear time, to prevent quadratic backtracking
  (ReDoS) on long colon runs (GHSA-fmxc-97cw-ph6p <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-fmxc-97cw-ph6p>_)
• Split strings into known words in linear time, to prevent quadratic parsing
  time on long runs of words like month names (GHSA-jxmm-w6rw-32ch <https://github.com/scrapinghub/dateparser/security/advisories/GHSA-jxmm-w6rw-32ch>_)

New features:

• Add a STRICT_DATE_ORDER setting that turns an explicit DATE_ORDER
  from a preference into a requirement: "year" reads a two-digit year
  only from the position DATE_ORDER gives it, and "all" also keeps
  the day and month where the order puts them (#1443)

• Add a TIMEZONE_ABBREVIATIONS setting to override the offset of
  timezone abbreviations, e.g. to read IST as India Standard Time
  instead of Israel Standard Time (#1415)

• Parse parts of the day, such as "tomorrow morning", "this evening" or
  "8 in the evening", at the times set by the new PARTS_OF_DAY setting;
  the new DateData.part_of_day attribute tells which one was found
  (#1473, #1562)

• Add DateData.parts, which lists the date parts (day, month, year,
  time…) read from the input rather than filled in from defaults (#1408)

• Add an add_period argument to search_dates() to also return the
  period of each date (#1437)

• Add a date_formats argument to search_dates(), to find dates
  written in the given formats; it requires strategy="ngram" and raises
  ValueError with the default "split" strategy (#1436)

• Add the French Republican calendar and the Minguo (Taiwanese) calendar
  (#1483, #1489)

• Parse calendar quarters, such as "2025Q1", "2025-Q3" or "Q4 2025", as a
  month of that quarter picked by PREFER_MONTH_OF_YEAR (by default, the
  month as far into the quarter as the current month is into its own)
  (#1417)

• Parse "last", "this" and "next" before a weekday in English, e.g. "next
  Friday", as the closest occurrence of that weekday before, on or after,
  and after the current date, whatever PREFER_DATES_FROM is, except for
  the abbreviations "wed", "sat" and "sun"; search_dates() now includes
  the word in its matches (#1401)

• Parse "the Nth of last/this/next month" as a day of that month (#1414)

• Parse Roman numeral years written in capitals, e.g. "5 March MDCCLXXVI";
  a short one such as "MMX" is read as a year only next to a month name,
  e.g. "April MMX" (#1482)

• Parse dates with a bare UTC offset, such as "+03" (#1475)

• Parse AM/PM written before the time, e.g. "PM 2:26" or
  "下午 02:26:26" (#1469)

• Parse ISO 8601 fractional seconds written with a decimal comma, e.g.
  "12:30:45,123" (#1467)

• Parse English number words from thirteen to ninety-nine, e.g.
  "forty-five seconds ago" (#1416)

• Parse "fortnight" and "century" in English relative dates, and read
  "a decade from now" as the future instead of the past (#1539)

• Match month and weekday names in date_formats in the given languages,
  so formats with literal text work, e.g. "el %d de %B de %Y" for "el 15 de
  junio de 2020" with languages=["es"] (#1486, #1552)

• When a date does not parse, try the other meanings of ambiguous words,
  such as Spanish "mar" (martes or marzo) (#1491), and read a token that
  is both a weekday and a month abbreviation as the weekday when that is
  what fits (#1476)

• In search_dates(), find dates written:

  • right after a label and a colon in Chinese or Japanese, e.g.
    "日期：2020年12月28日" (#1407)
  • next to a relative date without a comma (#1385)
  • joined by words that search skips, like "and" (#1488)
  • inside parentheses, square brackets or straight quotes, e.g.
    'He said "May 5, 2020"' or "(Friday)" (#1401, #1565)

Fixes:

• Take the current date, used to fill in the missing parts of a date such
  as "12:30" or "Nov" and to apply PREFER_DATES_FROM, from the
  TIMEZONE setting (the local time zone by default) instead of UTC
  (#1427)
• Keep the %z offset when parsing with date_formats, and return a
  timezone-aware date for it unless RETURN_AS_TIMEZONE_AWARE is
  False (#1424)
• Read year-first dates, such as "2021-01-13", as YMD unless DATE_ORDER
  starts with the year (#1419)
• Swap the day and the month when they do not fit an explicit
  DATE_ORDER, whatever its value, e.g. "01/13/2021" with DMY (#1431)
• Read a lone number next to a four-digit year as the month under any
  DATE_ORDER (#1472)
• Apply PREFER_DAY_OF_MONTH before PREFER_DATES_FROM picks the year
  (#1409), and apply PREFER_DAY_OF_MONTH and PREFER_MONTH_OF_YEAR
  to relative dates such as "last year" (#1413)
• Apply an explicit DATE_ORDER to every date that search_dates()
  finds, not only to the first one (#1535)
• Stop dropping extra numbers when parsing dates: a string with more numbers
  than a date has, such as "5 10 Jan 2020" or "03/04/2021 5", now returns
  None instead of a date built from some of them (#1480)
• Stop reading two adjacent numbers of a Russian numeric date as one
  compound ordinal, so parse("20 03 2021", languages=["ru"]) returns
  20 March 2021 instead of the 23rd of the current month (#1554)
• Return None for strings with only skipped words, such as "year" or
  "hour minute", instead of today, which also stops search_dates() from
  returning them (#1440, #1564)
• Stop ignoring numbers without a unit in relative dates, so "03/06/2020
  hour", "03/06/2020 day" or "01.02.2011 godz. 08:39" no longer parse as a
  number of hours or days ago (#1410)
• Stop reading "Nth day" as a number of days, e.g. "the 1st day of March
  2015" (#1412)
• Parse "at 22h" and "à 22h" as a time of day, not as hours ago; a bare
  "à 22h" is still read as English hours ago unless languages=["fr"] is
  given or the text has other French words, such as "demain à 22h" (#1441)
• Keep the future direction of a relative expression that search_dates()
  finds next to punctuation, e.g. "in 2 hours, thanks" (#1391)
• Stop French "sept" being read as 7 when it means September, e.g.
  "1 sept 2024" with languages=["fr"] and "le 3 sept à 10h" with or
  without languages; "sept" now means 7 only before a duration unit
  or a clock time ("sept jours", "sept heures", "sept:30"), so a day
  written as a word, as in "sept mai 2020", no longer parses, and a bare
  "sept" is read as September (#1353)
• Parse dates with a numeric offset followed by a redundant timezone
  abbreviation, e.g. "-0500 (CDT)", or with an abbreviation glued to the
  time, e.g. "2019-09-28WIB19:17:34" (#1348)
• Recognize UTC offsets written with Unicode dash-like characters, such as
  "−05:00" (#1474)
• Keep complete week dates (year, week number and weekday) when parsing
  with date_formats, including %G ISO week years (#1381)
• Fix negative timestamps with subseconds (#1387)
• Keep AM when searching for dates after a time (#1400)
• Keep full stops between digits in search_dates() sentences (#1471)
• Fix search_dates() dropping text next to a standalone accent mark
  (#1505)
• Fix search_dates() raising IndexError for the zh-Hans,
  zh-Hant and yue locales, including when it detects one of them in
  the text, e.g. "聽日" (#1401, #1407)
• In search_dates(), include the closing bracket in the matched text,
  e.g. "8 May 2020 (16:05)", and stop including separators such as "/" or
  "|" (#1401)
• Split a relative date from a numeric date that follows it in
  search_dates(), e.g. "3 days later on 2010-05-10" (#1430)
• Stop growing internal caches with every distinct RELATIVE_BASE value
  (#1442)
• Fix parse() and DateDataParser returning None when region
  does not match the language of the text, e.g.
  parse("12 mars 2020", languages=["fr"], region="MX") or
  parse("5 de marzo de 2020", region="US") (#1423)
• Fix dateparser.utils.get_logger() raising AttributeError when
  logging.config had not been imported (#1457)

Improvements:

• Speed up parse() by about 35% (#1544), skip locales that cannot
  change the outcome of a failed parse (#1390), and speed up import by
  compiling the timezone regexes lazily (#1405)
• Arabic: parse relative dates that use قبل for "ago" (#1397), Egyptian and
  informal ة→ه spellings (#1493), numbered Levantine month names such as
  "كانون 2" in the ar-SY, ar-JO, ar-LB, ar-PS and ar-IQ
  locales only, e.g. locales=["ar-SY"] or region="SY" (#1511),
  Arabic dates typed with Persian yeh and kaf and Persian dates typed with
  Arabic ones (#1423), and Arabic month names and Arabic-Indic digits in
  Hijri dates (#1432)
• Chinese: add the supplementary data of zh to zh-Hans and
  zh-Hant (#1407), and parse 后天, 大后天 and 大前天 as relative days
  (#1430)
• Chinese and Japanese: parse dates with spaces around 年, 月 and 日 (#1477,
  #1566)
• French: parse "midi" as noon, e.g. "demain à midi" ("après-midi" and
  "avant midi" are still not read as noon); this also applies when no
  languages are given, so search_dates() can return "MIDI" in an
  English text as 12:00 (#1536)
• Indonesian: add old-style month abbreviations, such as "Djan" or "Peb"
  (#1501)
• Korean: parse absolute dates written with 년, 월, 일, 시 and 분 (#1434)
• Slovak: parse "pozajtra" (#1430)
• Swedish: parse ordinal day suffixes like "24:e" and "1:a" (#1484)
• Thai: parse relative dates with spaced ago/in words (#1411)
• Vietnamese: parse dates with zero-padded months (#1338, #1563)
• Add type hints (#1457)
• Document search_dates() in the API reference (#1468), add a
  performance section (#1494), explain how DEFAULT_LANGUAGES relates to
  the languages argument (#1422) and how date_formats falls back to
  the other parsers (#1549)
• Show the whole README on PyPI (#1359)