mozilla-services/syncstorage-rs 0.22.0

on GitHub

Test

• molotov sync loadtests (#2052) (408a23fe)
• rollback transaction in tests relying on Spanner emulator (#2045) (6c6c0ffd)
• fix flake in test_users_with_the_same_batch_id_get_separate_data (#1981) (01edad76)

Breaking Changes

• switch to TIMESTAMPTZ (#1932) (5c022c04, breaks #)

Bug Fixes

• set new node's available = capacity (#2111) (33edb814)
• imeplement security guidelines for github actions (#2096) (47fe53e4)
• adjust tokenserver scripts per postgres (#2086) (7b765be5)
• re-add tokenserver/spanner scripts to the final docker (#2083) (f2b71995)
• get_storage_timestamp/lock_for_read should ignore PRETOUCH_DT (#2067) (0d780c74)
• move version.json to the root (#2041) (e8a392b0)
• correct the JWK env var examples/docs (#2022) (d26ca214)
• preserve the uid ordering by sorting in reverse (#2017) (574f3552)
• further downgrade mermaid, internal dep env problem (#1974) (a3ca41fe)
• update workflow not use script (#1972) (38fce2f3)
• point to tokenserver api (#1970) (d05cf818)
• address reserve and custom collections (#1950) (8b15ac53)
• preserve existing value on batch append if new value is null/empty (#1943) (424ab1e7)
• ensure Postgres return types and Rust value types match (#1940) (a94d702c)
• initializing the app once suffices (#1937) (afaafc57)
• set collection id col to auto-incr and fix get_collection_id (#1929) (9bc614d7)
• mariadb compatibility (b1ca7b32, closes #1753)
• tokenserver: use actual postgres post_user in release build (d5de9b14)

Refactor

• resolve vulns and upgrade (#1788) (88e7eb9c)
• extract a PgDb::check_quota method (4b399232)
• use u64 for DEFAULT_MAX_QUOTA_LIMIT (f8d63999)
• divvy up the db module (30bb0556)
• divvy up tokenserver-postgres's db module (eea08fb2)
• tokenserver-postgres models/orm_models/schema -> db (65becde0)
• divvy up tokenserver-mysql's db module (51e487d1)
• tokenserver-mysql models/pool/migrations -> db (992b1908)
• divvy up the db/batch impls into their own mods (7ecab8bb)
• models/batch/schema/diesel_ext -> db (4145a469)
• move syncstorage-mysql's error -> db-common (e8d89d7a)
• separate batch calls into a BatchDb trait (01566137)
• stream/support/BATCH_COMMIT.txt -> db (da27f9de)
• String -> &str (21bcc42f)
• divvy up the db impl into its own mod (84de3272)
• models/batch -> db (80358fe6)
• rmv last insert id, simplify queries, add check (#1841) (8e3f3670)
• move syncstorage's Db to async-trait (#1830) (d32c669b)
• kill SpannerDb's RefCell (#1826) (613f6ed3)
• kill r2d2 references, share From (bcf425c6)
• move shared tokenserver db into tokenserver-db-common (#1801) (40b21345)
• db: return only a SyncTimestamp from post_bsos db fns (730d6b81)
• metrics: make hostname in metrics optional (#1880) (9e0d3698)

Features

• adapt PoolState to usage of deadpool everywhere (#2128) (019bf46a)
• add FxA event webhook endpoint (#2108) (efa8c4b7)
• normalize use of chrono for time and datetime operations (#2125) (cdd0c3da)
• log when initializing a nodes table entry (#2104) (f2e18e2d)
• push postgres dockers to enterprise gar (#2100) (2823c2e8)
• upsert the first storage node record with env var (#2087) (0f759c6f)
• log when Sentry is not configured (#2073) (66d9d11e)
• postgres node type (#2076) (50a739b5)
• stick w/ Continuous Delivery style image tags (#2044) (9ac27a37)
• optimize postgres get_or_create_collection_id (#2028) (935b468c)
• migrate docker hub push to github actions (#2016) (efb70a1f)
• migrate to GitHub Actions for Spanner build and tests (#2015) (25c852a3)
• optimize post_bsos w/ MERGE INTO (eff8f805)
• migrate code checks to github actions (#2005) (3cca75a9)
• kill long removed browserid references (ab1887d2)
• optimize batch_commit w/ MERGE INTO (#2003) (9ed0d0c2)
• bootstrap the sync-1.5 service table entry (#1993) (0bc8c98d)
• get the timestamp from postgres (#1962) (0cca7c73)
• emit unexpected verify.py exceptions to sentry (#1951) (9fbeaee2)
• re-enable warnings (#1947) (daec1917)
• postgres docker and gar (#1945) (d4a3c24b)
• switch to TIMESTAMPTZ (#1932) (5c022c04, breaks #)
• UserIdentifier modification in lock and timestamp methods (#1927) (56c21e0f)
• support query logging via diesel's instrumentation (#1925) (cb8e620a)
• create post/put bsos postgres (#1911) (855066ce)
• impl {create,commit,append_to,get}_batch for Postgres (4208f037)
• quota methods (#1920) (278ffcc7)
• add the remaining postgres get methods (#1912) (1edfd870)
• add delete fns for Postgres db impls (f548ea1f)
• impl for update collection (#1900) (6b0b41c6)
• postgres read and write locks (#1891) (4d6fe26a)
• add get_bsos/bso_ids (#1899) (f3a33250)
• support systemd-journal logging (#1858) (7ba0b433)
• load and map collections impl (#1898) (455ad7ae)
• add script to purge expired items in Postgres (845a03b2)
• impl get collection and collection id methods (#1887) (ac10f5a7)
• postgres db pool and session implementation (#1875) (a65d4610)
• create schema and orm models for sync postgres (#1873) (7fa191e6)
• sync postgres schema (#1853) (0c119849)
• tokenserver script postgres support (#1857) (adbb0a65)
• add syncstorage-postgres (bcc5990e)
• postgres user methods (#1839) (d6e5f736)
• postgres node methods (#1828) (d7e737dd)
• postgres service methods (#1814) (daec270a)
• switch syncstorage to diesel-async (192d64c7)
• adapt the test suite to the generic dyn DbPool (#1808) (0400c2dd)
• create tokenserver postgres db trait (#1809) (8d8b79e1)
• create tokenserver postgres db pool (#1806) (0c3c06d5)

Chore

• rmv oauth tokenserver auth method (#2139) (23d04224)
• update test result/coverage filenames (#2113) (37350249)
• actions security updates (#2109) (f2e6cf5e)
• run gh workflows on pushes (#2105) (a66dfa71)
• rm .circleci (#2101) (8f289f3a)
• upgrade jsonwebtoken (#2097) (9db210d5)
• stop uploading test results to gcs from circleci (#2094) (c9fd339d)
• build test results file prefix when github (#2092) (dc07e833)
• stop checking for secrets.ETE_GCLOUD_SERVICE_KEY in workflows (#2090) (129c1aa0)
• update rust to 1.91 (#2082) (68081b2b)
• cargo upgrade deps (#2061) (b54129a3)
• poetry update python deps (#2063) (33d41fbf)
• update image names and jobs to syncserver (#2051) (f545cb3e)
• mv load tests (#2060) (ab47abaa)
• remove pushes to dockerhub (#2057) (cfb3e5c3)
• upgrade to Rust 2024 edition (#2048) (0ddc3b0a)
• use dorny/test-reporter for reporting gh action test results (#2036) (9f700d64)
• Update GAR Image Tagging to MozCloud Spec (#2029) (d2fda79d)
• migrate to GH action for Postgres build and tests (#2012) (6af6d5c6)
• move MySQL build and test to GH actions (#2011) (f125cf6b)
• kill remaining requirements.txt (5f8ab548)
• clean out OnDuplicateKeyUpdate mysql diesel extension (#2001) (ac614115)
• build image for postgres python utils (#1987) (e7b4ccdf)
• bump mbdook to 0.5.2 (#1988) (ba6cda12)
• update mdbook and mermaid (#1977) (1c12ca06)
• doc workflow fix (#1967) (6fca1c40)
• use real exit codes for Postgres tests in CI (#1944) (7bf49d79)
• run Postgres tests and clippy in CI (#1923) (6c6c06ff)
• adjust dependabot pr limit to 1 (#1918) (7103b3da)
• add healthcheck to prevent e2e tests starting too early (ad0f6776)
• tokenserver-db-postgres -> tokenserver-postgres (36657fd6)
• fix Cargo.lock (9b008f8d)
• async sync method suffix (#1821) (4a8ec959)
• allow overriding RUST_LOG (#1813) (f2c7f944)
• use environment value from syncserver settings for Sentry (204c13e7)
• CI: fetch MySQL public key from MIT (31e5e2ba)
• ci:
  • use gh actions to build and deploy to GAR and ghcr (#1976) (c46408b0)
  • configure CircleCI for postgres builds (30478687)
• deps:
  • bump quinn-proto from 0.11.13 to 0.11.14 (#2110) (40a8618b)
  • bump werkzeug in /tools/tokenserver/loadtests (#2077) (35b0a9dd)
  • bump flask in /tools/tokenserver/loadtests (#2078) (c556860a)
  • bump cryptography in /tools/integration_tests (#2072) (cc07bb51)
  • bump cryptography from 44.0.2 to 46.0.5 (#2054) (ab6ec0a0)
  • bump cryptography in /tools/tokenserver/loadtests (#2055) (d3880c56)
  • bump bytes from 1.10.1 to 1.11.1 (#2039) (854e6c5b)
  • bump pyasn1 from 0.6.1 to 0.6.2 in /tools/spanner (#2008) (cccdac90)
  • bump urllib3 from 2.6.0 to 2.6.3 in /tools/spanner (#1994) (ea0055df)
  • bump authlib in /tools/tokenserver/loadtests (#1992) (230b4a38)
  • bump urllib3 in /tools/integration_tests (#1985) (fbbd87e0)
  • bump urllib3 from 2.6.0 to 2.6.3 in /tools/tokenserver (#1983) (c13f7d7c)
  • bump urllib3 from 2.5.0 to 2.6.3 (#1982) (1a4a0e48)
  • bump urllib3 in /tools/integration_tests (#1978) (9612bc95)
  • bump urllib3 from 2.5.0 to 2.6.0 in /tools/tokenserver (4ab5dba8)
  • bump urllib3 from 2.5.0 to 2.6.0 in /tools/spanner (78ee28e7)
  • bump authlib in /tools/tokenserver/loadtests (207a010e)
  • bump authlib in /tools/tokenserver/loadtests (#1816) (f60189d9)
• deps-dev:
  • bump werkzeug in /tools/tokenserver/loadtests (#1990) (0e08ba60)
  • bump urllib3 in /tools/tokenserver/loadtests (#1984) (cba780e5)
  • bump urllib3 in /tools/tokenserver/loadtests (77836c6c)
  • bump werkzeug in /tools/tokenserver/loadtests (740b5cc9)
  • bump brotli in /tools/tokenserver/loadtests (7b907bdf)
• syncserver: break up extractors.rs (34041507)

Doc

• general bootstrapping instructions postgres (#2088) (03627ed3)
• content from archived repos (#2075) (a4758569)
• load test runs for locust and molotov (#2084) (30258c82)
• open api docs utoipa (#2023) (62bd7d2c)
• update docs for prepare-spanner (#2035) (9d0a6391)
• update settings list (#2030) (b9ea1191)
• add one-shot docker compose yaml to how-to (#2025) (646e516e)
• add how-to on deploying with docker compose (#2019) (11659d98)
• sync api docs github (#1986) (77868a38)
• architecture and system diagrams (#1973) (a108a0da)
• build deploy docs GitHub pages (#1965) (8b72a3b8)
• contributing guidelines fix (#1868) (9170b97f)
• update readme for project setup (494b1617)