Summary of Changes
Major Changes:
- LB IPAM: introduce the lbipam.cilium.io/sharing-permit-different-pods annotation to allow sharing an IP between services with externalTrafficPolicy=Local that select different pods. The annotation must be set to "true" on all services in the sharing group and is intended for deployments that steer traffic outside Cilium's built-in announcers. (#45819, @lexfrei)
Minor Changes:
- Add Helm values to configure Prometheus metrics container port names for the agent, operator, Envoy, Hubble, Hubble Relay, ClusterMesh API server, KVStoreMesh and etcd, supporting custom scrape configurations that require specific named ports. Existing names remain the defaults. (#48955, @gcampbell12)
- Add the internal path template matcher that later PRs use to emit a bounded
pathlabel on the Hubble HTTP metrics. No user-facing change on its own. (#47813, @amaanx86) - auth: remove deprecated mutual authentication (#48647, @mhofstetter)
- clang: Bump llvm version to v21.1.8 (#48425, @sayboras)
- Fix a "No route to host" window on node reboot where BGP advertised routes before BPF programs were attached to native devices. (#46948, @tkna)
- Fix conntrack garbage collection deleting CT entries that the datapath reopened between the GC batch read and the delete, which could cascade into NAT mapping deletion and SNAT source-port reallocation for live connections. (#46750, @gyutaeb)
- Hubble observer server hardening (#46290, @fristonio)
- Hubble Relay startup can now be composed by downstream binaries with additional server options. (#48461, @devodev)
- IPsec subnet encryption under ENI, Azure and AlibabaCloud now derives its pod subnets from the VPC CIDRs directly, fixing cases where they were empty (no encryption policies installed) depending on agent startup ordering. AlibabaCloud gains subnet encryption, and the ip-masq-agent non-masquerade CIDRs are no longer treated as pod subnets. (#48493, @HadrienPatte)
- LB IPAM: whitespace around the namespaces in the lbipam.cilium.io/sharing-cross-namespace annotation no longer prevents services from sharing an IP. (#48868, @lexfrei)
- Migrate Azure IPAM to the Multi-pool allocator (#48449, @jaredledvina)
- network driver: operator managed node configs (#47597, @bersoare)
- network driver: track DRA allocations separately from device inventory (#48755, @aarcamp)
- Setting "policyCidrMatchMode=nodes" no longer causes unnecessary node identities to be allocated, saving space in the IPCache and policy maps. (#48292, @squeed)
- The agent option
--fixed-identity-mappingis deprecated. It will be removed in a future release. (#48946, @squeed) - Track peer AllowedIPs as netip.Prefix (#47815, @Wise-Wizard)
Bugfixes:
- bpf: Fix ICMPv6 checksum handling after NAT of embedded SCTP packets. (#48309, @zihan001)
- bpf: icmp: use 576 byte of inner packet to avoid partial csum floor (#48578, @tommyp1ckles)
- bpf: lb: Do not drain terminating backends for UDP connections, preventing UDP packet drops during pod deletion and rolling updates.. (#48715, @ankitaojha123)
- ciliumendpointslice: fix flaky TestRegisterControllerNoCEPs (#48014, @elskow)
- daemon: fix a bug where L7 DNS policies could remain loaded in the datapath during agent shutdown, causing pods under a wildcard toFQDNs policy to lose DNS resolution for the duration of the restart. (#48747, @41ks)
- envoy: Corrently render tiers for specific port pass rules (#48854, @jrajahalme)
- envoy: handle listener additional address changes (#48712, @nezdolik)
- envoy: handle listener address changes in ADS (#48798, @nezdolik)
- envoy: track coalesced ADS snapshot updates (#48631, @nezdolik)
- Fix
proxy_protocollistener filter incorrectly injected into GAMMA CiliumEnvoyConfigs whengatewayAPI.enableProxyProtocolis enabled (#47555, @mkamadeus) - Fix cilium-cli clustermesh 'inspect-policy-default-cluster-local' command (#48390, @fristonio)
- Fix Gateway API ExternalAuth dropping request bodies when allowedResponseHeaders is unset (#48770, @kirgene)
- Fix incorrect deletion of new endpoint during GC iteration when reusing ID from a previously GC'd endpoint (#48905, @fristonio)
- Fix panic in agent when CiliumLocalRedirectPolicy
redirectBackend.toPortsis declared as an empty array. The agent will now reject such a policy. (#48491, @ajmmm) - Fix PMTU/ICMP replies for IPIP/Geneve DSR for GRO skbs (#48489, @borkmann)
- Fix rare data race in selector cache that could lead to agent panic (#47733, @odinuge)
- Fix restoration of terminating backends. If a terminating backend become active again during restart the backend was incorrectly kept as quarantined. (#48763, @joamaki)
- fix(envoyconfig): retry port allocation on duplicate-address rejections (#48624, @kokhlo)
- Fix: MCS EndpointSlice mirroring fails when service names exceed 63 characters (#48783, @weizhoublue)
- Fixed a cilium-agent crash (
fatal error: concurrent map iteration and map write) when an endpoint's policy was recomputed while incremental policy map changes were applied concurrently. Most commonly triggered by named-port policies under pod churn. (#48098, @nashant) - Fixes a bug that prevents cilium identity creation for pods with named ports when operator-managed-identity was enabled. (#48662, @tamilmani1989)
- Fixes a bug where Hubble flows may list the incorrect policy that allowed or denied a flow (#49086, @tsotne95)
- infraendpoints: Release the health IPv4 when the IPv6 allocation fails (#48816, @HadrienPatte)
- k8s/resource: fix leaked event finalizer on unobserved delete (#48785, @tklauser)
- Keep terminating backends in EDS when active backends exist and set their health to DRAINING so Envoy can drain in-flight traffic instead of resetting connections. (#46642, @komapa)
- operator: Do not emit warning when deleting already deleted unmanaged pod (#48765, @gandro)
- operator: fix
metrics listdefault address to work on IPv6-only clusters with dual stack nodes by trying both IPv4 and IPv6 loopback addresses (#48780, @41ks) - Restore ingress IPs from k8s / local node as a backup when they may be missing from the ipcache. (#47360, @jrajahalme)
- Set the resource version when updating CiliumCIDRGroup. (#47712, @bodgit)
- Sockets are no longer destroyed when a Service backend is terminated unless they are still connected to that backend. Unconnected UDP sockets, and sockets that have since disconnected or moved to another backend, used to have their local port silently changed. (#48725, @maxskokov)
- The cilium-operator now exits when its controller-runtime manager fails, instead of continuing to run with its controllers stopped. (#48385, @ryanjwong)
- wireguard: Fixed an issue where transient errors updating WireGuard peers could cause AllowedIPs to remain permanently out-of-sync with the IPCache, resulting in long-term traffic disruption. (#48664, @meel-ayush)
CI Changes:
- [ci] fix escaping in conformance ginkgo (#48991, @nebril)
- ci: Drop the empty floating tag line from CI image digests (#49070, @aanm)
- ci: Give Envoy Embedded conformance a bigger runner (#49018, @aanm)
- ci: Harden CI image build workflows (#48959, @ferozsalam)
- ci: keep UDPRouteWeightedRouting skipped in the profile job (#49005, @aanm)
- ci: login to docker registry for downgrade image wait (#48983, @nebril)
- ci: Name runner variables after their Ubuntu release and let Renovate bump them (#49065, @aanm)
- ci: stop setting removed mutual auth Helm value (#48964, @mhofstetter)
- contrib: Add workflow check-name collision linter (#48568, @christarazi)
- datapath: assert ARP off before up, not the link-local flags (#48610, @aanm)
- Fix the GitHub greeter so that it also welcomes first-time issue reporters. (#48579, @bitflicker64)
- gateway-api: cleanup unused helper functions (#47835, @asauber)
- gh: lint-bpf-checks: clean up disk when running BPF tests (#48601, @julianwiedmann)
- gha/conformance-multi-pool: use the local fake external targets (#48558, @aanm)
- gha: Derive the LVH package cache key once per run (#48656, @aanm)
- gha: make external fake targets image configurable (#48941, @giorio94)
- gha: Skip the duplicate CI image build on push to main (#48736, @aanm)
- gha: Tolerate cosign and SBOM failures on pull requests (#48737, @aanm)
- identity/cache: Fix data races flaking the test package (#48566, @christarazi)
- Increase test coverage in GKE, AKS and EKS conformance workflows. (#47220, @gentoo-root)
- loadbalancer/tests: Fix the graceful termination test restart (#48939, @aanm)
- Makefile: Order the bpf codegen ahead of the Go builds (#48644, @aanm)
- Migrate Conformance Cluster Mesh workflow to use fake external targets (#48250, @giorio94)
- nodeinit: wait for the kubelet wrapper before writing the bootstrap file (#48739, @aanm)
- policy/test: retry endpoint list until it reports ready (#49017, @aanm)
- renovate: Keep LLVM major version for stable branches (#48625, @sayboras)
- Skip embedded proxy conformance CI for non-functional changes (#47305, @AkashKumar7902)
- test: fix the never-taken retry on a partially read JSON line (#48190, @aanm)
- test: Wait for the testds DaemonSet before reading backend pod IPs (#48616, @christarazi)
- tests: Build only the packages a privileged run can execute (#48608, @aanm)
- Update CI image build workflow (#48220, @ferozsalam)
Misc Changes:
- .github: Add testing expectations to PR template (#49099, @joestringer)
- Add Aether to USERS.md with relevant details (#49073, @zozo6015)
- Add desired-device support for Linux VRF links and VRF-bound VLAN devices. (#48653, @harsimran-pabla)
- Add extension points for cilium-envoy Daemonset readiness, liveness and startup probes (#48421, @0xch4z)
- Allow Renovate to update Go to v1.26 on Cilium v1.18 and v1.19 (#48705, @ferozsalam)
- aws/ipam: Fix lock order inversion with the operator node (#48645, @aanm)
- bpf, datapath: switch
DROP_NOTIFYto runtime config (#48956, @tklauser) - bpf, datapath: switch socketlb externalIP mitigation to runtime config (#48649, @tklauser)
- bpf: convert HOST_NETNS_COOKIE to runtime config (#48499, @viktor-kurchenko)
- bpf: egressgw: clean up CT tuple initialization (#48629, @julianwiedmann)
- bpf: egressgw: prepare SNAT lookup for host traffic (#48585, @ysksuzuki)
- bpf: lxc: reuse
fraginfofromct_buffer(#48793, @tklauser) - bpf: lxc: switch cilium_nodeport_nat_buffer to auxvar (#48543, @julianwiedmann)
- bpf: move VLAN_FILTER host to runtime config (#48697, @viktor-kurchenko)
- bpf: nat46: remove dead csum_off assignment (#48458, @fds7280)
- bpf: only set ratelimit key and settings if needed (#48940, @tklauser)
- bpf: policy_log: move
send_policy_verdict_notify()variables to auxvar (#48802, @tklauser) - bpf: turn
ENABLE_SIP_VERIFICATIONinto runtime config (#48762, @andreaterzolo) - bpf: use
DSR_ENCAP_MODEinstead ofENABLE_DSR(#48981, @andreaterzolo) - bpf:policy: remove unnecessary __sync_fetch_and_add on percpu map (#48944, @smagnani96)
- build: handle git worktrees in documentation make targets (#48966, @utaal)
- chore(deps): update all github action dependencies (main) (#48590, @cilium-renovate[bot])
- chore(deps): update all github action dependencies (main) (#48669, @cilium-renovate[bot])
- chore(deps): update all github action dependencies (main) (#48877, @cilium-renovate[bot])
- chore(deps): update all github action dependencies (main) (#49033, @cilium-renovate[bot])
- chore(deps): update all lvh-images main (main) (patch) (#48666, @cilium-renovate[bot])
- chore(deps): update all lvh-images main (main) (patch) (#48874, @cilium-renovate[bot])
- chore(deps): update all lvh-images main (main) (patch) (#49027, @cilium-renovate[bot])
- chore(deps): update all-dependencies (main) (#48670, @cilium-renovate[bot])
- chore(deps): update all-dependencies (main) (#48878, @cilium-renovate[bot])
- chore(deps): update all-dependencies (main) (#48896, @cilium-renovate[bot])
- chore(deps): update all-dependencies (main) (#49026, @cilium-renovate[bot])
- chore(deps): update dependency bufbuild/buf to v1.73.0 (main) (#48727, @cilium-renovate[bot])
- chore(deps): update dependency cilium/cilium to v1.18.14 (main) (#48902, @cilium-renovate[bot])
- chore(deps): update dependency cilium/cilium to v1.19.8 (main) (#49028, @cilium-renovate[bot])
- chore(deps): update dependency cilium/cilium to v1.20.2 (main) (#48904, @cilium-renovate[bot])
- chore(deps): update dependency cilium/little-vm-helper to v0.0.31 (main) (#48787, @cilium-renovate[bot])
- chore(deps): update dependency protocolbuffers/protobuf to v36.2 (main) (#49031, @cilium-renovate[bot])
- chore(deps): update docker.io/library/golang:1.27.1 docker digest to 3680233 (main) (#48873, @cilium-renovate[bot])
- chore(deps): update docker.io/library/golang:1.27.1 docker digest to f44f6e8 (main) (#48665, @cilium-renovate[bot])
- chore(deps): update gcr.io/etcd-development/etcd docker tag to v3.7.2 (main) (#49029, @cilium-renovate[bot])
- chore(deps): update ghcr.io/renovatebot/renovate docker tag to v44.103.7 (main) (#48898, @cilium-renovate[bot])
- chore(deps): update ghcr.io/renovatebot/renovate docker tag to v44.104.0 (main) (#48903, @cilium-renovate[bot])
- chore(deps): update ghcr.io/renovatebot/renovate docker tag to v44.115.12 (main) (#49055, @cilium-renovate[bot])
- chore(deps): update ghcr.io/renovatebot/renovate docker tag to v44.115.13 (main) (#49059, @cilium-renovate[bot])
- chore(deps): update quay.io/cilium/cilium-envoy docker tag to v1.38.4-1789133559-b4174f07e1030109459e34436262323af5c3ef0f (main) (#48667, @cilium-renovate[bot])
- chore(deps): update quay.io/cilium/cilium-envoy docker tag to v1.38.4-1789396437-a6bb974d7ad015d17b46eb44fdd7bbfe8256c089 (main) (#48875, @cilium-renovate[bot])
- chore(deps): update quay.io/cilium/cilium-envoy docker tag to v1.38.4-1789992194-283c3f1fedbbc07e75e35f89b9a9ef93ffbe1bc3 (main) (#49030, @cilium-renovate[bot])
- chore(deps): update quay.io/cilium/cilium-envoy docker tag to v1.39.1-1790794634-fbfb16f5f9d4c09131542931999a2cbe79a863f8 (main) (#49116, @cilium-renovate[bot])
- chore(deps): update quay.io/goswagger/swagger docker tag to v0.36.6 (main) (#48668, @cilium-renovate[bot])
- ciliumendpointslice: Report no CID when the CEP has none (#48933, @aanm)
- ciliumendpointslice: Use controller-runtime priorityqueue (#48702, @alvaroaleman)
- cleanup: Replace deprecated strings.Title with x/text/cases (#48275, @abraaoteixeira)
- clustermesh: fix context cancellation race on remote connection restart (#48622, @giorio94)
- clustermesh: misc cleanups and improvements to the MCS EndpointSlice mirroring (#48520, @MrFreezeex)
- connectivity-check: move echo-b to the lower NodePort band (#48926, @aanm)
- datapath/loader: include BPF source in template hash (#48623, @tklauser)
- datapath: define
ENABLE_PER_PACKET_LBin terms ofis_defined()(#48866, @ekoops) - datapath: get rid of
LOCAL_DELIVERY_METRICS(#48640, @ekoops) - datapath: switch CT monitor report interval and flags to runtime config (#48595, @tklauser)
- datapath: turn
ENABLE_HEALTH_CHECKinto runtime config (#48713, @ekoops) - datapath: turn
ENABLE_SCTPinto runtime config (#48892, @ekoops) - datapath: turn
ENABLE_SOCKET_LB_FULLinto runtime config (#48914, @ekoops) - datapath: turn
ENABLE_VTEPinto runtime config (#48635, @ekoops) - datapath: turn
SERVICE_NO_BACKEND_RESPONSEinto runtime config (#48561, @ekoops) - datapath: turn
TRACE_SOCK_NOTIFYinto runtime config (#48628, @ekoops) - docs: add Hetzner Cloud native routing installation guide (#48615, @ocalzi)
- docs: document Envoy listener redirect and wildcard port in L4 policy (#46751, @PhilipSchmid)
- docs: fix typos in kvstore and deny policy pages (#48612, @magic-peach)
- Docs: list the conditions under which services with the same LB IPAM sharing key receive the same IP. (#48867, @lexfrei)
- docs: note on eBPF ClusterIP translation vs Docker raw-table protection (#47782, @Verbano-Tech)
- docs: remove outdated tip about disabling hubble before enabling UI (#48487, @ekoops)
- docs: remove stale mutual auth metrics mention (#48960, @mhofstetter)
- docs: Update clang version requirements (#48719, @sayboras)
- docs: Update kernel config requirements (#48594, @pchaigno)
- docs: use relative domain name for deathstar service instead of FQDN (#48448, @ekoops)
- envoy: add standalone NPDS unsubscribe regression test (#47818, @nezdolik)
- envoy: policy perf improvements (#48925, @jrajahalme)
- Fix data races on global state under parallel policy tests (#48929, @christarazi)
- Fix Envoy xDS listener handling across restoration, address changes, and rejected updates (#48897, @jrajahalme)
- fix(deps): update all go dependencies main (#48621, @aanm)
- fix(deps): update all go dependencies main (main) (#48696, @cilium-renovate[bot])
- fix(deps): update all go dependencies main (main) (#48698, @cilium-renovate[bot])
- fix(deps): update all go dependencies main (main) (#48876, @cilium-renovate[bot])
- fix(deps): update all go dependencies main (main) (#49032, @cilium-renovate[bot])
- fix(deps): update golang.org/x/exp digest to 85c1c22 (main) (#48699, @cilium-renovate[bot])
- fix(deps): update module github.com/aws/aws-sdk-go-v2/service/ec2 to v1.336.0 (main) (#49060, @cilium-renovate[bot])
- fix(deps): update module github.com/azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v11 to v12 (main) (#49034, @cilium-renovate[bot])
- fix(deps): update module github.com/cilium/hive to v1.0.6 (main) (#49054, @cilium-renovate[bot])
- fix(deps): update module github.com/google/go-github/v90 to v91 (main) (#48671, @cilium-renovate[bot])
- fix(deps): update module github.com/google/go-github/v91 to v92 (main) (#48728, @cilium-renovate[bot])
- fix(deps): update module github.com/gopacket/gopacket to v1.7.2 (main) (#48893, @cilium-renovate[bot])
- fix(route-manager): always set the initial route status to Done (#48522, @andreaterzolo)
- fix: disable auto-direct-node-routing in hybrid mode (#48547, @andreaterzolo)
- fqdn: Improve FQDN garbage collection and DNS response processing performance when many DNS names resolve to the same IP by avoiding repeated scans of deferred DNS names. (#48360, @vipul-21)
- gateway-api: centralize Gateway validation (#48602, @arybolovlev)
- gateway-api: fail Gateway reconcile on GatewayClass lookup errors (#48546, @mhofstetter)
- gateway-api: remove stale TODO comments in helper tests (#48605, @Sanjeevliv)
- GlobalIdentity: use labels.Labels, not LabelArray (#47429, @squeed)
- hive: Remove redundant netip.Prefix config decode hook (#49090, @HadrienPatte)
- In hybrid routing mode, skip SNAT for pod-to-remote-node traffic when source and destination belong to the same subnet topology group, preserving the original Pod source IP for same-subnet communication. (#45670, @OneSizeFitsQuorum)
- Increase bucket size range for endpoint/policy related latency histogram metrics (#48720, @fristonio)
- ipam, routing: Remove the unread VPC CIDRs from RoutingInfo and IPAM results (#48818, @HadrienPatte)
- ipam: centralize
CIDRAllocatorinterface underpkg/ipam/cidralloc(#48822, @ekoops) - ipam: Migrate
IPAMStatus.AssignedStaticIPtoip.Addr(#48971, @HadrienPatte) - ipam: Migrate IPAM map keys to netip (#48445, @HadrienPatte)
- ipcache/fake: migrate fake IPCache node event addresses to netip.Addr (#47683, @Dasmat13)
- ipcache: migrate CIDR filter to netip (#47779, @dlanov)
- ipmasq: Start the agent from its own cell, drop it from the IPAM path (#48949, @HadrienPatte)
- iptables: parameterize endpoint NOTRACK rule description and report every rule failure (#48252, @vipul-21)
- k8s/resource: Make
WithTransformusable across a relist (#48639, @HadrienPatte) - k8s/resource: remove transitory
Resource[T]toTable[T]conversion mechanism (#48791, @tklauser) - k8s: gateway-api: extract test helpers in k8s testutils package (#48901, @MrFreezeex)
- LB IPAM now reports distinct reasons when a service cannot share a requested IP because of a selector, instead of one message covering three cases. (#48869, @lexfrei)
- network driver: add README file with example configurations (#47389, @bersoare)
- node: Deduplicate node IP accessors and split them per address family (#48611, @HadrienPatte)
- node: Migrate
Address.IPfromnet.IPtonetip(#48663, @HadrienPatte) - operator/clustermesh : decouple clustermesh cell from observers (#48767, @glrf)
- operator: Give the node watcher cells ownership of their state (#48931, @HadrienPatte)
- operator: Transform the Pod informer to the fields actually read (#48750, @HadrienPatte)
- Permit toggling host endpoint drop notifications through the API (#49117, @weizhoublue)
- pkg/ipam: Improved the error message from cloud multipool allocators when they can't assign an IP because the node is waiting to get more IPs assigned to it by the operator (#48717, @41ks)
- pkg/loadbalancer: convert net.IP to netip.Addr (#48724, @utaal)
- Policy & friends: Use labels.Labels rather than labels.LabelArray (#46231, @squeed)
- policy: Add realized BPF policymap assertions to script tests (#48463, @christarazi)
- Prepare cloud endpoint rules for reconciliation (#48382, @pippolo84)
- Prepare for release v1.21.0-pre.2 (#48584, @cilium-release-bot[bot])
- Prune unreachable BPF functions in the Cilium agent before loading the program (#48609, @ti-mo)
- README: Update APAC community meeting time (#48722, @ysksuzuki)
- README: Update releases (#48596, @aanm)
- README: Update releases (#48759, @joestringer)
- renovate: Bump envoy version for main and 1.20 (#49109, @sayboras)
- renovate: Bump envoy version for v1.19 (#49122, @sayboras)
- renovate: Disable platform commits for Go module updates (#48627, @aanm)
- renovate: Enable OSV vulnerability alerts on stable branches (#48803, @aanm)
- renovate: Group golangci-lint with the Go toolchain (#48743, @aanm)
- renovate: Migrate the last custom manager and fail on warnings (#48642, @aanm)
- Share the identity and endpoint list formatters with the script tests (#48972, @christarazi)
- signal: remove SignalCTFillUp (#47573, @julianwiedmann)
- vendor,treewide: Bump StateDB to v0.9.2 and fix NewKeySet usage (#48726, @joamaki)
Docker Manifests
cilium
quay.io/cilium/cilium:v1.21.0-pre.3@sha256:e84ef5f4725a3085ce2fa49dd0df2053147ed373cbfc7b7f8120e776590f6fdc
clustermesh-apiserver
quay.io/cilium/clustermesh-apiserver:v1.21.0-pre.3@sha256:364e95e6dca80941080c2db59c18b4369ba532cf130045115a0924e1aa85dffa
hubble-relay
quay.io/cilium/hubble-relay:v1.21.0-pre.3@sha256:177078c69eeea1ba6e141e157a9d954122f33ebc12fcf4cef3cd9c72b51fa91b
operator-alibabacloud
quay.io/cilium/operator-alibabacloud:v1.21.0-pre.3@sha256:4bbfc2ef9cad8ce65def2d4cc43e49798656abeaf6f5112ceabf18e694a75383
operator-aws
quay.io/cilium/operator-aws:v1.21.0-pre.3@sha256:becf24a2158cb71424d04377a2339dabb5f99f6033a92ff14815c36dd34e997a
operator-azure
quay.io/cilium/operator-azure:v1.21.0-pre.3@sha256:8d917c81f243ab7476907dd377d9e798faab59711309bea8e726f0eb223061b7
operator-generic
quay.io/cilium/operator-generic:v1.21.0-pre.3@sha256:59545836b8fbcb1f44655419b5f6160f31f415cb5ae9b9c8866aafc397c6ff8a
operator
quay.io/cilium/operator:v1.21.0-pre.3@sha256:fad1dcccb5400f2957e275f31201fe5820a13f3dac3c28f779cd53fec91c63d2