DirektorBani/DataSafeS3 v1.6.0

on GitHub

[1.6.0] - 2026-10-15

VERSION is 1.6.0. This commit does not push the git tag. Images are published when release.yml runs on v1.6.0.

Added

• Migration wizard, KMS dashboard row, and encryption guides. Settings → Migrate is three steps: the object-copy command, the IAM command, then a local evidence.json. Source credentials are still not sent or stored. The Grafana overview row KMS shows datasafe_kms_available by node, p50/p95 of datasafe_kms_request_duration_seconds, DEK cache hit and miss rates, and datasafe_kms_jobs_objects_total. Vault series stay unpublished until Vault is enabled. Guides: encryption EN · RU, threat model in kms-vault EN · RU. The AIStor Free comparison cites the vendor license and pricing pages.

• storage-cli migrate from-minio iam. Imports MinIO builtin users, groups, service accounts, and canned policies through the admin API. MinIO secrets are not copied. New passwords and access keys are written to a mode 0600 file, or --keep-access-keys imports an operator file. Unsupported policy conditions, LDAP, and OIDC are WARN. On a cluster target, IAM writes retry HTTP 503 not cluster leader and follow GET /healthz. The report records signed ListBucket checks. Settings → Migrate prints the command and does not store source credentials. POST /api/v1/users/{id}/access-keys imports a key for that user.

• storage-cli migrate from-minio. Copies buckets from a MinIO-compatible endpoint: versions oldest-first, delete markers, Object Lock, retention, legal hold, tags, user metadata, content-type, prefix lifecycle expiration, and bucket policy statements this server can evaluate. Resume state is JSONL (--state, default ./.minio-migrate-state). A repeated run does not write the same source version twice. HTTP 503 not cluster leader is retried; several --target values pick the leader from GET /healthz. The report is summary.md, evidence.json (verdicts PASS/FAIL), and evidence.zip. Settings → Migrate prints the object command, the IAM command, and opens a local evidence file. Source credentials are not sent to the console. Guides: EN · RU. ADR 0007.

• Key rotation, rewrap, and crypto-shred. POST /api/v1/kms/keys/{id}/rotate stores the next key version and increments generation. New objects use that version immediately. Encrypt existing objects is a leader-only job: plaintext objects are encrypted, and an older wrapped data key is rewrapped without reading the object bytes. Repeating the job does not change an object already at the target version. Destroy key asks for the bucket name, marks the key destroyed, and a job deletes wrapped data keys. A read of any object in the bucket, a share link, and a presigned GET then return HTTP 403 Ключ шифрования уничтожен, файл недоступен навсегда. A destroyed share publishes ShareDownloaded with result=key_destroyed. A successful share or presigned decrypt records key_id and key_version. Progress is GET /api/v1/kms/jobs/{id}; the console downloads the JSON report. The counter datasafe_kms_jobs_objects_total is registered when the first object finishes. Postgres migration 029_kms_jobs.

• Cluster KMS gate. bash scripts/ha/run-kms-cluster-test.sh (and scripts/ha/run-kms-cluster-test.ps1) runs three storage nodes, Vault dev with an AppRole per node, MinIO, and a webhook receiver behind a load balancer. Ports stay off :9000 and :9001. The job cluster-kms-gate fails when any verdict in evidence.json is FAIL. The console encryption specs run against that balancer. Single-node RSS and startup limits are unchanged.

• Vault Transit for SSE-KMS (opt-in). The HTTP client starts only when STORAGE_KMS_PROVIDER=vault-transit and an address are set (environment or Settings → Vault (KMS)). Empty values open no connection, start no poller, and do not register KMS metrics. There is no Vault SDK. SSE-S3 and plaintext buckets do not call Vault. Until Vault is configured, aws:kms still wraps with the local server key. A cache miss while Vault is down fails the SSE-KMS read or write with HTTP 503 KMS unavailable and does not store a partial object. A cached data key still decrypts until STORAGE_KMS_DEK_CACHE_TTL (default 5m). Nodes drop cache entries when kms_keys.generation changes (STORAGE_KMS_POLL_INTERVAL, default 2s). The secret id and token are sealed in metadata even when field encryption is off, and are not returned by the settings API, /healthz, or Activity. The console page tests the connection, masks a saved secret, and lists each node's Vault status. A share of an SSE-KMS object while Vault is down returns Файл временно недоступен, попробуйте позже. Alert DataSafeKMSUnavailable fires when datasafe_kms_available stays 0 for 1 minute; that series exists only while Vault is enabled. Postgres migration 026. Guides: EN · RU.

Release notes

• v1.5.0 could lose an object row on crash during an unversioned overwrite. Postgres ran DELETE FROM objects and the following INSERT as separate autocommit statements. A process killed between them left the file on disk and deleted the metadata. 1.6.0 keeps that replace in one transaction. The row from before the crash remains. A rewrap job no longer reports completed when the metadata count is short of the count taken at the start.

• Update every node before enabling encryption. Finish the rolling upgrade of every storage node to 1.6.0 before bucket encryption is turned on. A live 1.5.0 peer makes that change HTTP 409 cluster upgrade in progress on a 1.6.0 node. A 1.5.0 process has no such gate and accepts PutBucketEncryption itself, so the call must not be left on the last old node. A 1.5.0 process cannot read a format-1 object.

• Known limit, not fixed: a node serves its own file on S3 GET without a checksum. When the object file is already on this disk, GetObject returns those bytes and does not compare them with the stored ETag or size. A corrupted local copy is HTTP 200. A copy accepted from a peer is checked against the metadata ETag before it is stored. This local read path is not. See encryption.md.

• Accepted limit: a 1.5.0 SSE object changed before its first 1.6.0 read. Classification stores plaintext: true and GET returns the bytes on disk. Those bytes are ciphertext. Fail-closed applies after a classification has succeeded, and to format-1 objects. The first read of a large unmarked object authenticates the whole file before the body (about 11s per 2GB). Background classification does that work after startup. Progress is logged (legacy classification progress, and legacy classification reading object at 32MB or more) and exposed as datasafe_kms_legacy_classify_objects_total and datasafe_kms_legacy_classify_current_bytes. The series are registered at process start at zero (result plaintext, format0, skipped, or error, and current bytes 0) and increment as objects are classified. Vault metrics stay unpublished until Vault is used.

Fixed

• Unversioned Postgres overwrite no longer drops the row. This bug was in v1.5.0. Replacing an object that is not versioned ran DELETE FROM objects and INSERT as two autocommit statements. A crash between them, including a leader killed during rewrap, removed the metadata row while the file stayed on disk. A resumed job could then report completed with a short count and zero errors. 1.6.0 runs both statements in one transaction, so a crash rolls the replace back and the previous row remains. A job whose present count is below the count taken at the start finishes completed_with_errors instead of completed. The cluster gate repeats kill-during-rewrap and checks that every object is still readable.

• Peer object copy is an internal API. STORAGE_INTERNAL_ADDR serves GET and PUT /api/v1/ha/object-replica. The public port and the load balancer do not. The HMAC covers method, path, body hash, expiry, nonce, and destination node id. A code expires within 30 seconds and is not accepted twice. A copy whose bytes do not match the metadata ETag (and, for an unsealed object, the size) is refused. A read whose row exists and whose bytes cannot be fetched from a peer is HTTP 503, not NoSuchKey.

• A migration journal does not skip a missing object. Resume copies a source version again when the state file says it was copied and the target no longer has it.

• A follower serves a share of an object stored on a peer. Each storage node keeps its own object directory. After a write, the leader copies the stored bytes to peers that have advertised an address. A local read, including GET /api/v1/public/share/{token}/download, whose metadata row exists and whose file is missing copies those stored bytes from a peer and then decrypts them on this node. S3 GET and HEAD of an object are proxied to the leader, and the original Host is kept. Guides: EN · RU.

• Sharing and the 1.5.0 upgrade. Anonymous public-read of a plaintext or SSE-S3 object is HTTP 200, or 403 when block-public-access is on. Anonymous SSE-KMS is 403 even when Vault is down or the key is destroyed. A presigned GET and a share link of SSE-KMS are 200 with key_id in Activity and ShareDownloaded while Vault is up, 503 KMS unavailable when a cache miss hits a down Vault, and 403 Ключ шифрования уничтожен, файл недоступен навсегда after the key is destroyed. Sharing disabled, and a share from a user without access, are 403 on every encryption state and write ShareBlocked. The public share page shows the administrator sentence instead of the sharing_disabled code. In English that page shows “The encryption key was destroyed. This file is gone for good.” after crypto-shred. GET /api/v1/public/share/{token} is HTTP 503 when an SSE-KMS data key cannot be unwrapped, so the page does not say the file is ready. A chunked SSE object written by the v1.5.0 image is plaintext on GET when this image has the same server key, including when STORAGE_SSE_LEGACY_FORMAT0 is unset. That body is never returned as ciphertext. Each unmarked 1.5.0 object is classified once: authenticated decryption stores format 0, and any other body stores plaintext: true. Zip, gzip, jpeg, and random bytes written without SSE stay readable on Bolt and Postgres with or without the flag, and those reads stream. A format-0 object that has authenticated fails closed if it is changed later. A blob that has never authenticated is not failed closed. A changed byte in an object written by 1.6.0 is HTTP 500 cipher: message authentication failed before the success status, and the connection is closed. The console bucket-encryption PUT with provider vault-transit is HTTP 409 cluster_upgrade while a 1.5.0 peer is visible, including when Vault is configured. The flag does not force the format-0 reader onto every unmarked object. Bolt and Postgres upgrade from 1.5.0 keep plaintext objects, SSE-S3, and webhooks; a volume with no SSE-KMS object rolls back to 1.5.0. A 3-node rolling upgrade keeps plaintext reads working. A live 1.5.0 follower is seen from its ha_leader_lock insert (migration 030, ha_leader_attempts) without a cluster.nodes row. An unknown peer version is treated as older than 1.6.0. A PUT with no encryption header stays plaintext while that peer is live; SSE-S3 requested in that window is format 0, and SSE-KMS is HTTP 409. Bucket encryption on S3 and in the console is HTTP 409 cluster upgrade in progress until the last node is 1.6.0, then aws:kms can be enabled. notePublicRead uses the TCP peer unless STORAGE_TRUSTED_PROXIES names that peer. The single-node +10% RSS and +1s startup limits are unchanged. Guides: EN upgrade · RU upgrade.

• MinIO IAM import rights. migrate from-minio iam reads groups with GET /minio/admin/v3/group?group=. An Allow that cannot be enforced exactly is dropped. A Deny with an unsupported condition is stored with that condition removed, so it denies unconditionally. Inline service-account policies do not inherit broader parent allows. Disabled and expired service accounts do not receive a key; a future expiration is stored. GET / is HTTP 403 only when the key owns no bucket, has no grant, and no policy allows s3:ListAllMyBuckets. A console owner or a grant holder still lists those buckets, and also buckets a policy makes visible, even when another policy names the key. An imported key with nothing owned or granted and no such allow is HTTP 403, the same as MinIO, not an empty 200. Administrators, operators, the bootstrap owner, and the console bucket list are unchanged. A policy for a bucket that does not exist is WARN, is not probed, and does not fail the policies or access verdict. A user who cannot be given a key is a WARN, and a ListBucket allow with aws:SourceIp passes the access probe when this host is outside the condition. Bucket policy statements are deduplicated in canonical form, so a second run does not grow the document. A lost leader is rediscovered from --target, and each key is written to the keys file as it is issued. aws:SourceIp is the TCP peer; STORAGE_TRUSTED_PROXIES names balancer CIDRs that may supply X-Forwarded-For. The import does not replace bucket grants or an existing keys file. A second run issues keys that were missing and merges bucket policy statements. A member of a disabled group is denied unconditionally, because MinIO returns no policy for that user. The report lists group status and service-account inline, disabled, and expiry results. CI compares a signed allow/deny matrix with pgsty/silo MinIO and fails if DataSafe allows a request MinIO denies.

• MinIO migration acceptance. SigV4 rejects an x-amz-* header that is not in SignedHeaders (HTTP 400) and does not store that metadata. Content-Type may be unsigned, which is what minio-go and a browser presigned PUT send. Lifecycle Filter/Prefix is stored; a transition, noncurrent, tag, or date rule is rejected instead of becoming a whole-bucket expiration. Delete markers stay in chronological order. Object retention keeps COMPLIANCE on the object when the bucket default is GOVERNANCE. ListObjects, ListObjectsV2, and version listing page at max-keys (capped at 1000) with a continuation token. The copier lists each target bucket once, and a follower forwards writes to the leader address from the node heartbeat (STORAGE_ADVERTISE_URL, otherwise the hostname and port). The promote-drill and feature-audit clients sign Content-Type on a body, which the HTTP library adds even for an empty CreateBucket. An ETag that differs, including a multipart ETag and an SSE-KMS ETag that is not the plaintext MD5, is checked with SHA-256. Source bucket encryption is not copied, and an empty source configuration does not turn off encryption the target already has. --verify-sample checks at least one object, secrets can come from the environment or a file, and the settings page shows a PowerShell command and per-bucket counts.

• Stock aws-cli 2 uploads. CRC64NVME uses the reflected polynomial 0x9A6C9329AC4BC9B5 (crc64.MakeTable). The check value of 123456789 is ae8b14860a799888. Trailers for CRC32, CRC32C, SHA-1, SHA-256, and CRC64NVME are checked. CI runs s3 cp of a small object, a 20MiB multipart object, and s3 sync with aws-cli 2 defaults and no checksum flags.

• CreateBucket keeps a bucket the caller already owns. The response is HTTP 409 BucketAlreadyOwnedByYou. Encryption, versioning, and lock configuration are not replaced. The migration encryption verdict fails when a target bucket that had default encryption loses it, or a copied object is stored without x-amz-server-side-encryption.

• Multipart verify on Postgres. CreateMultipartUpload user metadata and content type are stored (migration 028). A multipart ETag that does not match is checked by downloading the object and comparing SHA-256, including when x-amz-meta-datasafe-source-version is absent.

• ListMultipartUploads and Object Lock versioning. GET ?uploads lists an in-progress upload, including when the bucket storage key is not the logical name. A bucket with Object Lock reports versioning Enabled, including one created with x-amz-bucket-object-lock-enabled: true. Suspending that versioning is HTTP 409 InvalidBucketState.

• Job status and Vault connection test. A finished job with errors greater than zero is completed_with_errors, not completed. A failed Vault delete increments that count on the destroy response and on the stored job, with a single vault delete: prefix. The share page shows the destroyed-key sentence without an empty red box. The connection test does not call delete on datasafe-connection-test.

• Encrypt-existing and crypto-shred follow-up. Destroy is HTTP 403 for a bucket owner whose role is user and for an operator. A cluster administrator, a tenant administrator, and an explicit bucket-admin grant may destroy the key; MFA step-up still applies when that policy is on. GET /api/v1/public/share/{token} after destroy is HTTP 403 Ключ шифрования уничтожен, файл недоступен навсегда. Encrypt-existing writes Activity kms_job_started, kms_job_finished, and kms_job_failed (who, bucket, counts). holder_id is the node id. Plaintext objects are encrypted as a stream. A failed Vault Transit delete is stored on the job as vault_delete_error and shown in the console; new Transit keys are created with deletion_allowed. A metadata write that hits SQLSTATE 25006 or a dropped Postgres connection during a leader change is HTTP 503 not cluster leader with Retry-After, not HTTP 500. Encrypting an existing plaintext object changes its ETag. A local destroy erases wrapped data keys in the current metadata only; an older metadata backup still has them.

• Single-node gate baseline. Bolt idle RSS and startup are medians of interleaved runs of this build against ghcr.io/direktorbani/datasafe-storage-server:v1.5.0 on the same runner. Postgres is measured the same way, against that v1.5.0 image with STORAGE_METADATA_BACKEND=postgres, on a fresh database each run. The container is queried only after its init process has finished and SELECT 1 succeeds. Each comparison fails above baseline +10% RSS or baseline +1s startup.

• aws-chunked bodies are stored decoded. STREAMING-AWS4-HMAC-SHA256-PAYLOAD, STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER, and STREAMING-UNSIGNED-PAYLOAD-TRAILER are unwrapped before the object or part is written. Chunk signatures are checked. x-amz-decoded-content-length is the stored size. A trailing checksum that does not match is HTTP 400 BadDigest. GET ?location returns the bucket region. A client that reads it before PUT, including mc, no longer treats the object listing as a region.

• Object Lock on Postgres. Retention mode is stored without a NUL in the jsonb document, so a lock, retention, or legal hold no longer fails with unsupported Unicode escape sequence (SQLSTATE 22P05). The newest row in ListObjectVersions is IsLatest, including a delete marker. Rewriting an older version for retention does not take that flag away.

• Vault secret id and token are sealed and redacted. GET /api/v1/settings/system no longer returns kms.secret_id or kms.token. Those values are AES-GCM (kse:v1) under the server key, or the JWT secret when STORAGE_SSE_MASTER_KEY is empty, even when STORAGE_FIELD_ENCRYPTION_ENABLED is false. Field encryption still stores them as enc:v1. Settings, Activity export, /healthz, and a metadata backup do not contain the plaintext. PUT /api/v1/settings/system cannot replace them.

• A Postgres follower loads a Vault save within 2 seconds. Shared settings are reread on the 2 second poll, not only on the 30 second heartbeat. A Bolt node does not run that poll. Until the provider and address are set, the poll does not open a client or register KMS metrics.

• Save rejects Vault credentials that cannot encrypt. PUT /api/v1/kms/settings probes encrypt and decrypt first and returns HTTP 400, or 408 on timeout, without storing the body. The console Save button stays disabled until Test connection succeeds for the current form. The probe encrypts STORAGE_KMS_PROBE_KEY or STORAGE_KMS_DEFAULT_KEY when one is set. Otherwise it creates datasafe-connection-test if missing and does not delete it. The node policy does not need delete.

• Console download and /healthz follow Vault. An object download from the console returns HTTP 503 KMS unavailable when Vault is down. kms.available follows the latest probe, so a recovered node does not stay false. Standby health codes 429, 472, and 473 still count as up when login works.

• A Vault token that cannot be renewed still connects. vault server -dev answers renew-self for its root token with HTTP 400 token not renewable. That token is kept. HTTP 403, and any other HTTP 400, still reject it.

• Plaintext objects survive adding STORAGE_SSE_MASTER_KEY. 1.6.0 no longer marks every existing object as SSE-S3 format 0 just because a master key is set and the bucket is not empty. Unmarked plaintext is returned as stored, including zip, gzip, jpeg, and other binary objects. A format-0 blob that authenticates with the server key is marked format 0 and returned as plaintext, including when STORAGE_SSE_LEGACY_FORMAT0 is unset. A body that does not authenticate is marked plaintext and streamed. The variable does not force the format-0 reader. A legacy flag written by the first 1.6.0 builds, which inferred it from the object count, is cleared on the next start unless that variable is set. The key fingerprint is unchanged: a different key still refuses to start.

• Idle process does not start KMS. With no STORAGE_SSE_MASTER_KEY on Bolt, startup does not create kms_state and does not run the cluster heartbeat. A key that was set earlier and then cleared still refuses to start. The error says the key is missing and to set STORAGE_SSE_MASTER_KEY to the original key. A different key still reports SSE master key fingerprint mismatch. The heartbeat a node does write is at least 1.6.0, so an image stamped from the 1.5.0 VERSION file does not treat itself as a mixed-cluster peer. Postgres and HA nodes write that heartbeat even when the key is empty, so a live 1.5.0 peer can be told apart from them.

• Encryption stays off while a live 1.5.0 node is in the cluster. 1.5.0 does not write cluster_node_heartbeats. HTTP 409 cluster_upgrade covers an unexpired ha_leader_lock whose holder has no 1.6.0 heartbeat, a recent ha_leader_attempts row with no 1.6.0 heartbeat (migration 030), and a cluster-registry node whose GET /healthz version is below 1.6.0. A follower insert that does not win the lease is still recorded. An unknown version is treated as older than 1.6.0. cluster.nodes is not required. On Postgres or HA the console requires the checkbox “All nodes are upgraded to 1.6.0” before enabling. S3 PutBucketEncryption has no checkbox. A 1.5.0 leader accepts PutBucketEncryption with its own code when the call reaches that process, so upgrade every node first.

Container images

• ghcr.io/direktorbani/datasafe-storage-server:v1.6.0
• ghcr.io/direktorbani/datasafe-console:v1.6.0

CycloneDX SBOM files and cosign signatures are attached. See SECURITY.md for cosign verify instructions.