remix-run/remix file-storage-s3@1.1.0

file-storage-s3 v1.1.0

on GitHub

Minor Changes

• storage.set() now streams files to S3 instead of reading the whole file into memory first. Files larger than one part are sent with an S3 multipart upload, so memory use stays at about two parts per upload no matter how large the file is. Smaller files are still sent with a single PUT. Any File-shaped object works, including streaming sources that can only be read with stream() and don't know their size up front, which are sent to S3 as they're read. (#12070)

  Use the new partSize option to set the part size, which defaults to 8 MiB and must be at least 5 MiB, the S3 minimum. If a multipart upload fails, it is aborted so S3 does not keep the uploaded parts. Consider also adding a bucket lifecycle rule that removes incomplete multipart uploads, in case a process exits before it can abort one.

  let storage = createS3FileStorage({
    accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
    bucket: 'my-app-uploads',
    region: 'us-east-1',
    partSize: 16 * 1024 * 1024,
  })

  storage.put() uses the same upload path, but it still holds the whole file in memory because it returns a File with the stored contents. Use set() for large files.