S3RemoteStore

Store the encrypted DDO envelope in an S3 bucket — AWS S3, Exoscale SOS, MinIO or any other S3-compatible store.

Usage

Exoscale SOS, with both key pairs from the environment:

const store = new S3RemoteStore({ 
  endpoint: 'https://sos-de-fra-1.exo.io', // https://sos-<zone>.exo.io
  region: 'de-fra-1', // the zone
  bucket: 'my-ddos', 
  prefix: 'ddo/', 
  writeCredentials: { 
    accessKeyId: process.env.S3_WRITE_KEY!, 
    secretAccessKey: process.env.S3_WRITE_SECRET!
  }, 
  readCredentials: { 
    accessKeyId: process.env.S3_READ_KEY!, 
    secretAccessKey: process.env.S3_READ_SECRET!
  } 
}) 

A MinIO server that you reach on 127.0.0.1:9000 and the node reaches as minio:9000. endpoint may be plain http:// here because it is a loopback host; nodeEndpoint may always be http://, because only the node connects to it:

const store = new S3RemoteStore({
  endpoint: 'http://127.0.0.1:9000', // where you upload
  nodeEndpoint: 'http://minio:9000', // what the node is told
  forcePathStyle: true, // MinIO uses path-style addressing
  bucket: 'ddos',
  prefix: 'ddo/',
  writeCredentials: {
    accessKeyId: process.env.S3_WRITE_KEY!,
    secretAccessKey: process.env.S3_WRITE_SECRET!
  },
  readCredentials: {
    accessKeyId: process.env.S3_READ_KEY!,
    secretAccessKey: process.env.S3_READ_SECRET!
  }
})

It needs no node client and no AWS SDK: requests are signed with SigV4 over fetch and WebCrypto. See remote stores for the setup checklist.

Parameters

options

• Type: S3RemoteStoreOptions

Both credential options are S3Credentials, { accessKeyId: string; secretAccessKey: string }.

The constructor throws when:

• a required option is missing or blank (naming it);
• the read and write access key ids are the same, unless allowSharedCredentials is set;
• endpoint is plain http:// on a non-loopback host, unless allowInsecureTransport is set;
• endpoint or nodeEndpoint has a scheme other than http:// or https://;
• endpoint or nodeEndpoint carries a user name or password (user:password@host). Requests are signed with the key pairs and sent to the host alone, and the endpoint would otherwise reach the pointer (PublishResponse.stored.pointer included, where only the secret key is redacted) and error messages. The message does not repeat the URL;
• requestTimeoutMs is not a finite number of milliseconds above 0;
• bucket breaks the S3 naming rules (it goes into the request URL), or contains dots while endpoint is https:// with virtual-host addressing: bucket.with.dots.host does not match the endpoint's certificate, so set forcePathStyle: true;
• prefix has an empty, . or .. segment, starts with / or contains a backslash (it may end in /). S3 does not normalise keys but URL handling does, so the upload and the node's read would reach different objects;
• endpoint or nodeEndpoint is an IP address or localhost and the matching addressing style is virtual-host, which cannot work there (bucket.127.0.0.1). The message names the option to set: forcePathStyle: true, or nodeForcePathStyle: true.

put

Uploads the envelope with the write key to <prefix><DID hash>/<envelope sha256>.json and returns an S3Pointer, the same shape as ocean.js's S3FileObject:

const pointer: S3Pointer = {
  type: 's3',
  s3Access: {
    endpoint: 'http://minio:9000', // nodeEndpoint, or endpoint, with a lowercase scheme
    region: 'us-east-1',
    bucket: 'ddos',
    objectKey: 'ddo/<DID hash>/<envelope sha256>.json',
    accessKeyId: '<read key>', // readCredentials, never the write key
    secretAccessKey: '<read secret>', 
    forcePathStyle: true // nodeForcePathStyle, or forcePathStyle
  }
}

The <DID hash> is the hex part of the did:ope: DID. Object keys are deterministic, and an edit writes a sibling object instead of overwriting: the object an on-chain pointer refers to never changes, so its hash keeps matching, and a failed edit leaves the live version intact.

check

Called by publish, completePublish and edit before their first transaction. It:

1. uploads a small probe with the write key, at a random key shaped like a real object, <prefix>nautilus-store-check/<64 random hex>.json, so a policy that grants access only to <prefix>*/* is probed as well;
2. reads it back with the read key, as the node will (though through endpoint, not nodeEndpoint), and fails if the content differs;
3. makes sure the read key can neither PUT nor DELETE: it tries both with the read key, the PUT at another such key. Only an explicit 403 AccessDenied counts as "cannot"; any other answer, a bare 403 from a proxy included, fails the check, because it says nothing about the key. If the read key can write or delete, the check fails — or, with allowWritableReadKey, warns. Skipped when both keys are the same (allowSharedCredentials);
4. deletes every probe it wrote with the write key, also when a step above failed. A failed cleanup never hides the first error: both are thrown together as an AggregateError. A cleanup failure on its own fails the check too, since remove() would not work either. A probe that is already gone (404) counts as cleaned up.

check() does not try version-level actions. On a versioned bucket, a read key allowed s3:DeleteObjectVersion (or to change version ACLs or retention) passes it, so make sure its policy grants s3:GetObject and nothing that writes. Deleting a probe there leaves its version behind a delete marker; a lifecycle rule on <prefix>nautilus-store-check/ removes them.

S3 errors are turned into actionable messages: a signature mismatch (check the secret, region and forcePathStyle), an unknown access key, a bucket in another region (naming it, when S3 does), a redirect (not followed with a signed request; it means region or endpoint does not match the bucket), a missing bucket, a missing object, or access denied with the permission the key needs (s3:GetObject for the read key, s3:PutObject and s3:DeleteObject for the write key). Network errors and timeouts name the method, bucket, key, the key's role and the origin.

verify

await store.verify(pointer, expectedHash) 

Called by publish, completePublish and edit right before the metadata transaction. It reads the stored envelope back with the read key, which also proves the key in the pointer can read the object, and throws unless it hashes, the way the node hashes it, to the on-chain metadata hash. It reads through endpoint, not nodeEndpoint. Only pointers into this store's bucket and under its prefix are accepted.

remove

const { stored } = await nautilus.publish(asset)
 
// later, once the asset is revoked for good: a live asset needs this creation envelope
await store.remove(stored.pointer) 

Deletes the object behind a pointer this store returned, with the write key — for example PublishResponse.stored.pointer, whose secret is redacted (only bucket and objectKey are used), or an older version after an edit. Throws for anything but an s3 pointer into this store's bucket whose objectKey has the exact shape put() writes, <prefix><DID hash>/<sha256>.json, and for an object key with an empty, . or .. segment, a leading / or a backslash. An object that is already gone (404 NoSuchKey) counts as removed.

nautilus calls it itself only for an envelope whose metadata transaction was never sent, or mined and reverted; see RemoteStore.remove.

In the browser

Every request is a signed fetch with authorization, content-type, x-amz-content-sha256 and x-amz-date headers, so the browser sends a CORS preflight and the bucket must allow your app's origin for GET, PUT and DELETE with those headers. The host header is signed too; browsers set it themselves, to the same value, so the signature still matches. A minimal rule set for AWS S3:

{
  "CORSRules": [
    {
      "AllowedOrigins": ["https://app.example.com"],
      "AllowedMethods": ["GET", "PUT", "DELETE"],
      "AllowedHeaders": ["authorization", "content-type", "x-amz-content-sha256", "x-amz-date"],
      "MaxAgeSeconds": 3600
    }
  ]
}

See browser use for how to apply it on AWS, Exoscale SOS and MinIO, and why the write key must not ship in a public web app.