Module sds-object

The sds-object module provides S3-compatible object storage in a Deckhouse Kubernetes Platform (DKP) cluster. A user creates a Bucket resource and gets an S3 bucket without deploying or operating the storage themselves.

An administrator configures storage based on SeaweedFS or Ceph RGW and creates one or more ObjectStore classes for users. A user only needs to pick a suitable class — configuring the storage itself is not required.

Main Features

The module provides the following capabilities:

• Self-service creation of S3 buckets using the Bucket resource.
• Separate ReadOnly or ReadWrite credentials using BucketAccess. Credentials are rotated with the storage.deckhouse.io/rotate annotation; the BucketAccess resource does not need to be recreated.
• Multiple ObjectStore classes with different storage backends, parameters, and quotas. Limits can be set at the class or individual bucket level.
• Data retention after Bucket deletion with reclaimPolicy: Retain, with access restored when a Bucket with the same name is created again in the same namespace.
• Public read access through accessPolicy: PublicRead, versioning, lifecycle, Object Lock, and server-side object encryption.
• Access to buckets from outside the cluster through a published S3 endpoint.
• Resource status monitoring. Current status and error reasons are available in status, including conditions. For storage resources, status also includes capacity usage and data-integrity check results.

How the module works

Bucket provisioning and access work as follows:

1. An administrator creates a SeaweedFSStore or SDSElasticStore and configures the storage.
2. The administrator creates an ObjectStore, links it to the storage through storeRef, and defines default bucket parameters.
3. A user creates a Bucket and specifies the ObjectStore class in spec.objectStoreRef.
4. The module creates a related BucketContents resource and an S3 bucket in the selected storage. BucketContents records the created S3 bucket and the storage where its data resides.
5. The user creates a BucketAccess. The module creates a Secret containing the S3 endpoint, bucket name, and credentials.

When a Bucket is deleted, data handling is determined by reclaimPolicy. Retain preserves the data and BucketContents, while Delete removes the bucket and its data.

Changing or deleting an ObjectStore class does not change the storage used by an existing Bucket: the reference to the actual storage is retained in BucketContents.

See Usage for a step-by-step example.

Requirements

The module and selected storage must meet the following requirements:

• Deckhouse Kubernetes Platform version 1.72 or later.
• Storage based on Ceph RGW requires the sds-elastic module and a ready ElasticCluster.
• Storage based on SeaweedFS does not require a separate storage module.
• Multiple SeaweedFS filers require a shared PostgreSQL metadata store. You can use PostgreSQL managed by the managed-postgres module (metadataStore: Postgres) or an external PostgreSQL instance (metadataStore: External). metadataStore: LevelDB supports only one filer.

Limitations

The module has the following limitations:

• Binding an existing S3 bucket is not supported: a Bucket resource always provisions a new bucket.
• Sharing one bucket across multiple namespaces is not supported.
• quota.maxObjects is not enforced by SeaweedFS; quota.maxSize is enforced by both backends, while Ceph RGW also enforces the object-count limit.
• Object Lock can only be configured when a Bucket is created. It requires versioning to be enabled and cannot be used with reclaimPolicy: Delete.
• Key parameters that cannot be changed after creation include:
  • Bucket.spec.objectStoreRef and Bucket.spec.objectLock;
  • BucketAccess.spec.bucketRef;
  • ObjectStore.spec.storeRef;
  • SeaweedFSStore.spec.masters, storage.class, replication, metadataStore, and reclaimPolicy;
  • SDSElasticStore.spec.elasticClusterRef and reclaimPolicy. After server-side encryption is enabled in ServerManaged mode, it also cannot be disabled for an existing storage resource.
• SeaweedFSStore.spec.volumeServers can only be increased: removing a volume server removes its PersistentVolumeClaim (PVC) together with the objects stored on it; the data is not drained first.
• SeaweedFSStore.spec.replication accepts only the 00z form: all volume servers run in one Kubernetes cluster with the default SeaweedFS topology, so copies cannot be placed in another rack or data center.
• Ceph RGW metadata pools (SDSElasticStore.spec.metadataPool) are always replicated: erasure-coded pools do not support the omap operations required by RGW metadata.
• Disabling the module while managed storage and buckets still exist may make data unavailable or result in data loss.