Add presign support in OpenDAL so users can generate a pre-signed URL without leaking serect_key.


By default, all S3 objects are private. Only the object owner has permission to access them. However, the object owner can optionally share objects with others by creating a presigned URL, using their own security credentials, to grant time-limited permission to download the objects.

From Sharing objects using presigned URLs

We can use this presigned URL for:

  • Download the object within the expired time from a bucket directly
  • Upload content to the bucket on client-side

Adding this feature in OpenDAL will make users' lives easier to generate presigned URLs across different storage services.

The whole process would be:

Guide-level explanation

With this feature, our users can:

Generate presigned URL for downloading

let req = op.presign_read("path/to/file")?;
// req.method: GET
// req.url:<signature-value>  

Users can download this object directly from the s3 bucket. For example:

curl <generated_url> -O test.txt

Generate presigned URL for uploading

let req = op.presign_write("path/to/file")?;
// req.method: PUT
// req.url:<signature-value>  

Users can upload content directly to the s3 bucket. For example:

curl -X PUT <generated_url> -T "/tmp/test.txt"

Reference-level explanation

Accessor will add a new API presign:

pub trait Accessor {
    fn presign(&self, args: &OpPresign) -> Result<PresignedRequest> {..}

presign accepts OpPresign and returns Result<PresignedRequest>:

struct OpPresign {
    path: String,
    op: Operation,
    expire: time::Duration,

struct PresignedRequest {}

impl PresignedRequest {
    pub fn method(&self) -> &http::Method {..}
    pub fn url(&self) -> &http::Uri {..}

We are building a new struct to avoid leaking underlying implementations like hyper::Request<T> to users.

This feature will be a new capability in AccessorCapability as described in RFC-0409: Accessor Capabilities

Based on Accessor::presign, we will export public APIs in Operator:

impl Operator {
    fn presign_read(&self, path: &str) -> Result<PresignedRequest> {}
    fn presign_write(&self, path: &str) -> Result<PresignedRequest> {}

Although it's possible to generate URLs for create, delete, stat, and list, there are no obvious use-cases. So we will not add them to this proposal.



Rationale and alternatives

Query Sign Support Status

Prior art

awscli presign

AWS CLI has native presign support

> aws s3 presign s3://DOC-EXAMPLE-BUCKET/test2.txt

Refer to AWS CLI Command Reference for more information.

Unresolved questions


Future possibilities

  • Add stat/list/delete support