Struct opendal::ObjectReader

source ·
pub struct ObjectReader { /* private fields */ }
Expand description

ObjectReader is the public API for users.

Usage

ObjectReader implements the following APIs:

  • AsyncRead
  • AsyncSeek
  • Stream<Item = <io::Result<Bytes>>>

For reading data, we can use AsyncRead and Stream. The mainly different is where the copy happend.

AsyncRead requires user to prepare a buffer for ObjectReader to fill. And Stream will stream out a Bytes for user to decide when to copy it’s content.

For example, users may have their only CPU/IO bound workers and don’t want to do copy inside IO workers. They can use Stream to get a Bytes and consume it in side CPU workers inside.

Besides, Stream COULD reduce an extra copy if underlying reader is stream based (like services s3, azure which based on HTTP).

Notes

All implementions of ObjectReader should be zero cost. In our cases, which means others must pay the same cost for the same feature provide by us.

For examples, call read without seek should always act the same as calling read on plain reader.

Read is Seekable

We use internal AccessorHint::ReadIsSeekable to decide the most suitable implementations.

If there is a hint that ReadIsSeekable, we will open it with given args directy. Otherwise, we will pick a seekable reader implementation based on input range for it.

  • Some(offset), Some(size) => RangeReader
  • Some(offset), None and None, None => OffsetReader
  • None, Some(size) => get the total size first to convert as RangeReader

No matter which reader we use, we will make sure the read operation is zero cost.

Read is Streamable

We use internal AccessorHint::ReadIsStreamable to decide the most suitable implementations.

If there is a hint that ReadIsStreamable, we will use existing reader directly. Otherwise, we will use transform this reader as a stream.

Consume instead of Drop

Normally, if reader is seekable, we need to drop current reader and start a new read call.

We can consume the data if the seek position is close enough. For example, users try to seek to Current(1), we can just read the data can consume it.

In this way, we can reduce the extra cost of dropping reader.

Trait Implementations§

Attempt to read from the AsyncRead into buf. Read more
Attempt to read from the AsyncRead into bufs using vectored IO operations. Read more
Attempts to read from the AsyncRead into buf. Read more
Attempt to seek to an offset, in bytes, in a stream. Read more
Attempts to seek to an offset, in bytes, in a stream. Read more
Waits for a seek operation to complete. Read more
Read bytes asynchronously.
Seek asynchronously. Read more
Stream [Bytes] from underlying reader. Read more
Values yielded by the stream.
Attempt to pull out the next value of this stream, registering the current task for wakeup if the value is not yet available, and returning None if the stream is exhausted. Read more
Returns the bounds on the remaining length of the stream. Read more

Auto Trait Implementations§

Blanket Implementations§

Gets the TypeId of self. Read more
Creates an adaptor which will chain this stream with another. Read more
Tries to read some bytes directly into the given buf in asynchronous manner, returning a future type. Read more
Creates a future which will read from the AsyncRead into bufs using vectored IO operations. Read more
Creates a future which will read exactly enough bytes to fill buf, returning an error if end of file (EOF) is hit sooner. Read more
Creates a future which will read all the bytes from this AsyncRead. Read more
Creates a future which will read all the bytes from this AsyncRead. Read more
Helper method for splitting this read/write object into two halves. Read more
Creates an AsyncRead adapter which will read at most limit bytes from the underlying reader. Read more
Creates a new AsyncRead instance that chains this stream with next. Read more
Pulls some bytes from this source into the specified buffer, returning how many bytes were read. Read more
Pulls some bytes from this source into the specified buffer, advancing the buffer’s internal cursor. Read more
Reads the exact number of bytes required to fill buf. Read more
Reads an unsigned 8 bit integer from the underlying reader. Read more
Reads a signed 8 bit integer from the underlying reader. Read more
Reads an unsigned 16-bit integer in big-endian order from the underlying reader. Read more
Reads a signed 16-bit integer in big-endian order from the underlying reader. Read more
Reads an unsigned 32-bit integer in big-endian order from the underlying reader. Read more
Reads a signed 32-bit integer in big-endian order from the underlying reader. Read more
Reads an unsigned 64-bit integer in big-endian order from the underlying reader. Read more
Reads an signed 64-bit integer in big-endian order from the underlying reader. Read more
Reads an unsigned 128-bit integer in big-endian order from the underlying reader. Read more
Reads an signed 128-bit integer in big-endian order from the underlying reader. Read more
Reads an 32-bit floating point type in big-endian order from the underlying reader. Read more
Reads an 64-bit floating point type in big-endian order from the underlying reader. Read more
Reads an unsigned 16-bit integer in little-endian order from the underlying reader. Read more
Reads a signed 16-bit integer in little-endian order from the underlying reader. Read more
Reads an unsigned 32-bit integer in little-endian order from the underlying reader. Read more
Reads a signed 32-bit integer in little-endian order from the underlying reader. Read more
Reads an unsigned 64-bit integer in little-endian order from the underlying reader. Read more
Reads an signed 64-bit integer in little-endian order from the underlying reader. Read more
Reads an unsigned 128-bit integer in little-endian order from the underlying reader. Read more
Reads an signed 128-bit integer in little-endian order from the underlying reader. Read more
Reads an 32-bit floating point type in little-endian order from the underlying reader. Read more
Reads an 64-bit floating point type in little-endian order from the underlying reader. Read more
Reads all bytes until EOF in this source, placing them into buf. Read more
Reads all bytes until EOF in this source, appending them to buf. Read more
Creates an adaptor which reads at most limit bytes from it. Read more
Reads some bytes from the byte stream. Read more
Like [read()][AsyncReadExt::read()], except it reads into a slice of buffers. Read more
Reads the entire contents and appends them to a Vec. Read more
Reads the entire contents and appends them to a String. Read more
Reads the exact number of bytes required to fill buf. Read more
Creates an adapter which will read at most limit bytes from it. Read more
Converts this [AsyncRead] into a [Stream] of bytes. Read more
Creates an adapter which will chain this stream with another. Read more
Boxes the reader and changes its type to dyn AsyncRead + Send + 'a. Read more
Creates a future which will seek an IO object, and then yield the new position in the object and the object itself. Read more
Creates a future which will return the current seek position from the start of the stream. Read more
Creates a future which will seek an IO object, and then yield the new position in the object and the object itself. Read more
Creates a future which will rewind to the beginning of the stream. Read more
Creates a future which will return the current seek position from the start of the stream. Read more
Seeks to a new position in a byte stream. Read more
Immutably borrows from an owned value. Read more
Mutably borrows from an owned value. Read more
Applies the [Compat] adapter by value. Read more
Applies the [Compat] adapter by shared reference. Read more
Applies the [Compat] adapter by mutable reference. Read more
Convert a Stream of dns response into a Future yielding the first answer, discarding others if any. Read more

Returns the argument unchanged.

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Instruments this type with the current Span, returning an Instrumented wrapper. Read more

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

The alignment of pointer.
The type for initializers.
Initializes a with the given initializer. Read more
Dereferences the given pointer. Read more
Mutably dereferences the given pointer. Read more
Drops the object pointed to by the given pointer. Read more
Reads some bytes from the byte stream. Read more
Like read, except that it reads into a slice of buffers. Read more
Reads all bytes from the byte stream. Read more
Reads all bytes from the byte stream and appends them into a string. Read more
Reads the exact number of bytes required to fill buf. Read more
Creates an adaptor which will read at most limit bytes from it. Read more
Creates a “by reference” adaptor for this instance of Read. Read more
Transforms this Read instance to a Stream over its bytes. Read more
Creates an adaptor which will chain this stream with another. Read more
Should always be Self
Seeks to a new position in a byte stream. Read more
Creates a future that resolves to the next item in the stream. Read more
Converts this stream into a future of (next_item, tail_of_stream). If the stream terminates, then the next item is None. Read more
Maps this stream’s items to a different type, returning a new stream of the resulting type. Read more
Creates a stream which gives the current iteration count as well as the next value. Read more
Filters the values produced by this stream according to the provided asynchronous predicate. Read more
Filters the values produced by this stream while simultaneously mapping them to a different type according to the provided asynchronous closure. Read more
Computes from this stream’s items new items of a different type using an asynchronous closure. Read more
Transforms a stream into a collection, returning a future representing the result of that computation. Read more
Converts a stream of pairs into a future, which resolves to pair of containers. Read more
Concatenate all items of a stream into a single extendable destination, returning a future representing the end result. Read more
Drives the stream to completion, counting the number of items. Read more
Repeats a stream endlessly. Read more
Execute an accumulating asynchronous computation over a stream, collecting all the values into one final result. Read more
Execute predicate over asynchronous stream, and return true if any element in stream satisfied a predicate. Read more
Execute predicate over asynchronous stream, and return true if all element in stream satisfied a predicate. Read more
Flattens a stream of streams into just one continuous stream. Read more
Flattens a stream of streams into just one continuous stream. Polls inner streams concurrently. Read more
Maps a stream like [StreamExt::map] but flattens nested Streams. Read more
Maps a stream like [StreamExt::map] but flattens nested Streams and polls them concurrently, yielding items in any order, as they made available. Read more
Combinator similar to [StreamExt::fold] that holds internal state and produces a new stream. Read more
Skip elements on this stream while the provided asynchronous predicate resolves to true. Read more
Take elements from this stream while the provided asynchronous predicate resolves to true. Read more
Take elements from this stream until the provided future resolves. Read more
Runs this stream to completion, executing the provided asynchronous closure for each element on the stream. Read more
Runs this stream to completion, executing the provided asynchronous closure for each element on the stream concurrently as elements become available. Read more
Creates a new stream of at most n items of the underlying stream. Read more
Creates a new stream which skips n items of the underlying stream. Read more
Fuse a stream such that poll_next will never again be called once it has finished. This method can be used to turn any Stream into a FusedStream. Read more
Borrows a stream, rather than consuming it. Read more
Catches unwinding panics while polling the stream. Read more
Wrap the stream in a Box, pinning it. Read more
Wrap the stream in a Box, pinning it. Read more
An adaptor for creating a buffered list of pending futures. Read more
An adaptor for creating a buffered list of pending futures (unordered). Read more
An adapter for zipping two streams together. Read more
Adapter for chaining two streams. Read more
Creates a new stream which exposes a peek method. Read more
An adaptor for chunking up items of the stream inside a vector. Read more
An adaptor for chunking up ready items of the stream inside a vector. Read more
A future that completes after the given stream has been fully processed into the sink and the sink has been flushed and closed. Read more
Splits this Stream + Sink object into separate Sink and Stream objects. Read more
Do something with each item of this stream, afterwards passing it on. Read more
Wrap this stream in an Either stream, making it the left-hand variant of that Either. Read more
Wrap this stream in an Either stream, making it the right-hand variant of that Either. Read more
A convenience method for calling [Stream::poll_next] on Unpin stream types. Read more
Returns a Future that resolves when the next item in this stream is ready. Read more
Advances the stream and returns the next value. Read more
Creates a stream that yields its first n elements. Read more
Creates a stream that yields elements based on a predicate. Read more
Creates a stream that yields each stepth element. Read more
Takes two streams and creates a new stream over both in sequence. Read more
Creates an stream which copies all of its elements. Read more
Creates an stream which copies all of its elements. Read more
Creates a stream that yields the provided values infinitely and in order. Read more
Creates a stream that gives the current element’s count as well as the next value. Read more
Takes a closure and creates a stream that calls that closure on every element of this stream. Read more
A combinator that does something with each element in the stream, passing the value on. Read more
Returns the last element of the stream. Read more
Creates a stream which ends after the first None. Read more
Creates a stream that uses a predicate to determine if an element should be yielded. Read more
Both filters and maps a stream. Read more
Returns the element that gives the minimum value with respect to the specified key function. If several elements are equally minimum, the first element is returned. If the stream is empty, None is returned. Read more
Returns the element that gives the maximum value with respect to the specified key function. If several elements are equally maximum, the first element is returned. If the stream is empty, None is returned. Read more
Returns the element that gives the minimum value with respect to the specified comparison function. If several elements are equally minimum, the first element is returned. If the stream is empty, None is returned. Read more
Returns the element that gives the maximum value. If several elements are equally maximum, the first element is returned. If the stream is empty, None is returned. Read more
Returns the element that gives the minimum value. If several elements are equally minimum, the first element is returned. If the stream is empty, None is returned. Read more
Returns the element that gives the maximum value with respect to the specified comparison function. If several elements are equally maximum, the first element is returned. If the stream is empty, None is returned. Read more
Returns the nth element of the stream. Read more
Tests if every element of the stream matches a predicate. Read more
Searches for an element in a stream that satisfies a predicate. Read more
Applies function to the elements of stream and returns the first non-none result. Read more
A combinator that applies a function to every element in a stream producing a single, final value. Read more
Call a closure on each element of the stream. Read more
Tests if any element of the stream matches a predicate. Read more
A stream adaptor similar to fold that holds internal state and produces a new stream. Read more
Combinator that skips elements based on a predicate. Read more
Creates a combinator that skips the first n elements. Read more
A combinator that applies a function as long as it returns successfully, producing a single, final value. Immediately returns the error when the function returns unsuccessfully. Read more
Applies a falliable function to each element in a stream, stopping at first error and returning it. Read more
‘Zips up’ two streams into a single stream of pairs. Read more
Lexicographically compares the elements of this Stream with those of another. Read more
Searches for an element in a Stream that satisfies a predicate, returning its index. Read more
Lexicographically compares the elements of this Stream with those of another using ‘Ord’. Read more
Determines if the elements of this Stream are lexicographically not equal to those of another. Read more
Determines if the elements of this Stream are lexicographically greater than or equal to those of another. Read more
Determines if the elements of this Stream are lexicographically equal to those of another. Read more
Determines if the elements of this Stream are lexicographically greater than those of another. Read more
Determines if the elements of this Stream are lexicographically less or equal to those of another. Read more
Determines if the elements of this Stream are lexicographically less than those of another. Read more
A convenience for calling [Stream::poll_next()] on !Unpin types.
Retrieves the next item in the stream. Read more
Retrieves the next item in the stream. Read more
Counts the number of items in the stream. Read more
Maps items of the stream to new values using a closure. Read more
Maps items to streams and then concatenates them. Read more
Concatenates inner streams. Read more
Maps items of the stream to new values using an async closure. Read more
Keeps items of the stream for which predicate returns true. Read more
Filters and maps items of the stream using a closure. Read more
Takes only the first n items of the stream. Read more
Takes items while predicate returns true. Read more
Skips the first n items of the stream. Read more
Skips items while predicate returns true. Read more
Yields every stepth item. Read more
Appends another stream to the end of this one. Read more
Clones all items. Read more
Copies all items. Read more
Collects all items in the stream into a collection. Read more
Collects all items in the fallible stream into a collection. Read more
Partitions items into those for which predicate is true and those for which it is false, and then collects them into two collections. Read more
Accumulates a computation over the stream. Read more
Accumulates a fallible computation over the stream. Read more
Maps items of the stream to new values using a state value and a closure. Read more
Fuses the stream so that it stops yielding items after the first None. Read more
Repeats the stream from beginning to end, forever. Read more
Enumerates items, mapping them to (index, item). Read more
Calls a closure on each item and passes it on. Read more
Gets the nth item of the stream. Read more
Returns the last item in the stream. Read more
Finds the first item of the stream for which predicate returns true. Read more
Applies a closure to items in the stream and returns the first Some result. Read more
Finds the index of the first item of the stream for which predicate returns true. Read more
Tests if predicate returns true for all items in the stream. Read more
Tests if predicate returns true for any item in the stream. Read more
Calls a closure on each item of the stream. Read more
Calls a fallible closure on each item of the stream, stopping on first error. Read more
Zips up two streams into a single stream of pairs. Read more
Collects a stream of pairs into a pair of collections. Read more
Merges with other stream, preferring items from self whenever both streams are ready. Read more
Merges with other stream, with no preference for either stream when both are ready. Read more
Boxes the stream and changes its type to dyn Stream + Send + 'a. Read more
Boxes the stream and changes its type to dyn Stream + 'a. Read more
The type returned in the event of a conversion error.
Performs the conversion.
The type returned in the event of a conversion error.
Performs the conversion.
The type of successful values yielded by this future
The type of failures yielded by this future
Poll this TryStream as if it were a Stream. Read more
Wraps the current stream in a new stream which converts the error type into the one provided. Read more
Wraps the current stream in a new stream which maps the success value using the provided closure. Read more
Wraps the current stream in a new stream which maps the error value using the provided closure. Read more
Chain on a computation for when a value is ready, passing the successful results to the provided closure f. Read more
Chain on a computation for when an error happens, passing the erroneous result to the provided closure f. Read more
Do something with the success value of this stream, afterwards passing it on. Read more
Do something with the error value of this stream, afterwards passing it on. Read more
Wraps a [TryStream] into a type that implements Stream Read more
Creates a future that attempts to resolve the next item in the stream. If an error is encountered before the next item, the error is returned instead. Read more
Attempts to run this stream to completion, executing the provided asynchronous closure for each element on the stream. Read more
Skip elements on this stream while the provided asynchronous predicate resolves to true. Read more
Take elements on this stream while the provided asynchronous predicate resolves to true. Read more
Attempts to run this stream to completion, executing the provided asynchronous closure for each element on the stream concurrently as elements become available, exiting as soon as an error occurs. Read more
Attempt to transform a stream into a collection, returning a future representing the result of that computation. Read more
An adaptor for chunking up successful items of the stream inside a vector. Read more
Attempt to filter the values produced by this stream according to the provided asynchronous closure. Read more
Attempt to filter the values produced by this stream while simultaneously mapping them to a different type according to the provided asynchronous closure. Read more
Flattens a stream of streams into just one continuous stream. Read more
Attempt to execute an accumulating asynchronous computation over a stream, collecting all the values into one final result. Read more
Attempt to concatenate all items of a stream into a single extendable destination, returning a future representing the end result. Read more
Attempt to execute several futures from a stream concurrently (unordered). Read more
Attempt to execute several futures from a stream concurrently. Read more
A convenience method for calling [TryStream::try_poll_next] on Unpin stream types. Read more
Adapter that converts this stream into an AsyncBufRead. Read more
Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more