TaskPool.asyncBuf

Given a source range that is expensive to iterate over, returns an input range that asynchronously buffers the contents of source into a buffer of bufSize elements in a worker thread, while making previously buffered elements from a second buffer, also of size bufSize, available via the range interface of the returned object. The returned range has a length iff hasLength!S. asyncBuf is useful, for example, when performing expensive operations on the elements of ranges that represent data on a disk or network.

auto asyncBuf(S source, size_t bufSize)
class TaskPool
auto
asyncBuf
(
S
)
(
S source
,
size_t bufSize = 100
)
if (
isInputRange!S
)
auto asyncBuf(C1 next, C2 empty, size_t initialBufSize, size_t nBuffers)

Examples

1 import std.conv, std.stdio;
2 
3 void main()
4 {
5     // Fetch lines of a file in a background thread
6     // while processing previously fetched lines,
7     // dealing with byLine's buffer recycling by
8     // eagerly duplicating every line.
9     auto lines = File("foo.txt").byLine();
10     auto duped = std.algorithm.map!"a.idup"(lines);
11 
12     // Fetch more lines in the background while we
13     // process the lines already read into memory
14     // into a matrix of doubles.
15     double[][] matrix;
16     auto asyncReader = taskPool.asyncBuf(duped);
17 
18     foreach (line; asyncReader)
19     {
20         auto ls = line.split("\t");
21         matrix ~= to!(double[])(ls);
22     }
23 }

Exception Handling:

Any exceptions thrown while iterating over source are re-thrown on a call to popFront or, if thrown during construction, simply allowed to propagate to the caller.

TaskPool.asyncBuf

Examples

Meta

Source