132
|
1 /*******************************************************************************
|
|
2
|
|
3 copyright: Copyright (c) 2004 Kris Bell. All rights reserved
|
|
4
|
|
5 license: BSD style: $(LICENSE)
|
|
6
|
|
7 version: Mar 2004: Initial release
|
|
8 Dec 2006: Outback release
|
|
9
|
|
10 authors: Kris
|
|
11
|
|
12 *******************************************************************************/
|
|
13
|
|
14 module tango.io.Buffer;
|
|
15
|
|
16 private import tango.core.Exception;
|
|
17
|
|
18 public import tango.io.model.IBuffer,
|
|
19 tango.io.model.IConduit;
|
|
20
|
|
21 /******************************************************************************
|
|
22
|
|
23 ******************************************************************************/
|
|
24
|
|
25 extern (C)
|
|
26 {
|
|
27 protected void * memcpy (void *dst, void *src, uint);
|
|
28 }
|
|
29
|
|
30 /*******************************************************************************
|
|
31
|
|
32 Buffer is central concept in Tango I/O. Each buffer acts
|
|
33 as a queue (line) where items are removed from the front
|
|
34 and new items are added to the back. Buffers are modeled
|
|
35 by tango.io.model.IBuffer, and a concrete implementation
|
|
36 is provided by this class.
|
|
37
|
|
38 Buffer can be read from and written to directly, though
|
|
39 various data-converters and filters are often leveraged
|
|
40 to apply structure to what might otherwise be simple raw
|
|
41 data.
|
|
42
|
|
43 Buffers may also be tokenized by applying an Iterator.
|
|
44 This can be handy when one is dealing with text input,
|
|
45 and/or the content suits a more fluid format than most
|
|
46 typical converters support. Iterator tokens are mapped
|
|
47 directly onto buffer content (sliced), making them quite
|
|
48 efficient in practice. Like other types of buffer client,
|
|
49 multiple iterators can be mapped onto one common buffer
|
|
50 and access will be serialized.
|
|
51
|
|
52 Buffers are sometimes memory-only, in which case there
|
|
53 is nothing left to do when a client has consumed all the
|
|
54 content. Other buffers are themselves bound to an external
|
|
55 device called a conduit. When this is the case, a consumer
|
|
56 will eventually cause a buffer to reload via its associated
|
|
57 conduit and previous buffer content will be lost.
|
|
58
|
|
59 A similar approach is applied to clients which populate a
|
|
60 buffer, whereby the content of a full buffer will be flushed
|
|
61 to a bound conduit before continuing. Another variation is
|
|
62 that of a memory-mapped buffer, whereby the buffer content
|
|
63 is mapped directly to virtual memory exposed via the OS. This
|
|
64 can be used to address large files as an array of content.
|
|
65
|
|
66 Direct buffer manipulation typically involves appending,
|
|
67 as in the following example:
|
|
68 ---
|
|
69 // create a small buffer
|
|
70 auto buf = new Buffer (256);
|
|
71
|
|
72 auto foo = "to write some D";
|
|
73
|
|
74 // append some text directly to it
|
|
75 buf.append ("now is the time for all good men ").append(foo);
|
|
76 ---
|
|
77
|
|
78 Alternatively, one might use a formatter to append the buffer:
|
|
79 ---
|
|
80 auto output = new FormatOutput (new Buffer(256));
|
|
81 output.format ("now is the time for {} good men {}", 3, foo);
|
|
82 ---
|
|
83
|
|
84 A slice() method will return all valid content within a buffer.
|
|
85 GrowBuffer can be used instead, where one wishes to append beyond
|
|
86 a specified limit.
|
|
87
|
|
88 A common usage of a buffer is in conjunction with a conduit,
|
|
89 such as FileConduit. Each conduit exposes a preferred-size for
|
|
90 its associated buffers, utilized during buffer construction:
|
|
91 ---
|
|
92 auto file = new FileConduit ("file.name");
|
|
93 auto buf = new Buffer (file);
|
|
94 ---
|
|
95
|
|
96 However, this is typically hidden by higher level constructors
|
|
97 such as those exposed via the stream wrappers. For example:
|
|
98 ---
|
|
99 auto input = new DataInput (new FileInput("file.name"));
|
|
100 ---
|
|
101
|
|
102 There is indeed a buffer between the resultant stream and the
|
|
103 file source, but explicit buffer construction is unecessary in
|
|
104 common cases.
|
|
105
|
|
106 An Iterator is constructed in a similar manner, where you provide
|
|
107 it an input stream to operate upon. There's a variety of iterators
|
|
108 available in the tango.text.stream package, and they are templated
|
|
109 for each of utf8, utf16, and utf32. This example uses a line iterator
|
|
110 to sweep a text file:
|
|
111 ---
|
|
112 auto lines = new LineInput (new FileInput("file.name"));
|
|
113 foreach (line; lines)
|
|
114 Cout(line).newline;
|
|
115 ---
|
|
116
|
|
117 Buffers are useful for many purposes within Tango, but there
|
|
118 are times when it may be more appropriate to sidestep them. For
|
|
119 such cases, all conduit derivatives (such as FileConduit) support
|
|
120 direct array-based IO via a pair of read() and write() methods.
|
|
121
|
|
122 *******************************************************************************/
|
|
123
|
|
124 class Buffer : IBuffer
|
|
125 {
|
|
126 protected OutputStream sink; // optional data sink
|
|
127 protected InputStream source; // optional data source
|
|
128 protected void[] data; // the raw data buffer
|
|
129 protected uint index; // current read position
|
|
130 protected uint extent; // limit of valid content
|
|
131 protected uint dimension; // maximum extent of content
|
|
132
|
|
133
|
|
134 protected static char[] overflow = "output buffer is full";
|
|
135 protected static char[] underflow = "input buffer is empty";
|
|
136 protected static char[] eofRead = "end-of-flow whilst reading";
|
|
137 protected static char[] eofWrite = "end-of-flow whilst writing";
|
|
138
|
|
139 /***********************************************************************
|
|
140
|
|
141 Ensure the buffer remains valid between method calls
|
|
142
|
|
143 ***********************************************************************/
|
|
144
|
|
145 invariant
|
|
146 {
|
|
147 assert (index <= extent);
|
|
148 assert (extent <= dimension);
|
|
149 }
|
|
150
|
|
151 /***********************************************************************
|
|
152
|
|
153 Construct a buffer
|
|
154
|
|
155 Params:
|
|
156 conduit = the conduit to buffer
|
|
157
|
|
158 Remarks:
|
|
159 Construct a Buffer upon the provided conduit. A relevant
|
|
160 buffer size is supplied via the provided conduit.
|
|
161
|
|
162 ***********************************************************************/
|
|
163
|
|
164 this (IConduit conduit)
|
|
165 {
|
|
166 assert (conduit);
|
|
167
|
|
168 this (conduit.bufferSize);
|
|
169 setConduit (conduit);
|
|
170 }
|
|
171
|
|
172 /***********************************************************************
|
|
173
|
|
174 Construct a buffer
|
|
175
|
|
176 Params:
|
|
177 stream = an input stream
|
|
178 capacity = desired buffer capacity
|
|
179
|
|
180 Remarks:
|
|
181 Construct a Buffer upon the provided input stream.
|
|
182
|
|
183 ***********************************************************************/
|
|
184
|
|
185 this (InputStream stream, uint capacity)
|
|
186 {
|
|
187 this (capacity);
|
|
188 input = stream;
|
|
189 }
|
|
190
|
|
191 /***********************************************************************
|
|
192
|
|
193 Construct a buffer
|
|
194
|
|
195 Params:
|
|
196 stream = an output stream
|
|
197 capacity = desired buffer capacity
|
|
198
|
|
199 Remarks:
|
|
200 Construct a Buffer upon the provided output stream.
|
|
201
|
|
202 ***********************************************************************/
|
|
203
|
|
204 this (OutputStream stream, uint capacity)
|
|
205 {
|
|
206 this (capacity);
|
|
207 output = stream;
|
|
208 }
|
|
209
|
|
210 /***********************************************************************
|
|
211
|
|
212 Construct a buffer
|
|
213
|
|
214 Params:
|
|
215 capacity = the number of bytes to make available
|
|
216
|
|
217 Remarks:
|
|
218 Construct a Buffer with the specified number of bytes.
|
|
219
|
|
220 ***********************************************************************/
|
|
221
|
|
222 this (uint capacity = 0)
|
|
223 {
|
|
224 setContent (new ubyte[capacity], 0);
|
|
225 }
|
|
226
|
|
227 /***********************************************************************
|
|
228
|
|
229 Construct a buffer
|
|
230
|
|
231 Params:
|
|
232 data = the backing array to buffer within
|
|
233
|
|
234 Remarks:
|
|
235 Prime a buffer with an application-supplied array. All content
|
|
236 is considered valid for reading, and thus there is no writable
|
|
237 space initially available.
|
|
238
|
|
239 ***********************************************************************/
|
|
240
|
|
241 this (void[] data)
|
|
242 {
|
|
243 setContent (data, data.length);
|
|
244 }
|
|
245
|
|
246 /***********************************************************************
|
|
247
|
|
248 Construct a buffer
|
|
249
|
|
250 Params:
|
|
251 data = the backing array to buffer within
|
|
252 readable = the number of bytes initially made
|
|
253 readable
|
|
254
|
|
255 Remarks:
|
|
256 Prime buffer with an application-supplied array, and
|
|
257 indicate how much readable data is already there. A
|
|
258 write operation will begin writing immediately after
|
|
259 the existing readable content.
|
|
260
|
|
261 This is commonly used to attach a Buffer instance to
|
|
262 a local array.
|
|
263
|
|
264 ***********************************************************************/
|
|
265
|
|
266 this (void[] data, uint readable)
|
|
267 {
|
|
268 setContent (data, readable);
|
|
269 }
|
|
270
|
|
271 /***********************************************************************
|
|
272
|
|
273 Attempt to share an upstream Buffer, and create an instance
|
|
274 where there not one available.
|
|
275
|
|
276 Params:
|
|
277 stream = an input stream
|
|
278 size = a hint of the desired buffer size. Defaults to the
|
|
279 conduit-defined size
|
|
280
|
|
281 Remarks:
|
|
282 If an upstream Buffer instances is visible, it will be shared.
|
|
283 Otherwise, a new instance is created based upon the bufferSize
|
|
284 exposed by the stream endpoint (conduit).
|
|
285
|
|
286 ***********************************************************************/
|
|
287
|
|
288 static IBuffer share (InputStream stream, uint size=uint.max)
|
|
289 {
|
|
290 auto b = cast(Buffered) stream;
|
|
291 if (b)
|
|
292 return b.buffer;
|
|
293
|
|
294 if (size is uint.max)
|
|
295 size = stream.conduit.bufferSize;
|
|
296
|
|
297 return new Buffer (stream, size);
|
|
298 }
|
|
299
|
|
300 /***********************************************************************
|
|
301
|
|
302 Attempt to share an upstream Buffer, and create an instance
|
|
303 where there not one available.
|
|
304
|
|
305 Params:
|
|
306 stream = an output stream
|
|
307 size = a hint of the desired buffer size. Defaults to the
|
|
308 conduit-defined size
|
|
309
|
|
310 Remarks:
|
|
311 If an upstream Buffer instances is visible, it will be shared.
|
|
312 Otherwise, a new instance is created based upon the bufferSize
|
|
313 exposed by the stream endpoint (conduit).
|
|
314
|
|
315 ***********************************************************************/
|
|
316
|
|
317 static IBuffer share (OutputStream stream, uint size=uint.max)
|
|
318 {
|
|
319 auto b = cast(Buffered) stream;
|
|
320 if (b)
|
|
321 return b.buffer;
|
|
322
|
|
323 if (size is uint.max)
|
|
324 size = stream.conduit.bufferSize;
|
|
325
|
|
326 return new Buffer (stream, size);
|
|
327 }
|
|
328
|
|
329 /***********************************************************************
|
|
330
|
|
331 Reset the buffer content
|
|
332
|
|
333 Params:
|
|
334 data = the backing array to buffer within. All content
|
|
335 is considered valid
|
|
336
|
|
337 Returns:
|
|
338 the buffer instance
|
|
339
|
|
340 Remarks:
|
|
341 Set the backing array with all content readable. Writing
|
|
342 to this will either flush it to an associated conduit, or
|
|
343 raise an Eof condition. Use clear() to reset the content
|
|
344 (make it all writable).
|
|
345
|
|
346 ***********************************************************************/
|
|
347
|
|
348 IBuffer setContent (void[] data)
|
|
349 {
|
|
350 return setContent (data, data.length);
|
|
351 }
|
|
352
|
|
353 /***********************************************************************
|
|
354
|
|
355 Reset the buffer content
|
|
356
|
|
357 Params:
|
|
358 data = the backing array to buffer within
|
|
359 readable = the number of bytes within data considered
|
|
360 valid
|
|
361
|
|
362 Returns:
|
|
363 the buffer instance
|
|
364
|
|
365 Remarks:
|
|
366 Set the backing array with some content readable. Writing
|
|
367 to this will either flush it to an associated conduit, or
|
|
368 raise an Eof condition. Use clear() to reset the content
|
|
369 (make it all writable).
|
|
370
|
|
371 ***********************************************************************/
|
|
372
|
|
373 IBuffer setContent (void[] data, uint readable)
|
|
374 {
|
|
375 this.data = data;
|
|
376 this.extent = readable;
|
|
377 this.dimension = data.length;
|
|
378
|
|
379 // reset to start of input
|
|
380 this.index = 0;
|
|
381
|
|
382 return this;
|
|
383 }
|
|
384
|
|
385 /***********************************************************************
|
|
386
|
|
387 Access buffer content
|
|
388
|
|
389 Params:
|
|
390 size = number of bytes to access
|
|
391 eat = whether to consume the content or not
|
|
392
|
|
393 Returns:
|
|
394 the corresponding buffer slice when successful, or
|
|
395 null if there's not enough data available (Eof; Eob).
|
|
396
|
|
397 Remarks:
|
|
398 Read a slice of data from the buffer, loading from the
|
|
399 conduit as necessary. The specified number of bytes is
|
|
400 sliced from the buffer, and marked as having been read
|
|
401 when the 'eat' parameter is set true. When 'eat' is set
|
|
402 false, the read position is not adjusted.
|
|
403
|
|
404 Note that the slice cannot be larger than the size of
|
|
405 the buffer ~ use method fill(void[]) instead where you
|
|
406 simply want the content copied, or use conduit.read()
|
|
407 to extract directly from an attached conduit. Also note
|
|
408 that if you need to retain the slice, then it should be
|
|
409 .dup'd before the buffer is compressed or repopulated.
|
|
410
|
|
411 Examples:
|
|
412 ---
|
|
413 // create a buffer with some content
|
|
414 auto buffer = new Buffer ("hello world");
|
|
415
|
|
416 // consume everything unread
|
|
417 auto slice = buffer.slice (buffer.readable);
|
|
418 ---
|
|
419
|
|
420 ***********************************************************************/
|
|
421
|
|
422 void[] slice (uint size, bool eat = true)
|
|
423 {
|
|
424 if (size > readable)
|
|
425 {
|
|
426 if (source is null)
|
|
427 error (underflow);
|
|
428
|
|
429 // make some space? This will try to leave as much content
|
|
430 // in the buffer as possible, such that entire records may
|
|
431 // be aliased directly from within.
|
|
432 if (size > writable)
|
|
433 {
|
|
434 if (size > dimension)
|
|
435 error (underflow);
|
|
436 compress ();
|
|
437 }
|
|
438
|
|
439 // populate tail of buffer with new content
|
|
440 do {
|
|
441 if (fill(source) is IConduit.Eof)
|
|
442 error (eofRead);
|
|
443 } while (size > readable);
|
|
444 }
|
|
445
|
|
446 auto i = index;
|
|
447 if (eat)
|
|
448 index += size;
|
|
449 return data [i .. i + size];
|
|
450 }
|
|
451
|
|
452 /**********************************************************************
|
|
453
|
|
454 Fill the provided buffer. Returns the number of bytes
|
|
455 actually read, which will be less that dst.length when
|
|
456 Eof has been reached and IConduit.Eof thereafter
|
|
457
|
|
458 **********************************************************************/
|
|
459
|
|
460 uint fill (void[] dst)
|
|
461 {
|
|
462 uint len = 0;
|
|
463
|
|
464 while (len < dst.length)
|
|
465 {
|
|
466 uint i = read (dst [len .. $]);
|
|
467 if (i is IConduit.Eof)
|
|
468 return (len > 0) ? len : IConduit.Eof;
|
|
469 len += i;
|
|
470 }
|
|
471 return len;
|
|
472 }
|
|
473
|
|
474 /***********************************************************************
|
|
475
|
|
476 Copy buffer content into the provided dst
|
|
477
|
|
478 Params:
|
|
479 dst = destination of the content
|
|
480 bytes = size of dst
|
|
481
|
|
482 Returns:
|
|
483 A reference to the populated content
|
|
484
|
|
485 Remarks:
|
|
486 Fill the provided array with content. We try to satisfy
|
|
487 the request from the buffer content, and read directly
|
|
488 from an attached conduit where more is required.
|
|
489
|
|
490 ***********************************************************************/
|
|
491
|
|
492 void[] readExact (void* dst, uint bytes)
|
|
493 {
|
|
494 auto tmp = dst [0 .. bytes];
|
|
495 if (fill (tmp) != bytes)
|
|
496 error (eofRead);
|
|
497
|
|
498 return tmp;
|
|
499 }
|
|
500
|
|
501 /***********************************************************************
|
|
502
|
|
503 Append content
|
|
504
|
|
505 Params:
|
|
506 src = the content to _append
|
|
507
|
|
508 Returns a chaining reference if all content was written.
|
|
509 Throws an IOException indicating eof or eob if not.
|
|
510
|
|
511 Remarks:
|
|
512 Append an array to this buffer, and flush to the
|
|
513 conduit as necessary. This is often used in lieu of
|
|
514 a Writer.
|
|
515
|
|
516 ***********************************************************************/
|
|
517
|
|
518 IBuffer append (void[] src)
|
|
519 {
|
|
520 return append (src.ptr, src.length);
|
|
521 }
|
|
522
|
|
523 /***********************************************************************
|
|
524
|
|
525 Append content
|
|
526
|
|
527 Params:
|
|
528 src = the content to _append
|
|
529 length = the number of bytes in src
|
|
530
|
|
531 Returns a chaining reference if all content was written.
|
|
532 Throws an IOException indicating eof or eob if not.
|
|
533
|
|
534 Remarks:
|
|
535 Append an array to this buffer, and flush to the
|
|
536 conduit as necessary. This is often used in lieu of
|
|
537 a Writer.
|
|
538
|
|
539 ***********************************************************************/
|
|
540
|
|
541 IBuffer append (void* src, uint length)
|
|
542 {
|
|
543 if (length > writable)
|
|
544 // can we write externally?
|
|
545 if (sink)
|
|
546 {
|
|
547 flush ();
|
|
548
|
|
549 // check for pathological case
|
|
550 if (length > dimension)
|
|
551 {
|
|
552 do {
|
|
553 auto written = sink.write (src [0 .. length]);
|
|
554 if (written is IConduit.Eof)
|
|
555 error (eofWrite);
|
|
556 src += written, length -= written;
|
|
557 } while (length > dimension);
|
|
558 }
|
|
559 }
|
|
560 else
|
|
561 error (overflow);
|
|
562
|
|
563 copy (src, length);
|
|
564 return this;
|
|
565 }
|
|
566
|
|
567 /***********************************************************************
|
|
568
|
|
569 Append content
|
|
570
|
|
571 Params:
|
|
572 other = a buffer with content available
|
|
573
|
|
574 Returns:
|
|
575 Returns a chaining reference if all content was written.
|
|
576 Throws an IOException indicating eof or eob if not.
|
|
577
|
|
578 Remarks:
|
|
579 Append another buffer to this one, and flush to the
|
|
580 conduit as necessary. This is often used in lieu of
|
|
581 a Writer.
|
|
582
|
|
583 ***********************************************************************/
|
|
584
|
|
585 IBuffer append (IBuffer other)
|
|
586 {
|
|
587 return append (other.slice);
|
|
588 }
|
|
589
|
|
590 /***********************************************************************
|
|
591
|
|
592 Consume content from a producer
|
|
593
|
|
594 Params:
|
|
595 The content to consume. This is consumed verbatim, and in
|
|
596 raw binary format ~ no implicit conversions are performed.
|
|
597
|
|
598 Remarks:
|
|
599 This is often used in lieu of a Writer, and enables simple
|
|
600 classes, such as FilePath and Uri, to emit content directly
|
|
601 into a buffer (thus avoiding potential heap activity)
|
|
602
|
|
603 Examples:
|
|
604 ---
|
|
605 auto path = new FilePath (somepath);
|
|
606
|
|
607 path.produce (&buffer.consume);
|
|
608 ---
|
|
609
|
|
610 ***********************************************************************/
|
|
611
|
|
612 void consume (void[] x)
|
|
613 {
|
|
614 append (x);
|
|
615 }
|
|
616
|
|
617 /***********************************************************************
|
|
618
|
|
619 Retrieve the valid content
|
|
620
|
|
621 Returns:
|
|
622 a void[] slice of the buffer
|
|
623
|
|
624 Remarks:
|
|
625 Return a void[] slice of the buffer, from the current position
|
|
626 up to the limit of valid content. The content remains in the
|
|
627 buffer for future extraction.
|
|
628
|
|
629 ***********************************************************************/
|
|
630
|
|
631 void[] slice ()
|
|
632 {
|
|
633 return data [index .. extent];
|
|
634 }
|
|
635
|
|
636 /***********************************************************************
|
|
637
|
|
638 Move the current read location
|
|
639
|
|
640 Params:
|
|
641 size = the number of bytes to move
|
|
642
|
|
643 Returns:
|
|
644 Returns true if successful, false otherwise.
|
|
645
|
|
646 Remarks:
|
|
647 Skip ahead by the specified number of bytes, streaming from
|
|
648 the associated conduit as necessary.
|
|
649
|
|
650 Can also reverse the read position by 'size' bytes, when size
|
|
651 is negative. This may be used to support lookahead operations.
|
|
652 Note that a negative size will fail where there is not sufficient
|
|
653 content available in the buffer (can't _skip beyond the beginning).
|
|
654
|
|
655 ***********************************************************************/
|
|
656
|
|
657 bool skip (int size)
|
|
658 {
|
|
659 if (size < 0)
|
|
660 {
|
|
661 size = -size;
|
|
662 if (index >= size)
|
|
663 {
|
|
664 index -= size;
|
|
665 return true;
|
|
666 }
|
|
667 return false;
|
|
668 }
|
|
669 return slice(size) !is null;
|
|
670 }
|
|
671
|
|
672 /***********************************************************************
|
|
673
|
|
674 Iterator support
|
|
675
|
|
676 Params:
|
|
677 scan = the delagate to invoke with the current content
|
|
678
|
|
679 Returns:
|
|
680 Returns true if a token was isolated, false otherwise.
|
|
681
|
|
682 Remarks:
|
|
683 Upon success, the delegate should return the byte-based
|
|
684 index of the consumed pattern (tail end of it). Failure
|
|
685 to match a pattern should be indicated by returning an
|
|
686 IConduit.Eof
|
|
687
|
|
688 Each pattern is expected to be stripped of the delimiter.
|
|
689 An end-of-file condition causes trailing content to be
|
|
690 placed into the token. Requests made beyond Eof result
|
|
691 in empty matches (length is zero).
|
|
692
|
|
693 Note that additional iterator and/or reader instances
|
|
694 will operate in lockstep when bound to a common buffer.
|
|
695
|
|
696 ***********************************************************************/
|
|
697
|
|
698 bool next (uint delegate (void[]) scan)
|
|
699 {
|
|
700 while (read(scan) is IConduit.Eof)
|
|
701 // not found - are we streaming?
|
|
702 if (source)
|
|
703 {
|
|
704 // did we start at the beginning?
|
|
705 if (position)
|
|
706 // nope - move partial token to start of buffer
|
|
707 compress ();
|
|
708 else
|
|
709 // no more space in the buffer?
|
|
710 if (writable is 0)
|
|
711 error ("Token is too large to fit within buffer");
|
|
712
|
|
713 // read another chunk of data
|
|
714 if (fill(source) is IConduit.Eof)
|
|
715 return false;
|
|
716 }
|
|
717 else
|
|
718 return false;
|
|
719
|
|
720 return true;
|
|
721 }
|
|
722
|
|
723 /***********************************************************************
|
|
724
|
|
725 Available content
|
|
726
|
|
727 Remarks:
|
|
728 Return count of _readable bytes remaining in buffer. This is
|
|
729 calculated simply as limit() - position()
|
|
730
|
|
731 ***********************************************************************/
|
|
732
|
|
733 uint readable ()
|
|
734 {
|
|
735 return extent - index;
|
|
736 }
|
|
737
|
|
738 /***********************************************************************
|
|
739
|
|
740 Available space
|
|
741
|
|
742 Remarks:
|
|
743 Return count of _writable bytes available in buffer. This is
|
|
744 calculated simply as capacity() - limit()
|
|
745
|
|
746 ***********************************************************************/
|
|
747
|
|
748 uint writable ()
|
|
749 {
|
|
750 return dimension - extent;
|
|
751 }
|
|
752
|
|
753 /***********************************************************************
|
|
754
|
|
755 Write into this buffer
|
|
756
|
|
757 Params:
|
|
758 dg = the callback to provide buffer access to
|
|
759
|
|
760 Returns:
|
|
761 Returns whatever the delegate returns.
|
|
762
|
|
763 Remarks:
|
|
764 Exposes the raw data buffer at the current _write position,
|
|
765 The delegate is provided with a void[] representing space
|
|
766 available within the buffer at the current _write position.
|
|
767
|
|
768 The delegate should return the appropriate number of bytes
|
|
769 if it writes valid content, or IConduit.Eof on error.
|
|
770
|
|
771 ***********************************************************************/
|
|
772
|
|
773 uint write (uint delegate (void[]) dg)
|
|
774 {
|
|
775 int count = dg (data [extent..dimension]);
|
|
776
|
|
777 if (count != IConduit.Eof)
|
|
778 {
|
|
779 extent += count;
|
|
780 assert (extent <= dimension);
|
|
781 }
|
|
782 return count;
|
|
783 }
|
|
784
|
|
785 /***********************************************************************
|
|
786
|
|
787 Read directly from this buffer
|
|
788
|
|
789 Params:
|
|
790 dg = callback to provide buffer access to
|
|
791
|
|
792 Returns:
|
|
793 Returns whatever the delegate returns.
|
|
794
|
|
795 Remarks:
|
|
796 Exposes the raw data buffer at the current _read position. The
|
|
797 delegate is provided with a void[] representing the available
|
|
798 data, and should return zero to leave the current _read position
|
|
799 intact.
|
|
800
|
|
801 If the delegate consumes data, it should return the number of
|
|
802 bytes consumed; or IConduit.Eof to indicate an error.
|
|
803
|
|
804 ***********************************************************************/
|
|
805
|
|
806 uint read (uint delegate (void[]) dg)
|
|
807 {
|
|
808 int count = dg (data [index..extent]);
|
|
809
|
|
810 if (count != IConduit.Eof)
|
|
811 {
|
|
812 index += count;
|
|
813 assert (index <= extent);
|
|
814 }
|
|
815 return count;
|
|
816 }
|
|
817
|
|
818 /***********************************************************************
|
|
819
|
|
820 Compress buffer space
|
|
821
|
|
822 Returns:
|
|
823 the buffer instance
|
|
824
|
|
825 Remarks:
|
|
826 If we have some data left after an export, move it to
|
|
827 front-of-buffer and set position to be just after the
|
|
828 remains. This is for supporting certain conduits which
|
|
829 choose to write just the initial portion of a request.
|
|
830
|
|
831 Limit is set to the amount of data remaining. Position
|
|
832 is always reset to zero.
|
|
833
|
|
834 ***********************************************************************/
|
|
835
|
|
836 IBuffer compress ()
|
|
837 {
|
|
838 uint r = readable ();
|
|
839
|
|
840 if (index > 0 && r > 0)
|
|
841 // content may overlap ...
|
|
842 memcpy (&data[0], &data[index], r);
|
|
843
|
|
844 index = 0;
|
|
845 extent = r;
|
|
846 return this;
|
|
847 }
|
|
848
|
|
849 /***********************************************************************
|
|
850
|
|
851 Fill buffer from the specific conduit
|
|
852
|
|
853 Returns:
|
|
854 Returns the number of bytes read, or Conduit.Eof
|
|
855
|
|
856 Remarks:
|
|
857 Try to _fill the available buffer with content from the
|
|
858 specified conduit. We try to read as much as possible
|
|
859 by clearing the buffer when all current content has been
|
|
860 eaten. If there is no space available, nothing will be
|
|
861 read.
|
|
862
|
|
863 ***********************************************************************/
|
|
864
|
|
865 uint fill (InputStream src)
|
|
866 {
|
|
867 if (src is null)
|
|
868 return IConduit.Eof;
|
|
869
|
|
870 if (readable is 0)
|
|
871 index = extent = 0; // same as clear(), but without chain
|
|
872 else
|
|
873 if (writable is 0)
|
|
874 return 0;
|
|
875
|
|
876 return write (&src.read);
|
|
877 }
|
|
878
|
|
879 /***********************************************************************
|
|
880
|
|
881 Drain buffer content to the specific conduit
|
|
882
|
|
883 Returns:
|
|
884 Returns the number of bytes written
|
|
885
|
|
886 Remarks:
|
|
887 Write as much of the buffer that the associated conduit
|
|
888 can consume. The conduit is not obliged to consume all
|
|
889 content, so some may remain within the buffer.
|
|
890
|
|
891 Throws an IOException on premature Eof.
|
|
892
|
|
893 ***********************************************************************/
|
|
894
|
|
895 final uint drain (OutputStream dst)
|
|
896 {
|
|
897 if (dst is null)
|
|
898 return IConduit.Eof;
|
|
899
|
|
900 uint ret = read (&dst.write);
|
|
901 if (ret is IConduit.Eof)
|
|
902 error (eofWrite);
|
|
903
|
|
904 compress ();
|
|
905 return ret;
|
|
906 }
|
|
907
|
|
908 /***********************************************************************
|
|
909
|
|
910 Truncate buffer content
|
|
911
|
|
912 Remarks:
|
|
913 Truncate the buffer within its extent. Returns true if
|
|
914 the new length is valid, false otherwise.
|
|
915
|
|
916 ***********************************************************************/
|
|
917
|
|
918 bool truncate (uint length)
|
|
919 {
|
|
920 if (length <= data.length)
|
|
921 {
|
|
922 extent = length;
|
|
923 return true;
|
|
924 }
|
|
925 return false;
|
|
926 }
|
|
927
|
|
928 /***********************************************************************
|
|
929
|
|
930 Access buffer limit
|
|
931
|
|
932 Returns:
|
|
933 Returns the limit of readable content within this buffer.
|
|
934
|
|
935 Remarks:
|
|
936 Each buffer has a capacity, a limit, and a position. The
|
|
937 capacity is the maximum content a buffer can contain, limit
|
|
938 represents the extent of valid content, and position marks
|
|
939 the current read location.
|
|
940
|
|
941 ***********************************************************************/
|
|
942
|
|
943 uint limit ()
|
|
944 {
|
|
945 return extent;
|
|
946 }
|
|
947
|
|
948 /***********************************************************************
|
|
949
|
|
950 Access buffer capacity
|
|
951
|
|
952 Returns:
|
|
953 Returns the maximum capacity of this buffer
|
|
954
|
|
955 Remarks:
|
|
956 Each buffer has a capacity, a limit, and a position. The
|
|
957 capacity is the maximum content a buffer can contain, limit
|
|
958 represents the extent of valid content, and position marks
|
|
959 the current read location.
|
|
960
|
|
961 ***********************************************************************/
|
|
962
|
|
963 uint capacity ()
|
|
964 {
|
|
965 return dimension;
|
|
966 }
|
|
967
|
|
968 /***********************************************************************
|
|
969
|
|
970 Access buffer read position
|
|
971
|
|
972 Returns:
|
|
973 Returns the current read-position within this buffer
|
|
974
|
|
975 Remarks:
|
|
976 Each buffer has a capacity, a limit, and a position. The
|
|
977 capacity is the maximum content a buffer can contain, limit
|
|
978 represents the extent of valid content, and position marks
|
|
979 the current read location.
|
|
980
|
|
981 ***********************************************************************/
|
|
982
|
|
983 uint position ()
|
|
984 {
|
|
985 return index;
|
|
986 }
|
|
987
|
|
988 /***********************************************************************
|
|
989
|
|
990 Set external conduit
|
|
991
|
|
992 Params:
|
|
993 conduit = the conduit to attach to
|
|
994
|
|
995 Remarks:
|
|
996 Sets the external conduit associated with this buffer.
|
|
997
|
|
998 Buffers do not require an external conduit to operate, but
|
|
999 it can be convenient to associate one. For example, methods
|
|
1000 fill() & drain() use it to import/export content as necessary.
|
|
1001
|
|
1002 ***********************************************************************/
|
|
1003
|
|
1004 IBuffer setConduit (IConduit conduit)
|
|
1005 {
|
|
1006 sink = conduit.output;
|
|
1007 source = conduit.input;
|
|
1008 return this;
|
|
1009 }
|
|
1010
|
|
1011 /***********************************************************************
|
|
1012
|
|
1013 Set output stream
|
|
1014
|
|
1015 Params:
|
|
1016 sink = the stream to attach to
|
|
1017
|
|
1018 Remarks:
|
|
1019 Sets the external output stream associated with this buffer.
|
|
1020
|
|
1021 Buffers do not require an external stream to operate, but
|
|
1022 it can be convenient to associate one. For example, methods
|
|
1023 fill & drain use them to import/export content as necessary.
|
|
1024
|
|
1025 ***********************************************************************/
|
|
1026
|
|
1027 final IBuffer output (OutputStream sink)
|
|
1028 {
|
|
1029 this.sink = sink;
|
|
1030 return this;
|
|
1031 }
|
|
1032
|
|
1033 /***********************************************************************
|
|
1034
|
|
1035 Set input stream
|
|
1036
|
|
1037 Params:
|
|
1038 source = the stream to attach to
|
|
1039
|
|
1040 Remarks:
|
|
1041 Sets the external input stream associated with this buffer.
|
|
1042
|
|
1043 Buffers do not require an external stream to operate, but
|
|
1044 it can be convenient to associate one. For example, methods
|
|
1045 fill & drain use them to import/export content as necessary.
|
|
1046
|
|
1047 ***********************************************************************/
|
|
1048
|
|
1049 final IBuffer input (InputStream source)
|
|
1050 {
|
|
1051 this.source = source;
|
|
1052 return this;
|
|
1053 }
|
|
1054
|
|
1055 /***********************************************************************
|
|
1056
|
|
1057 Access buffer content
|
|
1058
|
|
1059 Remarks:
|
|
1060 Return the entire backing array. Exposed for subclass usage
|
|
1061 only
|
|
1062
|
|
1063 ***********************************************************************/
|
|
1064
|
|
1065 protected void[] getContent ()
|
|
1066 {
|
|
1067 return data;
|
|
1068 }
|
|
1069
|
|
1070 /***********************************************************************
|
|
1071
|
|
1072 Copy content into buffer
|
|
1073
|
|
1074 Params:
|
|
1075 src = the soure of the content
|
|
1076 size = the length of content at src
|
|
1077
|
|
1078 Remarks:
|
|
1079 Bulk _copy of data from 'src'. The new content is made
|
|
1080 available for reading. This is exposed for subclass use
|
|
1081 only
|
|
1082
|
|
1083 ***********************************************************************/
|
|
1084
|
|
1085 protected void copy (void *src, uint size)
|
|
1086 {
|
|
1087 // avoid "out of bounds" test on zero size
|
|
1088 if (size)
|
|
1089 {
|
|
1090 // content may overlap ...
|
|
1091 memcpy (&data[extent], src, size);
|
|
1092 extent += size;
|
|
1093 }
|
|
1094 }
|
|
1095
|
|
1096 /***********************************************************************
|
|
1097
|
|
1098 Cast to a target type without invoking the wrath of the
|
|
1099 runtime checks for misalignment. Instead, we truncate the
|
|
1100 array length
|
|
1101
|
|
1102 ***********************************************************************/
|
|
1103
|
|
1104 static T[] convert(T)(void[] x)
|
|
1105 {
|
|
1106 return (cast(T*) x.ptr) [0 .. (x.length / T.sizeof)];
|
|
1107 }
|
|
1108
|
|
1109
|
|
1110
|
|
1111 /**********************************************************************/
|
|
1112 /*********************** Buffered Interface ***************************/
|
|
1113 /**********************************************************************/
|
|
1114
|
|
1115 IBuffer buffer ()
|
|
1116 {
|
|
1117 return this;
|
|
1118 }
|
|
1119
|
|
1120
|
|
1121 /**********************************************************************/
|
|
1122 /******************** Stream & Conduit Interfaces *********************/
|
|
1123 /**********************************************************************/
|
|
1124
|
|
1125
|
|
1126 /***********************************************************************
|
|
1127
|
|
1128 Return the name of this conduit
|
|
1129
|
|
1130 ***********************************************************************/
|
|
1131
|
|
1132 override char[] toString ()
|
|
1133 {
|
|
1134 return "<buffer>";
|
|
1135 }
|
|
1136
|
|
1137 /***********************************************************************
|
|
1138
|
|
1139 Generic IOException thrower
|
|
1140
|
|
1141 Params:
|
|
1142 msg = a text message describing the exception reason
|
|
1143
|
|
1144 Remarks:
|
|
1145 Throw an IOException with the provided message
|
|
1146
|
|
1147 ***********************************************************************/
|
|
1148
|
|
1149 final void error (char[] msg)
|
|
1150 {
|
|
1151 throw new IOException (msg);
|
|
1152 }
|
|
1153
|
|
1154 /***********************************************************************
|
|
1155
|
|
1156 Flush all buffer content to the specific conduit
|
|
1157
|
|
1158 Remarks:
|
|
1159 Flush the contents of this buffer. This will block until
|
|
1160 all content is actually flushed via the associated conduit,
|
|
1161 whereas drain() will not.
|
|
1162
|
|
1163 Do nothing where a conduit is not attached, enabling memory
|
|
1164 buffers to treat flush as a noop.
|
|
1165
|
|
1166 Throws an IOException on premature Eof.
|
|
1167
|
|
1168 ***********************************************************************/
|
|
1169
|
|
1170 override OutputStream flush ()
|
|
1171 {
|
|
1172 if (sink)
|
|
1173 {
|
|
1174 while (readable() > 0)
|
|
1175 drain (sink);
|
|
1176
|
|
1177 // flush the filter chain also
|
|
1178 sink.flush;
|
|
1179 }
|
|
1180 return this;
|
|
1181 }
|
|
1182
|
|
1183 /***********************************************************************
|
|
1184
|
|
1185 Clear buffer content
|
|
1186
|
|
1187 Remarks:
|
|
1188 Reset 'position' and 'limit' to zero. This effectively
|
|
1189 clears all content from the buffer.
|
|
1190
|
|
1191 ***********************************************************************/
|
|
1192
|
|
1193 override InputStream clear ()
|
|
1194 {
|
|
1195 index = extent = 0;
|
|
1196
|
|
1197 // clear the filter chain also
|
|
1198 if (source)
|
|
1199 source.clear;
|
|
1200 return this;
|
|
1201 }
|
|
1202
|
|
1203 /***********************************************************************
|
|
1204
|
|
1205 Copy content via this buffer from the provided src
|
|
1206 conduit.
|
|
1207
|
|
1208 Remarks:
|
|
1209 The src conduit has its content transferred through
|
|
1210 this buffer via a series of fill & drain operations,
|
|
1211 until there is no more content available. The buffer
|
|
1212 content should be explicitly flushed by the caller.
|
|
1213
|
|
1214 Throws an IOException on premature eof
|
|
1215
|
|
1216 ***********************************************************************/
|
|
1217
|
|
1218 override OutputStream copy (InputStream src)
|
|
1219 {
|
|
1220 while (fill(src) != IConduit.Eof)
|
|
1221 // don't drain until we actually need to
|
|
1222 if (writable is 0)
|
|
1223 if (sink)
|
|
1224 drain (sink);
|
|
1225 else
|
|
1226 error (overflow);
|
|
1227 return this;
|
|
1228 }
|
|
1229
|
|
1230 /***********************************************************************
|
|
1231
|
|
1232 Transfer content into the provided dst
|
|
1233
|
|
1234 Params:
|
|
1235 dst = destination of the content
|
|
1236
|
|
1237 Returns:
|
|
1238 return the number of bytes read, which may be less than
|
|
1239 dst.length. Eof is returned when no further content is
|
|
1240 available.
|
|
1241
|
|
1242 Remarks:
|
|
1243 Populates the provided array with content. We try to
|
|
1244 satisfy the request from the buffer content, and read
|
|
1245 directly from an attached conduit when the buffer is
|
|
1246 empty.
|
|
1247
|
|
1248 ***********************************************************************/
|
|
1249
|
|
1250 override uint read (void[] dst)
|
|
1251 {
|
|
1252 uint content = readable();
|
|
1253 if (content)
|
|
1254 {
|
|
1255 if (content >= dst.length)
|
|
1256 content = dst.length;
|
|
1257
|
|
1258 // transfer buffer content
|
|
1259 dst [0 .. content] = data [index .. index + content];
|
|
1260 index += content;
|
|
1261 }
|
|
1262 else
|
|
1263 if (source)
|
|
1264 {
|
|
1265 // pathological cases read directly from conduit
|
|
1266 if (dst.length > dimension)
|
|
1267 content = source.read (dst);
|
|
1268 else
|
|
1269 // keep buffer partially populated
|
|
1270 if ((content = fill(source)) != IConduit.Eof && content > 0)
|
|
1271 content = read (dst);
|
|
1272 }
|
|
1273 else
|
|
1274 content = IConduit.Eof;
|
|
1275 return content;
|
|
1276 }
|
|
1277
|
|
1278 /***********************************************************************
|
|
1279
|
|
1280 Emulate OutputStream.write()
|
|
1281
|
|
1282 Params:
|
|
1283 src = the content to write
|
|
1284
|
|
1285 Returns:
|
|
1286 return the number of bytes written, which may be less than
|
|
1287 provided (conceptually).
|
|
1288
|
|
1289 Remarks:
|
|
1290 Appends src content to the buffer, flushing to an attached
|
|
1291 conduit as necessary. An IOException is thrown upon write
|
|
1292 failure.
|
|
1293
|
|
1294 ***********************************************************************/
|
|
1295
|
|
1296 override uint write (void[] src)
|
|
1297 {
|
|
1298 append (src.ptr, src.length);
|
|
1299 return src.length;
|
|
1300 }
|
|
1301
|
|
1302 /***********************************************************************
|
|
1303
|
|
1304 Access configured conduit
|
|
1305
|
|
1306 Returns:
|
|
1307 Returns the conduit associated with this buffer. Returns
|
|
1308 null if the buffer is purely memory based; that is, it's
|
|
1309 not backed by some external medium.
|
|
1310
|
|
1311 Remarks:
|
|
1312 Buffers do not require an external conduit to operate, but
|
|
1313 it can be convenient to associate one. For example, methods
|
|
1314 fill() & drain() use it to import/export content as necessary.
|
|
1315
|
|
1316 ***********************************************************************/
|
|
1317
|
|
1318 final override IConduit conduit ()
|
|
1319 {
|
|
1320 if (sink)
|
|
1321 return sink.conduit;
|
|
1322 else
|
|
1323 if (source)
|
|
1324 return source.conduit;
|
|
1325 return this;
|
|
1326 }
|
|
1327
|
|
1328 /***********************************************************************
|
|
1329
|
|
1330 Return a preferred size for buffering conduit I/O
|
|
1331
|
|
1332 ***********************************************************************/
|
|
1333
|
|
1334 final override uint bufferSize ()
|
|
1335 {
|
|
1336 return 32 * 1024;
|
|
1337 }
|
|
1338
|
|
1339 /***********************************************************************
|
|
1340
|
|
1341 Is the conduit alive?
|
|
1342
|
|
1343 ***********************************************************************/
|
|
1344
|
|
1345 final override bool isAlive ()
|
|
1346 {
|
|
1347 return true;
|
|
1348 }
|
|
1349
|
|
1350 /***********************************************************************
|
|
1351
|
|
1352 Exposes configured output stream
|
|
1353
|
|
1354 Returns:
|
|
1355 Returns the OutputStream associated with this buffer. Returns
|
|
1356 null if the buffer is not attached to an output; that is, it's
|
|
1357 not backed by some external medium.
|
|
1358
|
|
1359 Remarks:
|
|
1360 Buffers do not require an external stream to operate, but
|
|
1361 it can be convenient to associate them. For example, methods
|
|
1362 fill & drain use them to import/export content as necessary.
|
|
1363
|
|
1364 ***********************************************************************/
|
|
1365
|
|
1366 final OutputStream output ()
|
|
1367 {
|
|
1368 return sink;
|
|
1369 }
|
|
1370
|
|
1371 /***********************************************************************
|
|
1372
|
|
1373 Exposes configured input stream
|
|
1374
|
|
1375 Returns:
|
|
1376 Returns the InputStream associated with this buffer. Returns
|
|
1377 null if the buffer is not attached to an input; that is, it's
|
|
1378 not backed by some external medium.
|
|
1379
|
|
1380 Remarks:
|
|
1381 Buffers do not require an external stream to operate, but
|
|
1382 it can be convenient to associate them. For example, methods
|
|
1383 fill & drain use them to import/export content as necessary.
|
|
1384
|
|
1385 ***********************************************************************/
|
|
1386
|
|
1387 final InputStream input ()
|
|
1388 {
|
|
1389 return source;
|
|
1390 }
|
|
1391
|
|
1392 /***********************************************************************
|
|
1393
|
|
1394 Release external resources
|
|
1395
|
|
1396 ***********************************************************************/
|
|
1397
|
|
1398 final override void detach ()
|
|
1399 {
|
|
1400 }
|
|
1401
|
|
1402 /***********************************************************************
|
|
1403
|
|
1404 Close the stream
|
|
1405
|
|
1406 Remarks:
|
|
1407 Propagate request to an attached OutputStream (this is a
|
|
1408 requirement for the OutputStream interface)
|
|
1409
|
|
1410 ***********************************************************************/
|
|
1411
|
|
1412 override void close ()
|
|
1413 {
|
|
1414 if (sink)
|
|
1415 sink.close;
|
|
1416 else
|
|
1417 if (source)
|
|
1418 source.close;
|
|
1419 }
|
|
1420 }
|