Mercurial > projects > ldc
annotate tango/tango/io/Buffer.d @ 136:0e28624814e8 trunk
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
author | lindquist |
---|---|
date | Thu, 17 Jan 2008 03:15:12 +0100 |
parents | 1700239cab2e |
children | ce7b81fb957f |
rev | line source |
---|---|
132 | 1 /******************************************************************************* |
2 | |
3 copyright: Copyright (c) 2004 Kris Bell. All rights reserved | |
4 | |
5 license: BSD style: $(LICENSE) | |
6 | |
7 version: Mar 2004: Initial release | |
8 Dec 2006: Outback release | |
9 | |
10 authors: Kris | |
11 | |
12 *******************************************************************************/ | |
13 | |
14 module tango.io.Buffer; | |
15 | |
16 private import tango.core.Exception; | |
17 | |
18 public import tango.io.model.IBuffer, | |
19 tango.io.model.IConduit; | |
20 | |
21 /****************************************************************************** | |
22 | |
23 ******************************************************************************/ | |
24 | |
25 extern (C) | |
26 { | |
27 protected void * memcpy (void *dst, void *src, uint); | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
28 private int printf(char*, ...); |
132 | 29 } |
30 | |
31 /******************************************************************************* | |
32 | |
33 Buffer is central concept in Tango I/O. Each buffer acts | |
34 as a queue (line) where items are removed from the front | |
35 and new items are added to the back. Buffers are modeled | |
36 by tango.io.model.IBuffer, and a concrete implementation | |
37 is provided by this class. | |
38 | |
39 Buffer can be read from and written to directly, though | |
40 various data-converters and filters are often leveraged | |
41 to apply structure to what might otherwise be simple raw | |
42 data. | |
43 | |
44 Buffers may also be tokenized by applying an Iterator. | |
45 This can be handy when one is dealing with text input, | |
46 and/or the content suits a more fluid format than most | |
47 typical converters support. Iterator tokens are mapped | |
48 directly onto buffer content (sliced), making them quite | |
49 efficient in practice. Like other types of buffer client, | |
50 multiple iterators can be mapped onto one common buffer | |
51 and access will be serialized. | |
52 | |
53 Buffers are sometimes memory-only, in which case there | |
54 is nothing left to do when a client has consumed all the | |
55 content. Other buffers are themselves bound to an external | |
56 device called a conduit. When this is the case, a consumer | |
57 will eventually cause a buffer to reload via its associated | |
58 conduit and previous buffer content will be lost. | |
59 | |
60 A similar approach is applied to clients which populate a | |
61 buffer, whereby the content of a full buffer will be flushed | |
62 to a bound conduit before continuing. Another variation is | |
63 that of a memory-mapped buffer, whereby the buffer content | |
64 is mapped directly to virtual memory exposed via the OS. This | |
65 can be used to address large files as an array of content. | |
66 | |
67 Direct buffer manipulation typically involves appending, | |
68 as in the following example: | |
69 --- | |
70 // create a small buffer | |
71 auto buf = new Buffer (256); | |
72 | |
73 auto foo = "to write some D"; | |
74 | |
75 // append some text directly to it | |
76 buf.append ("now is the time for all good men ").append(foo); | |
77 --- | |
78 | |
79 Alternatively, one might use a formatter to append the buffer: | |
80 --- | |
81 auto output = new FormatOutput (new Buffer(256)); | |
82 output.format ("now is the time for {} good men {}", 3, foo); | |
83 --- | |
84 | |
85 A slice() method will return all valid content within a buffer. | |
86 GrowBuffer can be used instead, where one wishes to append beyond | |
87 a specified limit. | |
88 | |
89 A common usage of a buffer is in conjunction with a conduit, | |
90 such as FileConduit. Each conduit exposes a preferred-size for | |
91 its associated buffers, utilized during buffer construction: | |
92 --- | |
93 auto file = new FileConduit ("file.name"); | |
94 auto buf = new Buffer (file); | |
95 --- | |
96 | |
97 However, this is typically hidden by higher level constructors | |
98 such as those exposed via the stream wrappers. For example: | |
99 --- | |
100 auto input = new DataInput (new FileInput("file.name")); | |
101 --- | |
102 | |
103 There is indeed a buffer between the resultant stream and the | |
104 file source, but explicit buffer construction is unecessary in | |
105 common cases. | |
106 | |
107 An Iterator is constructed in a similar manner, where you provide | |
108 it an input stream to operate upon. There's a variety of iterators | |
109 available in the tango.text.stream package, and they are templated | |
110 for each of utf8, utf16, and utf32. This example uses a line iterator | |
111 to sweep a text file: | |
112 --- | |
113 auto lines = new LineInput (new FileInput("file.name")); | |
114 foreach (line; lines) | |
115 Cout(line).newline; | |
116 --- | |
117 | |
118 Buffers are useful for many purposes within Tango, but there | |
119 are times when it may be more appropriate to sidestep them. For | |
120 such cases, all conduit derivatives (such as FileConduit) support | |
121 direct array-based IO via a pair of read() and write() methods. | |
122 | |
123 *******************************************************************************/ | |
124 | |
125 class Buffer : IBuffer | |
126 { | |
127 protected OutputStream sink; // optional data sink | |
128 protected InputStream source; // optional data source | |
129 protected void[] data; // the raw data buffer | |
130 protected uint index; // current read position | |
131 protected uint extent; // limit of valid content | |
132 protected uint dimension; // maximum extent of content | |
133 | |
134 | |
135 protected static char[] overflow = "output buffer is full"; | |
136 protected static char[] underflow = "input buffer is empty"; | |
137 protected static char[] eofRead = "end-of-flow whilst reading"; | |
138 protected static char[] eofWrite = "end-of-flow whilst writing"; | |
139 | |
140 /*********************************************************************** | |
141 | |
142 Ensure the buffer remains valid between method calls | |
143 | |
144 ***********************************************************************/ | |
145 | |
146 invariant | |
147 { | |
148 assert (index <= extent); | |
149 assert (extent <= dimension); | |
150 } | |
151 | |
152 /*********************************************************************** | |
153 | |
154 Construct a buffer | |
155 | |
156 Params: | |
157 conduit = the conduit to buffer | |
158 | |
159 Remarks: | |
160 Construct a Buffer upon the provided conduit. A relevant | |
161 buffer size is supplied via the provided conduit. | |
162 | |
163 ***********************************************************************/ | |
164 | |
165 this (IConduit conduit) | |
166 { | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
167 printf("Buffer.this(%p)\n", conduit); |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
168 assert (conduit !is null); |
132 | 169 assert (conduit); |
170 | |
171 this (conduit.bufferSize); | |
172 setConduit (conduit); | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
173 |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
174 assert(this !is null); |
132 | 175 } |
176 | |
177 /*********************************************************************** | |
178 | |
179 Construct a buffer | |
180 | |
181 Params: | |
182 stream = an input stream | |
183 capacity = desired buffer capacity | |
184 | |
185 Remarks: | |
186 Construct a Buffer upon the provided input stream. | |
187 | |
188 ***********************************************************************/ | |
189 | |
190 this (InputStream stream, uint capacity) | |
191 { | |
192 this (capacity); | |
193 input = stream; | |
194 } | |
195 | |
196 /*********************************************************************** | |
197 | |
198 Construct a buffer | |
199 | |
200 Params: | |
201 stream = an output stream | |
202 capacity = desired buffer capacity | |
203 | |
204 Remarks: | |
205 Construct a Buffer upon the provided output stream. | |
206 | |
207 ***********************************************************************/ | |
208 | |
209 this (OutputStream stream, uint capacity) | |
210 { | |
211 this (capacity); | |
212 output = stream; | |
213 } | |
214 | |
215 /*********************************************************************** | |
216 | |
217 Construct a buffer | |
218 | |
219 Params: | |
220 capacity = the number of bytes to make available | |
221 | |
222 Remarks: | |
223 Construct a Buffer with the specified number of bytes. | |
224 | |
225 ***********************************************************************/ | |
226 | |
227 this (uint capacity = 0) | |
228 { | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
229 setContent (new ubyte[capacity], 0); |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
230 assert(this !is null); |
132 | 231 } |
232 | |
233 /*********************************************************************** | |
234 | |
235 Construct a buffer | |
236 | |
237 Params: | |
238 data = the backing array to buffer within | |
239 | |
240 Remarks: | |
241 Prime a buffer with an application-supplied array. All content | |
242 is considered valid for reading, and thus there is no writable | |
243 space initially available. | |
244 | |
245 ***********************************************************************/ | |
246 | |
247 this (void[] data) | |
248 { | |
249 setContent (data, data.length); | |
250 } | |
251 | |
252 /*********************************************************************** | |
253 | |
254 Construct a buffer | |
255 | |
256 Params: | |
257 data = the backing array to buffer within | |
258 readable = the number of bytes initially made | |
259 readable | |
260 | |
261 Remarks: | |
262 Prime buffer with an application-supplied array, and | |
263 indicate how much readable data is already there. A | |
264 write operation will begin writing immediately after | |
265 the existing readable content. | |
266 | |
267 This is commonly used to attach a Buffer instance to | |
268 a local array. | |
269 | |
270 ***********************************************************************/ | |
271 | |
272 this (void[] data, uint readable) | |
273 { | |
274 setContent (data, readable); | |
275 } | |
276 | |
277 /*********************************************************************** | |
278 | |
279 Attempt to share an upstream Buffer, and create an instance | |
280 where there not one available. | |
281 | |
282 Params: | |
283 stream = an input stream | |
284 size = a hint of the desired buffer size. Defaults to the | |
285 conduit-defined size | |
286 | |
287 Remarks: | |
288 If an upstream Buffer instances is visible, it will be shared. | |
289 Otherwise, a new instance is created based upon the bufferSize | |
290 exposed by the stream endpoint (conduit). | |
291 | |
292 ***********************************************************************/ | |
293 | |
294 static IBuffer share (InputStream stream, uint size=uint.max) | |
295 { | |
296 auto b = cast(Buffered) stream; | |
297 if (b) | |
298 return b.buffer; | |
299 | |
300 if (size is uint.max) | |
301 size = stream.conduit.bufferSize; | |
302 | |
303 return new Buffer (stream, size); | |
304 } | |
305 | |
306 /*********************************************************************** | |
307 | |
308 Attempt to share an upstream Buffer, and create an instance | |
309 where there not one available. | |
310 | |
311 Params: | |
312 stream = an output stream | |
313 size = a hint of the desired buffer size. Defaults to the | |
314 conduit-defined size | |
315 | |
316 Remarks: | |
317 If an upstream Buffer instances is visible, it will be shared. | |
318 Otherwise, a new instance is created based upon the bufferSize | |
319 exposed by the stream endpoint (conduit). | |
320 | |
321 ***********************************************************************/ | |
322 | |
323 static IBuffer share (OutputStream stream, uint size=uint.max) | |
324 { | |
325 auto b = cast(Buffered) stream; | |
326 if (b) | |
327 return b.buffer; | |
328 | |
329 if (size is uint.max) | |
330 size = stream.conduit.bufferSize; | |
331 | |
332 return new Buffer (stream, size); | |
333 } | |
334 | |
335 /*********************************************************************** | |
336 | |
337 Reset the buffer content | |
338 | |
339 Params: | |
340 data = the backing array to buffer within. All content | |
341 is considered valid | |
342 | |
343 Returns: | |
344 the buffer instance | |
345 | |
346 Remarks: | |
347 Set the backing array with all content readable. Writing | |
348 to this will either flush it to an associated conduit, or | |
349 raise an Eof condition. Use clear() to reset the content | |
350 (make it all writable). | |
351 | |
352 ***********************************************************************/ | |
353 | |
354 IBuffer setContent (void[] data) | |
355 { | |
356 return setContent (data, data.length); | |
357 } | |
358 | |
359 /*********************************************************************** | |
360 | |
361 Reset the buffer content | |
362 | |
363 Params: | |
364 data = the backing array to buffer within | |
365 readable = the number of bytes within data considered | |
366 valid | |
367 | |
368 Returns: | |
369 the buffer instance | |
370 | |
371 Remarks: | |
372 Set the backing array with some content readable. Writing | |
373 to this will either flush it to an associated conduit, or | |
374 raise an Eof condition. Use clear() to reset the content | |
375 (make it all writable). | |
376 | |
377 ***********************************************************************/ | |
378 | |
379 IBuffer setContent (void[] data, uint readable) | |
380 { | |
381 this.data = data; | |
382 this.extent = readable; | |
383 this.dimension = data.length; | |
384 | |
385 // reset to start of input | |
386 this.index = 0; | |
387 | |
388 return this; | |
389 } | |
390 | |
391 /*********************************************************************** | |
392 | |
393 Access buffer content | |
394 | |
395 Params: | |
396 size = number of bytes to access | |
397 eat = whether to consume the content or not | |
398 | |
399 Returns: | |
400 the corresponding buffer slice when successful, or | |
401 null if there's not enough data available (Eof; Eob). | |
402 | |
403 Remarks: | |
404 Read a slice of data from the buffer, loading from the | |
405 conduit as necessary. The specified number of bytes is | |
406 sliced from the buffer, and marked as having been read | |
407 when the 'eat' parameter is set true. When 'eat' is set | |
408 false, the read position is not adjusted. | |
409 | |
410 Note that the slice cannot be larger than the size of | |
411 the buffer ~ use method fill(void[]) instead where you | |
412 simply want the content copied, or use conduit.read() | |
413 to extract directly from an attached conduit. Also note | |
414 that if you need to retain the slice, then it should be | |
415 .dup'd before the buffer is compressed or repopulated. | |
416 | |
417 Examples: | |
418 --- | |
419 // create a buffer with some content | |
420 auto buffer = new Buffer ("hello world"); | |
421 | |
422 // consume everything unread | |
423 auto slice = buffer.slice (buffer.readable); | |
424 --- | |
425 | |
426 ***********************************************************************/ | |
427 | |
428 void[] slice (uint size, bool eat = true) | |
429 { | |
430 if (size > readable) | |
431 { | |
432 if (source is null) | |
433 error (underflow); | |
434 | |
435 // make some space? This will try to leave as much content | |
436 // in the buffer as possible, such that entire records may | |
437 // be aliased directly from within. | |
438 if (size > writable) | |
439 { | |
440 if (size > dimension) | |
441 error (underflow); | |
442 compress (); | |
443 } | |
444 | |
445 // populate tail of buffer with new content | |
446 do { | |
447 if (fill(source) is IConduit.Eof) | |
448 error (eofRead); | |
449 } while (size > readable); | |
450 } | |
451 | |
452 auto i = index; | |
453 if (eat) | |
454 index += size; | |
455 return data [i .. i + size]; | |
456 } | |
457 | |
458 /********************************************************************** | |
459 | |
460 Fill the provided buffer. Returns the number of bytes | |
461 actually read, which will be less that dst.length when | |
462 Eof has been reached and IConduit.Eof thereafter | |
463 | |
464 **********************************************************************/ | |
465 | |
466 uint fill (void[] dst) | |
467 { | |
468 uint len = 0; | |
469 | |
470 while (len < dst.length) | |
471 { | |
472 uint i = read (dst [len .. $]); | |
473 if (i is IConduit.Eof) | |
474 return (len > 0) ? len : IConduit.Eof; | |
475 len += i; | |
476 } | |
477 return len; | |
478 } | |
479 | |
480 /*********************************************************************** | |
481 | |
482 Copy buffer content into the provided dst | |
483 | |
484 Params: | |
485 dst = destination of the content | |
486 bytes = size of dst | |
487 | |
488 Returns: | |
489 A reference to the populated content | |
490 | |
491 Remarks: | |
492 Fill the provided array with content. We try to satisfy | |
493 the request from the buffer content, and read directly | |
494 from an attached conduit where more is required. | |
495 | |
496 ***********************************************************************/ | |
497 | |
498 void[] readExact (void* dst, uint bytes) | |
499 { | |
500 auto tmp = dst [0 .. bytes]; | |
501 if (fill (tmp) != bytes) | |
502 error (eofRead); | |
503 | |
504 return tmp; | |
505 } | |
506 | |
507 /*********************************************************************** | |
508 | |
509 Append content | |
510 | |
511 Params: | |
512 src = the content to _append | |
513 | |
514 Returns a chaining reference if all content was written. | |
515 Throws an IOException indicating eof or eob if not. | |
516 | |
517 Remarks: | |
518 Append an array to this buffer, and flush to the | |
519 conduit as necessary. This is often used in lieu of | |
520 a Writer. | |
521 | |
522 ***********************************************************************/ | |
523 | |
524 IBuffer append (void[] src) | |
525 { | |
526 return append (src.ptr, src.length); | |
527 } | |
528 | |
529 /*********************************************************************** | |
530 | |
531 Append content | |
532 | |
533 Params: | |
534 src = the content to _append | |
535 length = the number of bytes in src | |
536 | |
537 Returns a chaining reference if all content was written. | |
538 Throws an IOException indicating eof or eob if not. | |
539 | |
540 Remarks: | |
541 Append an array to this buffer, and flush to the | |
542 conduit as necessary. This is often used in lieu of | |
543 a Writer. | |
544 | |
545 ***********************************************************************/ | |
546 | |
547 IBuffer append (void* src, uint length) | |
548 { | |
549 if (length > writable) | |
550 // can we write externally? | |
551 if (sink) | |
552 { | |
553 flush (); | |
554 | |
555 // check for pathological case | |
556 if (length > dimension) | |
557 { | |
558 do { | |
559 auto written = sink.write (src [0 .. length]); | |
560 if (written is IConduit.Eof) | |
561 error (eofWrite); | |
562 src += written, length -= written; | |
563 } while (length > dimension); | |
564 } | |
565 } | |
566 else | |
567 error (overflow); | |
568 | |
569 copy (src, length); | |
570 return this; | |
571 } | |
572 | |
573 /*********************************************************************** | |
574 | |
575 Append content | |
576 | |
577 Params: | |
578 other = a buffer with content available | |
579 | |
580 Returns: | |
581 Returns a chaining reference if all content was written. | |
582 Throws an IOException indicating eof or eob if not. | |
583 | |
584 Remarks: | |
585 Append another buffer to this one, and flush to the | |
586 conduit as necessary. This is often used in lieu of | |
587 a Writer. | |
588 | |
589 ***********************************************************************/ | |
590 | |
591 IBuffer append (IBuffer other) | |
592 { | |
593 return append (other.slice); | |
594 } | |
595 | |
596 /*********************************************************************** | |
597 | |
598 Consume content from a producer | |
599 | |
600 Params: | |
601 The content to consume. This is consumed verbatim, and in | |
602 raw binary format ~ no implicit conversions are performed. | |
603 | |
604 Remarks: | |
605 This is often used in lieu of a Writer, and enables simple | |
606 classes, such as FilePath and Uri, to emit content directly | |
607 into a buffer (thus avoiding potential heap activity) | |
608 | |
609 Examples: | |
610 --- | |
611 auto path = new FilePath (somepath); | |
612 | |
613 path.produce (&buffer.consume); | |
614 --- | |
615 | |
616 ***********************************************************************/ | |
617 | |
618 void consume (void[] x) | |
619 { | |
620 append (x); | |
621 } | |
622 | |
623 /*********************************************************************** | |
624 | |
625 Retrieve the valid content | |
626 | |
627 Returns: | |
628 a void[] slice of the buffer | |
629 | |
630 Remarks: | |
631 Return a void[] slice of the buffer, from the current position | |
632 up to the limit of valid content. The content remains in the | |
633 buffer for future extraction. | |
634 | |
635 ***********************************************************************/ | |
636 | |
637 void[] slice () | |
638 { | |
639 return data [index .. extent]; | |
640 } | |
641 | |
642 /*********************************************************************** | |
643 | |
644 Move the current read location | |
645 | |
646 Params: | |
647 size = the number of bytes to move | |
648 | |
649 Returns: | |
650 Returns true if successful, false otherwise. | |
651 | |
652 Remarks: | |
653 Skip ahead by the specified number of bytes, streaming from | |
654 the associated conduit as necessary. | |
655 | |
656 Can also reverse the read position by 'size' bytes, when size | |
657 is negative. This may be used to support lookahead operations. | |
658 Note that a negative size will fail where there is not sufficient | |
659 content available in the buffer (can't _skip beyond the beginning). | |
660 | |
661 ***********************************************************************/ | |
662 | |
663 bool skip (int size) | |
664 { | |
665 if (size < 0) | |
666 { | |
667 size = -size; | |
668 if (index >= size) | |
669 { | |
670 index -= size; | |
671 return true; | |
672 } | |
673 return false; | |
674 } | |
675 return slice(size) !is null; | |
676 } | |
677 | |
678 /*********************************************************************** | |
679 | |
680 Iterator support | |
681 | |
682 Params: | |
683 scan = the delagate to invoke with the current content | |
684 | |
685 Returns: | |
686 Returns true if a token was isolated, false otherwise. | |
687 | |
688 Remarks: | |
689 Upon success, the delegate should return the byte-based | |
690 index of the consumed pattern (tail end of it). Failure | |
691 to match a pattern should be indicated by returning an | |
692 IConduit.Eof | |
693 | |
694 Each pattern is expected to be stripped of the delimiter. | |
695 An end-of-file condition causes trailing content to be | |
696 placed into the token. Requests made beyond Eof result | |
697 in empty matches (length is zero). | |
698 | |
699 Note that additional iterator and/or reader instances | |
700 will operate in lockstep when bound to a common buffer. | |
701 | |
702 ***********************************************************************/ | |
703 | |
704 bool next (uint delegate (void[]) scan) | |
705 { | |
706 while (read(scan) is IConduit.Eof) | |
707 // not found - are we streaming? | |
708 if (source) | |
709 { | |
710 // did we start at the beginning? | |
711 if (position) | |
712 // nope - move partial token to start of buffer | |
713 compress (); | |
714 else | |
715 // no more space in the buffer? | |
716 if (writable is 0) | |
717 error ("Token is too large to fit within buffer"); | |
718 | |
719 // read another chunk of data | |
720 if (fill(source) is IConduit.Eof) | |
721 return false; | |
722 } | |
723 else | |
724 return false; | |
725 | |
726 return true; | |
727 } | |
728 | |
729 /*********************************************************************** | |
730 | |
731 Available content | |
732 | |
733 Remarks: | |
734 Return count of _readable bytes remaining in buffer. This is | |
735 calculated simply as limit() - position() | |
736 | |
737 ***********************************************************************/ | |
738 | |
739 uint readable () | |
740 { | |
741 return extent - index; | |
742 } | |
743 | |
744 /*********************************************************************** | |
745 | |
746 Available space | |
747 | |
748 Remarks: | |
749 Return count of _writable bytes available in buffer. This is | |
750 calculated simply as capacity() - limit() | |
751 | |
752 ***********************************************************************/ | |
753 | |
754 uint writable () | |
755 { | |
756 return dimension - extent; | |
757 } | |
758 | |
759 /*********************************************************************** | |
760 | |
761 Write into this buffer | |
762 | |
763 Params: | |
764 dg = the callback to provide buffer access to | |
765 | |
766 Returns: | |
767 Returns whatever the delegate returns. | |
768 | |
769 Remarks: | |
770 Exposes the raw data buffer at the current _write position, | |
771 The delegate is provided with a void[] representing space | |
772 available within the buffer at the current _write position. | |
773 | |
774 The delegate should return the appropriate number of bytes | |
775 if it writes valid content, or IConduit.Eof on error. | |
776 | |
777 ***********************************************************************/ | |
778 | |
779 uint write (uint delegate (void[]) dg) | |
780 { | |
781 int count = dg (data [extent..dimension]); | |
782 | |
783 if (count != IConduit.Eof) | |
784 { | |
785 extent += count; | |
786 assert (extent <= dimension); | |
787 } | |
788 return count; | |
789 } | |
790 | |
791 /*********************************************************************** | |
792 | |
793 Read directly from this buffer | |
794 | |
795 Params: | |
796 dg = callback to provide buffer access to | |
797 | |
798 Returns: | |
799 Returns whatever the delegate returns. | |
800 | |
801 Remarks: | |
802 Exposes the raw data buffer at the current _read position. The | |
803 delegate is provided with a void[] representing the available | |
804 data, and should return zero to leave the current _read position | |
805 intact. | |
806 | |
807 If the delegate consumes data, it should return the number of | |
808 bytes consumed; or IConduit.Eof to indicate an error. | |
809 | |
810 ***********************************************************************/ | |
811 | |
812 uint read (uint delegate (void[]) dg) | |
813 { | |
814 int count = dg (data [index..extent]); | |
815 | |
816 if (count != IConduit.Eof) | |
817 { | |
818 index += count; | |
819 assert (index <= extent); | |
820 } | |
821 return count; | |
822 } | |
823 | |
824 /*********************************************************************** | |
825 | |
826 Compress buffer space | |
827 | |
828 Returns: | |
829 the buffer instance | |
830 | |
831 Remarks: | |
832 If we have some data left after an export, move it to | |
833 front-of-buffer and set position to be just after the | |
834 remains. This is for supporting certain conduits which | |
835 choose to write just the initial portion of a request. | |
836 | |
837 Limit is set to the amount of data remaining. Position | |
838 is always reset to zero. | |
839 | |
840 ***********************************************************************/ | |
841 | |
842 IBuffer compress () | |
843 { | |
844 uint r = readable (); | |
845 | |
846 if (index > 0 && r > 0) | |
847 // content may overlap ... | |
848 memcpy (&data[0], &data[index], r); | |
849 | |
850 index = 0; | |
851 extent = r; | |
852 return this; | |
853 } | |
854 | |
855 /*********************************************************************** | |
856 | |
857 Fill buffer from the specific conduit | |
858 | |
859 Returns: | |
860 Returns the number of bytes read, or Conduit.Eof | |
861 | |
862 Remarks: | |
863 Try to _fill the available buffer with content from the | |
864 specified conduit. We try to read as much as possible | |
865 by clearing the buffer when all current content has been | |
866 eaten. If there is no space available, nothing will be | |
867 read. | |
868 | |
869 ***********************************************************************/ | |
870 | |
871 uint fill (InputStream src) | |
872 { | |
873 if (src is null) | |
874 return IConduit.Eof; | |
875 | |
876 if (readable is 0) | |
877 index = extent = 0; // same as clear(), but without chain | |
878 else | |
879 if (writable is 0) | |
880 return 0; | |
881 | |
882 return write (&src.read); | |
883 } | |
884 | |
885 /*********************************************************************** | |
886 | |
887 Drain buffer content to the specific conduit | |
888 | |
889 Returns: | |
890 Returns the number of bytes written | |
891 | |
892 Remarks: | |
893 Write as much of the buffer that the associated conduit | |
894 can consume. The conduit is not obliged to consume all | |
895 content, so some may remain within the buffer. | |
896 | |
897 Throws an IOException on premature Eof. | |
898 | |
899 ***********************************************************************/ | |
900 | |
901 final uint drain (OutputStream dst) | |
902 { | |
903 if (dst is null) | |
904 return IConduit.Eof; | |
905 | |
906 uint ret = read (&dst.write); | |
907 if (ret is IConduit.Eof) | |
908 error (eofWrite); | |
909 | |
910 compress (); | |
911 return ret; | |
912 } | |
913 | |
914 /*********************************************************************** | |
915 | |
916 Truncate buffer content | |
917 | |
918 Remarks: | |
919 Truncate the buffer within its extent. Returns true if | |
920 the new length is valid, false otherwise. | |
921 | |
922 ***********************************************************************/ | |
923 | |
924 bool truncate (uint length) | |
925 { | |
926 if (length <= data.length) | |
927 { | |
928 extent = length; | |
929 return true; | |
930 } | |
931 return false; | |
932 } | |
933 | |
934 /*********************************************************************** | |
935 | |
936 Access buffer limit | |
937 | |
938 Returns: | |
939 Returns the limit of readable content within this buffer. | |
940 | |
941 Remarks: | |
942 Each buffer has a capacity, a limit, and a position. The | |
943 capacity is the maximum content a buffer can contain, limit | |
944 represents the extent of valid content, and position marks | |
945 the current read location. | |
946 | |
947 ***********************************************************************/ | |
948 | |
949 uint limit () | |
950 { | |
951 return extent; | |
952 } | |
953 | |
954 /*********************************************************************** | |
955 | |
956 Access buffer capacity | |
957 | |
958 Returns: | |
959 Returns the maximum capacity of this buffer | |
960 | |
961 Remarks: | |
962 Each buffer has a capacity, a limit, and a position. The | |
963 capacity is the maximum content a buffer can contain, limit | |
964 represents the extent of valid content, and position marks | |
965 the current read location. | |
966 | |
967 ***********************************************************************/ | |
968 | |
969 uint capacity () | |
970 { | |
971 return dimension; | |
972 } | |
973 | |
974 /*********************************************************************** | |
975 | |
976 Access buffer read position | |
977 | |
978 Returns: | |
979 Returns the current read-position within this buffer | |
980 | |
981 Remarks: | |
982 Each buffer has a capacity, a limit, and a position. The | |
983 capacity is the maximum content a buffer can contain, limit | |
984 represents the extent of valid content, and position marks | |
985 the current read location. | |
986 | |
987 ***********************************************************************/ | |
988 | |
989 uint position () | |
990 { | |
991 return index; | |
992 } | |
993 | |
994 /*********************************************************************** | |
995 | |
996 Set external conduit | |
997 | |
998 Params: | |
999 conduit = the conduit to attach to | |
1000 | |
1001 Remarks: | |
1002 Sets the external conduit associated with this buffer. | |
1003 | |
1004 Buffers do not require an external conduit to operate, but | |
1005 it can be convenient to associate one. For example, methods | |
1006 fill() & drain() use it to import/export content as necessary. | |
1007 | |
1008 ***********************************************************************/ | |
1009 | |
1010 IBuffer setConduit (IConduit conduit) | |
1011 { | |
1012 sink = conduit.output; | |
1013 source = conduit.input; | |
1014 return this; | |
1015 } | |
1016 | |
1017 /*********************************************************************** | |
1018 | |
1019 Set output stream | |
1020 | |
1021 Params: | |
1022 sink = the stream to attach to | |
1023 | |
1024 Remarks: | |
1025 Sets the external output stream associated with this buffer. | |
1026 | |
1027 Buffers do not require an external stream to operate, but | |
1028 it can be convenient to associate one. For example, methods | |
1029 fill & drain use them to import/export content as necessary. | |
1030 | |
1031 ***********************************************************************/ | |
1032 | |
1033 final IBuffer output (OutputStream sink) | |
1034 { | |
1035 this.sink = sink; | |
1036 return this; | |
1037 } | |
1038 | |
1039 /*********************************************************************** | |
1040 | |
1041 Set input stream | |
1042 | |
1043 Params: | |
1044 source = the stream to attach to | |
1045 | |
1046 Remarks: | |
1047 Sets the external input stream associated with this buffer. | |
1048 | |
1049 Buffers do not require an external stream to operate, but | |
1050 it can be convenient to associate one. For example, methods | |
1051 fill & drain use them to import/export content as necessary. | |
1052 | |
1053 ***********************************************************************/ | |
1054 | |
1055 final IBuffer input (InputStream source) | |
1056 { | |
1057 this.source = source; | |
1058 return this; | |
1059 } | |
1060 | |
1061 /*********************************************************************** | |
1062 | |
1063 Access buffer content | |
1064 | |
1065 Remarks: | |
1066 Return the entire backing array. Exposed for subclass usage | |
1067 only | |
1068 | |
1069 ***********************************************************************/ | |
1070 | |
1071 protected void[] getContent () | |
1072 { | |
1073 return data; | |
1074 } | |
1075 | |
1076 /*********************************************************************** | |
1077 | |
1078 Copy content into buffer | |
1079 | |
1080 Params: | |
1081 src = the soure of the content | |
1082 size = the length of content at src | |
1083 | |
1084 Remarks: | |
1085 Bulk _copy of data from 'src'. The new content is made | |
1086 available for reading. This is exposed for subclass use | |
1087 only | |
1088 | |
1089 ***********************************************************************/ | |
1090 | |
1091 protected void copy (void *src, uint size) | |
1092 { | |
1093 // avoid "out of bounds" test on zero size | |
1094 if (size) | |
1095 { | |
1096 // content may overlap ... | |
1097 memcpy (&data[extent], src, size); | |
1098 extent += size; | |
1099 } | |
1100 } | |
1101 | |
1102 /*********************************************************************** | |
1103 | |
1104 Cast to a target type without invoking the wrath of the | |
1105 runtime checks for misalignment. Instead, we truncate the | |
1106 array length | |
1107 | |
1108 ***********************************************************************/ | |
1109 | |
1110 static T[] convert(T)(void[] x) | |
1111 { | |
1112 return (cast(T*) x.ptr) [0 .. (x.length / T.sizeof)]; | |
1113 } | |
1114 | |
1115 | |
1116 | |
1117 /**********************************************************************/ | |
1118 /*********************** Buffered Interface ***************************/ | |
1119 /**********************************************************************/ | |
1120 | |
1121 IBuffer buffer () | |
1122 { | |
1123 return this; | |
1124 } | |
1125 | |
1126 | |
1127 /**********************************************************************/ | |
1128 /******************** Stream & Conduit Interfaces *********************/ | |
1129 /**********************************************************************/ | |
1130 | |
1131 | |
1132 /*********************************************************************** | |
1133 | |
1134 Return the name of this conduit | |
1135 | |
1136 ***********************************************************************/ | |
1137 | |
1138 override char[] toString () | |
1139 { | |
1140 return "<buffer>"; | |
1141 } | |
1142 | |
1143 /*********************************************************************** | |
1144 | |
1145 Generic IOException thrower | |
1146 | |
1147 Params: | |
1148 msg = a text message describing the exception reason | |
1149 | |
1150 Remarks: | |
1151 Throw an IOException with the provided message | |
1152 | |
1153 ***********************************************************************/ | |
1154 | |
1155 final void error (char[] msg) | |
1156 { | |
1157 throw new IOException (msg); | |
1158 } | |
1159 | |
1160 /*********************************************************************** | |
1161 | |
1162 Flush all buffer content to the specific conduit | |
1163 | |
1164 Remarks: | |
1165 Flush the contents of this buffer. This will block until | |
1166 all content is actually flushed via the associated conduit, | |
1167 whereas drain() will not. | |
1168 | |
1169 Do nothing where a conduit is not attached, enabling memory | |
1170 buffers to treat flush as a noop. | |
1171 | |
1172 Throws an IOException on premature Eof. | |
1173 | |
1174 ***********************************************************************/ | |
1175 | |
1176 override OutputStream flush () | |
1177 { | |
1178 if (sink) | |
1179 { | |
1180 while (readable() > 0) | |
1181 drain (sink); | |
1182 | |
1183 // flush the filter chain also | |
1184 sink.flush; | |
1185 } | |
1186 return this; | |
1187 } | |
1188 | |
1189 /*********************************************************************** | |
1190 | |
1191 Clear buffer content | |
1192 | |
1193 Remarks: | |
1194 Reset 'position' and 'limit' to zero. This effectively | |
1195 clears all content from the buffer. | |
1196 | |
1197 ***********************************************************************/ | |
1198 | |
1199 override InputStream clear () | |
1200 { | |
1201 index = extent = 0; | |
1202 | |
1203 // clear the filter chain also | |
1204 if (source) | |
1205 source.clear; | |
1206 return this; | |
1207 } | |
1208 | |
1209 /*********************************************************************** | |
1210 | |
1211 Copy content via this buffer from the provided src | |
1212 conduit. | |
1213 | |
1214 Remarks: | |
1215 The src conduit has its content transferred through | |
1216 this buffer via a series of fill & drain operations, | |
1217 until there is no more content available. The buffer | |
1218 content should be explicitly flushed by the caller. | |
1219 | |
1220 Throws an IOException on premature eof | |
1221 | |
1222 ***********************************************************************/ | |
1223 | |
1224 override OutputStream copy (InputStream src) | |
1225 { | |
1226 while (fill(src) != IConduit.Eof) | |
1227 // don't drain until we actually need to | |
1228 if (writable is 0) | |
1229 if (sink) | |
1230 drain (sink); | |
1231 else | |
1232 error (overflow); | |
1233 return this; | |
1234 } | |
1235 | |
1236 /*********************************************************************** | |
1237 | |
1238 Transfer content into the provided dst | |
1239 | |
1240 Params: | |
1241 dst = destination of the content | |
1242 | |
1243 Returns: | |
1244 return the number of bytes read, which may be less than | |
1245 dst.length. Eof is returned when no further content is | |
1246 available. | |
1247 | |
1248 Remarks: | |
1249 Populates the provided array with content. We try to | |
1250 satisfy the request from the buffer content, and read | |
1251 directly from an attached conduit when the buffer is | |
1252 empty. | |
1253 | |
1254 ***********************************************************************/ | |
1255 | |
1256 override uint read (void[] dst) | |
1257 { | |
1258 uint content = readable(); | |
1259 if (content) | |
1260 { | |
1261 if (content >= dst.length) | |
1262 content = dst.length; | |
1263 | |
1264 // transfer buffer content | |
1265 dst [0 .. content] = data [index .. index + content]; | |
1266 index += content; | |
1267 } | |
1268 else | |
1269 if (source) | |
1270 { | |
1271 // pathological cases read directly from conduit | |
1272 if (dst.length > dimension) | |
1273 content = source.read (dst); | |
1274 else | |
1275 // keep buffer partially populated | |
1276 if ((content = fill(source)) != IConduit.Eof && content > 0) | |
1277 content = read (dst); | |
1278 } | |
1279 else | |
1280 content = IConduit.Eof; | |
1281 return content; | |
1282 } | |
1283 | |
1284 /*********************************************************************** | |
1285 | |
1286 Emulate OutputStream.write() | |
1287 | |
1288 Params: | |
1289 src = the content to write | |
1290 | |
1291 Returns: | |
1292 return the number of bytes written, which may be less than | |
1293 provided (conceptually). | |
1294 | |
1295 Remarks: | |
1296 Appends src content to the buffer, flushing to an attached | |
1297 conduit as necessary. An IOException is thrown upon write | |
1298 failure. | |
1299 | |
1300 ***********************************************************************/ | |
1301 | |
1302 override uint write (void[] src) | |
1303 { | |
1304 append (src.ptr, src.length); | |
1305 return src.length; | |
1306 } | |
1307 | |
1308 /*********************************************************************** | |
1309 | |
1310 Access configured conduit | |
1311 | |
1312 Returns: | |
1313 Returns the conduit associated with this buffer. Returns | |
1314 null if the buffer is purely memory based; that is, it's | |
1315 not backed by some external medium. | |
1316 | |
1317 Remarks: | |
1318 Buffers do not require an external conduit to operate, but | |
1319 it can be convenient to associate one. For example, methods | |
1320 fill() & drain() use it to import/export content as necessary. | |
1321 | |
1322 ***********************************************************************/ | |
1323 | |
1324 final override IConduit conduit () | |
1325 { | |
1326 if (sink) | |
1327 return sink.conduit; | |
1328 else | |
1329 if (source) | |
1330 return source.conduit; | |
1331 return this; | |
1332 } | |
1333 | |
1334 /*********************************************************************** | |
1335 | |
1336 Return a preferred size for buffering conduit I/O | |
1337 | |
1338 ***********************************************************************/ | |
1339 | |
1340 final override uint bufferSize () | |
1341 { | |
1342 return 32 * 1024; | |
1343 } | |
1344 | |
1345 /*********************************************************************** | |
1346 | |
1347 Is the conduit alive? | |
1348 | |
1349 ***********************************************************************/ | |
1350 | |
1351 final override bool isAlive () | |
1352 { | |
1353 return true; | |
1354 } | |
1355 | |
1356 /*********************************************************************** | |
1357 | |
1358 Exposes configured output stream | |
1359 | |
1360 Returns: | |
1361 Returns the OutputStream associated with this buffer. Returns | |
1362 null if the buffer is not attached to an output; that is, it's | |
1363 not backed by some external medium. | |
1364 | |
1365 Remarks: | |
1366 Buffers do not require an external stream to operate, but | |
1367 it can be convenient to associate them. For example, methods | |
1368 fill & drain use them to import/export content as necessary. | |
1369 | |
1370 ***********************************************************************/ | |
1371 | |
1372 final OutputStream output () | |
1373 { | |
1374 return sink; | |
1375 } | |
1376 | |
1377 /*********************************************************************** | |
1378 | |
1379 Exposes configured input stream | |
1380 | |
1381 Returns: | |
1382 Returns the InputStream associated with this buffer. Returns | |
1383 null if the buffer is not attached to an input; that is, it's | |
1384 not backed by some external medium. | |
1385 | |
1386 Remarks: | |
1387 Buffers do not require an external stream to operate, but | |
1388 it can be convenient to associate them. For example, methods | |
1389 fill & drain use them to import/export content as necessary. | |
1390 | |
1391 ***********************************************************************/ | |
1392 | |
1393 final InputStream input () | |
1394 { | |
1395 return source; | |
1396 } | |
1397 | |
1398 /*********************************************************************** | |
1399 | |
1400 Release external resources | |
1401 | |
1402 ***********************************************************************/ | |
1403 | |
1404 final override void detach () | |
1405 { | |
1406 } | |
1407 | |
1408 /*********************************************************************** | |
1409 | |
1410 Close the stream | |
1411 | |
1412 Remarks: | |
1413 Propagate request to an attached OutputStream (this is a | |
1414 requirement for the OutputStream interface) | |
1415 | |
1416 ***********************************************************************/ | |
1417 | |
1418 override void close () | |
1419 { | |
1420 if (sink) | |
1421 sink.close; | |
1422 else | |
1423 if (source) | |
1424 source.close; | |
1425 } | |
1426 } |