Mercurial > projects > ldc
annotate tango/tango/io/Buffer.d @ 138:aeddd4d533b3 trunk
[svn r142] minor fix to dynamic casts.
added a few missed files.
author | lindquist |
---|---|
date | Fri, 18 Jan 2008 20:13:19 +0100 |
parents | ce7b81fb957f |
children | 0ab29b838084 |
rev | line source |
---|---|
132 | 1 /******************************************************************************* |
2 | |
3 copyright: Copyright (c) 2004 Kris Bell. All rights reserved | |
4 | |
5 license: BSD style: $(LICENSE) | |
6 | |
7 version: Mar 2004: Initial release | |
8 Dec 2006: Outback release | |
9 | |
10 authors: Kris | |
11 | |
12 *******************************************************************************/ | |
13 | |
14 module tango.io.Buffer; | |
15 | |
16 private import tango.core.Exception; | |
17 | |
18 public import tango.io.model.IBuffer, | |
19 tango.io.model.IConduit; | |
20 | |
21 /****************************************************************************** | |
22 | |
23 ******************************************************************************/ | |
24 | |
25 extern (C) | |
26 { | |
27 protected void * memcpy (void *dst, void *src, uint); | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
28 private int printf(char*, ...); |
132 | 29 } |
30 | |
31 /******************************************************************************* | |
32 | |
33 Buffer is central concept in Tango I/O. Each buffer acts | |
34 as a queue (line) where items are removed from the front | |
35 and new items are added to the back. Buffers are modeled | |
36 by tango.io.model.IBuffer, and a concrete implementation | |
37 is provided by this class. | |
38 | |
39 Buffer can be read from and written to directly, though | |
40 various data-converters and filters are often leveraged | |
41 to apply structure to what might otherwise be simple raw | |
42 data. | |
43 | |
44 Buffers may also be tokenized by applying an Iterator. | |
45 This can be handy when one is dealing with text input, | |
46 and/or the content suits a more fluid format than most | |
47 typical converters support. Iterator tokens are mapped | |
48 directly onto buffer content (sliced), making them quite | |
49 efficient in practice. Like other types of buffer client, | |
50 multiple iterators can be mapped onto one common buffer | |
51 and access will be serialized. | |
52 | |
53 Buffers are sometimes memory-only, in which case there | |
54 is nothing left to do when a client has consumed all the | |
55 content. Other buffers are themselves bound to an external | |
56 device called a conduit. When this is the case, a consumer | |
57 will eventually cause a buffer to reload via its associated | |
58 conduit and previous buffer content will be lost. | |
59 | |
60 A similar approach is applied to clients which populate a | |
61 buffer, whereby the content of a full buffer will be flushed | |
62 to a bound conduit before continuing. Another variation is | |
63 that of a memory-mapped buffer, whereby the buffer content | |
64 is mapped directly to virtual memory exposed via the OS. This | |
65 can be used to address large files as an array of content. | |
66 | |
67 Direct buffer manipulation typically involves appending, | |
68 as in the following example: | |
69 --- | |
70 // create a small buffer | |
71 auto buf = new Buffer (256); | |
72 | |
73 auto foo = "to write some D"; | |
74 | |
75 // append some text directly to it | |
76 buf.append ("now is the time for all good men ").append(foo); | |
77 --- | |
78 | |
79 Alternatively, one might use a formatter to append the buffer: | |
80 --- | |
81 auto output = new FormatOutput (new Buffer(256)); | |
82 output.format ("now is the time for {} good men {}", 3, foo); | |
83 --- | |
84 | |
85 A slice() method will return all valid content within a buffer. | |
86 GrowBuffer can be used instead, where one wishes to append beyond | |
87 a specified limit. | |
88 | |
89 A common usage of a buffer is in conjunction with a conduit, | |
90 such as FileConduit. Each conduit exposes a preferred-size for | |
91 its associated buffers, utilized during buffer construction: | |
92 --- | |
93 auto file = new FileConduit ("file.name"); | |
94 auto buf = new Buffer (file); | |
95 --- | |
96 | |
97 However, this is typically hidden by higher level constructors | |
98 such as those exposed via the stream wrappers. For example: | |
99 --- | |
100 auto input = new DataInput (new FileInput("file.name")); | |
101 --- | |
102 | |
103 There is indeed a buffer between the resultant stream and the | |
104 file source, but explicit buffer construction is unecessary in | |
105 common cases. | |
106 | |
107 An Iterator is constructed in a similar manner, where you provide | |
108 it an input stream to operate upon. There's a variety of iterators | |
109 available in the tango.text.stream package, and they are templated | |
110 for each of utf8, utf16, and utf32. This example uses a line iterator | |
111 to sweep a text file: | |
112 --- | |
113 auto lines = new LineInput (new FileInput("file.name")); | |
114 foreach (line; lines) | |
115 Cout(line).newline; | |
116 --- | |
117 | |
118 Buffers are useful for many purposes within Tango, but there | |
119 are times when it may be more appropriate to sidestep them. For | |
120 such cases, all conduit derivatives (such as FileConduit) support | |
121 direct array-based IO via a pair of read() and write() methods. | |
122 | |
123 *******************************************************************************/ | |
124 | |
125 class Buffer : IBuffer | |
126 { | |
127 protected OutputStream sink; // optional data sink | |
128 protected InputStream source; // optional data source | |
129 protected void[] data; // the raw data buffer | |
130 protected uint index; // current read position | |
131 protected uint extent; // limit of valid content | |
132 protected uint dimension; // maximum extent of content | |
133 | |
134 | |
135 protected static char[] overflow = "output buffer is full"; | |
136 protected static char[] underflow = "input buffer is empty"; | |
137 protected static char[] eofRead = "end-of-flow whilst reading"; | |
138 protected static char[] eofWrite = "end-of-flow whilst writing"; | |
139 | |
140 /*********************************************************************** | |
141 | |
142 Ensure the buffer remains valid between method calls | |
143 | |
144 ***********************************************************************/ | |
145 | |
146 invariant | |
147 { | |
148 assert (index <= extent); | |
149 assert (extent <= dimension); | |
150 } | |
151 | |
152 /*********************************************************************** | |
153 | |
154 Construct a buffer | |
155 | |
156 Params: | |
157 conduit = the conduit to buffer | |
158 | |
159 Remarks: | |
160 Construct a Buffer upon the provided conduit. A relevant | |
161 buffer size is supplied via the provided conduit. | |
162 | |
163 ***********************************************************************/ | |
164 | |
165 this (IConduit conduit) | |
166 { | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
167 printf("Buffer.this(%p)\n", conduit); |
137 | 168 printf("assert (conduit !is null);\n"); |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
169 assert (conduit !is null); |
137 | 170 printf("assert (conduit);\n", conduit); |
132 | 171 assert (conduit); |
172 | |
137 | 173 printf("this (conduit.bufferSize(%p));\n", conduit); |
174 printf("cast(Object)conduit = %p\n", cast(Object)conduit); | |
132 | 175 this (conduit.bufferSize); |
176 setConduit (conduit); | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
177 |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
178 assert(this !is null); |
132 | 179 } |
180 | |
181 /*********************************************************************** | |
182 | |
183 Construct a buffer | |
184 | |
185 Params: | |
186 stream = an input stream | |
187 capacity = desired buffer capacity | |
188 | |
189 Remarks: | |
190 Construct a Buffer upon the provided input stream. | |
191 | |
192 ***********************************************************************/ | |
193 | |
194 this (InputStream stream, uint capacity) | |
195 { | |
196 this (capacity); | |
197 input = stream; | |
198 } | |
199 | |
200 /*********************************************************************** | |
201 | |
202 Construct a buffer | |
203 | |
204 Params: | |
205 stream = an output stream | |
206 capacity = desired buffer capacity | |
207 | |
208 Remarks: | |
209 Construct a Buffer upon the provided output stream. | |
210 | |
211 ***********************************************************************/ | |
212 | |
213 this (OutputStream stream, uint capacity) | |
214 { | |
215 this (capacity); | |
216 output = stream; | |
217 } | |
218 | |
219 /*********************************************************************** | |
220 | |
221 Construct a buffer | |
222 | |
223 Params: | |
224 capacity = the number of bytes to make available | |
225 | |
226 Remarks: | |
227 Construct a Buffer with the specified number of bytes. | |
228 | |
229 ***********************************************************************/ | |
230 | |
231 this (uint capacity = 0) | |
232 { | |
137 | 233 printf("Buffer.this(%p, %u)\n", this, capacity); |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
234 setContent (new ubyte[capacity], 0); |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
235 assert(this !is null); |
132 | 236 } |
237 | |
238 /*********************************************************************** | |
239 | |
240 Construct a buffer | |
241 | |
242 Params: | |
243 data = the backing array to buffer within | |
244 | |
245 Remarks: | |
246 Prime a buffer with an application-supplied array. All content | |
247 is considered valid for reading, and thus there is no writable | |
248 space initially available. | |
249 | |
250 ***********************************************************************/ | |
251 | |
252 this (void[] data) | |
253 { | |
254 setContent (data, data.length); | |
255 } | |
256 | |
257 /*********************************************************************** | |
258 | |
259 Construct a buffer | |
260 | |
261 Params: | |
262 data = the backing array to buffer within | |
263 readable = the number of bytes initially made | |
264 readable | |
265 | |
266 Remarks: | |
267 Prime buffer with an application-supplied array, and | |
268 indicate how much readable data is already there. A | |
269 write operation will begin writing immediately after | |
270 the existing readable content. | |
271 | |
272 This is commonly used to attach a Buffer instance to | |
273 a local array. | |
274 | |
275 ***********************************************************************/ | |
276 | |
277 this (void[] data, uint readable) | |
278 { | |
279 setContent (data, readable); | |
280 } | |
281 | |
282 /*********************************************************************** | |
283 | |
284 Attempt to share an upstream Buffer, and create an instance | |
285 where there not one available. | |
286 | |
287 Params: | |
288 stream = an input stream | |
289 size = a hint of the desired buffer size. Defaults to the | |
290 conduit-defined size | |
291 | |
292 Remarks: | |
293 If an upstream Buffer instances is visible, it will be shared. | |
294 Otherwise, a new instance is created based upon the bufferSize | |
295 exposed by the stream endpoint (conduit). | |
296 | |
297 ***********************************************************************/ | |
298 | |
299 static IBuffer share (InputStream stream, uint size=uint.max) | |
300 { | |
301 auto b = cast(Buffered) stream; | |
302 if (b) | |
303 return b.buffer; | |
304 | |
305 if (size is uint.max) | |
306 size = stream.conduit.bufferSize; | |
307 | |
308 return new Buffer (stream, size); | |
309 } | |
310 | |
311 /*********************************************************************** | |
312 | |
313 Attempt to share an upstream Buffer, and create an instance | |
314 where there not one available. | |
315 | |
316 Params: | |
317 stream = an output stream | |
318 size = a hint of the desired buffer size. Defaults to the | |
319 conduit-defined size | |
320 | |
321 Remarks: | |
322 If an upstream Buffer instances is visible, it will be shared. | |
323 Otherwise, a new instance is created based upon the bufferSize | |
324 exposed by the stream endpoint (conduit). | |
325 | |
326 ***********************************************************************/ | |
327 | |
328 static IBuffer share (OutputStream stream, uint size=uint.max) | |
329 { | |
330 auto b = cast(Buffered) stream; | |
331 if (b) | |
332 return b.buffer; | |
333 | |
334 if (size is uint.max) | |
335 size = stream.conduit.bufferSize; | |
336 | |
337 return new Buffer (stream, size); | |
338 } | |
339 | |
340 /*********************************************************************** | |
341 | |
342 Reset the buffer content | |
343 | |
344 Params: | |
345 data = the backing array to buffer within. All content | |
346 is considered valid | |
347 | |
348 Returns: | |
349 the buffer instance | |
350 | |
351 Remarks: | |
352 Set the backing array with all content readable. Writing | |
353 to this will either flush it to an associated conduit, or | |
354 raise an Eof condition. Use clear() to reset the content | |
355 (make it all writable). | |
356 | |
357 ***********************************************************************/ | |
358 | |
359 IBuffer setContent (void[] data) | |
360 { | |
361 return setContent (data, data.length); | |
362 } | |
363 | |
364 /*********************************************************************** | |
365 | |
366 Reset the buffer content | |
367 | |
368 Params: | |
369 data = the backing array to buffer within | |
370 readable = the number of bytes within data considered | |
371 valid | |
372 | |
373 Returns: | |
374 the buffer instance | |
375 | |
376 Remarks: | |
377 Set the backing array with some content readable. Writing | |
378 to this will either flush it to an associated conduit, or | |
379 raise an Eof condition. Use clear() to reset the content | |
380 (make it all writable). | |
381 | |
382 ***********************************************************************/ | |
383 | |
384 IBuffer setContent (void[] data, uint readable) | |
385 { | |
386 this.data = data; | |
387 this.extent = readable; | |
388 this.dimension = data.length; | |
389 | |
390 // reset to start of input | |
391 this.index = 0; | |
392 | |
393 return this; | |
394 } | |
395 | |
396 /*********************************************************************** | |
397 | |
398 Access buffer content | |
399 | |
400 Params: | |
401 size = number of bytes to access | |
402 eat = whether to consume the content or not | |
403 | |
404 Returns: | |
405 the corresponding buffer slice when successful, or | |
406 null if there's not enough data available (Eof; Eob). | |
407 | |
408 Remarks: | |
409 Read a slice of data from the buffer, loading from the | |
410 conduit as necessary. The specified number of bytes is | |
411 sliced from the buffer, and marked as having been read | |
412 when the 'eat' parameter is set true. When 'eat' is set | |
413 false, the read position is not adjusted. | |
414 | |
415 Note that the slice cannot be larger than the size of | |
416 the buffer ~ use method fill(void[]) instead where you | |
417 simply want the content copied, or use conduit.read() | |
418 to extract directly from an attached conduit. Also note | |
419 that if you need to retain the slice, then it should be | |
420 .dup'd before the buffer is compressed or repopulated. | |
421 | |
422 Examples: | |
423 --- | |
424 // create a buffer with some content | |
425 auto buffer = new Buffer ("hello world"); | |
426 | |
427 // consume everything unread | |
428 auto slice = buffer.slice (buffer.readable); | |
429 --- | |
430 | |
431 ***********************************************************************/ | |
432 | |
433 void[] slice (uint size, bool eat = true) | |
434 { | |
435 if (size > readable) | |
436 { | |
437 if (source is null) | |
438 error (underflow); | |
439 | |
440 // make some space? This will try to leave as much content | |
441 // in the buffer as possible, such that entire records may | |
442 // be aliased directly from within. | |
443 if (size > writable) | |
444 { | |
445 if (size > dimension) | |
446 error (underflow); | |
447 compress (); | |
448 } | |
449 | |
450 // populate tail of buffer with new content | |
451 do { | |
452 if (fill(source) is IConduit.Eof) | |
453 error (eofRead); | |
454 } while (size > readable); | |
455 } | |
456 | |
457 auto i = index; | |
458 if (eat) | |
459 index += size; | |
460 return data [i .. i + size]; | |
461 } | |
462 | |
463 /********************************************************************** | |
464 | |
465 Fill the provided buffer. Returns the number of bytes | |
466 actually read, which will be less that dst.length when | |
467 Eof has been reached and IConduit.Eof thereafter | |
468 | |
469 **********************************************************************/ | |
470 | |
471 uint fill (void[] dst) | |
472 { | |
473 uint len = 0; | |
474 | |
475 while (len < dst.length) | |
476 { | |
477 uint i = read (dst [len .. $]); | |
478 if (i is IConduit.Eof) | |
479 return (len > 0) ? len : IConduit.Eof; | |
480 len += i; | |
481 } | |
482 return len; | |
483 } | |
484 | |
485 /*********************************************************************** | |
486 | |
487 Copy buffer content into the provided dst | |
488 | |
489 Params: | |
490 dst = destination of the content | |
491 bytes = size of dst | |
492 | |
493 Returns: | |
494 A reference to the populated content | |
495 | |
496 Remarks: | |
497 Fill the provided array with content. We try to satisfy | |
498 the request from the buffer content, and read directly | |
499 from an attached conduit where more is required. | |
500 | |
501 ***********************************************************************/ | |
502 | |
503 void[] readExact (void* dst, uint bytes) | |
504 { | |
505 auto tmp = dst [0 .. bytes]; | |
506 if (fill (tmp) != bytes) | |
507 error (eofRead); | |
508 | |
509 return tmp; | |
510 } | |
511 | |
512 /*********************************************************************** | |
513 | |
514 Append content | |
515 | |
516 Params: | |
517 src = the content to _append | |
518 | |
519 Returns a chaining reference if all content was written. | |
520 Throws an IOException indicating eof or eob if not. | |
521 | |
522 Remarks: | |
523 Append an array to this buffer, and flush to the | |
524 conduit as necessary. This is often used in lieu of | |
525 a Writer. | |
526 | |
527 ***********************************************************************/ | |
528 | |
529 IBuffer append (void[] src) | |
530 { | |
531 return append (src.ptr, src.length); | |
532 } | |
533 | |
534 /*********************************************************************** | |
535 | |
536 Append content | |
537 | |
538 Params: | |
539 src = the content to _append | |
540 length = the number of bytes in src | |
541 | |
542 Returns a chaining reference if all content was written. | |
543 Throws an IOException indicating eof or eob if not. | |
544 | |
545 Remarks: | |
546 Append an array to this buffer, and flush to the | |
547 conduit as necessary. This is often used in lieu of | |
548 a Writer. | |
549 | |
550 ***********************************************************************/ | |
551 | |
552 IBuffer append (void* src, uint length) | |
138 | 553 { |
554 printf("Buffer.append(%p, %u)\n", src, length); | |
132 | 555 if (length > writable) |
556 // can we write externally? | |
557 if (sink) | |
558 { | |
559 flush (); | |
560 | |
561 // check for pathological case | |
562 if (length > dimension) | |
563 { | |
564 do { | |
565 auto written = sink.write (src [0 .. length]); | |
566 if (written is IConduit.Eof) | |
567 error (eofWrite); | |
568 src += written, length -= written; | |
569 } while (length > dimension); | |
570 } | |
571 } | |
572 else | |
573 error (overflow); | |
138 | 574 printf(" copying\n"); |
132 | 575 copy (src, length); |
138 | 576 printf("returning\n"); |
132 | 577 return this; |
578 } | |
579 | |
580 /*********************************************************************** | |
581 | |
582 Append content | |
583 | |
584 Params: | |
585 other = a buffer with content available | |
586 | |
587 Returns: | |
588 Returns a chaining reference if all content was written. | |
589 Throws an IOException indicating eof or eob if not. | |
590 | |
591 Remarks: | |
592 Append another buffer to this one, and flush to the | |
593 conduit as necessary. This is often used in lieu of | |
594 a Writer. | |
595 | |
596 ***********************************************************************/ | |
597 | |
598 IBuffer append (IBuffer other) | |
599 { | |
600 return append (other.slice); | |
601 } | |
602 | |
603 /*********************************************************************** | |
604 | |
605 Consume content from a producer | |
606 | |
607 Params: | |
608 The content to consume. This is consumed verbatim, and in | |
609 raw binary format ~ no implicit conversions are performed. | |
610 | |
611 Remarks: | |
612 This is often used in lieu of a Writer, and enables simple | |
613 classes, such as FilePath and Uri, to emit content directly | |
614 into a buffer (thus avoiding potential heap activity) | |
615 | |
616 Examples: | |
617 --- | |
618 auto path = new FilePath (somepath); | |
619 | |
620 path.produce (&buffer.consume); | |
621 --- | |
622 | |
623 ***********************************************************************/ | |
624 | |
625 void consume (void[] x) | |
626 { | |
627 append (x); | |
628 } | |
629 | |
630 /*********************************************************************** | |
631 | |
632 Retrieve the valid content | |
633 | |
634 Returns: | |
635 a void[] slice of the buffer | |
636 | |
637 Remarks: | |
638 Return a void[] slice of the buffer, from the current position | |
639 up to the limit of valid content. The content remains in the | |
640 buffer for future extraction. | |
641 | |
642 ***********************************************************************/ | |
643 | |
644 void[] slice () | |
645 { | |
646 return data [index .. extent]; | |
647 } | |
648 | |
649 /*********************************************************************** | |
650 | |
651 Move the current read location | |
652 | |
653 Params: | |
654 size = the number of bytes to move | |
655 | |
656 Returns: | |
657 Returns true if successful, false otherwise. | |
658 | |
659 Remarks: | |
660 Skip ahead by the specified number of bytes, streaming from | |
661 the associated conduit as necessary. | |
662 | |
663 Can also reverse the read position by 'size' bytes, when size | |
664 is negative. This may be used to support lookahead operations. | |
665 Note that a negative size will fail where there is not sufficient | |
666 content available in the buffer (can't _skip beyond the beginning). | |
667 | |
668 ***********************************************************************/ | |
669 | |
670 bool skip (int size) | |
671 { | |
672 if (size < 0) | |
673 { | |
674 size = -size; | |
675 if (index >= size) | |
676 { | |
677 index -= size; | |
678 return true; | |
679 } | |
680 return false; | |
681 } | |
682 return slice(size) !is null; | |
683 } | |
684 | |
685 /*********************************************************************** | |
686 | |
687 Iterator support | |
688 | |
689 Params: | |
690 scan = the delagate to invoke with the current content | |
691 | |
692 Returns: | |
693 Returns true if a token was isolated, false otherwise. | |
694 | |
695 Remarks: | |
696 Upon success, the delegate should return the byte-based | |
697 index of the consumed pattern (tail end of it). Failure | |
698 to match a pattern should be indicated by returning an | |
699 IConduit.Eof | |
700 | |
701 Each pattern is expected to be stripped of the delimiter. | |
702 An end-of-file condition causes trailing content to be | |
703 placed into the token. Requests made beyond Eof result | |
704 in empty matches (length is zero). | |
705 | |
706 Note that additional iterator and/or reader instances | |
707 will operate in lockstep when bound to a common buffer. | |
708 | |
709 ***********************************************************************/ | |
710 | |
711 bool next (uint delegate (void[]) scan) | |
712 { | |
713 while (read(scan) is IConduit.Eof) | |
714 // not found - are we streaming? | |
715 if (source) | |
716 { | |
717 // did we start at the beginning? | |
718 if (position) | |
719 // nope - move partial token to start of buffer | |
720 compress (); | |
721 else | |
722 // no more space in the buffer? | |
723 if (writable is 0) | |
724 error ("Token is too large to fit within buffer"); | |
725 | |
726 // read another chunk of data | |
727 if (fill(source) is IConduit.Eof) | |
728 return false; | |
729 } | |
730 else | |
731 return false; | |
732 | |
733 return true; | |
734 } | |
735 | |
736 /*********************************************************************** | |
737 | |
738 Available content | |
739 | |
740 Remarks: | |
741 Return count of _readable bytes remaining in buffer. This is | |
742 calculated simply as limit() - position() | |
743 | |
744 ***********************************************************************/ | |
745 | |
746 uint readable () | |
747 { | |
748 return extent - index; | |
749 } | |
750 | |
751 /*********************************************************************** | |
752 | |
753 Available space | |
754 | |
755 Remarks: | |
756 Return count of _writable bytes available in buffer. This is | |
757 calculated simply as capacity() - limit() | |
758 | |
759 ***********************************************************************/ | |
760 | |
761 uint writable () | |
762 { | |
763 return dimension - extent; | |
764 } | |
765 | |
766 /*********************************************************************** | |
767 | |
768 Write into this buffer | |
769 | |
770 Params: | |
771 dg = the callback to provide buffer access to | |
772 | |
773 Returns: | |
774 Returns whatever the delegate returns. | |
775 | |
776 Remarks: | |
777 Exposes the raw data buffer at the current _write position, | |
778 The delegate is provided with a void[] representing space | |
779 available within the buffer at the current _write position. | |
780 | |
781 The delegate should return the appropriate number of bytes | |
782 if it writes valid content, or IConduit.Eof on error. | |
783 | |
784 ***********************************************************************/ | |
785 | |
786 uint write (uint delegate (void[]) dg) | |
787 { | |
788 int count = dg (data [extent..dimension]); | |
789 | |
790 if (count != IConduit.Eof) | |
791 { | |
792 extent += count; | |
793 assert (extent <= dimension); | |
794 } | |
795 return count; | |
796 } | |
797 | |
798 /*********************************************************************** | |
799 | |
800 Read directly from this buffer | |
801 | |
802 Params: | |
803 dg = callback to provide buffer access to | |
804 | |
805 Returns: | |
806 Returns whatever the delegate returns. | |
807 | |
808 Remarks: | |
809 Exposes the raw data buffer at the current _read position. The | |
810 delegate is provided with a void[] representing the available | |
811 data, and should return zero to leave the current _read position | |
812 intact. | |
813 | |
814 If the delegate consumes data, it should return the number of | |
815 bytes consumed; or IConduit.Eof to indicate an error. | |
816 | |
817 ***********************************************************************/ | |
818 | |
819 uint read (uint delegate (void[]) dg) | |
820 { | |
821 int count = dg (data [index..extent]); | |
822 | |
823 if (count != IConduit.Eof) | |
824 { | |
825 index += count; | |
826 assert (index <= extent); | |
827 } | |
828 return count; | |
829 } | |
830 | |
831 /*********************************************************************** | |
832 | |
833 Compress buffer space | |
834 | |
835 Returns: | |
836 the buffer instance | |
837 | |
838 Remarks: | |
839 If we have some data left after an export, move it to | |
840 front-of-buffer and set position to be just after the | |
841 remains. This is for supporting certain conduits which | |
842 choose to write just the initial portion of a request. | |
843 | |
844 Limit is set to the amount of data remaining. Position | |
845 is always reset to zero. | |
846 | |
847 ***********************************************************************/ | |
848 | |
849 IBuffer compress () | |
850 { | |
851 uint r = readable (); | |
852 | |
853 if (index > 0 && r > 0) | |
854 // content may overlap ... | |
855 memcpy (&data[0], &data[index], r); | |
856 | |
857 index = 0; | |
858 extent = r; | |
859 return this; | |
860 } | |
861 | |
862 /*********************************************************************** | |
863 | |
864 Fill buffer from the specific conduit | |
865 | |
866 Returns: | |
867 Returns the number of bytes read, or Conduit.Eof | |
868 | |
869 Remarks: | |
870 Try to _fill the available buffer with content from the | |
871 specified conduit. We try to read as much as possible | |
872 by clearing the buffer when all current content has been | |
873 eaten. If there is no space available, nothing will be | |
874 read. | |
875 | |
876 ***********************************************************************/ | |
877 | |
878 uint fill (InputStream src) | |
879 { | |
880 if (src is null) | |
881 return IConduit.Eof; | |
882 | |
883 if (readable is 0) | |
884 index = extent = 0; // same as clear(), but without chain | |
885 else | |
886 if (writable is 0) | |
887 return 0; | |
888 | |
889 return write (&src.read); | |
890 } | |
891 | |
892 /*********************************************************************** | |
893 | |
894 Drain buffer content to the specific conduit | |
895 | |
896 Returns: | |
897 Returns the number of bytes written | |
898 | |
899 Remarks: | |
900 Write as much of the buffer that the associated conduit | |
901 can consume. The conduit is not obliged to consume all | |
902 content, so some may remain within the buffer. | |
903 | |
904 Throws an IOException on premature Eof. | |
905 | |
906 ***********************************************************************/ | |
907 | |
908 final uint drain (OutputStream dst) | |
909 { | |
910 if (dst is null) | |
911 return IConduit.Eof; | |
912 | |
913 uint ret = read (&dst.write); | |
914 if (ret is IConduit.Eof) | |
915 error (eofWrite); | |
916 | |
917 compress (); | |
918 return ret; | |
919 } | |
920 | |
921 /*********************************************************************** | |
922 | |
923 Truncate buffer content | |
924 | |
925 Remarks: | |
926 Truncate the buffer within its extent. Returns true if | |
927 the new length is valid, false otherwise. | |
928 | |
929 ***********************************************************************/ | |
930 | |
931 bool truncate (uint length) | |
932 { | |
933 if (length <= data.length) | |
934 { | |
935 extent = length; | |
936 return true; | |
937 } | |
938 return false; | |
939 } | |
940 | |
941 /*********************************************************************** | |
942 | |
943 Access buffer limit | |
944 | |
945 Returns: | |
946 Returns the limit of readable content within this buffer. | |
947 | |
948 Remarks: | |
949 Each buffer has a capacity, a limit, and a position. The | |
950 capacity is the maximum content a buffer can contain, limit | |
951 represents the extent of valid content, and position marks | |
952 the current read location. | |
953 | |
954 ***********************************************************************/ | |
955 | |
956 uint limit () | |
957 { | |
958 return extent; | |
959 } | |
960 | |
961 /*********************************************************************** | |
962 | |
963 Access buffer capacity | |
964 | |
965 Returns: | |
966 Returns the maximum capacity of this buffer | |
967 | |
968 Remarks: | |
969 Each buffer has a capacity, a limit, and a position. The | |
970 capacity is the maximum content a buffer can contain, limit | |
971 represents the extent of valid content, and position marks | |
972 the current read location. | |
973 | |
974 ***********************************************************************/ | |
975 | |
976 uint capacity () | |
977 { | |
978 return dimension; | |
979 } | |
980 | |
981 /*********************************************************************** | |
982 | |
983 Access buffer read position | |
984 | |
985 Returns: | |
986 Returns the current read-position within this buffer | |
987 | |
988 Remarks: | |
989 Each buffer has a capacity, a limit, and a position. The | |
990 capacity is the maximum content a buffer can contain, limit | |
991 represents the extent of valid content, and position marks | |
992 the current read location. | |
993 | |
994 ***********************************************************************/ | |
995 | |
996 uint position () | |
997 { | |
998 return index; | |
999 } | |
1000 | |
1001 /*********************************************************************** | |
1002 | |
1003 Set external conduit | |
1004 | |
1005 Params: | |
1006 conduit = the conduit to attach to | |
1007 | |
1008 Remarks: | |
1009 Sets the external conduit associated with this buffer. | |
1010 | |
1011 Buffers do not require an external conduit to operate, but | |
1012 it can be convenient to associate one. For example, methods | |
1013 fill() & drain() use it to import/export content as necessary. | |
1014 | |
1015 ***********************************************************************/ | |
1016 | |
1017 IBuffer setConduit (IConduit conduit) | |
1018 { | |
1019 sink = conduit.output; | |
1020 source = conduit.input; | |
1021 return this; | |
1022 } | |
1023 | |
1024 /*********************************************************************** | |
1025 | |
1026 Set output stream | |
1027 | |
1028 Params: | |
1029 sink = the stream to attach to | |
1030 | |
1031 Remarks: | |
1032 Sets the external output stream associated with this buffer. | |
1033 | |
1034 Buffers do not require an external stream to operate, but | |
1035 it can be convenient to associate one. For example, methods | |
1036 fill & drain use them to import/export content as necessary. | |
1037 | |
1038 ***********************************************************************/ | |
1039 | |
1040 final IBuffer output (OutputStream sink) | |
1041 { | |
1042 this.sink = sink; | |
1043 return this; | |
1044 } | |
1045 | |
1046 /*********************************************************************** | |
1047 | |
1048 Set input stream | |
1049 | |
1050 Params: | |
1051 source = the stream to attach to | |
1052 | |
1053 Remarks: | |
1054 Sets the external input stream associated with this buffer. | |
1055 | |
1056 Buffers do not require an external stream to operate, but | |
1057 it can be convenient to associate one. For example, methods | |
1058 fill & drain use them to import/export content as necessary. | |
1059 | |
1060 ***********************************************************************/ | |
1061 | |
1062 final IBuffer input (InputStream source) | |
1063 { | |
1064 this.source = source; | |
1065 return this; | |
1066 } | |
1067 | |
1068 /*********************************************************************** | |
1069 | |
1070 Access buffer content | |
1071 | |
1072 Remarks: | |
1073 Return the entire backing array. Exposed for subclass usage | |
1074 only | |
1075 | |
1076 ***********************************************************************/ | |
1077 | |
1078 protected void[] getContent () | |
1079 { | |
1080 return data; | |
1081 } | |
1082 | |
1083 /*********************************************************************** | |
1084 | |
1085 Copy content into buffer | |
1086 | |
1087 Params: | |
1088 src = the soure of the content | |
1089 size = the length of content at src | |
1090 | |
1091 Remarks: | |
1092 Bulk _copy of data from 'src'. The new content is made | |
1093 available for reading. This is exposed for subclass use | |
1094 only | |
1095 | |
1096 ***********************************************************************/ | |
1097 | |
1098 protected void copy (void *src, uint size) | |
1099 { | |
138 | 1100 printf("Buffer.copy(%p, %u)\n", src, size); |
132 | 1101 // avoid "out of bounds" test on zero size |
1102 if (size) | |
1103 { | |
1104 // content may overlap ... | |
1105 memcpy (&data[extent], src, size); | |
1106 extent += size; | |
1107 } | |
138 | 1108 printf(" copy done\n"); |
132 | 1109 } |
1110 | |
1111 /*********************************************************************** | |
1112 | |
1113 Cast to a target type without invoking the wrath of the | |
1114 runtime checks for misalignment. Instead, we truncate the | |
1115 array length | |
1116 | |
1117 ***********************************************************************/ | |
1118 | |
1119 static T[] convert(T)(void[] x) | |
1120 { | |
1121 return (cast(T*) x.ptr) [0 .. (x.length / T.sizeof)]; | |
1122 } | |
1123 | |
1124 | |
1125 | |
1126 /**********************************************************************/ | |
1127 /*********************** Buffered Interface ***************************/ | |
1128 /**********************************************************************/ | |
1129 | |
1130 IBuffer buffer () | |
1131 { | |
1132 return this; | |
1133 } | |
1134 | |
1135 | |
1136 /**********************************************************************/ | |
1137 /******************** Stream & Conduit Interfaces *********************/ | |
1138 /**********************************************************************/ | |
1139 | |
1140 | |
1141 /*********************************************************************** | |
1142 | |
1143 Return the name of this conduit | |
1144 | |
1145 ***********************************************************************/ | |
1146 | |
1147 override char[] toString () | |
1148 { | |
1149 return "<buffer>"; | |
1150 } | |
1151 | |
1152 /*********************************************************************** | |
1153 | |
1154 Generic IOException thrower | |
1155 | |
1156 Params: | |
1157 msg = a text message describing the exception reason | |
1158 | |
1159 Remarks: | |
1160 Throw an IOException with the provided message | |
1161 | |
1162 ***********************************************************************/ | |
1163 | |
1164 final void error (char[] msg) | |
1165 { | |
1166 throw new IOException (msg); | |
1167 } | |
1168 | |
1169 /*********************************************************************** | |
1170 | |
1171 Flush all buffer content to the specific conduit | |
1172 | |
1173 Remarks: | |
1174 Flush the contents of this buffer. This will block until | |
1175 all content is actually flushed via the associated conduit, | |
1176 whereas drain() will not. | |
1177 | |
1178 Do nothing where a conduit is not attached, enabling memory | |
1179 buffers to treat flush as a noop. | |
1180 | |
1181 Throws an IOException on premature Eof. | |
1182 | |
1183 ***********************************************************************/ | |
1184 | |
1185 override OutputStream flush () | |
1186 { | |
1187 if (sink) | |
1188 { | |
1189 while (readable() > 0) | |
1190 drain (sink); | |
1191 | |
1192 // flush the filter chain also | |
1193 sink.flush; | |
1194 } | |
1195 return this; | |
1196 } | |
1197 | |
1198 /*********************************************************************** | |
1199 | |
1200 Clear buffer content | |
1201 | |
1202 Remarks: | |
1203 Reset 'position' and 'limit' to zero. This effectively | |
1204 clears all content from the buffer. | |
1205 | |
1206 ***********************************************************************/ | |
1207 | |
1208 override InputStream clear () | |
1209 { | |
1210 index = extent = 0; | |
1211 | |
1212 // clear the filter chain also | |
1213 if (source) | |
1214 source.clear; | |
1215 return this; | |
1216 } | |
1217 | |
1218 /*********************************************************************** | |
1219 | |
1220 Copy content via this buffer from the provided src | |
1221 conduit. | |
1222 | |
1223 Remarks: | |
1224 The src conduit has its content transferred through | |
1225 this buffer via a series of fill & drain operations, | |
1226 until there is no more content available. The buffer | |
1227 content should be explicitly flushed by the caller. | |
1228 | |
1229 Throws an IOException on premature eof | |
1230 | |
1231 ***********************************************************************/ | |
1232 | |
1233 override OutputStream copy (InputStream src) | |
1234 { | |
1235 while (fill(src) != IConduit.Eof) | |
1236 // don't drain until we actually need to | |
1237 if (writable is 0) | |
1238 if (sink) | |
1239 drain (sink); | |
1240 else | |
1241 error (overflow); | |
1242 return this; | |
1243 } | |
1244 | |
1245 /*********************************************************************** | |
1246 | |
1247 Transfer content into the provided dst | |
1248 | |
1249 Params: | |
1250 dst = destination of the content | |
1251 | |
1252 Returns: | |
1253 return the number of bytes read, which may be less than | |
1254 dst.length. Eof is returned when no further content is | |
1255 available. | |
1256 | |
1257 Remarks: | |
1258 Populates the provided array with content. We try to | |
1259 satisfy the request from the buffer content, and read | |
1260 directly from an attached conduit when the buffer is | |
1261 empty. | |
1262 | |
1263 ***********************************************************************/ | |
1264 | |
1265 override uint read (void[] dst) | |
1266 { | |
1267 uint content = readable(); | |
1268 if (content) | |
1269 { | |
1270 if (content >= dst.length) | |
1271 content = dst.length; | |
1272 | |
1273 // transfer buffer content | |
1274 dst [0 .. content] = data [index .. index + content]; | |
1275 index += content; | |
1276 } | |
1277 else | |
1278 if (source) | |
1279 { | |
1280 // pathological cases read directly from conduit | |
1281 if (dst.length > dimension) | |
1282 content = source.read (dst); | |
1283 else | |
1284 // keep buffer partially populated | |
1285 if ((content = fill(source)) != IConduit.Eof && content > 0) | |
1286 content = read (dst); | |
1287 } | |
1288 else | |
1289 content = IConduit.Eof; | |
1290 return content; | |
1291 } | |
1292 | |
1293 /*********************************************************************** | |
1294 | |
1295 Emulate OutputStream.write() | |
1296 | |
1297 Params: | |
1298 src = the content to write | |
1299 | |
1300 Returns: | |
1301 return the number of bytes written, which may be less than | |
1302 provided (conceptually). | |
1303 | |
1304 Remarks: | |
1305 Appends src content to the buffer, flushing to an attached | |
1306 conduit as necessary. An IOException is thrown upon write | |
1307 failure. | |
1308 | |
1309 ***********************************************************************/ | |
1310 | |
1311 override uint write (void[] src) | |
1312 { | |
1313 append (src.ptr, src.length); | |
1314 return src.length; | |
1315 } | |
1316 | |
1317 /*********************************************************************** | |
1318 | |
1319 Access configured conduit | |
1320 | |
1321 Returns: | |
1322 Returns the conduit associated with this buffer. Returns | |
1323 null if the buffer is purely memory based; that is, it's | |
1324 not backed by some external medium. | |
1325 | |
1326 Remarks: | |
1327 Buffers do not require an external conduit to operate, but | |
1328 it can be convenient to associate one. For example, methods | |
1329 fill() & drain() use it to import/export content as necessary. | |
1330 | |
1331 ***********************************************************************/ | |
1332 | |
1333 final override IConduit conduit () | |
1334 { | |
1335 if (sink) | |
1336 return sink.conduit; | |
1337 else | |
1338 if (source) | |
1339 return source.conduit; | |
1340 return this; | |
1341 } | |
1342 | |
1343 /*********************************************************************** | |
1344 | |
1345 Return a preferred size for buffering conduit I/O | |
1346 | |
1347 ***********************************************************************/ | |
1348 | |
1349 final override uint bufferSize () | |
1350 { | |
1351 return 32 * 1024; | |
1352 } | |
1353 | |
1354 /*********************************************************************** | |
1355 | |
1356 Is the conduit alive? | |
1357 | |
1358 ***********************************************************************/ | |
1359 | |
1360 final override bool isAlive () | |
1361 { | |
1362 return true; | |
1363 } | |
1364 | |
1365 /*********************************************************************** | |
1366 | |
1367 Exposes configured output stream | |
1368 | |
1369 Returns: | |
1370 Returns the OutputStream associated with this buffer. Returns | |
1371 null if the buffer is not attached to an output; that is, it's | |
1372 not backed by some external medium. | |
1373 | |
1374 Remarks: | |
1375 Buffers do not require an external stream to operate, but | |
1376 it can be convenient to associate them. For example, methods | |
1377 fill & drain use them to import/export content as necessary. | |
1378 | |
1379 ***********************************************************************/ | |
1380 | |
1381 final OutputStream output () | |
1382 { | |
1383 return sink; | |
1384 } | |
1385 | |
1386 /*********************************************************************** | |
1387 | |
1388 Exposes configured input stream | |
1389 | |
1390 Returns: | |
1391 Returns the InputStream associated with this buffer. Returns | |
1392 null if the buffer is not attached to an input; that is, it's | |
1393 not backed by some external medium. | |
1394 | |
1395 Remarks: | |
1396 Buffers do not require an external stream to operate, but | |
1397 it can be convenient to associate them. For example, methods | |
1398 fill & drain use them to import/export content as necessary. | |
1399 | |
1400 ***********************************************************************/ | |
1401 | |
1402 final InputStream input () | |
1403 { | |
1404 return source; | |
1405 } | |
1406 | |
1407 /*********************************************************************** | |
1408 | |
1409 Release external resources | |
1410 | |
1411 ***********************************************************************/ | |
1412 | |
1413 final override void detach () | |
1414 { | |
1415 } | |
1416 | |
1417 /*********************************************************************** | |
1418 | |
1419 Close the stream | |
1420 | |
1421 Remarks: | |
1422 Propagate request to an attached OutputStream (this is a | |
1423 requirement for the OutputStream interface) | |
1424 | |
1425 ***********************************************************************/ | |
1426 | |
1427 override void close () | |
1428 { | |
1429 if (sink) | |
1430 sink.close; | |
1431 else | |
1432 if (source) | |
1433 source.close; | |
1434 } | |
1435 } |