Mercurial > projects > ldc
annotate tango/tango/io/Buffer.d @ 137:ce7b81fb957f trunk
[svn r141] fixed more problems with classinfo
moved more IR state out of the AST classes
author | lindquist |
---|---|
date | Fri, 18 Jan 2008 16:42:16 +0100 |
parents | 0e28624814e8 |
children | aeddd4d533b3 |
rev | line source |
---|---|
132 | 1 /******************************************************************************* |
2 | |
3 copyright: Copyright (c) 2004 Kris Bell. All rights reserved | |
4 | |
5 license: BSD style: $(LICENSE) | |
6 | |
7 version: Mar 2004: Initial release | |
8 Dec 2006: Outback release | |
9 | |
10 authors: Kris | |
11 | |
12 *******************************************************************************/ | |
13 | |
14 module tango.io.Buffer; | |
15 | |
16 private import tango.core.Exception; | |
17 | |
18 public import tango.io.model.IBuffer, | |
19 tango.io.model.IConduit; | |
20 | |
21 /****************************************************************************** | |
22 | |
23 ******************************************************************************/ | |
24 | |
25 extern (C) | |
26 { | |
27 protected void * memcpy (void *dst, void *src, uint); | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
28 private int printf(char*, ...); |
132 | 29 } |
30 | |
31 /******************************************************************************* | |
32 | |
33 Buffer is central concept in Tango I/O. Each buffer acts | |
34 as a queue (line) where items are removed from the front | |
35 and new items are added to the back. Buffers are modeled | |
36 by tango.io.model.IBuffer, and a concrete implementation | |
37 is provided by this class. | |
38 | |
39 Buffer can be read from and written to directly, though | |
40 various data-converters and filters are often leveraged | |
41 to apply structure to what might otherwise be simple raw | |
42 data. | |
43 | |
44 Buffers may also be tokenized by applying an Iterator. | |
45 This can be handy when one is dealing with text input, | |
46 and/or the content suits a more fluid format than most | |
47 typical converters support. Iterator tokens are mapped | |
48 directly onto buffer content (sliced), making them quite | |
49 efficient in practice. Like other types of buffer client, | |
50 multiple iterators can be mapped onto one common buffer | |
51 and access will be serialized. | |
52 | |
53 Buffers are sometimes memory-only, in which case there | |
54 is nothing left to do when a client has consumed all the | |
55 content. Other buffers are themselves bound to an external | |
56 device called a conduit. When this is the case, a consumer | |
57 will eventually cause a buffer to reload via its associated | |
58 conduit and previous buffer content will be lost. | |
59 | |
60 A similar approach is applied to clients which populate a | |
61 buffer, whereby the content of a full buffer will be flushed | |
62 to a bound conduit before continuing. Another variation is | |
63 that of a memory-mapped buffer, whereby the buffer content | |
64 is mapped directly to virtual memory exposed via the OS. This | |
65 can be used to address large files as an array of content. | |
66 | |
67 Direct buffer manipulation typically involves appending, | |
68 as in the following example: | |
69 --- | |
70 // create a small buffer | |
71 auto buf = new Buffer (256); | |
72 | |
73 auto foo = "to write some D"; | |
74 | |
75 // append some text directly to it | |
76 buf.append ("now is the time for all good men ").append(foo); | |
77 --- | |
78 | |
79 Alternatively, one might use a formatter to append the buffer: | |
80 --- | |
81 auto output = new FormatOutput (new Buffer(256)); | |
82 output.format ("now is the time for {} good men {}", 3, foo); | |
83 --- | |
84 | |
85 A slice() method will return all valid content within a buffer. | |
86 GrowBuffer can be used instead, where one wishes to append beyond | |
87 a specified limit. | |
88 | |
89 A common usage of a buffer is in conjunction with a conduit, | |
90 such as FileConduit. Each conduit exposes a preferred-size for | |
91 its associated buffers, utilized during buffer construction: | |
92 --- | |
93 auto file = new FileConduit ("file.name"); | |
94 auto buf = new Buffer (file); | |
95 --- | |
96 | |
97 However, this is typically hidden by higher level constructors | |
98 such as those exposed via the stream wrappers. For example: | |
99 --- | |
100 auto input = new DataInput (new FileInput("file.name")); | |
101 --- | |
102 | |
103 There is indeed a buffer between the resultant stream and the | |
104 file source, but explicit buffer construction is unecessary in | |
105 common cases. | |
106 | |
107 An Iterator is constructed in a similar manner, where you provide | |
108 it an input stream to operate upon. There's a variety of iterators | |
109 available in the tango.text.stream package, and they are templated | |
110 for each of utf8, utf16, and utf32. This example uses a line iterator | |
111 to sweep a text file: | |
112 --- | |
113 auto lines = new LineInput (new FileInput("file.name")); | |
114 foreach (line; lines) | |
115 Cout(line).newline; | |
116 --- | |
117 | |
118 Buffers are useful for many purposes within Tango, but there | |
119 are times when it may be more appropriate to sidestep them. For | |
120 such cases, all conduit derivatives (such as FileConduit) support | |
121 direct array-based IO via a pair of read() and write() methods. | |
122 | |
123 *******************************************************************************/ | |
124 | |
125 class Buffer : IBuffer | |
126 { | |
127 protected OutputStream sink; // optional data sink | |
128 protected InputStream source; // optional data source | |
129 protected void[] data; // the raw data buffer | |
130 protected uint index; // current read position | |
131 protected uint extent; // limit of valid content | |
132 protected uint dimension; // maximum extent of content | |
133 | |
134 | |
135 protected static char[] overflow = "output buffer is full"; | |
136 protected static char[] underflow = "input buffer is empty"; | |
137 protected static char[] eofRead = "end-of-flow whilst reading"; | |
138 protected static char[] eofWrite = "end-of-flow whilst writing"; | |
139 | |
140 /*********************************************************************** | |
141 | |
142 Ensure the buffer remains valid between method calls | |
143 | |
144 ***********************************************************************/ | |
145 | |
146 invariant | |
147 { | |
148 assert (index <= extent); | |
149 assert (extent <= dimension); | |
150 } | |
151 | |
152 /*********************************************************************** | |
153 | |
154 Construct a buffer | |
155 | |
156 Params: | |
157 conduit = the conduit to buffer | |
158 | |
159 Remarks: | |
160 Construct a Buffer upon the provided conduit. A relevant | |
161 buffer size is supplied via the provided conduit. | |
162 | |
163 ***********************************************************************/ | |
164 | |
165 this (IConduit conduit) | |
166 { | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
167 printf("Buffer.this(%p)\n", conduit); |
137 | 168 printf("assert (conduit !is null);\n"); |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
169 assert (conduit !is null); |
137 | 170 printf("assert (conduit);\n", conduit); |
132 | 171 assert (conduit); |
172 | |
137 | 173 printf("this (conduit.bufferSize(%p));\n", conduit); |
174 printf("cast(Object)conduit = %p\n", cast(Object)conduit); | |
132 | 175 this (conduit.bufferSize); |
176 setConduit (conduit); | |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
177 |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
178 assert(this !is null); |
132 | 179 } |
180 | |
181 /*********************************************************************** | |
182 | |
183 Construct a buffer | |
184 | |
185 Params: | |
186 stream = an input stream | |
187 capacity = desired buffer capacity | |
188 | |
189 Remarks: | |
190 Construct a Buffer upon the provided input stream. | |
191 | |
192 ***********************************************************************/ | |
193 | |
194 this (InputStream stream, uint capacity) | |
195 { | |
196 this (capacity); | |
197 input = stream; | |
198 } | |
199 | |
200 /*********************************************************************** | |
201 | |
202 Construct a buffer | |
203 | |
204 Params: | |
205 stream = an output stream | |
206 capacity = desired buffer capacity | |
207 | |
208 Remarks: | |
209 Construct a Buffer upon the provided output stream. | |
210 | |
211 ***********************************************************************/ | |
212 | |
213 this (OutputStream stream, uint capacity) | |
214 { | |
215 this (capacity); | |
216 output = stream; | |
217 } | |
218 | |
219 /*********************************************************************** | |
220 | |
221 Construct a buffer | |
222 | |
223 Params: | |
224 capacity = the number of bytes to make available | |
225 | |
226 Remarks: | |
227 Construct a Buffer with the specified number of bytes. | |
228 | |
229 ***********************************************************************/ | |
230 | |
231 this (uint capacity = 0) | |
232 { | |
137 | 233 printf("Buffer.this(%p, %u)\n", this, capacity); |
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
234 setContent (new ubyte[capacity], 0); |
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
235 assert(this !is null); |
132 | 236 } |
237 | |
238 /*********************************************************************** | |
239 | |
240 Construct a buffer | |
241 | |
242 Params: | |
243 data = the backing array to buffer within | |
244 | |
245 Remarks: | |
246 Prime a buffer with an application-supplied array. All content | |
247 is considered valid for reading, and thus there is no writable | |
248 space initially available. | |
249 | |
250 ***********************************************************************/ | |
251 | |
252 this (void[] data) | |
253 { | |
254 setContent (data, data.length); | |
255 } | |
256 | |
257 /*********************************************************************** | |
258 | |
259 Construct a buffer | |
260 | |
261 Params: | |
262 data = the backing array to buffer within | |
263 readable = the number of bytes initially made | |
264 readable | |
265 | |
266 Remarks: | |
267 Prime buffer with an application-supplied array, and | |
268 indicate how much readable data is already there. A | |
269 write operation will begin writing immediately after | |
270 the existing readable content. | |
271 | |
272 This is commonly used to attach a Buffer instance to | |
273 a local array. | |
274 | |
275 ***********************************************************************/ | |
276 | |
277 this (void[] data, uint readable) | |
278 { | |
279 setContent (data, readable); | |
280 } | |
281 | |
282 /*********************************************************************** | |
283 | |
284 Attempt to share an upstream Buffer, and create an instance | |
285 where there not one available. | |
286 | |
287 Params: | |
288 stream = an input stream | |
289 size = a hint of the desired buffer size. Defaults to the | |
290 conduit-defined size | |
291 | |
292 Remarks: | |
293 If an upstream Buffer instances is visible, it will be shared. | |
294 Otherwise, a new instance is created based upon the bufferSize | |
295 exposed by the stream endpoint (conduit). | |
296 | |
297 ***********************************************************************/ | |
298 | |
299 static IBuffer share (InputStream stream, uint size=uint.max) | |
300 { | |
301 auto b = cast(Buffered) stream; | |
302 if (b) | |
303 return b.buffer; | |
304 | |
305 if (size is uint.max) | |
306 size = stream.conduit.bufferSize; | |
307 | |
308 return new Buffer (stream, size); | |
309 } | |
310 | |
311 /*********************************************************************** | |
312 | |
313 Attempt to share an upstream Buffer, and create an instance | |
314 where there not one available. | |
315 | |
316 Params: | |
317 stream = an output stream | |
318 size = a hint of the desired buffer size. Defaults to the | |
319 conduit-defined size | |
320 | |
321 Remarks: | |
322 If an upstream Buffer instances is visible, it will be shared. | |
323 Otherwise, a new instance is created based upon the bufferSize | |
324 exposed by the stream endpoint (conduit). | |
325 | |
326 ***********************************************************************/ | |
327 | |
328 static IBuffer share (OutputStream stream, uint size=uint.max) | |
329 { | |
330 auto b = cast(Buffered) stream; | |
331 if (b) | |
332 return b.buffer; | |
333 | |
334 if (size is uint.max) | |
335 size = stream.conduit.bufferSize; | |
336 | |
337 return new Buffer (stream, size); | |
338 } | |
339 | |
340 /*********************************************************************** | |
341 | |
342 Reset the buffer content | |
343 | |
344 Params: | |
345 data = the backing array to buffer within. All content | |
346 is considered valid | |
347 | |
348 Returns: | |
349 the buffer instance | |
350 | |
351 Remarks: | |
352 Set the backing array with all content readable. Writing | |
353 to this will either flush it to an associated conduit, or | |
354 raise an Eof condition. Use clear() to reset the content | |
355 (make it all writable). | |
356 | |
357 ***********************************************************************/ | |
358 | |
359 IBuffer setContent (void[] data) | |
360 { | |
361 return setContent (data, data.length); | |
362 } | |
363 | |
364 /*********************************************************************** | |
365 | |
366 Reset the buffer content | |
367 | |
368 Params: | |
369 data = the backing array to buffer within | |
370 readable = the number of bytes within data considered | |
371 valid | |
372 | |
373 Returns: | |
374 the buffer instance | |
375 | |
376 Remarks: | |
377 Set the backing array with some content readable. Writing | |
378 to this will either flush it to an associated conduit, or | |
379 raise an Eof condition. Use clear() to reset the content | |
380 (make it all writable). | |
381 | |
382 ***********************************************************************/ | |
383 | |
384 IBuffer setContent (void[] data, uint readable) | |
385 { | |
386 this.data = data; | |
387 this.extent = readable; | |
388 this.dimension = data.length; | |
389 | |
390 // reset to start of input | |
391 this.index = 0; | |
392 | |
393 return this; | |
394 } | |
395 | |
396 /*********************************************************************** | |
397 | |
398 Access buffer content | |
399 | |
400 Params: | |
401 size = number of bytes to access | |
402 eat = whether to consume the content or not | |
403 | |
404 Returns: | |
405 the corresponding buffer slice when successful, or | |
406 null if there's not enough data available (Eof; Eob). | |
407 | |
408 Remarks: | |
409 Read a slice of data from the buffer, loading from the | |
410 conduit as necessary. The specified number of bytes is | |
411 sliced from the buffer, and marked as having been read | |
412 when the 'eat' parameter is set true. When 'eat' is set | |
413 false, the read position is not adjusted. | |
414 | |
415 Note that the slice cannot be larger than the size of | |
416 the buffer ~ use method fill(void[]) instead where you | |
417 simply want the content copied, or use conduit.read() | |
418 to extract directly from an attached conduit. Also note | |
419 that if you need to retain the slice, then it should be | |
420 .dup'd before the buffer is compressed or repopulated. | |
421 | |
422 Examples: | |
423 --- | |
424 // create a buffer with some content | |
425 auto buffer = new Buffer ("hello world"); | |
426 | |
427 // consume everything unread | |
428 auto slice = buffer.slice (buffer.readable); | |
429 --- | |
430 | |
431 ***********************************************************************/ | |
432 | |
433 void[] slice (uint size, bool eat = true) | |
434 { | |
435 if (size > readable) | |
436 { | |
437 if (source is null) | |
438 error (underflow); | |
439 | |
440 // make some space? This will try to leave as much content | |
441 // in the buffer as possible, such that entire records may | |
442 // be aliased directly from within. | |
443 if (size > writable) | |
444 { | |
445 if (size > dimension) | |
446 error (underflow); | |
447 compress (); | |
448 } | |
449 | |
450 // populate tail of buffer with new content | |
451 do { | |
452 if (fill(source) is IConduit.Eof) | |
453 error (eofRead); | |
454 } while (size > readable); | |
455 } | |
456 | |
457 auto i = index; | |
458 if (eat) | |
459 index += size; | |
460 return data [i .. i + size]; | |
461 } | |
462 | |
463 /********************************************************************** | |
464 | |
465 Fill the provided buffer. Returns the number of bytes | |
466 actually read, which will be less that dst.length when | |
467 Eof has been reached and IConduit.Eof thereafter | |
468 | |
469 **********************************************************************/ | |
470 | |
471 uint fill (void[] dst) | |
472 { | |
473 uint len = 0; | |
474 | |
475 while (len < dst.length) | |
476 { | |
477 uint i = read (dst [len .. $]); | |
478 if (i is IConduit.Eof) | |
479 return (len > 0) ? len : IConduit.Eof; | |
480 len += i; | |
481 } | |
482 return len; | |
483 } | |
484 | |
485 /*********************************************************************** | |
486 | |
487 Copy buffer content into the provided dst | |
488 | |
489 Params: | |
490 dst = destination of the content | |
491 bytes = size of dst | |
492 | |
493 Returns: | |
494 A reference to the populated content | |
495 | |
496 Remarks: | |
497 Fill the provided array with content. We try to satisfy | |
498 the request from the buffer content, and read directly | |
499 from an attached conduit where more is required. | |
500 | |
501 ***********************************************************************/ | |
502 | |
503 void[] readExact (void* dst, uint bytes) | |
504 { | |
505 auto tmp = dst [0 .. bytes]; | |
506 if (fill (tmp) != bytes) | |
507 error (eofRead); | |
508 | |
509 return tmp; | |
510 } | |
511 | |
512 /*********************************************************************** | |
513 | |
514 Append content | |
515 | |
516 Params: | |
517 src = the content to _append | |
518 | |
519 Returns a chaining reference if all content was written. | |
520 Throws an IOException indicating eof or eob if not. | |
521 | |
522 Remarks: | |
523 Append an array to this buffer, and flush to the | |
524 conduit as necessary. This is often used in lieu of | |
525 a Writer. | |
526 | |
527 ***********************************************************************/ | |
528 | |
529 IBuffer append (void[] src) | |
530 { | |
531 return append (src.ptr, src.length); | |
532 } | |
533 | |
534 /*********************************************************************** | |
535 | |
536 Append content | |
537 | |
538 Params: | |
539 src = the content to _append | |
540 length = the number of bytes in src | |
541 | |
542 Returns a chaining reference if all content was written. | |
543 Throws an IOException indicating eof or eob if not. | |
544 | |
545 Remarks: | |
546 Append an array to this buffer, and flush to the | |
547 conduit as necessary. This is often used in lieu of | |
548 a Writer. | |
549 | |
550 ***********************************************************************/ | |
551 | |
552 IBuffer append (void* src, uint length) | |
553 { | |
554 if (length > writable) | |
555 // can we write externally? | |
556 if (sink) | |
557 { | |
558 flush (); | |
559 | |
560 // check for pathological case | |
561 if (length > dimension) | |
562 { | |
563 do { | |
564 auto written = sink.write (src [0 .. length]); | |
565 if (written is IConduit.Eof) | |
566 error (eofWrite); | |
567 src += written, length -= written; | |
568 } while (length > dimension); | |
569 } | |
570 } | |
571 else | |
572 error (overflow); | |
573 | |
574 copy (src, length); | |
575 return this; | |
576 } | |
577 | |
578 /*********************************************************************** | |
579 | |
580 Append content | |
581 | |
582 Params: | |
583 other = a buffer with content available | |
584 | |
585 Returns: | |
586 Returns a chaining reference if all content was written. | |
587 Throws an IOException indicating eof or eob if not. | |
588 | |
589 Remarks: | |
590 Append another buffer to this one, and flush to the | |
591 conduit as necessary. This is often used in lieu of | |
592 a Writer. | |
593 | |
594 ***********************************************************************/ | |
595 | |
596 IBuffer append (IBuffer other) | |
597 { | |
598 return append (other.slice); | |
599 } | |
600 | |
601 /*********************************************************************** | |
602 | |
603 Consume content from a producer | |
604 | |
605 Params: | |
606 The content to consume. This is consumed verbatim, and in | |
607 raw binary format ~ no implicit conversions are performed. | |
608 | |
609 Remarks: | |
610 This is often used in lieu of a Writer, and enables simple | |
611 classes, such as FilePath and Uri, to emit content directly | |
612 into a buffer (thus avoiding potential heap activity) | |
613 | |
614 Examples: | |
615 --- | |
616 auto path = new FilePath (somepath); | |
617 | |
618 path.produce (&buffer.consume); | |
619 --- | |
620 | |
621 ***********************************************************************/ | |
622 | |
623 void consume (void[] x) | |
624 { | |
625 append (x); | |
626 } | |
627 | |
628 /*********************************************************************** | |
629 | |
630 Retrieve the valid content | |
631 | |
632 Returns: | |
633 a void[] slice of the buffer | |
634 | |
635 Remarks: | |
636 Return a void[] slice of the buffer, from the current position | |
637 up to the limit of valid content. The content remains in the | |
638 buffer for future extraction. | |
639 | |
640 ***********************************************************************/ | |
641 | |
642 void[] slice () | |
643 { | |
644 return data [index .. extent]; | |
645 } | |
646 | |
647 /*********************************************************************** | |
648 | |
649 Move the current read location | |
650 | |
651 Params: | |
652 size = the number of bytes to move | |
653 | |
654 Returns: | |
655 Returns true if successful, false otherwise. | |
656 | |
657 Remarks: | |
658 Skip ahead by the specified number of bytes, streaming from | |
659 the associated conduit as necessary. | |
660 | |
661 Can also reverse the read position by 'size' bytes, when size | |
662 is negative. This may be used to support lookahead operations. | |
663 Note that a negative size will fail where there is not sufficient | |
664 content available in the buffer (can't _skip beyond the beginning). | |
665 | |
666 ***********************************************************************/ | |
667 | |
668 bool skip (int size) | |
669 { | |
670 if (size < 0) | |
671 { | |
672 size = -size; | |
673 if (index >= size) | |
674 { | |
675 index -= size; | |
676 return true; | |
677 } | |
678 return false; | |
679 } | |
680 return slice(size) !is null; | |
681 } | |
682 | |
683 /*********************************************************************** | |
684 | |
685 Iterator support | |
686 | |
687 Params: | |
688 scan = the delagate to invoke with the current content | |
689 | |
690 Returns: | |
691 Returns true if a token was isolated, false otherwise. | |
692 | |
693 Remarks: | |
694 Upon success, the delegate should return the byte-based | |
695 index of the consumed pattern (tail end of it). Failure | |
696 to match a pattern should be indicated by returning an | |
697 IConduit.Eof | |
698 | |
699 Each pattern is expected to be stripped of the delimiter. | |
700 An end-of-file condition causes trailing content to be | |
701 placed into the token. Requests made beyond Eof result | |
702 in empty matches (length is zero). | |
703 | |
704 Note that additional iterator and/or reader instances | |
705 will operate in lockstep when bound to a common buffer. | |
706 | |
707 ***********************************************************************/ | |
708 | |
709 bool next (uint delegate (void[]) scan) | |
710 { | |
711 while (read(scan) is IConduit.Eof) | |
712 // not found - are we streaming? | |
713 if (source) | |
714 { | |
715 // did we start at the beginning? | |
716 if (position) | |
717 // nope - move partial token to start of buffer | |
718 compress (); | |
719 else | |
720 // no more space in the buffer? | |
721 if (writable is 0) | |
722 error ("Token is too large to fit within buffer"); | |
723 | |
724 // read another chunk of data | |
725 if (fill(source) is IConduit.Eof) | |
726 return false; | |
727 } | |
728 else | |
729 return false; | |
730 | |
731 return true; | |
732 } | |
733 | |
734 /*********************************************************************** | |
735 | |
736 Available content | |
737 | |
738 Remarks: | |
739 Return count of _readable bytes remaining in buffer. This is | |
740 calculated simply as limit() - position() | |
741 | |
742 ***********************************************************************/ | |
743 | |
744 uint readable () | |
745 { | |
746 return extent - index; | |
747 } | |
748 | |
749 /*********************************************************************** | |
750 | |
751 Available space | |
752 | |
753 Remarks: | |
754 Return count of _writable bytes available in buffer. This is | |
755 calculated simply as capacity() - limit() | |
756 | |
757 ***********************************************************************/ | |
758 | |
759 uint writable () | |
760 { | |
761 return dimension - extent; | |
762 } | |
763 | |
764 /*********************************************************************** | |
765 | |
766 Write into this buffer | |
767 | |
768 Params: | |
769 dg = the callback to provide buffer access to | |
770 | |
771 Returns: | |
772 Returns whatever the delegate returns. | |
773 | |
774 Remarks: | |
775 Exposes the raw data buffer at the current _write position, | |
776 The delegate is provided with a void[] representing space | |
777 available within the buffer at the current _write position. | |
778 | |
779 The delegate should return the appropriate number of bytes | |
780 if it writes valid content, or IConduit.Eof on error. | |
781 | |
782 ***********************************************************************/ | |
783 | |
784 uint write (uint delegate (void[]) dg) | |
785 { | |
786 int count = dg (data [extent..dimension]); | |
787 | |
788 if (count != IConduit.Eof) | |
789 { | |
790 extent += count; | |
791 assert (extent <= dimension); | |
792 } | |
793 return count; | |
794 } | |
795 | |
796 /*********************************************************************** | |
797 | |
798 Read directly from this buffer | |
799 | |
800 Params: | |
801 dg = callback to provide buffer access to | |
802 | |
803 Returns: | |
804 Returns whatever the delegate returns. | |
805 | |
806 Remarks: | |
807 Exposes the raw data buffer at the current _read position. The | |
808 delegate is provided with a void[] representing the available | |
809 data, and should return zero to leave the current _read position | |
810 intact. | |
811 | |
812 If the delegate consumes data, it should return the number of | |
813 bytes consumed; or IConduit.Eof to indicate an error. | |
814 | |
815 ***********************************************************************/ | |
816 | |
817 uint read (uint delegate (void[]) dg) | |
818 { | |
819 int count = dg (data [index..extent]); | |
820 | |
821 if (count != IConduit.Eof) | |
822 { | |
823 index += count; | |
824 assert (index <= extent); | |
825 } | |
826 return count; | |
827 } | |
828 | |
829 /*********************************************************************** | |
830 | |
831 Compress buffer space | |
832 | |
833 Returns: | |
834 the buffer instance | |
835 | |
836 Remarks: | |
837 If we have some data left after an export, move it to | |
838 front-of-buffer and set position to be just after the | |
839 remains. This is for supporting certain conduits which | |
840 choose to write just the initial portion of a request. | |
841 | |
842 Limit is set to the amount of data remaining. Position | |
843 is always reset to zero. | |
844 | |
845 ***********************************************************************/ | |
846 | |
847 IBuffer compress () | |
848 { | |
849 uint r = readable (); | |
850 | |
851 if (index > 0 && r > 0) | |
852 // content may overlap ... | |
853 memcpy (&data[0], &data[index], r); | |
854 | |
855 index = 0; | |
856 extent = r; | |
857 return this; | |
858 } | |
859 | |
860 /*********************************************************************** | |
861 | |
862 Fill buffer from the specific conduit | |
863 | |
864 Returns: | |
865 Returns the number of bytes read, or Conduit.Eof | |
866 | |
867 Remarks: | |
868 Try to _fill the available buffer with content from the | |
869 specified conduit. We try to read as much as possible | |
870 by clearing the buffer when all current content has been | |
871 eaten. If there is no space available, nothing will be | |
872 read. | |
873 | |
874 ***********************************************************************/ | |
875 | |
876 uint fill (InputStream src) | |
877 { | |
878 if (src is null) | |
879 return IConduit.Eof; | |
880 | |
881 if (readable is 0) | |
882 index = extent = 0; // same as clear(), but without chain | |
883 else | |
884 if (writable is 0) | |
885 return 0; | |
886 | |
887 return write (&src.read); | |
888 } | |
889 | |
890 /*********************************************************************** | |
891 | |
892 Drain buffer content to the specific conduit | |
893 | |
894 Returns: | |
895 Returns the number of bytes written | |
896 | |
897 Remarks: | |
898 Write as much of the buffer that the associated conduit | |
899 can consume. The conduit is not obliged to consume all | |
900 content, so some may remain within the buffer. | |
901 | |
902 Throws an IOException on premature Eof. | |
903 | |
904 ***********************************************************************/ | |
905 | |
906 final uint drain (OutputStream dst) | |
907 { | |
908 if (dst is null) | |
909 return IConduit.Eof; | |
910 | |
911 uint ret = read (&dst.write); | |
912 if (ret is IConduit.Eof) | |
913 error (eofWrite); | |
914 | |
915 compress (); | |
916 return ret; | |
917 } | |
918 | |
919 /*********************************************************************** | |
920 | |
921 Truncate buffer content | |
922 | |
923 Remarks: | |
924 Truncate the buffer within its extent. Returns true if | |
925 the new length is valid, false otherwise. | |
926 | |
927 ***********************************************************************/ | |
928 | |
929 bool truncate (uint length) | |
930 { | |
931 if (length <= data.length) | |
932 { | |
933 extent = length; | |
934 return true; | |
935 } | |
936 return false; | |
937 } | |
938 | |
939 /*********************************************************************** | |
940 | |
941 Access buffer limit | |
942 | |
943 Returns: | |
944 Returns the limit of readable content within this buffer. | |
945 | |
946 Remarks: | |
947 Each buffer has a capacity, a limit, and a position. The | |
948 capacity is the maximum content a buffer can contain, limit | |
949 represents the extent of valid content, and position marks | |
950 the current read location. | |
951 | |
952 ***********************************************************************/ | |
953 | |
954 uint limit () | |
955 { | |
956 return extent; | |
957 } | |
958 | |
959 /*********************************************************************** | |
960 | |
961 Access buffer capacity | |
962 | |
963 Returns: | |
964 Returns the maximum capacity of this buffer | |
965 | |
966 Remarks: | |
967 Each buffer has a capacity, a limit, and a position. The | |
968 capacity is the maximum content a buffer can contain, limit | |
969 represents the extent of valid content, and position marks | |
970 the current read location. | |
971 | |
972 ***********************************************************************/ | |
973 | |
974 uint capacity () | |
975 { | |
976 return dimension; | |
977 } | |
978 | |
979 /*********************************************************************** | |
980 | |
981 Access buffer read position | |
982 | |
983 Returns: | |
984 Returns the current read-position within this buffer | |
985 | |
986 Remarks: | |
987 Each buffer has a capacity, a limit, and a position. The | |
988 capacity is the maximum content a buffer can contain, limit | |
989 represents the extent of valid content, and position marks | |
990 the current read location. | |
991 | |
992 ***********************************************************************/ | |
993 | |
994 uint position () | |
995 { | |
996 return index; | |
997 } | |
998 | |
999 /*********************************************************************** | |
1000 | |
1001 Set external conduit | |
1002 | |
1003 Params: | |
1004 conduit = the conduit to attach to | |
1005 | |
1006 Remarks: | |
1007 Sets the external conduit associated with this buffer. | |
1008 | |
1009 Buffers do not require an external conduit to operate, but | |
1010 it can be convenient to associate one. For example, methods | |
1011 fill() & drain() use it to import/export content as necessary. | |
1012 | |
1013 ***********************************************************************/ | |
1014 | |
1015 IBuffer setConduit (IConduit conduit) | |
1016 { | |
1017 sink = conduit.output; | |
1018 source = conduit.input; | |
1019 return this; | |
1020 } | |
1021 | |
1022 /*********************************************************************** | |
1023 | |
1024 Set output stream | |
1025 | |
1026 Params: | |
1027 sink = the stream to attach to | |
1028 | |
1029 Remarks: | |
1030 Sets the external output stream associated with this buffer. | |
1031 | |
1032 Buffers do not require an external stream to operate, but | |
1033 it can be convenient to associate one. For example, methods | |
1034 fill & drain use them to import/export content as necessary. | |
1035 | |
1036 ***********************************************************************/ | |
1037 | |
1038 final IBuffer output (OutputStream sink) | |
1039 { | |
1040 this.sink = sink; | |
1041 return this; | |
1042 } | |
1043 | |
1044 /*********************************************************************** | |
1045 | |
1046 Set input stream | |
1047 | |
1048 Params: | |
1049 source = the stream to attach to | |
1050 | |
1051 Remarks: | |
1052 Sets the external input stream associated with this buffer. | |
1053 | |
1054 Buffers do not require an external stream to operate, but | |
1055 it can be convenient to associate one. For example, methods | |
1056 fill & drain use them to import/export content as necessary. | |
1057 | |
1058 ***********************************************************************/ | |
1059 | |
1060 final IBuffer input (InputStream source) | |
1061 { | |
1062 this.source = source; | |
1063 return this; | |
1064 } | |
1065 | |
1066 /*********************************************************************** | |
1067 | |
1068 Access buffer content | |
1069 | |
1070 Remarks: | |
1071 Return the entire backing array. Exposed for subclass usage | |
1072 only | |
1073 | |
1074 ***********************************************************************/ | |
1075 | |
1076 protected void[] getContent () | |
1077 { | |
1078 return data; | |
1079 } | |
1080 | |
1081 /*********************************************************************** | |
1082 | |
1083 Copy content into buffer | |
1084 | |
1085 Params: | |
1086 src = the soure of the content | |
1087 size = the length of content at src | |
1088 | |
1089 Remarks: | |
1090 Bulk _copy of data from 'src'. The new content is made | |
1091 available for reading. This is exposed for subclass use | |
1092 only | |
1093 | |
1094 ***********************************************************************/ | |
1095 | |
1096 protected void copy (void *src, uint size) | |
1097 { | |
1098 // avoid "out of bounds" test on zero size | |
1099 if (size) | |
1100 { | |
1101 // content may overlap ... | |
1102 memcpy (&data[extent], src, size); | |
1103 extent += size; | |
1104 } | |
1105 } | |
1106 | |
1107 /*********************************************************************** | |
1108 | |
1109 Cast to a target type without invoking the wrath of the | |
1110 runtime checks for misalignment. Instead, we truncate the | |
1111 array length | |
1112 | |
1113 ***********************************************************************/ | |
1114 | |
1115 static T[] convert(T)(void[] x) | |
1116 { | |
1117 return (cast(T*) x.ptr) [0 .. (x.length / T.sizeof)]; | |
1118 } | |
1119 | |
1120 | |
1121 | |
1122 /**********************************************************************/ | |
1123 /*********************** Buffered Interface ***************************/ | |
1124 /**********************************************************************/ | |
1125 | |
1126 IBuffer buffer () | |
1127 { | |
1128 return this; | |
1129 } | |
1130 | |
1131 | |
1132 /**********************************************************************/ | |
1133 /******************** Stream & Conduit Interfaces *********************/ | |
1134 /**********************************************************************/ | |
1135 | |
1136 | |
1137 /*********************************************************************** | |
1138 | |
1139 Return the name of this conduit | |
1140 | |
1141 ***********************************************************************/ | |
1142 | |
1143 override char[] toString () | |
1144 { | |
1145 return "<buffer>"; | |
1146 } | |
1147 | |
1148 /*********************************************************************** | |
1149 | |
1150 Generic IOException thrower | |
1151 | |
1152 Params: | |
1153 msg = a text message describing the exception reason | |
1154 | |
1155 Remarks: | |
1156 Throw an IOException with the provided message | |
1157 | |
1158 ***********************************************************************/ | |
1159 | |
1160 final void error (char[] msg) | |
1161 { | |
1162 throw new IOException (msg); | |
1163 } | |
1164 | |
1165 /*********************************************************************** | |
1166 | |
1167 Flush all buffer content to the specific conduit | |
1168 | |
1169 Remarks: | |
1170 Flush the contents of this buffer. This will block until | |
1171 all content is actually flushed via the associated conduit, | |
1172 whereas drain() will not. | |
1173 | |
1174 Do nothing where a conduit is not attached, enabling memory | |
1175 buffers to treat flush as a noop. | |
1176 | |
1177 Throws an IOException on premature Eof. | |
1178 | |
1179 ***********************************************************************/ | |
1180 | |
1181 override OutputStream flush () | |
1182 { | |
1183 if (sink) | |
1184 { | |
1185 while (readable() > 0) | |
1186 drain (sink); | |
1187 | |
1188 // flush the filter chain also | |
1189 sink.flush; | |
1190 } | |
1191 return this; | |
1192 } | |
1193 | |
1194 /*********************************************************************** | |
1195 | |
1196 Clear buffer content | |
1197 | |
1198 Remarks: | |
1199 Reset 'position' and 'limit' to zero. This effectively | |
1200 clears all content from the buffer. | |
1201 | |
1202 ***********************************************************************/ | |
1203 | |
1204 override InputStream clear () | |
1205 { | |
1206 index = extent = 0; | |
1207 | |
1208 // clear the filter chain also | |
1209 if (source) | |
1210 source.clear; | |
1211 return this; | |
1212 } | |
1213 | |
1214 /*********************************************************************** | |
1215 | |
1216 Copy content via this buffer from the provided src | |
1217 conduit. | |
1218 | |
1219 Remarks: | |
1220 The src conduit has its content transferred through | |
1221 this buffer via a series of fill & drain operations, | |
1222 until there is no more content available. The buffer | |
1223 content should be explicitly flushed by the caller. | |
1224 | |
1225 Throws an IOException on premature eof | |
1226 | |
1227 ***********************************************************************/ | |
1228 | |
1229 override OutputStream copy (InputStream src) | |
1230 { | |
1231 while (fill(src) != IConduit.Eof) | |
1232 // don't drain until we actually need to | |
1233 if (writable is 0) | |
1234 if (sink) | |
1235 drain (sink); | |
1236 else | |
1237 error (overflow); | |
1238 return this; | |
1239 } | |
1240 | |
1241 /*********************************************************************** | |
1242 | |
1243 Transfer content into the provided dst | |
1244 | |
1245 Params: | |
1246 dst = destination of the content | |
1247 | |
1248 Returns: | |
1249 return the number of bytes read, which may be less than | |
1250 dst.length. Eof is returned when no further content is | |
1251 available. | |
1252 | |
1253 Remarks: | |
1254 Populates the provided array with content. We try to | |
1255 satisfy the request from the buffer content, and read | |
1256 directly from an attached conduit when the buffer is | |
1257 empty. | |
1258 | |
1259 ***********************************************************************/ | |
1260 | |
1261 override uint read (void[] dst) | |
1262 { | |
1263 uint content = readable(); | |
1264 if (content) | |
1265 { | |
1266 if (content >= dst.length) | |
1267 content = dst.length; | |
1268 | |
1269 // transfer buffer content | |
1270 dst [0 .. content] = data [index .. index + content]; | |
1271 index += content; | |
1272 } | |
1273 else | |
1274 if (source) | |
1275 { | |
1276 // pathological cases read directly from conduit | |
1277 if (dst.length > dimension) | |
1278 content = source.read (dst); | |
1279 else | |
1280 // keep buffer partially populated | |
1281 if ((content = fill(source)) != IConduit.Eof && content > 0) | |
1282 content = read (dst); | |
1283 } | |
1284 else | |
1285 content = IConduit.Eof; | |
1286 return content; | |
1287 } | |
1288 | |
1289 /*********************************************************************** | |
1290 | |
1291 Emulate OutputStream.write() | |
1292 | |
1293 Params: | |
1294 src = the content to write | |
1295 | |
1296 Returns: | |
1297 return the number of bytes written, which may be less than | |
1298 provided (conceptually). | |
1299 | |
1300 Remarks: | |
1301 Appends src content to the buffer, flushing to an attached | |
1302 conduit as necessary. An IOException is thrown upon write | |
1303 failure. | |
1304 | |
1305 ***********************************************************************/ | |
1306 | |
1307 override uint write (void[] src) | |
1308 { | |
1309 append (src.ptr, src.length); | |
1310 return src.length; | |
1311 } | |
1312 | |
1313 /*********************************************************************** | |
1314 | |
1315 Access configured conduit | |
1316 | |
1317 Returns: | |
1318 Returns the conduit associated with this buffer. Returns | |
1319 null if the buffer is purely memory based; that is, it's | |
1320 not backed by some external medium. | |
1321 | |
1322 Remarks: | |
1323 Buffers do not require an external conduit to operate, but | |
1324 it can be convenient to associate one. For example, methods | |
1325 fill() & drain() use it to import/export content as necessary. | |
1326 | |
1327 ***********************************************************************/ | |
1328 | |
1329 final override IConduit conduit () | |
1330 { | |
1331 if (sink) | |
1332 return sink.conduit; | |
1333 else | |
1334 if (source) | |
1335 return source.conduit; | |
1336 return this; | |
1337 } | |
1338 | |
1339 /*********************************************************************** | |
1340 | |
1341 Return a preferred size for buffering conduit I/O | |
1342 | |
1343 ***********************************************************************/ | |
1344 | |
1345 final override uint bufferSize () | |
1346 { | |
1347 return 32 * 1024; | |
1348 } | |
1349 | |
1350 /*********************************************************************** | |
1351 | |
1352 Is the conduit alive? | |
1353 | |
1354 ***********************************************************************/ | |
1355 | |
1356 final override bool isAlive () | |
1357 { | |
1358 return true; | |
1359 } | |
1360 | |
1361 /*********************************************************************** | |
1362 | |
1363 Exposes configured output stream | |
1364 | |
1365 Returns: | |
1366 Returns the OutputStream associated with this buffer. Returns | |
1367 null if the buffer is not attached to an output; that is, it's | |
1368 not backed by some external medium. | |
1369 | |
1370 Remarks: | |
1371 Buffers do not require an external stream to operate, but | |
1372 it can be convenient to associate them. For example, methods | |
1373 fill & drain use them to import/export content as necessary. | |
1374 | |
1375 ***********************************************************************/ | |
1376 | |
1377 final OutputStream output () | |
1378 { | |
1379 return sink; | |
1380 } | |
1381 | |
1382 /*********************************************************************** | |
1383 | |
1384 Exposes configured input stream | |
1385 | |
1386 Returns: | |
1387 Returns the InputStream associated with this buffer. Returns | |
1388 null if the buffer is not attached to an input; that is, it's | |
1389 not backed by some external medium. | |
1390 | |
1391 Remarks: | |
1392 Buffers do not require an external stream to operate, but | |
1393 it can be convenient to associate them. For example, methods | |
1394 fill & drain use them to import/export content as necessary. | |
1395 | |
1396 ***********************************************************************/ | |
1397 | |
1398 final InputStream input () | |
1399 { | |
1400 return source; | |
1401 } | |
1402 | |
1403 /*********************************************************************** | |
1404 | |
1405 Release external resources | |
1406 | |
1407 ***********************************************************************/ | |
1408 | |
1409 final override void detach () | |
1410 { | |
1411 } | |
1412 | |
1413 /*********************************************************************** | |
1414 | |
1415 Close the stream | |
1416 | |
1417 Remarks: | |
1418 Propagate request to an attached OutputStream (this is a | |
1419 requirement for the OutputStream interface) | |
1420 | |
1421 ***********************************************************************/ | |
1422 | |
1423 override void close () | |
1424 { | |
1425 if (sink) | |
1426 sink.close; | |
1427 else | |
1428 if (source) | |
1429 source.close; | |
1430 } | |
1431 } |