Mercurial > projects > ldc
annotate tango/tango/io/Buffer.d @ 341:1bb99290e03a trunk
[svn r362] Started merging the old 'test' dir as well as the newer 'tangotests' dir into 'tests/mini' and 'tests/minicomplex'.
| author | lindquist |
|---|---|
| date | Sun, 13 Jul 2008 02:51:19 +0200 |
| parents | 0ab29b838084 |
| children |
| rev | line source |
|---|---|
| 132 | 1 /******************************************************************************* |
| 2 | |
| 3 copyright: Copyright (c) 2004 Kris Bell. All rights reserved | |
| 4 | |
| 5 license: BSD style: $(LICENSE) | |
| 6 | |
| 7 version: Mar 2004: Initial release | |
| 8 Dec 2006: Outback release | |
| 9 | |
| 10 authors: Kris | |
| 11 | |
| 12 *******************************************************************************/ | |
| 13 | |
| 14 module tango.io.Buffer; | |
| 15 | |
| 16 private import tango.core.Exception; | |
| 17 | |
| 18 public import tango.io.model.IBuffer, | |
| 19 tango.io.model.IConduit; | |
| 20 | |
| 21 /****************************************************************************** | |
| 22 | |
| 23 ******************************************************************************/ | |
| 24 | |
| 25 extern (C) | |
| 26 { | |
| 27 protected void * memcpy (void *dst, void *src, uint); | |
| 28 } | |
| 29 | |
| 30 /******************************************************************************* | |
| 31 | |
| 32 Buffer is central concept in Tango I/O. Each buffer acts | |
| 33 as a queue (line) where items are removed from the front | |
| 34 and new items are added to the back. Buffers are modeled | |
| 35 by tango.io.model.IBuffer, and a concrete implementation | |
| 36 is provided by this class. | |
| 37 | |
| 38 Buffer can be read from and written to directly, though | |
| 39 various data-converters and filters are often leveraged | |
| 40 to apply structure to what might otherwise be simple raw | |
| 41 data. | |
| 42 | |
| 43 Buffers may also be tokenized by applying an Iterator. | |
| 44 This can be handy when one is dealing with text input, | |
| 45 and/or the content suits a more fluid format than most | |
| 46 typical converters support. Iterator tokens are mapped | |
| 47 directly onto buffer content (sliced), making them quite | |
| 48 efficient in practice. Like other types of buffer client, | |
| 49 multiple iterators can be mapped onto one common buffer | |
| 50 and access will be serialized. | |
| 51 | |
| 52 Buffers are sometimes memory-only, in which case there | |
| 53 is nothing left to do when a client has consumed all the | |
| 54 content. Other buffers are themselves bound to an external | |
| 55 device called a conduit. When this is the case, a consumer | |
| 56 will eventually cause a buffer to reload via its associated | |
| 57 conduit and previous buffer content will be lost. | |
| 58 | |
| 59 A similar approach is applied to clients which populate a | |
| 60 buffer, whereby the content of a full buffer will be flushed | |
| 61 to a bound conduit before continuing. Another variation is | |
| 62 that of a memory-mapped buffer, whereby the buffer content | |
| 63 is mapped directly to virtual memory exposed via the OS. This | |
| 64 can be used to address large files as an array of content. | |
| 65 | |
| 66 Direct buffer manipulation typically involves appending, | |
| 67 as in the following example: | |
| 68 --- | |
| 69 // create a small buffer | |
| 70 auto buf = new Buffer (256); | |
| 71 | |
| 72 auto foo = "to write some D"; | |
| 73 | |
| 74 // append some text directly to it | |
| 75 buf.append ("now is the time for all good men ").append(foo); | |
| 76 --- | |
| 77 | |
| 78 Alternatively, one might use a formatter to append the buffer: | |
| 79 --- | |
| 80 auto output = new FormatOutput (new Buffer(256)); | |
| 81 output.format ("now is the time for {} good men {}", 3, foo); | |
| 82 --- | |
| 83 | |
| 84 A slice() method will return all valid content within a buffer. | |
| 85 GrowBuffer can be used instead, where one wishes to append beyond | |
| 86 a specified limit. | |
| 87 | |
| 88 A common usage of a buffer is in conjunction with a conduit, | |
| 89 such as FileConduit. Each conduit exposes a preferred-size for | |
| 90 its associated buffers, utilized during buffer construction: | |
| 91 --- | |
| 92 auto file = new FileConduit ("file.name"); | |
| 93 auto buf = new Buffer (file); | |
| 94 --- | |
| 95 | |
| 96 However, this is typically hidden by higher level constructors | |
| 97 such as those exposed via the stream wrappers. For example: | |
| 98 --- | |
| 99 auto input = new DataInput (new FileInput("file.name")); | |
| 100 --- | |
| 101 | |
| 102 There is indeed a buffer between the resultant stream and the | |
| 103 file source, but explicit buffer construction is unecessary in | |
| 104 common cases. | |
| 105 | |
| 106 An Iterator is constructed in a similar manner, where you provide | |
| 107 it an input stream to operate upon. There's a variety of iterators | |
| 108 available in the tango.text.stream package, and they are templated | |
| 109 for each of utf8, utf16, and utf32. This example uses a line iterator | |
| 110 to sweep a text file: | |
| 111 --- | |
| 112 auto lines = new LineInput (new FileInput("file.name")); | |
| 113 foreach (line; lines) | |
| 114 Cout(line).newline; | |
| 115 --- | |
| 116 | |
| 117 Buffers are useful for many purposes within Tango, but there | |
| 118 are times when it may be more appropriate to sidestep them. For | |
| 119 such cases, all conduit derivatives (such as FileConduit) support | |
| 120 direct array-based IO via a pair of read() and write() methods. | |
| 121 | |
| 122 *******************************************************************************/ | |
| 123 | |
| 124 class Buffer : IBuffer | |
| 125 { | |
| 126 protected OutputStream sink; // optional data sink | |
| 127 protected InputStream source; // optional data source | |
| 128 protected void[] data; // the raw data buffer | |
| 129 protected uint index; // current read position | |
| 130 protected uint extent; // limit of valid content | |
| 131 protected uint dimension; // maximum extent of content | |
| 132 | |
| 133 | |
| 134 protected static char[] overflow = "output buffer is full"; | |
| 135 protected static char[] underflow = "input buffer is empty"; | |
| 136 protected static char[] eofRead = "end-of-flow whilst reading"; | |
| 137 protected static char[] eofWrite = "end-of-flow whilst writing"; | |
| 138 | |
| 139 /*********************************************************************** | |
| 140 | |
| 141 Ensure the buffer remains valid between method calls | |
| 142 | |
| 143 ***********************************************************************/ | |
| 144 | |
| 145 invariant | |
| 146 { | |
| 147 assert (index <= extent); | |
| 148 assert (extent <= dimension); | |
| 149 } | |
| 150 | |
| 151 /*********************************************************************** | |
| 152 | |
| 153 Construct a buffer | |
| 154 | |
| 155 Params: | |
| 156 conduit = the conduit to buffer | |
| 157 | |
| 158 Remarks: | |
| 159 Construct a Buffer upon the provided conduit. A relevant | |
| 160 buffer size is supplied via the provided conduit. | |
| 161 | |
| 162 ***********************************************************************/ | |
| 163 | |
| 164 this (IConduit conduit) | |
| 165 { | |
| 166 assert (conduit); | |
| 167 | |
| 168 this (conduit.bufferSize); | |
| 169 setConduit (conduit); | |
| 170 } | |
| 171 | |
| 172 /*********************************************************************** | |
| 173 | |
| 174 Construct a buffer | |
| 175 | |
| 176 Params: | |
| 177 stream = an input stream | |
| 178 capacity = desired buffer capacity | |
| 179 | |
| 180 Remarks: | |
| 181 Construct a Buffer upon the provided input stream. | |
| 182 | |
| 183 ***********************************************************************/ | |
| 184 | |
| 185 this (InputStream stream, uint capacity) | |
| 186 { | |
| 187 this (capacity); | |
| 188 input = stream; | |
| 189 } | |
| 190 | |
| 191 /*********************************************************************** | |
| 192 | |
| 193 Construct a buffer | |
| 194 | |
| 195 Params: | |
| 196 stream = an output stream | |
| 197 capacity = desired buffer capacity | |
| 198 | |
| 199 Remarks: | |
| 200 Construct a Buffer upon the provided output stream. | |
| 201 | |
| 202 ***********************************************************************/ | |
| 203 | |
| 204 this (OutputStream stream, uint capacity) | |
| 205 { | |
| 206 this (capacity); | |
| 207 output = stream; | |
| 208 } | |
| 209 | |
| 210 /*********************************************************************** | |
| 211 | |
| 212 Construct a buffer | |
| 213 | |
| 214 Params: | |
| 215 capacity = the number of bytes to make available | |
| 216 | |
| 217 Remarks: | |
| 218 Construct a Buffer with the specified number of bytes. | |
| 219 | |
| 220 ***********************************************************************/ | |
| 221 | |
| 222 this (uint capacity = 0) | |
| 223 { | |
|
136
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
224 setContent (new ubyte[capacity], 0); |
|
0e28624814e8
[svn r140] did a lot of the work towards being able to pass multiple modules on the command line. not complete yet though
lindquist
parents:
132
diff
changeset
|
225 assert(this !is null); |
| 132 | 226 } |
| 227 | |
| 228 /*********************************************************************** | |
| 229 | |
| 230 Construct a buffer | |
| 231 | |
| 232 Params: | |
| 233 data = the backing array to buffer within | |
| 234 | |
| 235 Remarks: | |
| 236 Prime a buffer with an application-supplied array. All content | |
| 237 is considered valid for reading, and thus there is no writable | |
| 238 space initially available. | |
| 239 | |
| 240 ***********************************************************************/ | |
| 241 | |
| 242 this (void[] data) | |
| 243 { | |
| 244 setContent (data, data.length); | |
| 245 } | |
| 246 | |
| 247 /*********************************************************************** | |
| 248 | |
| 249 Construct a buffer | |
| 250 | |
| 251 Params: | |
| 252 data = the backing array to buffer within | |
| 253 readable = the number of bytes initially made | |
| 254 readable | |
| 255 | |
| 256 Remarks: | |
| 257 Prime buffer with an application-supplied array, and | |
| 258 indicate how much readable data is already there. A | |
| 259 write operation will begin writing immediately after | |
| 260 the existing readable content. | |
| 261 | |
| 262 This is commonly used to attach a Buffer instance to | |
| 263 a local array. | |
| 264 | |
| 265 ***********************************************************************/ | |
| 266 | |
| 267 this (void[] data, uint readable) | |
| 268 { | |
| 269 setContent (data, readable); | |
| 270 } | |
| 271 | |
| 272 /*********************************************************************** | |
| 273 | |
| 274 Attempt to share an upstream Buffer, and create an instance | |
| 275 where there not one available. | |
| 276 | |
| 277 Params: | |
| 278 stream = an input stream | |
| 279 size = a hint of the desired buffer size. Defaults to the | |
| 280 conduit-defined size | |
| 281 | |
| 282 Remarks: | |
| 283 If an upstream Buffer instances is visible, it will be shared. | |
| 284 Otherwise, a new instance is created based upon the bufferSize | |
| 285 exposed by the stream endpoint (conduit). | |
| 286 | |
| 287 ***********************************************************************/ | |
| 288 | |
| 289 static IBuffer share (InputStream stream, uint size=uint.max) | |
| 290 { | |
| 291 auto b = cast(Buffered) stream; | |
| 292 if (b) | |
| 293 return b.buffer; | |
| 294 | |
| 295 if (size is uint.max) | |
| 296 size = stream.conduit.bufferSize; | |
| 297 | |
| 298 return new Buffer (stream, size); | |
| 299 } | |
| 300 | |
| 301 /*********************************************************************** | |
| 302 | |
| 303 Attempt to share an upstream Buffer, and create an instance | |
| 304 where there not one available. | |
| 305 | |
| 306 Params: | |
| 307 stream = an output stream | |
| 308 size = a hint of the desired buffer size. Defaults to the | |
| 309 conduit-defined size | |
| 310 | |
| 311 Remarks: | |
| 312 If an upstream Buffer instances is visible, it will be shared. | |
| 313 Otherwise, a new instance is created based upon the bufferSize | |
| 314 exposed by the stream endpoint (conduit). | |
| 315 | |
| 316 ***********************************************************************/ | |
| 317 | |
| 318 static IBuffer share (OutputStream stream, uint size=uint.max) | |
| 319 { | |
| 320 auto b = cast(Buffered) stream; | |
| 321 if (b) | |
| 322 return b.buffer; | |
| 323 | |
| 324 if (size is uint.max) | |
| 325 size = stream.conduit.bufferSize; | |
| 326 | |
| 327 return new Buffer (stream, size); | |
| 328 } | |
| 329 | |
| 330 /*********************************************************************** | |
| 331 | |
| 332 Reset the buffer content | |
| 333 | |
| 334 Params: | |
| 335 data = the backing array to buffer within. All content | |
| 336 is considered valid | |
| 337 | |
| 338 Returns: | |
| 339 the buffer instance | |
| 340 | |
| 341 Remarks: | |
| 342 Set the backing array with all content readable. Writing | |
| 343 to this will either flush it to an associated conduit, or | |
| 344 raise an Eof condition. Use clear() to reset the content | |
| 345 (make it all writable). | |
| 346 | |
| 347 ***********************************************************************/ | |
| 348 | |
| 349 IBuffer setContent (void[] data) | |
| 350 { | |
| 351 return setContent (data, data.length); | |
| 352 } | |
| 353 | |
| 354 /*********************************************************************** | |
| 355 | |
| 356 Reset the buffer content | |
| 357 | |
| 358 Params: | |
| 359 data = the backing array to buffer within | |
| 360 readable = the number of bytes within data considered | |
| 361 valid | |
| 362 | |
| 363 Returns: | |
| 364 the buffer instance | |
| 365 | |
| 366 Remarks: | |
| 367 Set the backing array with some content readable. Writing | |
| 368 to this will either flush it to an associated conduit, or | |
| 369 raise an Eof condition. Use clear() to reset the content | |
| 370 (make it all writable). | |
| 371 | |
| 372 ***********************************************************************/ | |
| 373 | |
| 374 IBuffer setContent (void[] data, uint readable) | |
| 375 { | |
| 376 this.data = data; | |
| 377 this.extent = readable; | |
| 378 this.dimension = data.length; | |
| 379 | |
| 380 // reset to start of input | |
| 381 this.index = 0; | |
| 382 | |
| 383 return this; | |
| 384 } | |
| 385 | |
| 386 /*********************************************************************** | |
| 387 | |
| 388 Access buffer content | |
| 389 | |
| 390 Params: | |
| 391 size = number of bytes to access | |
| 392 eat = whether to consume the content or not | |
| 393 | |
| 394 Returns: | |
| 395 the corresponding buffer slice when successful, or | |
| 396 null if there's not enough data available (Eof; Eob). | |
| 397 | |
| 398 Remarks: | |
| 399 Read a slice of data from the buffer, loading from the | |
| 400 conduit as necessary. The specified number of bytes is | |
| 401 sliced from the buffer, and marked as having been read | |
| 402 when the 'eat' parameter is set true. When 'eat' is set | |
| 403 false, the read position is not adjusted. | |
| 404 | |
| 405 Note that the slice cannot be larger than the size of | |
| 406 the buffer ~ use method fill(void[]) instead where you | |
| 407 simply want the content copied, or use conduit.read() | |
| 408 to extract directly from an attached conduit. Also note | |
| 409 that if you need to retain the slice, then it should be | |
| 410 .dup'd before the buffer is compressed or repopulated. | |
| 411 | |
| 412 Examples: | |
| 413 --- | |
| 414 // create a buffer with some content | |
| 415 auto buffer = new Buffer ("hello world"); | |
| 416 | |
| 417 // consume everything unread | |
| 418 auto slice = buffer.slice (buffer.readable); | |
| 419 --- | |
| 420 | |
| 421 ***********************************************************************/ | |
| 422 | |
| 423 void[] slice (uint size, bool eat = true) | |
| 424 { | |
| 425 if (size > readable) | |
| 426 { | |
| 427 if (source is null) | |
| 428 error (underflow); | |
| 429 | |
| 430 // make some space? This will try to leave as much content | |
| 431 // in the buffer as possible, such that entire records may | |
| 432 // be aliased directly from within. | |
| 433 if (size > writable) | |
| 434 { | |
| 435 if (size > dimension) | |
| 436 error (underflow); | |
| 437 compress (); | |
| 438 } | |
| 439 | |
| 440 // populate tail of buffer with new content | |
| 441 do { | |
| 442 if (fill(source) is IConduit.Eof) | |
| 443 error (eofRead); | |
| 444 } while (size > readable); | |
| 445 } | |
| 446 | |
| 447 auto i = index; | |
| 448 if (eat) | |
| 449 index += size; | |
| 450 return data [i .. i + size]; | |
| 451 } | |
| 452 | |
| 453 /********************************************************************** | |
| 454 | |
| 455 Fill the provided buffer. Returns the number of bytes | |
| 456 actually read, which will be less that dst.length when | |
| 457 Eof has been reached and IConduit.Eof thereafter | |
| 458 | |
| 459 **********************************************************************/ | |
| 460 | |
| 461 uint fill (void[] dst) | |
| 462 { | |
| 463 uint len = 0; | |
| 464 | |
| 465 while (len < dst.length) | |
| 466 { | |
| 467 uint i = read (dst [len .. $]); | |
| 468 if (i is IConduit.Eof) | |
| 469 return (len > 0) ? len : IConduit.Eof; | |
| 470 len += i; | |
| 471 } | |
| 472 return len; | |
| 473 } | |
| 474 | |
| 475 /*********************************************************************** | |
| 476 | |
| 477 Copy buffer content into the provided dst | |
| 478 | |
| 479 Params: | |
| 480 dst = destination of the content | |
| 481 bytes = size of dst | |
| 482 | |
| 483 Returns: | |
| 484 A reference to the populated content | |
| 485 | |
| 486 Remarks: | |
| 487 Fill the provided array with content. We try to satisfy | |
| 488 the request from the buffer content, and read directly | |
| 489 from an attached conduit where more is required. | |
| 490 | |
| 491 ***********************************************************************/ | |
| 492 | |
| 493 void[] readExact (void* dst, uint bytes) | |
| 494 { | |
| 495 auto tmp = dst [0 .. bytes]; | |
| 496 if (fill (tmp) != bytes) | |
| 497 error (eofRead); | |
| 498 | |
| 499 return tmp; | |
| 500 } | |
| 501 | |
| 502 /*********************************************************************** | |
| 503 | |
| 504 Append content | |
| 505 | |
| 506 Params: | |
| 507 src = the content to _append | |
| 508 | |
| 509 Returns a chaining reference if all content was written. | |
| 510 Throws an IOException indicating eof or eob if not. | |
| 511 | |
| 512 Remarks: | |
| 513 Append an array to this buffer, and flush to the | |
| 514 conduit as necessary. This is often used in lieu of | |
| 515 a Writer. | |
| 516 | |
| 517 ***********************************************************************/ | |
| 518 | |
| 519 IBuffer append (void[] src) | |
| 520 { | |
| 521 return append (src.ptr, src.length); | |
| 522 } | |
| 523 | |
| 524 /*********************************************************************** | |
| 525 | |
| 526 Append content | |
| 527 | |
| 528 Params: | |
| 529 src = the content to _append | |
| 530 length = the number of bytes in src | |
| 531 | |
| 532 Returns a chaining reference if all content was written. | |
| 533 Throws an IOException indicating eof or eob if not. | |
| 534 | |
| 535 Remarks: | |
| 536 Append an array to this buffer, and flush to the | |
| 537 conduit as necessary. This is often used in lieu of | |
| 538 a Writer. | |
| 539 | |
| 540 ***********************************************************************/ | |
| 541 | |
| 542 IBuffer append (void* src, uint length) | |
| 138 | 543 { |
| 132 | 544 if (length > writable) |
| 545 // can we write externally? | |
| 546 if (sink) | |
| 547 { | |
| 548 flush (); | |
| 549 | |
| 550 // check for pathological case | |
| 551 if (length > dimension) | |
| 552 { | |
| 553 do { | |
| 554 auto written = sink.write (src [0 .. length]); | |
| 555 if (written is IConduit.Eof) | |
| 556 error (eofWrite); | |
| 557 src += written, length -= written; | |
| 558 } while (length > dimension); | |
| 559 } | |
| 560 } | |
| 561 else | |
| 562 error (overflow); | |
| 563 copy (src, length); | |
| 564 return this; | |
| 565 } | |
| 566 | |
| 567 /*********************************************************************** | |
| 568 | |
| 569 Append content | |
| 570 | |
| 571 Params: | |
| 572 other = a buffer with content available | |
| 573 | |
| 574 Returns: | |
| 575 Returns a chaining reference if all content was written. | |
| 576 Throws an IOException indicating eof or eob if not. | |
| 577 | |
| 578 Remarks: | |
| 579 Append another buffer to this one, and flush to the | |
| 580 conduit as necessary. This is often used in lieu of | |
| 581 a Writer. | |
| 582 | |
| 583 ***********************************************************************/ | |
| 584 | |
| 585 IBuffer append (IBuffer other) | |
| 586 { | |
| 587 return append (other.slice); | |
| 588 } | |
| 589 | |
| 590 /*********************************************************************** | |
| 591 | |
| 592 Consume content from a producer | |
| 593 | |
| 594 Params: | |
| 595 The content to consume. This is consumed verbatim, and in | |
| 596 raw binary format ~ no implicit conversions are performed. | |
| 597 | |
| 598 Remarks: | |
| 599 This is often used in lieu of a Writer, and enables simple | |
| 600 classes, such as FilePath and Uri, to emit content directly | |
| 601 into a buffer (thus avoiding potential heap activity) | |
| 602 | |
| 603 Examples: | |
| 604 --- | |
| 605 auto path = new FilePath (somepath); | |
| 606 | |
| 607 path.produce (&buffer.consume); | |
| 608 --- | |
| 609 | |
| 610 ***********************************************************************/ | |
| 611 | |
| 612 void consume (void[] x) | |
| 613 { | |
| 614 append (x); | |
| 615 } | |
| 616 | |
| 617 /*********************************************************************** | |
| 618 | |
| 619 Retrieve the valid content | |
| 620 | |
| 621 Returns: | |
| 622 a void[] slice of the buffer | |
| 623 | |
| 624 Remarks: | |
| 625 Return a void[] slice of the buffer, from the current position | |
| 626 up to the limit of valid content. The content remains in the | |
| 627 buffer for future extraction. | |
| 628 | |
| 629 ***********************************************************************/ | |
| 630 | |
| 631 void[] slice () | |
| 632 { | |
| 633 return data [index .. extent]; | |
| 634 } | |
| 635 | |
| 636 /*********************************************************************** | |
| 637 | |
| 638 Move the current read location | |
| 639 | |
| 640 Params: | |
| 641 size = the number of bytes to move | |
| 642 | |
| 643 Returns: | |
| 644 Returns true if successful, false otherwise. | |
| 645 | |
| 646 Remarks: | |
| 647 Skip ahead by the specified number of bytes, streaming from | |
| 648 the associated conduit as necessary. | |
| 649 | |
| 650 Can also reverse the read position by 'size' bytes, when size | |
| 651 is negative. This may be used to support lookahead operations. | |
| 652 Note that a negative size will fail where there is not sufficient | |
| 653 content available in the buffer (can't _skip beyond the beginning). | |
| 654 | |
| 655 ***********************************************************************/ | |
| 656 | |
| 657 bool skip (int size) | |
| 658 { | |
| 659 if (size < 0) | |
| 660 { | |
| 661 size = -size; | |
| 662 if (index >= size) | |
| 663 { | |
| 664 index -= size; | |
| 665 return true; | |
| 666 } | |
| 667 return false; | |
| 668 } | |
| 669 return slice(size) !is null; | |
| 670 } | |
| 671 | |
| 672 /*********************************************************************** | |
| 673 | |
| 674 Iterator support | |
| 675 | |
| 676 Params: | |
| 677 scan = the delagate to invoke with the current content | |
| 678 | |
| 679 Returns: | |
| 680 Returns true if a token was isolated, false otherwise. | |
| 681 | |
| 682 Remarks: | |
| 683 Upon success, the delegate should return the byte-based | |
| 684 index of the consumed pattern (tail end of it). Failure | |
| 685 to match a pattern should be indicated by returning an | |
| 686 IConduit.Eof | |
| 687 | |
| 688 Each pattern is expected to be stripped of the delimiter. | |
| 689 An end-of-file condition causes trailing content to be | |
| 690 placed into the token. Requests made beyond Eof result | |
| 691 in empty matches (length is zero). | |
| 692 | |
| 693 Note that additional iterator and/or reader instances | |
| 694 will operate in lockstep when bound to a common buffer. | |
| 695 | |
| 696 ***********************************************************************/ | |
| 697 | |
| 698 bool next (uint delegate (void[]) scan) | |
| 699 { | |
| 700 while (read(scan) is IConduit.Eof) | |
| 701 // not found - are we streaming? | |
| 702 if (source) | |
| 703 { | |
| 704 // did we start at the beginning? | |
| 705 if (position) | |
| 706 // nope - move partial token to start of buffer | |
| 707 compress (); | |
| 708 else | |
| 709 // no more space in the buffer? | |
| 710 if (writable is 0) | |
| 711 error ("Token is too large to fit within buffer"); | |
| 712 | |
| 713 // read another chunk of data | |
| 714 if (fill(source) is IConduit.Eof) | |
| 715 return false; | |
| 716 } | |
| 717 else | |
| 718 return false; | |
| 719 | |
| 720 return true; | |
| 721 } | |
| 722 | |
| 723 /*********************************************************************** | |
| 724 | |
| 725 Available content | |
| 726 | |
| 727 Remarks: | |
| 728 Return count of _readable bytes remaining in buffer. This is | |
| 729 calculated simply as limit() - position() | |
| 730 | |
| 731 ***********************************************************************/ | |
| 732 | |
| 733 uint readable () | |
| 734 { | |
| 735 return extent - index; | |
| 736 } | |
| 737 | |
| 738 /*********************************************************************** | |
| 739 | |
| 740 Available space | |
| 741 | |
| 742 Remarks: | |
| 743 Return count of _writable bytes available in buffer. This is | |
| 744 calculated simply as capacity() - limit() | |
| 745 | |
| 746 ***********************************************************************/ | |
| 747 | |
| 748 uint writable () | |
| 749 { | |
| 750 return dimension - extent; | |
| 751 } | |
| 752 | |
| 753 /*********************************************************************** | |
| 754 | |
| 755 Write into this buffer | |
| 756 | |
| 757 Params: | |
| 758 dg = the callback to provide buffer access to | |
| 759 | |
| 760 Returns: | |
| 761 Returns whatever the delegate returns. | |
| 762 | |
| 763 Remarks: | |
| 764 Exposes the raw data buffer at the current _write position, | |
| 765 The delegate is provided with a void[] representing space | |
| 766 available within the buffer at the current _write position. | |
| 767 | |
| 768 The delegate should return the appropriate number of bytes | |
| 769 if it writes valid content, or IConduit.Eof on error. | |
| 770 | |
| 771 ***********************************************************************/ | |
| 772 | |
| 773 uint write (uint delegate (void[]) dg) | |
| 774 { | |
| 775 int count = dg (data [extent..dimension]); | |
| 776 | |
| 777 if (count != IConduit.Eof) | |
| 778 { | |
| 779 extent += count; | |
| 780 assert (extent <= dimension); | |
| 781 } | |
| 782 return count; | |
| 783 } | |
| 784 | |
| 785 /*********************************************************************** | |
| 786 | |
| 787 Read directly from this buffer | |
| 788 | |
| 789 Params: | |
| 790 dg = callback to provide buffer access to | |
| 791 | |
| 792 Returns: | |
| 793 Returns whatever the delegate returns. | |
| 794 | |
| 795 Remarks: | |
| 796 Exposes the raw data buffer at the current _read position. The | |
| 797 delegate is provided with a void[] representing the available | |
| 798 data, and should return zero to leave the current _read position | |
| 799 intact. | |
| 800 | |
| 801 If the delegate consumes data, it should return the number of | |
| 802 bytes consumed; or IConduit.Eof to indicate an error. | |
| 803 | |
| 804 ***********************************************************************/ | |
| 805 | |
| 806 uint read (uint delegate (void[]) dg) | |
| 807 { | |
| 808 int count = dg (data [index..extent]); | |
| 809 | |
| 810 if (count != IConduit.Eof) | |
| 811 { | |
| 812 index += count; | |
| 813 assert (index <= extent); | |
| 814 } | |
| 815 return count; | |
| 816 } | |
| 817 | |
| 818 /*********************************************************************** | |
| 819 | |
| 820 Compress buffer space | |
| 821 | |
| 822 Returns: | |
| 823 the buffer instance | |
| 824 | |
| 825 Remarks: | |
| 826 If we have some data left after an export, move it to | |
| 827 front-of-buffer and set position to be just after the | |
| 828 remains. This is for supporting certain conduits which | |
| 829 choose to write just the initial portion of a request. | |
| 830 | |
| 831 Limit is set to the amount of data remaining. Position | |
| 832 is always reset to zero. | |
| 833 | |
| 834 ***********************************************************************/ | |
| 835 | |
| 836 IBuffer compress () | |
| 837 { | |
| 838 uint r = readable (); | |
| 839 | |
| 840 if (index > 0 && r > 0) | |
| 841 // content may overlap ... | |
| 842 memcpy (&data[0], &data[index], r); | |
| 843 | |
| 844 index = 0; | |
| 845 extent = r; | |
| 846 return this; | |
| 847 } | |
| 848 | |
| 849 /*********************************************************************** | |
| 850 | |
| 851 Fill buffer from the specific conduit | |
| 852 | |
| 853 Returns: | |
| 854 Returns the number of bytes read, or Conduit.Eof | |
| 855 | |
| 856 Remarks: | |
| 857 Try to _fill the available buffer with content from the | |
| 858 specified conduit. We try to read as much as possible | |
| 859 by clearing the buffer when all current content has been | |
| 860 eaten. If there is no space available, nothing will be | |
| 861 read. | |
| 862 | |
| 863 ***********************************************************************/ | |
| 864 | |
| 865 uint fill (InputStream src) | |
| 866 { | |
| 867 if (src is null) | |
| 868 return IConduit.Eof; | |
| 869 | |
| 870 if (readable is 0) | |
| 871 index = extent = 0; // same as clear(), but without chain | |
| 872 else | |
| 873 if (writable is 0) | |
| 874 return 0; | |
| 875 | |
| 876 return write (&src.read); | |
| 877 } | |
| 878 | |
| 879 /*********************************************************************** | |
| 880 | |
| 881 Drain buffer content to the specific conduit | |
| 882 | |
| 883 Returns: | |
| 884 Returns the number of bytes written | |
| 885 | |
| 886 Remarks: | |
| 887 Write as much of the buffer that the associated conduit | |
| 888 can consume. The conduit is not obliged to consume all | |
| 889 content, so some may remain within the buffer. | |
| 890 | |
| 891 Throws an IOException on premature Eof. | |
| 892 | |
| 893 ***********************************************************************/ | |
| 894 | |
| 895 final uint drain (OutputStream dst) | |
| 896 { | |
| 897 if (dst is null) | |
| 898 return IConduit.Eof; | |
| 899 | |
| 900 uint ret = read (&dst.write); | |
| 901 if (ret is IConduit.Eof) | |
| 902 error (eofWrite); | |
| 903 | |
| 904 compress (); | |
| 905 return ret; | |
| 906 } | |
| 907 | |
| 908 /*********************************************************************** | |
| 909 | |
| 910 Truncate buffer content | |
| 911 | |
| 912 Remarks: | |
| 913 Truncate the buffer within its extent. Returns true if | |
| 914 the new length is valid, false otherwise. | |
| 915 | |
| 916 ***********************************************************************/ | |
| 917 | |
| 918 bool truncate (uint length) | |
| 919 { | |
| 920 if (length <= data.length) | |
| 921 { | |
| 922 extent = length; | |
| 923 return true; | |
| 924 } | |
| 925 return false; | |
| 926 } | |
| 927 | |
| 928 /*********************************************************************** | |
| 929 | |
| 930 Access buffer limit | |
| 931 | |
| 932 Returns: | |
| 933 Returns the limit of readable content within this buffer. | |
| 934 | |
| 935 Remarks: | |
| 936 Each buffer has a capacity, a limit, and a position. The | |
| 937 capacity is the maximum content a buffer can contain, limit | |
| 938 represents the extent of valid content, and position marks | |
| 939 the current read location. | |
| 940 | |
| 941 ***********************************************************************/ | |
| 942 | |
| 943 uint limit () | |
| 944 { | |
| 945 return extent; | |
| 946 } | |
| 947 | |
| 948 /*********************************************************************** | |
| 949 | |
| 950 Access buffer capacity | |
| 951 | |
| 952 Returns: | |
| 953 Returns the maximum capacity of this buffer | |
| 954 | |
| 955 Remarks: | |
| 956 Each buffer has a capacity, a limit, and a position. The | |
| 957 capacity is the maximum content a buffer can contain, limit | |
| 958 represents the extent of valid content, and position marks | |
| 959 the current read location. | |
| 960 | |
| 961 ***********************************************************************/ | |
| 962 | |
| 963 uint capacity () | |
| 964 { | |
| 965 return dimension; | |
| 966 } | |
| 967 | |
| 968 /*********************************************************************** | |
| 969 | |
| 970 Access buffer read position | |
| 971 | |
| 972 Returns: | |
| 973 Returns the current read-position within this buffer | |
| 974 | |
| 975 Remarks: | |
| 976 Each buffer has a capacity, a limit, and a position. The | |
| 977 capacity is the maximum content a buffer can contain, limit | |
| 978 represents the extent of valid content, and position marks | |
| 979 the current read location. | |
| 980 | |
| 981 ***********************************************************************/ | |
| 982 | |
| 983 uint position () | |
| 984 { | |
| 985 return index; | |
| 986 } | |
| 987 | |
| 988 /*********************************************************************** | |
| 989 | |
| 990 Set external conduit | |
| 991 | |
| 992 Params: | |
| 993 conduit = the conduit to attach to | |
| 994 | |
| 995 Remarks: | |
| 996 Sets the external conduit associated with this buffer. | |
| 997 | |
| 998 Buffers do not require an external conduit to operate, but | |
| 999 it can be convenient to associate one. For example, methods | |
| 1000 fill() & drain() use it to import/export content as necessary. | |
| 1001 | |
| 1002 ***********************************************************************/ | |
| 1003 | |
| 1004 IBuffer setConduit (IConduit conduit) | |
| 1005 { | |
| 1006 sink = conduit.output; | |
| 1007 source = conduit.input; | |
| 1008 return this; | |
| 1009 } | |
| 1010 | |
| 1011 /*********************************************************************** | |
| 1012 | |
| 1013 Set output stream | |
| 1014 | |
| 1015 Params: | |
| 1016 sink = the stream to attach to | |
| 1017 | |
| 1018 Remarks: | |
| 1019 Sets the external output stream associated with this buffer. | |
| 1020 | |
| 1021 Buffers do not require an external stream to operate, but | |
| 1022 it can be convenient to associate one. For example, methods | |
| 1023 fill & drain use them to import/export content as necessary. | |
| 1024 | |
| 1025 ***********************************************************************/ | |
| 1026 | |
| 1027 final IBuffer output (OutputStream sink) | |
| 1028 { | |
| 1029 this.sink = sink; | |
| 1030 return this; | |
| 1031 } | |
| 1032 | |
| 1033 /*********************************************************************** | |
| 1034 | |
| 1035 Set input stream | |
| 1036 | |
| 1037 Params: | |
| 1038 source = the stream to attach to | |
| 1039 | |
| 1040 Remarks: | |
| 1041 Sets the external input stream associated with this buffer. | |
| 1042 | |
| 1043 Buffers do not require an external stream to operate, but | |
| 1044 it can be convenient to associate one. For example, methods | |
| 1045 fill & drain use them to import/export content as necessary. | |
| 1046 | |
| 1047 ***********************************************************************/ | |
| 1048 | |
| 1049 final IBuffer input (InputStream source) | |
| 1050 { | |
| 1051 this.source = source; | |
| 1052 return this; | |
| 1053 } | |
| 1054 | |
| 1055 /*********************************************************************** | |
| 1056 | |
| 1057 Access buffer content | |
| 1058 | |
| 1059 Remarks: | |
| 1060 Return the entire backing array. Exposed for subclass usage | |
| 1061 only | |
| 1062 | |
| 1063 ***********************************************************************/ | |
| 1064 | |
| 1065 protected void[] getContent () | |
| 1066 { | |
| 1067 return data; | |
| 1068 } | |
| 1069 | |
| 1070 /*********************************************************************** | |
| 1071 | |
| 1072 Copy content into buffer | |
| 1073 | |
| 1074 Params: | |
| 1075 src = the soure of the content | |
| 1076 size = the length of content at src | |
| 1077 | |
| 1078 Remarks: | |
| 1079 Bulk _copy of data from 'src'. The new content is made | |
| 1080 available for reading. This is exposed for subclass use | |
| 1081 only | |
| 1082 | |
| 1083 ***********************************************************************/ | |
| 1084 | |
| 1085 protected void copy (void *src, uint size) | |
| 1086 { | |
| 1087 // avoid "out of bounds" test on zero size | |
| 1088 if (size) | |
| 1089 { | |
| 1090 // content may overlap ... | |
| 1091 memcpy (&data[extent], src, size); | |
| 1092 extent += size; | |
| 1093 } | |
| 1094 } | |
| 1095 | |
| 1096 /*********************************************************************** | |
| 1097 | |
| 1098 Cast to a target type without invoking the wrath of the | |
| 1099 runtime checks for misalignment. Instead, we truncate the | |
| 1100 array length | |
| 1101 | |
| 1102 ***********************************************************************/ | |
| 1103 | |
| 1104 static T[] convert(T)(void[] x) | |
| 1105 { | |
| 1106 return (cast(T*) x.ptr) [0 .. (x.length / T.sizeof)]; | |
| 1107 } | |
| 1108 | |
| 1109 | |
| 1110 | |
| 1111 /**********************************************************************/ | |
| 1112 /*********************** Buffered Interface ***************************/ | |
| 1113 /**********************************************************************/ | |
| 1114 | |
| 1115 IBuffer buffer () | |
| 1116 { | |
| 1117 return this; | |
| 1118 } | |
| 1119 | |
| 1120 | |
| 1121 /**********************************************************************/ | |
| 1122 /******************** Stream & Conduit Interfaces *********************/ | |
| 1123 /**********************************************************************/ | |
| 1124 | |
| 1125 | |
| 1126 /*********************************************************************** | |
| 1127 | |
| 1128 Return the name of this conduit | |
| 1129 | |
| 1130 ***********************************************************************/ | |
| 1131 | |
| 1132 override char[] toString () | |
| 1133 { | |
| 1134 return "<buffer>"; | |
| 1135 } | |
| 1136 | |
| 1137 /*********************************************************************** | |
| 1138 | |
| 1139 Generic IOException thrower | |
| 1140 | |
| 1141 Params: | |
| 1142 msg = a text message describing the exception reason | |
| 1143 | |
| 1144 Remarks: | |
| 1145 Throw an IOException with the provided message | |
| 1146 | |
| 1147 ***********************************************************************/ | |
| 1148 | |
| 1149 final void error (char[] msg) | |
| 1150 { | |
| 1151 throw new IOException (msg); | |
| 1152 } | |
| 1153 | |
| 1154 /*********************************************************************** | |
| 1155 | |
| 1156 Flush all buffer content to the specific conduit | |
| 1157 | |
| 1158 Remarks: | |
| 1159 Flush the contents of this buffer. This will block until | |
| 1160 all content is actually flushed via the associated conduit, | |
| 1161 whereas drain() will not. | |
| 1162 | |
| 1163 Do nothing where a conduit is not attached, enabling memory | |
| 1164 buffers to treat flush as a noop. | |
| 1165 | |
| 1166 Throws an IOException on premature Eof. | |
| 1167 | |
| 1168 ***********************************************************************/ | |
| 1169 | |
| 1170 override OutputStream flush () | |
| 1171 { | |
| 1172 if (sink) | |
| 1173 { | |
| 1174 while (readable() > 0) | |
| 1175 drain (sink); | |
| 1176 | |
| 1177 // flush the filter chain also | |
| 1178 sink.flush; | |
| 1179 } | |
| 1180 return this; | |
| 1181 } | |
| 1182 | |
| 1183 /*********************************************************************** | |
| 1184 | |
| 1185 Clear buffer content | |
| 1186 | |
| 1187 Remarks: | |
| 1188 Reset 'position' and 'limit' to zero. This effectively | |
| 1189 clears all content from the buffer. | |
| 1190 | |
| 1191 ***********************************************************************/ | |
| 1192 | |
| 1193 override InputStream clear () | |
| 1194 { | |
| 1195 index = extent = 0; | |
| 1196 | |
| 1197 // clear the filter chain also | |
| 1198 if (source) | |
| 1199 source.clear; | |
| 1200 return this; | |
| 1201 } | |
| 1202 | |
| 1203 /*********************************************************************** | |
| 1204 | |
| 1205 Copy content via this buffer from the provided src | |
| 1206 conduit. | |
| 1207 | |
| 1208 Remarks: | |
| 1209 The src conduit has its content transferred through | |
| 1210 this buffer via a series of fill & drain operations, | |
| 1211 until there is no more content available. The buffer | |
| 1212 content should be explicitly flushed by the caller. | |
| 1213 | |
| 1214 Throws an IOException on premature eof | |
| 1215 | |
| 1216 ***********************************************************************/ | |
| 1217 | |
| 1218 override OutputStream copy (InputStream src) | |
| 1219 { | |
| 1220 while (fill(src) != IConduit.Eof) | |
| 1221 // don't drain until we actually need to | |
| 1222 if (writable is 0) | |
| 1223 if (sink) | |
| 1224 drain (sink); | |
| 1225 else | |
| 1226 error (overflow); | |
| 1227 return this; | |
| 1228 } | |
| 1229 | |
| 1230 /*********************************************************************** | |
| 1231 | |
| 1232 Transfer content into the provided dst | |
| 1233 | |
| 1234 Params: | |
| 1235 dst = destination of the content | |
| 1236 | |
| 1237 Returns: | |
| 1238 return the number of bytes read, which may be less than | |
| 1239 dst.length. Eof is returned when no further content is | |
| 1240 available. | |
| 1241 | |
| 1242 Remarks: | |
| 1243 Populates the provided array with content. We try to | |
| 1244 satisfy the request from the buffer content, and read | |
| 1245 directly from an attached conduit when the buffer is | |
| 1246 empty. | |
| 1247 | |
| 1248 ***********************************************************************/ | |
| 1249 | |
| 1250 override uint read (void[] dst) | |
| 1251 { | |
| 1252 uint content = readable(); | |
| 1253 if (content) | |
| 1254 { | |
| 1255 if (content >= dst.length) | |
| 1256 content = dst.length; | |
| 1257 | |
| 1258 // transfer buffer content | |
| 1259 dst [0 .. content] = data [index .. index + content]; | |
| 1260 index += content; | |
| 1261 } | |
| 1262 else | |
| 1263 if (source) | |
| 1264 { | |
| 1265 // pathological cases read directly from conduit | |
| 1266 if (dst.length > dimension) | |
| 1267 content = source.read (dst); | |
| 1268 else | |
| 1269 // keep buffer partially populated | |
| 1270 if ((content = fill(source)) != IConduit.Eof && content > 0) | |
| 1271 content = read (dst); | |
| 1272 } | |
| 1273 else | |
| 1274 content = IConduit.Eof; | |
| 1275 return content; | |
| 1276 } | |
| 1277 | |
| 1278 /*********************************************************************** | |
| 1279 | |
| 1280 Emulate OutputStream.write() | |
| 1281 | |
| 1282 Params: | |
| 1283 src = the content to write | |
| 1284 | |
| 1285 Returns: | |
| 1286 return the number of bytes written, which may be less than | |
| 1287 provided (conceptually). | |
| 1288 | |
| 1289 Remarks: | |
| 1290 Appends src content to the buffer, flushing to an attached | |
| 1291 conduit as necessary. An IOException is thrown upon write | |
| 1292 failure. | |
| 1293 | |
| 1294 ***********************************************************************/ | |
| 1295 | |
| 1296 override uint write (void[] src) | |
| 1297 { | |
| 1298 append (src.ptr, src.length); | |
| 1299 return src.length; | |
| 1300 } | |
| 1301 | |
| 1302 /*********************************************************************** | |
| 1303 | |
| 1304 Access configured conduit | |
| 1305 | |
| 1306 Returns: | |
| 1307 Returns the conduit associated with this buffer. Returns | |
| 1308 null if the buffer is purely memory based; that is, it's | |
| 1309 not backed by some external medium. | |
| 1310 | |
| 1311 Remarks: | |
| 1312 Buffers do not require an external conduit to operate, but | |
| 1313 it can be convenient to associate one. For example, methods | |
| 1314 fill() & drain() use it to import/export content as necessary. | |
| 1315 | |
| 1316 ***********************************************************************/ | |
| 1317 | |
| 1318 final override IConduit conduit () | |
| 1319 { | |
| 1320 if (sink) | |
| 1321 return sink.conduit; | |
| 1322 else | |
| 1323 if (source) | |
| 1324 return source.conduit; | |
| 1325 return this; | |
| 1326 } | |
| 1327 | |
| 1328 /*********************************************************************** | |
| 1329 | |
| 1330 Return a preferred size for buffering conduit I/O | |
| 1331 | |
| 1332 ***********************************************************************/ | |
| 1333 | |
| 1334 final override uint bufferSize () | |
| 1335 { | |
| 1336 return 32 * 1024; | |
| 1337 } | |
| 1338 | |
| 1339 /*********************************************************************** | |
| 1340 | |
| 1341 Is the conduit alive? | |
| 1342 | |
| 1343 ***********************************************************************/ | |
| 1344 | |
| 1345 final override bool isAlive () | |
| 1346 { | |
| 1347 return true; | |
| 1348 } | |
| 1349 | |
| 1350 /*********************************************************************** | |
| 1351 | |
| 1352 Exposes configured output stream | |
| 1353 | |
| 1354 Returns: | |
| 1355 Returns the OutputStream associated with this buffer. Returns | |
| 1356 null if the buffer is not attached to an output; that is, it's | |
| 1357 not backed by some external medium. | |
| 1358 | |
| 1359 Remarks: | |
| 1360 Buffers do not require an external stream to operate, but | |
| 1361 it can be convenient to associate them. For example, methods | |
| 1362 fill & drain use them to import/export content as necessary. | |
| 1363 | |
| 1364 ***********************************************************************/ | |
| 1365 | |
| 1366 final OutputStream output () | |
| 1367 { | |
| 1368 return sink; | |
| 1369 } | |
| 1370 | |
| 1371 /*********************************************************************** | |
| 1372 | |
| 1373 Exposes configured input stream | |
| 1374 | |
| 1375 Returns: | |
| 1376 Returns the InputStream associated with this buffer. Returns | |
| 1377 null if the buffer is not attached to an input; that is, it's | |
| 1378 not backed by some external medium. | |
| 1379 | |
| 1380 Remarks: | |
| 1381 Buffers do not require an external stream to operate, but | |
| 1382 it can be convenient to associate them. For example, methods | |
| 1383 fill & drain use them to import/export content as necessary. | |
| 1384 | |
| 1385 ***********************************************************************/ | |
| 1386 | |
| 1387 final InputStream input () | |
| 1388 { | |
| 1389 return source; | |
| 1390 } | |
| 1391 | |
| 1392 /*********************************************************************** | |
| 1393 | |
| 1394 Release external resources | |
| 1395 | |
| 1396 ***********************************************************************/ | |
| 1397 | |
| 1398 final override void detach () | |
| 1399 { | |
| 1400 } | |
| 1401 | |
| 1402 /*********************************************************************** | |
| 1403 | |
| 1404 Close the stream | |
| 1405 | |
| 1406 Remarks: | |
| 1407 Propagate request to an attached OutputStream (this is a | |
| 1408 requirement for the OutputStream interface) | |
| 1409 | |
| 1410 ***********************************************************************/ | |
| 1411 | |
| 1412 override void close () | |
| 1413 { | |
| 1414 if (sink) | |
| 1415 sink.close; | |
| 1416 else | |
| 1417 if (source) | |
| 1418 source.close; | |
| 1419 } | |
| 1420 } |