Mercurial > projects > mde
annotate mde/mergetag/read.d @ 11:b940f267419e
Options class created & changes to mergetag exception messages.
Options class created (barebones). Loading/saving from Init.
Init no longer runs cleanup functions after initialisation failure.
Improved mergetag exception messages & error reporting.
committer: Diggory Hardy <diggory.hardy@gmail.com>
| author | Diggory Hardy <diggory.hardy@gmail.com> |
|---|---|
| date | Thu, 21 Feb 2008 09:05:33 +0000 |
| parents | 4c3575400769 |
| children | bff0d802cb7d |
| rev | line source |
|---|---|
| 0 | 1 /************************************************************************************************** |
| 2 * This module contains all reading functions, for both binary and text MergeTag files. | |
| 3 * | |
| 4 * It publically imports mde.mergetag.dataset. | |
| 5 *************************************************************************************************/ | |
| 6 | |
| 7 module mde.mergetag.read; | |
| 8 | |
| 9 // package imports | |
| 10 public import mde.mergetag.dataset; | |
|
10
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
11 import mde.mergetag.defaultdata; |
|
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
12 import mde.mergetag.exception; |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
13 |
|
10
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
14 import tango.core.Exception; |
| 0 | 15 |
| 16 // tango imports | |
| 17 import tango.io.UnicodeFile; | |
| 18 import Util = tango.text.Util; | |
| 19 import ConvInt = tango.text.convert.Integer; | |
| 20 import tango.util.collection.model.View : View; | |
| 21 import tango.util.collection.ArrayBag : ArrayBag; | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
22 import tango.util.collection.HashSet : HashSet; |
| 0 | 23 import tango.util.log.Log : Log, Logger; |
| 24 | |
| 25 // TODO: allow compressing with zlib for both binary and text? (.mtz, .mtt, .mtb extensions) | |
| 26 | |
| 27 /** | |
| 28 * Class for reading a file. | |
| 29 * | |
| 30 * Use as: | |
| 31 * ----------------------- | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
32 * Reader foo; |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
33 * try { |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
34 * foo = new Reader("foo.mtt"); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
35 * foo.read(); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
36 * } |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
37 * catch (MTException) {} |
| 0 | 38 * // get your data from foo.dataset. |
| 39 * ----------------------- | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
40 * |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
41 * Throws: |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
42 * $(TABLE |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
43 * $(TR $(TH Exception) $(TH Thrown when)) |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
44 * $(TR $(TD MTFileIOException) $(TD An error occurs while opening the file)) |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
45 * $(TR $(TD MTFileFormatException) $(TD The file doesn't start with a recognised header/version)) |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
46 * $(TR $(TD MTSyntaxException) $(TD A file syntax error occurs)) |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
47 * $(TR $(TD MTException) $(TD An unexpected error occurs)) |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
48 * ) |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
49 * Note that all exceptions extend MTException and when any exception is thrown the class is |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
50 * rendered unusable: any subsequent calls to read will be ignored. |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
51 * |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
52 * Threading: Separate instances of Reader should be thread-safe provided access to the same |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
53 * dataset is synchronized; i.e. no two readers refering to the same dataset should run |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
54 * simultaneously. (The Reader class could be made thread-safe w.r.t. datasets, but |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
55 * performance-wise I doubt it would be worth it.) |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
56 * Do not run a single instance of Reader in multiple threads simultaneously. |
| 0 | 57 */ |
| 58 class Reader | |
| 59 { | |
| 60 //BEGIN DATA | |
| 61 /** | |
| 62 A container for all read data. | |
| 63 | |
| 64 This may be accessed from here; however it may be preferable to use an external reference | |
| 65 (passed to the class on initialisation). | |
| 66 */ | |
| 67 DataSet dataset; | |
| 68 | |
| 69 /** A function for creating new DataSections within the dataset. | |
| 70 * | |
| 71 * Allows a user-made class to be used in the DataSet instead of DefaultData. | |
| 72 * | |
| 73 * This works by supplying a function which returns a reference to an instance of a class | |
| 74 * implementing DataSection. The function is passed the ID of the new section and may use this | |
| 75 * to use different DataSection classes for different sections. | |
| 76 */ | |
| 77 DataSection function (ID) dataSecCreator = null; | |
| 78 | |
| 79 private: | |
| 80 static Logger logger; | |
| 81 | |
| 82 // Non-static symbols: | |
|
7
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
83 final char[] ErrFile; // added after ErrInFile to do the same without the "in " bit. |
| 0 | 84 final char[] ErrInFile; // something like "in \"path/file.mtt\"" |
| 85 | |
| 86 final char[] fbuf; // file is read into this | |
|
1
18491334a525
Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
0
diff
changeset
|
87 MTFormatVersion.VERS fileVer = MTFormatVersion.VERS.INVALID; // Remains INVALID until set otherwise by CTOR. |
| 0 | 88 |
| 89 uint endOfHeader; | |
| 90 bool allRead = false; // true if endOfHeader == fbuf.length or read([]) has run | |
| 91 bool fatal = false; // a fatal file error occured; don't try to recover | |
| 92 /* If the file is scanned for sections, the starting position of all sections are stored | |
| 93 * in secTable. If this is empty, either no sections exist (and endOfHeader == fbuf.length) | |
| 94 * or a section scan has not been run (read() with no section names doesn't need to do so). | |
| 95 */ | |
| 96 struct SecMD { // sec meta data | |
| 97 static SecMD opCall (uint _pos, bool _read) { | |
| 98 SecMD ret; | |
| 99 ret.pos = _pos; | |
| 100 ret.read = _read; | |
| 101 return ret; | |
| 102 } | |
| 103 uint pos; // position to start reading | |
| 104 bool read; // true if already read | |
| 105 } | |
| 106 SecMD [ID] secTable; | |
| 107 //END DATA | |
| 108 | |
| 109 //BEGIN METHODS: CTOR / DTOR | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
110 static this () { |
| 0 | 111 logger = Log.getLogger ("mde.mergetag.read.Reader"); |
| 112 } | |
| 113 | |
| 114 /** Tries to open file path and read it into a buffer. | |
| 115 * | |
| 116 * Params: | |
| 117 * path = The name or FilePath of the file to open. | |
| 118 * Standard extensions are .mtt and .mtb for text and binary files respectively. | |
| 119 * dataset_ = If null create a new DataSet, else use existing DataSet *dataset_ and merge read | |
| 120 * data into it. | |
| 121 * rdHeader = If true, read the header like a standard section. Doesn't read the header by | |
| 122 * default since if it's not requested it's likely not wanted. | |
| 123 * | |
| 124 * Memory: | |
| 125 * This currently works by loading the whole file into memory at once. This should be fine most | |
| 126 * of the time, but could potentially be a problem. Changing this would mean significantly | |
| 127 * changes to the way the code works. | |
| 128 */ | |
| 129 /* Ideas for implementing a partial-loading memory model: | |
| 130 * Use a conduit directly. | |
| 131 * Use a fiber to do the parsing; let it switch back when it runs out of memory. | |
| 132 * Redesign the code so it never needs to look backwards in the buffer? | |
| 133 * | |
| 134 * Major problem: reading only some sections and keeping references to other sections | |
| 135 * would no longer be possible. | |
| 136 */ | |
| 137 public this (char[] path, DataSet* dataset_ = null, bool rdHeader = false) { | |
| 138 this (new FilePath (path), dataset_, rdHeader); | |
| 139 } | |
| 140 /** ditto */ | |
| 141 public this (PathView path, DataSet* dataset_ = null, bool rdHeader = false) { | |
| 142 // Create a dataset or use an existing one | |
| 143 if (dataset_) dataset = *dataset_; | |
| 144 else dataset = new DataSet(); | |
| 145 | |
| 146 // Open & read the file | |
| 147 try { // Supports unicode files with a BOM; defaults to UTF8 when there isn't a BOM: | |
| 148 scope file = new UnicodeFile!(char) (path, Encoding.Unknown); | |
| 149 fbuf = cast(char[]) file.read(); | |
| 150 } catch (Exception e) { | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
151 throwMTErr ("Error reading file: " ~ e.msg, new MTFileIOException); |
| 0 | 152 } |
| 153 // Remember the file name so that we can report errors (somewhat) informatively: | |
|
7
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
154 ErrFile = path.path ~ path.file; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
155 ErrInFile = " in \"" ~ ErrFile ~ '"'; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
156 |
| 0 | 157 // Version checking & matching header section tag: |
| 158 if (fbuf.length < 6 || fbuf[0] != '{' || fbuf[1] != 'M' || fbuf[2] != 'T' || fbuf[5] != '}') | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
159 throwMTErr("Not a valid MergeTag text file" ~ ErrInFile, new MTFileFormatException); |
| 0 | 160 fileVer = MTFormatVersion.parseString (fbuf[3..5]); |
| 161 if (fileVer == MTFormatVersion.VERS.INVALID) | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
162 throwMTErr("Unrecognised MergeTag version: MT" ~ fbuf[3..5] ~ ErrInFile, new MTFileFormatException); |
| 0 | 163 |
| 164 // Header reading/skipping: | |
| 165 if (rdHeader) { // only bother actually reading it if it was requested | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
166 // If already existing, merge. |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
167 if (!dataset.header) dataset.header = new DefaultData; |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
168 endOfHeader = parseSection (6, cast(DataSection*) &dataset.header); |
| 0 | 169 } |
| 170 else endOfHeader = parseSection (6,null); | |
| 171 } | |
| 172 //END METHODS: CTOR / DTOR | |
| 173 | |
| 174 //BEGIN METHODS: PUBLIC | |
| 175 /// Scans for sections if not already done and returns a list of IDs. | |
| 176 public uint[] getSectionNames () { | |
| 177 if (fatal) return []; | |
| 178 if (!secTable.length) | |
| 179 for (uint pos = endOfHeader; pos < fbuf.length;) { | |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
180 ID id = fbufReadSecMarker (pos); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
181 secTable[id] = SecMD(pos,false); // add to table |
| 0 | 182 pos = parseSection (pos, null); |
| 183 } | |
| 184 return cast(uint[]) secTable.keys; | |
| 185 } | |
| 186 | |
| 187 /** Reads (some) sections of the file into data. Note that sections will never be _read twice. | |
| 188 * | |
| 189 * To be more accurate, the file is copied into a buffer by this(). read() then parses the | |
| 190 * contents of this buffer, and stores the contents in dataset. | |
| 191 * | |
| 192 * Each section read is stored in a DataSection class. By default this is an instance of | |
| 193 * DefaultData; this can be customised (see setDataSectionCreator). | |
| 194 * | |
| 195 * If secSet is non-empty, reading is restricted to sections given in secSet, otherwise all | |
| 196 * sections are read. Sections given in secSet but not found in the file are not reported as an | |
| 197 * error. Suggested: supply a HashSet!(uint) as the View!(ID). An ArrayBag!(ID) as used is not a | |
| 198 * good choice, except that in this case it's empty. | |
| 199 * | |
| 200 * Merging: | |
| 201 * Where a section already exists in the DataSet (when either the section is given more than | |
| 202 * once in the file, or it was read from a different file by another reader) it is merged. | |
| 203 * Entries already in the DataSet take priority. | |
| 204 * | |
| 205 * Performance: | |
| 206 * Note that loading only desired sections like this still parses the sections not | |
| 207 * read (although it does not try to understand the type or data fields), so there is only a | |
| 208 * small performance advantage to this where other sections do exist in the file. There is also | |
| 209 * some overhead in only partially reading the file to keep track of where other sections are so | |
| 210 * that the entire file need not be re-read if further (or all remaining) sections are read | |
| 211 * later. | |
| 212 */ | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
213 public void read (ID[] secSet) { |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
214 HashSet!(ID) hs; |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
215 foreach (id; secSet) hs.add(id); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
216 read (hs); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
217 } |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
218 public void read (View!(ID) secSet = new ArrayBag!(ID)) { /** ditto */ |
|
7
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
219 /* Look for a section; return it if it exists otherwise create a new section: |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
220 * use dataSecCreator if it exists or just create a DefaultData if not. |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
221 */ |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
222 DataSection getOrCreateSec (ID id) { |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
223 DataSection* i = id in dataset.sec; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
224 if (i) return *i; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
225 return (dataset.sec[id] = (dataSecCreator !is null) ? dataSecCreator(id) : new DefaultData); |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
226 } |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
227 |
| 0 | 228 if (allRead || fatal) return; // never do anything in either case |
| 229 if (secSet.size) { | |
| 230 if (secTable.length) { | |
| 231 foreach (ID id; secSet) { | |
| 232 SecMD* psmd = id in secTable; | |
| 233 if (psmd && !psmd.read) { // may not exist | |
| 234 DataSection ds = getOrCreateSec (id); | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
235 parseSection (psmd.pos, &ds); |
| 0 | 236 psmd.read = true; |
| 237 } | |
| 238 } | |
| 239 } else { | |
| 240 for (uint pos = endOfHeader; pos < fbuf.length;) { | |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
241 ID id = fbufReadSecMarker (pos); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
242 secTable[id] = SecMD(pos,false); // add to table |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
243 if (secSet.contains(id)) { |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
244 DataSection ds = getOrCreateSec (id); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
245 pos = parseSection (pos, &ds); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
246 secTable[id].read = true; |
| 0 | 247 } |
| 248 } | |
| 249 } | |
| 250 } else { | |
| 251 if (secTable.length) { | |
| 252 foreach (ID id, ref SecMD smd; secTable) { | |
| 253 if (!smd.read) { | |
| 254 DataSection ds = getOrCreateSec (id); | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
255 parseSection (smd.pos, &ds); |
| 0 | 256 smd.read = true; |
| 257 } | |
| 258 } | |
| 259 } else { // this time we don't need to use secTable | |
| 260 for (uint pos = endOfHeader; pos < fbuf.length;) { | |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
261 ID id = fbufReadSecMarker (pos); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
262 DataSection ds = getOrCreateSec (id); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
263 pos = parseSection (pos, &ds); |
| 0 | 264 } |
| 265 } | |
| 266 allRead = true; | |
| 267 } | |
| 268 } | |
| 269 //END METHODS: PUBLIC | |
| 270 | |
| 271 //BEGIN METHODS: PRIVATE | |
| 272 /* Reads a section, starting from index pos, finishing at the next section marker (returning | |
| 273 the position of the start of the marker). pos should start after the section marker. | |
| 274 | |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
275 After analysing tags, the function passes the type, ID and data to addTag. |
| 0 | 276 |
| 277 NOTE: from performance tests on indexing char[]'s and dereferencing char*'s, the char*'s are | |
| 278 slightly faster, but a tiny difference isn't worth the extra effort/risk of using char*'s. | |
| 279 */ | |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
280 private uint parseSection (uint pos, DataSection* dsec) { |
|
7
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
281 /* Searches fbuf starting from start to find one of <=>| and stops at its index. |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
282 |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
283 If quotable then be quote-aware for single and double quotes. |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
284 Note: there's no length restriction for the content of the quote since it could be a single |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
285 non-ascii UTF-8 char which would look like several chars. |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
286 */ |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
287 void fbufLocateDataTagChar (inout uint pos, bool quotable) { |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
288 for (; pos < fbuf.length; ++pos) { |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
289 if ((fbuf[pos] >= '<' && fbuf[pos] <= '>') || fbuf[pos] == '|') return; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
290 else if (quotable) { |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
291 char c = fbuf[pos]; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
292 if (c == '\'' || c == '"') { |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
293 ++pos; |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
294 while (fbuf[pos] != c) { |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
295 if (fbuf[pos] == '\\') ++pos; // escape seq. |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
296 fbufIncrement(pos); |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
297 } |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
298 } |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
299 } |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
300 } |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
301 } |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
302 |
| 0 | 303 bool comment = false; // preceding char was ! |
| 304 for (; pos < fbuf.length; ++pos) { | |
| 305 if (Util.isSpace(fbuf[pos])) continue; // whitespace | |
| 306 else if (fbuf[pos] == '<') { // data tag | |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
307 char[] ErrDTAG = "Bad data tag format: not <type|id=data>" ~ ErrInFile; |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
308 |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
309 fbufIncrement (pos); |
| 0 | 310 |
| 311 // Type section of tag: | |
| 312 uint pos_s = pos; | |
| 313 fbufLocateDataTagChar (pos, false); // find end of type section | |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
314 if (fbuf[pos] != '|') throwMTErr (ErrDTAG, new MTSyntaxException); |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
315 char[] type = fbuf[pos_s..pos]; |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
316 |
| 0 | 317 fbufIncrement (pos); |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
318 |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
319 // ID section of tag: |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
320 pos_s = pos; |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
321 fbufLocateDataTagChar (pos, false); // find end of type section |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
322 if (fbuf[pos] != '=') throwMTErr (ErrDTAG, new MTSyntaxException); |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
323 ID tagID = cast(ID) fbuf[pos_s..pos]; |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
324 |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
325 fbufIncrement (pos); |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
326 |
| 0 | 327 // Data section of tag: |
| 328 pos_s = pos; | |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
329 fbufLocateDataTagChar (pos, true); // find end of data section |
|
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
330 if (fbuf[pos] != '>') throwMTErr (ErrDTAG, new MTSyntaxException); |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
331 char[] data = fbuf[pos_s..pos]; |
| 0 | 332 |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
333 if (!comment && dsec != null) { |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
334 type = Util.trim(type); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
335 try { |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
336 dsec.addTag (type, tagID, data); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
337 } |
|
10
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
338 catch (TextException e) { |
|
7
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
339 logger.warn ("While reading " ~ ErrFile ~ ":"); // following a parse error |
|
b544c3a7c9ca
Some changes to exceptions and a few more debug commands.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
6
diff
changeset
|
340 logger.warn (e.msg); |
|
4
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
341 } |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
342 catch (MTUnknownTypeException e) { |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
343 logger.warn ("Unsupported type \"" ~ type ~ "\" " ~ ErrInFile /*~ ":"*/); |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
344 //logger.warn (e.msg); needless; the above includes enough details. |
|
9a990644948c
Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
3
diff
changeset
|
345 } |
|
10
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
346 catch (Exception e) { |
|
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
347 logger.error ("Unknown error occured" ~ ErrInFile ~ ':'); |
|
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
348 logger.error (e.msg); |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
349 throwMTErr (e.msg); // Fatal to Reader |
|
6
dcb24afa0dce
Some fixes from mde/text/format.d unittests plus a few more fixes.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
4
diff
changeset
|
350 } |
| 0 | 351 } else comment = false; // cancel comment status now |
| 352 } | |
| 353 else if (fbuf[pos] == '{') { | |
| 354 if (comment) { // simple block comment | |
| 355 uint depth = 0; // depth of embedded comment blocks | |
| 356 while (true) { | |
| 357 fbufIncrement (pos); | |
| 358 if (fbuf[pos] == '}') { | |
| 359 if (depth == 0) break; | |
| 360 else --depth; | |
| 361 } else if (fbuf[pos] == '{') | |
| 362 ++depth; | |
| 363 } | |
| 364 comment = false; // end of this comment | |
| 365 } else { | |
| 366 return pos; // next section coming up; we are done | |
| 367 } | |
| 368 } | |
| 369 else if (fbuf[pos] == '!') { // possibly a comment; check next char | |
| 370 comment = true; // starting a comment (or an error) | |
| 371 // variable is reset at end of comment | |
| 372 } else // must be an error | |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
373 throwMTErr ("Invalid character (or sequence starting \"!\") outside of tag" ~ ErrInFile, new MTSyntaxException); |
| 0 | 374 } |
| 375 // if code execution reaches here, we're at EOF | |
| 376 // possible error: last character was ! (but don't bother checking since it's inconsequential) | |
| 377 return pos; | |
| 378 } | |
| 379 | |
| 380 /* Parses fbuf for a section marker. Already knows fbuf[pos] == '{'. | |
| 381 */ | |
| 382 private ID fbufReadSecMarker (inout uint pos) { | |
| 383 // at this point pos is whatever a parseSection run returned | |
| 384 // since we haven't hit EOF, fbuf[pos] MUST be '{' so no need to check | |
| 385 fbufIncrement(pos); | |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
386 |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
387 uint start = pos; |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
388 for (; pos < fbuf.length; ++pos) |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
389 if (fbuf[pos] == '}' || fbuf[pos] == '{') break; |
|
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
390 |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
391 if (fbuf[pos] != '}') throwMTErr ("Bad section tag format: not {id}" ~ ErrInFile, new MTSyntaxException); |
|
8
f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
7
diff
changeset
|
392 ID id = cast(ID) fbuf[start..pos]; |
| 0 | 393 fbufIncrement(pos); |
| 394 return id; | |
| 395 } | |
| 396 | |
| 397 /* Increments pos and checks it hasn't hit fbuf.length . */ | |
| 398 private void fbufIncrement(inout uint pos) { | |
| 399 ++pos; | |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
400 if (pos >= fbuf.length) throwMTErr("Unexpected EOF" ~ ErrInFile, new MTSyntaxException); |
| 0 | 401 } |
| 402 | |
|
11
b940f267419e
Options class created & changes to mergetag exception messages.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
10
diff
changeset
|
403 private void throwMTErr (char[] msg, MTException exc = new MTException) { |
| 0 | 404 fatal = true; // if anyone catches the error and tries to do anything --- we're dead now |
| 405 logger.error (msg); // report the error | |
| 406 throw exc; // and signal our error | |
| 407 } | |
| 408 //END METHODS: PRIVATE | |
| 409 | |
|
10
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
410 /+ A unittest here is really not practical since a file must be read from. |
|
4c3575400769
DefaultData largely rewritten with unittest, SDL input event handling completed with unittest, changes to Init threading.
Diggory Hardy <diggory.hardy@gmail.com>
parents:
9
diff
changeset
|
411 + A unittest is included in defaultdata.d . |
| 0 | 412 unittest {} |
| 413 +/ | |
| 414 } |