projects/mde: mde/text/parse.d annotate

annotate mde/text/parse.d @ 4:9a990644948c

Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. committer: Diggory Hardy <diggory.hardy@gmail.com>

author	Diggory Hardy <diggory.hardy@gmail.com>
date	Sun, 06 Jan 2008 17:38:51 +0000
parents	485c98ecbd91
children	dcb24afa0dce

rev	line source
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	1 /**************************************************************************************************
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	2 * This contains templates for converting a char[] to various data-types.
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	3 *
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	4 * Authors: Diggory Hardy, diggory.hardy@gmail.com
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	5 * Copyright: Copyright © 2007 Diggory Hardy.
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	6 * License: Licensed under the Academic Free License version 3.0
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	7 *
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	8 * This module basically implements the following templated function for $(B most) basic D types:
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	9 * bool, byte, short, int, long, ubyte, ushort, uint, ulong, float, double, real, char.
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	10 * It also supports arrays and associative arrays of any supported type (including of other arrays)
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	11 * and has special handling for strings (char[]) and binary (ubyte[]) data-types.
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	12 * -----------------------------
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	13 * T parse(T) (char[] source);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	14 * -----------------------------
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	15 *
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	16 * The syntax is mostly the same used by D without any prefixes/suffixes (except 0x, 0b & 0o base
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	17 * specifiers). The following escape sequences are supported for strings and characters: \' \" \\
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	18 * \a \b \f \n \r \t \v . Associative array literals use the same syntax as D, described here:
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	19 * $(LINK http://www.digitalmars.com/d/expression.html#AssocArrayLiteral).
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	20 *
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	21 * There are also a few utility functions defined; the public ones have their own documentation.
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	22 *
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	23 * On errors, a warning is logged and an TextParseException is thrown. No other exceptions should
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	24 * be thrown and none thrown from functions used outside this module.
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	25 *************************************************************************************************/
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	26 module mde.text.parse;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	27
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	28 // package imports
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	29 import mde.text.exception;
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	30 import mde.text.util : postTrim;
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	31
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	32 // tango imports
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	33 import cInt = tango.text.convert.Integer;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	34 import cFloat = tango.text.convert.Float;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	35 import Util = tango.text.Util;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	36 import tango.util.log.Log : Log, Logger;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	37
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	38 private Logger logger;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	39 static this () {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	40 logger = Log.getLogger ("mde.text.parse");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	41 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	42
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	43 //BEGIN parse templates
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	44 // Associative arrays
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	45 T[S] parse(T : T[S], S) (char[] src) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	46 src = Util.trim(src);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	47 if (src.length < 2 \|\| src[0] != '[' \|\| src[$-1] != ']')
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	48 throwException ("Invalid associative array: not [a:x, ..., c:z]");
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	49
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	50 T[S] ret;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	51 foreach (char[] pair; split (src[1..$-1])) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	52 uint i = 0;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	53 while (i < pair.length) { // advance to the ':'
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	54 char c = pair[i];
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	55 if (c == ':') break;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	56 if (c == '\'' \|\| c == '"') { // string or character
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	57 ++i;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	58 while (i < pair.length && pair[i] != c) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	59 if (pair[i] == '\\') ++i; // escape seq.
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	60 ++i;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	61 } // Doesn't throw if no terminal quote at end of pair (in this case an error is thrown anyway)
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	62 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	63 ++i;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	64 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	65 if (i == pair.length) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	66 debug logger.trace ("In pair: " ~ pair);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	67 throwException ("Invalid key:value pair in associative array literal");
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	68 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	69 debug logger.trace ("pair is: " ~ pair[0..i] ~ " : " ~ pair[i+1..$]);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	70 ret[parse!(S) (pair[0..i])] = parse!(T) (pair[i+1..$]);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	71 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	72 return ret;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	73 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	74 unittest {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	75 char[][char] X = parse!(char[][char]) (`['a':"animal", 'b':['b','u','s']]`);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	76 char[][char] Y = ['a':cast(char[])"animal", 'b':['b','u','s']];
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	77
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	78 //FIXME: when the compiler's fixed...
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	79 // just assert (X == Y)
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	80 assert (X.length == Y.length);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	81 assert (X.keys == Y.keys);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	82 assert (X.values == Y.values);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	83 //X.rehash; Y.rehash; // doesn't make a difference
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	84 //assert (X == Y); // fails
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	85 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	86
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	87 // Arrays
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	88 T[] parse(T : T[]) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	89 src = Util.trim(src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	90 if (src.length >= 2 && src[0] == '[' && src[$-1] == ']') return toArray!(T[]) (src);
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	91 throwException ("Invalid array: not [x, ..., z]");
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	92 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	93 T parse(T : char[]) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	94 src = Util.trim(src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	95 if (src.length >= 2 && src[0] == '"' && src[$-1] == '"') {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	96 src = src[1..$-1];
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	97 T ret;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	98 ret.length = src.length; // maximum length; retract to actual length later
3 485c98ecbd91 text.parse: fixed a small bug with char[]'s. Diggory Hardy <diggory.hardy@gmail.com> parents: 1 diff changeset	99 uint i = 0;
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	100 for (uint t = 0; t < src.length;) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	101 // process a block of non-escaped characters
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	102 uint s = t;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	103 while (t < src.length && src[t] != '\\') ++t; // non-escaped characters
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	104 uint j = i + t - s;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	105 ret[i..j] = src[s..t]; // copy a block
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	106 i = j;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	107
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	108 // process a block of escaped characters
3 485c98ecbd91 text.parse: fixed a small bug with char[]'s. Diggory Hardy <diggory.hardy@gmail.com> parents: 1 diff changeset	109 while (t < src.length && src[t] == '\\') {
485c98ecbd91 text.parse: fixed a small bug with char[]'s. Diggory Hardy <diggory.hardy@gmail.com> parents: 1 diff changeset	110 t++;
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	111 if (t == src.length) throwException (`Warning: string ends \" !`); // next char is "
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	112 ret[i++] = replaceEscapedChar (src[t++]); // throws if it's invalid
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	113 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	114 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	115 return ret[0..i];
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	116 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	117 else if (src.length >= 2 && src[0] == '[' && src[$-1] == ']') return toArray!(T) (src);
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	118 throwException ("Invalid string: not quoted (\"*\") or char array (['a',...,'c'])");
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	119 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	120 T parse(T : ubyte[]) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	121 src = Util.trim(src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	122 // Standard case:
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	123 if (src.length >= 2 && src[0] == '[' && src[$-1] == ']') return toArray!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	124 // Special case: sequence of hex digits, each pair of which is a ubyte
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	125 if (src.length % 2 == 1) throwException ("Invalid binary: odd number of chars");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	126 T ret;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	127 ret.length = src.length / 2; // exact
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	128 for (uint i, pos; pos + 1 < src.length; ++i) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	129 ubyte x = readHexChar(src, pos) << 4;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	130 x \|= readHexChar(src, pos);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	131 ret[i] = x;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	132 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	133 return ret;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	134 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	135 unittest {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	136 assert (parse!(double[]) (`[1.0,1.0e-10]`) == [1.0, 1.0e-10]); // generic array stuff
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	137 assert (parse!(double[]) (`[ ]`) == cast(double[]) []); // empty array
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	138
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	139 // char[] and char conversions, with commas, escape sequences and multichar UTF8 characters:
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	140 assert (parse!(char[][]) (`[ ".\"", [',','\''] ,"!\b€" ]`) == [ ".\"".dup, [',','\''] ,"!\b€" ]);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	141
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	142 assert (parse!(ubyte[]) (`01F2AC`) == cast(ubyte[]) [0x01, 0xF2, 0xAC]); // ubyte[] special notation
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	143 assert (parse!(ubyte[]) (`[01 ,0xF2, 0xAC]`) == cast(ubyte[]) [0x01, 0xF2, 0xAC]); // ubyte[] std notation
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	144 }
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	145
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	146 T parse(T : char) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	147 src = Util.trim(src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	148 if (src.length < 3 \|\| src[0] != '\'' \|\| src[$-1] != '\'')
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	149 throwException ("Invalid char: not quoted ('c')");
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	150 if (src[1] != '\\' && src.length == 3) return src[1]; // Either non escaped
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	151 if (src.length == 4) return replaceEscapedChar (src[2]); // Or escaped
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	152
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	153 // Report various errors; warnings for likely and difficult to tell cases:
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	154 /+ This was caused by a bug. Shouldn't occur now normally.
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	155 if (src[1] == '\\' && src.length == 3) throwException (`Warning: \' in char! There's currently no support for this during tokenising. Thus your input's probably been garbled!`); // next char is ' +/
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	156 // Warn in case it's a multibyte UTF-8 character:
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	157 if (src[1] & 0xC0u) throwException ("Invalid char: too long (non-ASCII UTF-8 characters cannot be read as a single character)");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	158 throwException ("Invalid char: too long");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	159 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	160 // unittest covered above
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	161
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	162 T parse(T : bool) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	163 src = Util.trim(src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	164 if (src == "true") return true;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	165 if (src == "false") return false;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	166 uint pos;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	167 while (src.length > pos && src[pos] == '0') ++pos; // strip leading zeros
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	168 if (src.length == pos && pos > 0) return false;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	169 if (src.length == pos + 1 && src[pos] == '1') return true;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	170 throwException ("Invalid bool: not true or false and doesn't evaluate to 0 or 1");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	171 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	172 unittest {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	173 assert (parse!(bool[]) (`[true,false,01,00]`) == cast(bool[]) [1,0,1,0]);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	174 }
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	175
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	176 T parse(T : byte) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	177 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	178 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	179 T parse(T : short) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	180 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	181 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	182 T parse(T : int) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	183 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	184 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	185 T parse(T : long) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	186 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	187 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	188 T parse(T : ubyte) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	189 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	190 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	191 T parse(T : ushort) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	192 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	193 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	194 T parse(T : uint) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	195 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	196 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	197 T parse(T : ulong) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	198 return toTInt!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	199 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	200 unittest {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	201 assert (parse!(byte) ("-5") == cast(byte) -5);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	202 // annoyingly, octal syntax differs from D (blame tango):
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	203 assert (parse!(uint[]) ("[0b0100,0o724,0xFa59c,0xFFFFFFFF,0]") == [0b0100u,0724,0xFa59c,0xFFFFFFFF,0]);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	204 }
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	205
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	206 T parse(T : float) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	207 return toTFloat!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	208 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	209 T parse(T : double) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	210 return toTFloat!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	211 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	212 T parse(T : real) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	213 return toTFloat!(T) (src);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	214 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	215 unittest {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	216 assert (parse!(float) ("0.0") == 0.0f);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	217 assert (parse!(double) ("-1e25") == -1e25);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	218 assert (parse!(real) ("5.24e-269") == cast(real) 5.24e-269);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	219 }
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	220 //END parse templates
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	221
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	222 //BEGIN Utility funcs
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	223 /** Templated read-int function to read (un)signed 1-4 byte integers.
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	224 *
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	225 * Actually a reimplementation of tango.text.convert.Integer toLong and parse functions.
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	226 */
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	227 TInt toTInt(TInt) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	228 const char[] INT_OUT_OF_RANGE = "Integer out of range";
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	229 bool sign;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	230 uint radix, ate, ate2;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	231
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	232 ate = cInt.trim (src, sign, radix);
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	233 if (ate == src.length) throwException ("Invalid integer: no digits");
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	234 ulong val = cInt.convert (src[ate..$], radix, &ate2);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	235 ate += ate2;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	236
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	237 while (ate < src.length) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	238 if (src[ate] == ' ' \|\| src[ate] == '\t') ++ate;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	239 else throwException ("Invalid integer");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	240 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	241
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	242 if (val > TInt.max) throwException (INT_OUT_OF_RANGE);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	243 if (sign) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	244 long sval = cast(long) -val;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	245 if (sval > TInt.min) return cast(TInt) sval;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	246 else throwException (INT_OUT_OF_RANGE);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	247 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	248 return cast(TInt) val;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	249 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	250
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	251 /** Basically a reimplementation of tango.text.convert.Float.toFloat which checks for trailing
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	252 * whitespace before throwing an exception for overlong input and throws my exception class
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	253 * when it does. */
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	254 TFloat toTFloat(TFloat) (char[] src) {
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	255 src = postTrim (src);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	256 if (src == "") throwException ("Invalid float: no digits");
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	257 uint ate;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	258
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	259 TFloat x = cFloat.parse (src, &ate);
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	260 return x;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	261 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	262
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	263 /** Splits a string into substrings separated by '$(B ,)' with support for characters and strings
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	264 * containing escape sequences and for embedded arrays ($(B [...])).
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	265 *
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	266 * Empty strings may get returned. */
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	267 char[][] split (char[] src) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	268 src = Util.trim (src);
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	269 if (src == "") return []; // empty array: no elements when no data
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	270
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	271 uint depth = 0; // surface depth (embedded arrays)
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	272 char[][] ret;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	273 ret.length = src.length / 3; // unlikely to need a longer array
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	274 uint k = 0; // current split piece
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	275 uint i = 0, j = 0; // current read location, start of current piece
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	276
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	277 while (i < src.length) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	278 char c = src[i];
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	279 if (c == '\'' \|\| c == '"') { // string or character
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	280 ++i;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	281 while (i < src.length && src[i] != c) {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	282 if (src[i] == '\\') ++i; // escape seq.
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	283 ++i;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	284 } // Doesn't throw if no terminal quote at end of src, but this should be caught later.
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	285 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	286 else if (c == '[') ++depth;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	287 else if (c == ']') {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	288 if (depth) --depth;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	289 else throwException ("Invalid array literal: closes before end of data item.");
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	290 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	291 else if (c == ',' && depth == 0) { // only if not an embedded array
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	292 if (ret.length <= k) ret.length = ret.length * 2;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	293 ret[k++] = src[j..i]; // add this piece and increment k
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	294 j = i + 1;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	295 }
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	296 ++i;
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	297 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	298 if (ret.length <= k) ret.length = k + 1;
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	299 ret[k] = src[j..i]; // add final piece (i >= j)
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	300 return ret[0..k+1];
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	301 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	302
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	303 /* Throws an exception on invalid escape sequences. Supported escape sequences are the following
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	304 * subset of those supported by D: \" \' \\ \a \b \f \n \r \t \v
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	305 */
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	306 private char replaceEscapedChar (char c)
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	307 {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	308 static char[char] escChars;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	309 static bool escCharsFilled; // will be initialised false
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	310
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	311 if (!escCharsFilled) {
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	312 // map of all supported escape sequences (cannot be static?)
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	313 escChars = ['"' : '"', '\'' : '\'',
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	314 '\\' : '\\', 'a' : '\a',
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	315 'b' : '\b', 'f' : '\f',
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	316 'n' : '\n', 'r' : '\r',
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	317 't' : '\t', 'v' : '\v'];
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	318 escCharsFilled = true;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	319 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	320
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	321 char* r = c in escChars;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	322 if (r != null) return *r;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	323
3 485c98ecbd91 text.parse: fixed a small bug with char[]'s. Diggory Hardy <diggory.hardy@gmail.com> parents: 1 diff changeset	324 throwException ("Invalid escape sequence: \\"~c); // we didn't return, so something failed
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	325 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	326
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	327 // Reads one hex char: [0-9A-Fa-f]. Otherwise throws an exception. Doesn't check src.length.
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	328 private ubyte readHexChar (char[] src, inout uint pos) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	329 ubyte x;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	330 if (src[pos] >= '0' && src[pos] <= '9') x = src[pos] - '0';
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	331 else if (src[pos] >= 'A' && src[pos] <= 'F') x = src[pos] - 'A' + 10;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	332 else if (src[pos] >= 'a' && src[pos] <= 'f') x = src[pos] - 'a' + 10;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	333 else throwException ("Invalid hex digit.");
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	334 ++pos;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	335 return x;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	336 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	337
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	338 // Generic array reader
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	339 // Assumes input is of form "[xxxxx]" (i.e. first and last chars are '[', ']' and length >= 2).
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	340 private T[] toArray(T : T[]) (char[] src) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	341 T[] ret = new T[16]; // avoid unnecessary allocations
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	342 uint i = 0;
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	343 foreach (char[] element; split(src[1..$-1])) {
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	344 if (i == ret.length) ret.length = ret.length * 2;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	345 ret[i] = parse!(T) (element);
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	346 ++i;
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	347 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	348 return ret[0..i];
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	349 }
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	350
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	351 private void throwException (char[] msg) {
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	352 logger.warn (msg); // only small errors are trapped here
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	353 throw new TextParseException ();
18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	354 }
4 9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	355
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	356 unittest {
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	357 // all utility functions should be well-enough used not to need testing
9a990644948c Many changes: upgraded to tango 0.99.4, reorganised mde/input, large changes to mde/mergetag and mde/init, separated off test/MTTest.d and more. Diggory Hardy <diggory.hardy@gmail.com> parents: 3 diff changeset	358 }
1 18491334a525 Finished format.d and parse.d modules; moved to mde/text. Partway implementing mde.mergetag.write.TextWriter. Diggory Hardy <diggory.hardy@gmail.com> parents: diff changeset	359 //END Utility funcs

Mercurial > projects > mde

annotate mde/text/parse.d @ 4:9a990644948c