Mercurial > projects > mde
changeset 8:f63f4f41a2dc
Big changes to init; got some way towards input event support; changed mergetag ID to char[] from uint.
committer: Diggory Hardy <diggory.hardy@gmail.com>
| author | Diggory Hardy <diggory.hardy@gmail.com> |
|---|---|
| date | Fri, 25 Jan 2008 18:17:38 +0000 |
| parents | b544c3a7c9ca |
| children | 1885a9080f2a |
| files | conf/input.mtt doc/policies.txt dsss.conf mde/events.d mde/exception.d mde/init.d mde/input/config.d mde/input/core.d.old mde/input/eventstream.d.old mde/input/exception.d mde/input/input.d mde/mde.d mde/mergetag/dataset.d mde/mergetag/doc/file-format-text.txt mde/mergetag/doc/issues.txt mde/mergetag/exception.d mde/mergetag/old-code/dataset.d.old mde/mergetag/old-code/typeSpec.d.old mde/mergetag/read.d mde/mergetag/write.d mde/test.d mde/text/format.d mde/text/parse.d |
| diffstat | 23 files changed, 267 insertions(+), 881 deletions(-) [+] |
line wrap: on
line diff
--- a/conf/input.mtt Wed Jan 16 12:48:07 2008 +0000 +++ b/conf/input.mtt Fri Jan 25 18:17:38 2008 +0000 @@ -1,4 +1,4 @@ {MT01} -!<uint[]|0=[1]> -{0} -<uint[][uint]|0=[0x20000000 : [0x100, 3] ]> +!<string[]|Configs=["Std"]> +{Default} +<uint[][uint]|B=[0x20000000 : [0x100, 3] ]>
--- a/doc/policies.txt Wed Jan 16 12:48:07 2008 +0000 +++ b/doc/policies.txt Fri Jan 25 18:17:38 2008 +0000 @@ -5,25 +5,42 @@ Coding conventions: Mostly stick to those provided in the D specification. Generally indent with four spaces and use tabs to align comments. Aim to break long lines at around 100 chars (particularly with documentation); this isn't essential but provides a good guide and keeps text looking reasonable. With code, however, breaking lines doesn't always produce better-looking code. -Spelling: Use british or american (or other) spellings as you like, but BE CONSISTANT at least for code symbols within packages. For text output to the user there need be no convention until internationalisation support is built-in. As far as internationalisation/localisation is concerned, does it make sense to translate log messages or not? (They are going to be seen by end-users, but will largely be used by developers.) +Identifiers: as for the D spec, use descriptive words for identifiers, although try to keep them from being overlong. Use capital letters to show separate words, not _s. E.g.: file, readFile; not: rdFile, read_file, readFileUsingMyMethodNow. Again, this is only really a guideline. + +Module/file names: unless you have a good reason, keep names all lower-case. And if you're programming on windows, make sure you always use the correct capitalisation (yeah, of course...). + +Spelling: Use british or american (or other) spellings as you like, but BE CONSISTANT at least for code symbols within packages (i.e. if you write your own package you choose the spelling, and for comments it doesn't matter, unless it's actually refering to a symbol). For text output to the user there need be no convention until internationalisation support is built-in. As far as internationalisation/localisation is concerned, does it make sense to translate log messages or not? (They are going to be seen by end-users, but will largely be used by developers.) Package design principle: use a separate package for each module of the engine. In most packages where there is only one module (file) imported by other parts of the engine, that module should have the same name as the package and be designed to have a standardised interface to the package so that the package could be replaced with another as a drop-in replacement (written with the same interface). Of course in many cases it may not be possible to swich one package for another quite this easily, but holding to this principle should at least minimise the amount of work necessary when doing so. +Module imports: modules should publically import dependancy modules likely to be needed by dependant modules, in particular the package-level exception module (where used & use by dependants is expected). + + Engine-wide initialisation and cleanup should be handled or invoked by mde.init.Init's CTOR and DTOR methods where this is viable. Logging should be handled by tango's Logger class. A logger with the name mde.package.module or mde.package.module.X where X is a symbol within the module should be used for each module. Thrown errors should, where documented, be documented with a log message; an exception message may be used to produce the final log message but must be output via a log message. In general the levels should be used as follows: - Trace Where required or thought highly useful for debugging - Info Sparingly, for informational purposes (e.g. when parsing a file). Should not generally be used repetitively (within loops, etc.). - Warn For small errors which can be overlooked, even if they MAY cause bigger problems later. + Trace Where required or thought highly useful for debugging, and only compiled in debugging mode. + Info Sparingly, for informational purposes (e.g. when parsing a file). Should not generally be used repetitively (within loops, etc.). Not for reporting unexpected behaviour. + Warn For small errors which can be overlooked, even if they MAY cause bigger problems later. I.e. something unexpected, but not necessarily a major problem, happens. Error For errors which directly: • cut-short a (reasonably large) operation (e.g. reading a file). • cause a significant change in program operation, but do not directly cause the program to terminate. Fatal For errors directly (i.e. definately and almost immediately) ending the program. +For all levels bar trace, messages should if possible be understandable to end users, while (for warn and above) including enough information to fix the problem, including code symbols if necessary. +Thus: + Trace output should only be available when compiled in debug mode. + When run by an end-user (with info-level logging enabled), + • info messages normally occur and should be understandable to end users; + • warn messages may occur, and may or may not indicate problems; + • error messages indicate that something big is wrong, and if the program still runs it is unlikely to be usable as intended; + • fatal messages indicate a problem preventing the program from running. + -Thrown errors should use an exception class specific to at least the package involved to enable specific catching of errors. Exception classes should be defined within a module exception.d in the package directory. Exception classes should all contain a this() CTOR and possibly a this(char[] msg) CTOR. + +Thrown errors should use an exception class specific to at least the package involved to enable specific catching of errors. Exception classes should be defined within a module exception.d in the package directory. Exception classes should generally follow the conventions within mde/exception.d to aid in providing reasonable error messages.
--- a/dsss.conf Wed Jan 16 12:48:07 2008 +0000 +++ b/dsss.conf Fri Jan 25 18:17:38 2008 +0000 @@ -3,6 +3,6 @@ target=bin/mde #[mde/mergetag] #[mde/input] -[test/MTTest.d] -target=bin/MTTest +#[test/MTTest.d] +#target=bin/MTTest
--- a/mde/events.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/events.d Fri Jan 25 18:17:38 2008 +0000 @@ -7,9 +7,16 @@ import derelict.sdl.events; +import tango.util.log.Log : Log, Logger; + +private Logger logger; +static this() { + logger = Log.getLogger ("mde.events"); +} + static bool run = true; -static this () { +void addEventsSchedule () { Scheduler.perFrame (&pollEvents); } @@ -18,10 +25,11 @@ while (SDL_PollEvent (&event)) { switch (event.type) { case SDL_QUIT: + logger.info ("Quit requested"); run = false; - break; + break; default: - Input.instance() (event); + input (event); } } }
--- a/mde/exception.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/exception.d Fri Jan 25 18:17:38 2008 +0000 @@ -25,9 +25,3 @@ super(msg); } } - -class DynamicLibraryLoadException : initException { - this (char[] msg) { - super("when loading dynamic library: " ~ msg); - } -}
--- a/mde/init.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/init.d Fri Jan 25 18:17:38 2008 +0000 @@ -5,14 +5,17 @@ *************************************************************************************************/ module mde.init; -import mde.exception; +public import mde.exception; +import mde.events; import mde.input.input; +import mde.input.joystick; // tango imports import tango.core.Thread; import tango.util.log.Log : Log, Logger; import tango.util.log.ConsoleAppender : ConsoleAppender; +import tango.stdc.stringz : fromUtf8z; import derelict.sdl.sdl; import derelict.util.exception; @@ -42,7 +45,7 @@ */ scope class Init { - static Logger logger; + private static Logger logger; static this() { logger = Log.getLogger ("mde.init.Init"); } @@ -51,8 +54,7 @@ * * Runs general initialisation code, in a threaded manner where this isn't difficult. * - * If this fails by throwing an exception, it must run necessary cleanup first since the DTOR - * cannot be run. */ + * If any init fails, it must run necessary cleanup first since the DTOR cannot be run. */ /* In a single-threaded function this could be done with: * scope(failure) cleanup; * This won't work with a threaded init function since any threads completing succesfully will @@ -62,46 +64,100 @@ */ this() { + /* Initialisation functions. + * + * These should each handle a separate area of initialisation such that these functions could + * be run simultaneously in separate threads. */ + + bool initFailure = false; // Set true if something goes wrong and we need to abort. + void setFailure () { // For synchronization, although may be unnecessary + synchronized initFailure = true; + } + void delegate() [] initFuncs = [ + delegate void() { + // Inits SDL and related stuff (joystick). + try { + // SDL Joystick, used by mde.input + DerelictSDL.load(); + } catch (DerelictException de) { + logger.fatal ("Loading dynamic library failed:"); + logger.fatal (de.msg); + + setFailure (); // abort + return; + } + logger.info ("Derelict: loaded SDL"); + + if (SDL_Init (SDL_INIT_VIDEO | SDL_INIT_JOYSTICK)) { + logger.fatal ("SDL initialisation failed:"); + char* msg = SDL_GetError (); + logger.fatal (msg ? fromUtf8z(msg) : "no reason available"); + + setFailure (); // abort + return; + } + + SDL_SetVideoMode (800, 600, 32, 0); + + addCleanupFct (&cleanupSDL); + logger.info ("SDL initialised"); + + openJoysticks (); // after SDL init + addCleanupFct (&closeJoysticks); + } + ]; + // Start all threads - ThreadGroup tg = new ThreadGroup; - tg.create(&initSDL); + ThreadGroup tg = new ThreadGroup; // can't fail since it does nothing (tango 0.99.4) (unless out of memory − anyway should be safe to throw at this point) + try { // creating/starting threads can fail + foreach (func; initFuncs) tg.create(func); + } catch (Exception e) { // should be a ThreadException, but catch all Exceptions + logger.warn ("Caught exception while trying to create threads:"); + logger.warn (e.msg); + logger.warn ("Will continue in a non-threaded manner."); + + foreach (func; initFuncs) func(); + } // Do some initialisation in the main thread - Input.instance.loadConfig (0); + input.loadConfig (); // (may also create instance) + addEventsSchedule (); - // Wait for all threads to complete - try { - tg.joinAll (true); // rethrows any exceptions + // Wait for all threads to complete. + // If something went wrong, we still need to do this before cleaning up. + foreach (t; tg) { + try { + t.join (true); + /+ Will only catch thread exceptions; but even so something still went badly wrong so we want the same functionality. + } catch (ThreadException e) { // Any threading exception + +/ + } catch (Exception e) { // Any other exception, i.e. caught from thread. + // Relying on catching exceptions thrown by other threads is a bad idea. + // Hence all threads should catch their own exceptions and return safely. + + logger.fatal ("Exception caught during init:"); + logger.fatal (e.msg); + logger.fatal ("Please report this; it should NEVER happen."); + + setFailure (); // abort (but join other threads first) + } } - catch (initException e) { // Any problems? + + if (initFailure) { // All cleanup-on-failure must be done here. runCleanupFcts(); - throw e; // Rethrow. Warning: if multiple threads throw exceptions, only one gets returned. + + // Throw an exception to signal failure and prevent DTOR from also running. + throw new initException ("Initialisation failed due to above exceptions."); } - } - /* Initialisation functions. + /** DTOR - runs cleanup. * - * These should each handle a separate area of initialisation such that these functions could - * be run simultaneously in separate threads. */ - void initSDL () { - try { - // SDL Joystick, used by mde.input - DerelictSDL.load(); - } catch (DerelictException de) { - throw new DynamicLibraryLoadException (de.msg); - } - logger.info ("Derelict: loaded SDL"); - - SDL_Init (SDL_INIT_TIMER | SDL_INIT_JOYSTICK); - addCleanupFct (&cleanupSDL); - logger.info ("SDL initialised"); - } - + * Currently unthreaded; probably might as well stay that way. */ ~this() { - runCleanupFcts(); + runCleanupFcts(); // if threading, note not all functions can be called simultaeneously } /* Cleanup Functions.
--- a/mde/input/config.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/input/config.d Fri Jan 25 18:17:38 2008 +0000 @@ -3,15 +3,15 @@ debug import mde.text.format; -import mde.input.exception; +public import mde.input.exception; -import mde.mergetag.read; +import MT = mde.mergetag.read; import mde.text.parse; import tango.util.log.Log : Log, Logger; import tango.util.collection.TreeBag : TreeBag; -Logger logger; +private Logger logger; static this() { logger = Log.getLogger ("mde.input.config"); } @@ -21,7 +21,7 @@ * * Class extends DataSection so that it can be loaded by mergetag easily. */ -class Config : DataSection +class Config : MT.DataSection { alias uint[] outQueue; // This is the type for the out queue config data. /** Button event type bit-codes @@ -86,8 +86,7 @@ char[] name; /// Name for user to save this under. uint[] inheritants; /// Other profiles to inherit. - // FIXME: using uint IDs really isn't nice... - static Config[uint] configs; /// All configs loaded by load(). + static Config[char[]] configs; /// All configs loaded by load(). private static TreeBag!(char[]) loadedFiles; // all filenames load tried to read //BEGIN File loading/saving code @@ -99,41 +98,54 @@ static void load (char[] filename) { if (loadedFiles.contains (filename)) return; // forget it; already done that loadedFiles.add (filename); - Reader file; + MT.Reader file; try { - file = new Reader(filename, null, true); // open and read header + file = new MT.Reader(filename, null, true); // open and read header // TODO: also load user-config file file.dataSecCreator = - function DataSection (ID) { return new Config; }; + function MT.DataSection (MT.ID) { return new Config; }; - enum : ID { CONFIGS } - ID[] configs; // active config sections (may not exist) - uint[]* configs_p = CONFIGS in file.dataset.header._uintA; + // D2.0: enum MT.ID CONFIGS = "Configs"; + const MT.ID CONFIGS = cast(MT.ID)"Configs"; + MT.ID[] configs; // active config sections (may not exist) + MT.ID[]* configs_p = cast(MT.ID[]*) (CONFIGS in file.dataset.header._stringA); - if (configs_p) file.read(cast(ID[]) *configs_p); // restrict to this set IF a restriction was given + if (configs_p) file.read(*configs_p); // restrict to this set IF a restriction was given else file.read(); // otherwise read all } - catch (MTException) { + catch (MT.MTException) { logger.error ("Unable to load configs from: " ~ filename); throw new ConfigLoadException; } - // FIXME: don't override configs if not empty - configs = cast (Config[uint]) file.dataset.sec; - // NOTE: this is in some ways dangerous (assuming all DataSections are Configs), but they should be. + + // NOTE: It is in some ways a bad idea assuming all DataSections are Configs, but they should be. + if (!configs) { // these are the first Configs loaded + configs = cast (Config[char[]]) file.dataset.sec; + } else { // add to existing Configs, replacing ones with same ID + foreach (i, sec; file.dataset.sec) { + Config c = cast(Config) sec; + if (c) configs[i] = c; // Check, because we don't want null entries in configs + else debug logger.warn ("Ended up with DataSection of wrong type; this should never happen."); + } + } + debug { char tmp[128] = void; - logger.info (logger.format (tmp, "Loaded {} config sections.", configs.length)); + logger.trace (logger.format (tmp, "Loaded {} config sections.", configs.length)); foreach (id, cfg; configs) { logger.trace ("Section "~format!(uint)(id)~": " ~ format!(uint[][uint])(cfg.button) ~ " (" ~ format!(uint)(cfg.dnbc) ~ ")"); } } } - private enum QUEUE : ID { BUTTON, AXIS, MOUSE } + // D2.0: private enum QUEUE : MT.ID { BUTTON = "B", AXIS = "A", MOUSE = "M" } + private struct QUEUE { + static const MT.ID BUTTON = cast(MT.ID)"B", AXIS = cast(MT.ID)"A", MOUSE = cast(MT.ID)"M"; + } private this() {} // Private since this class should only be created from here. - void addTag (char[] tp, ID id, char[] dt) { + void addTag (char[] tp, MT.ID id, char[] dt) { if (tp == "uint[][uint]") { if (id == QUEUE.BUTTON) { button = cast(outQueue[uint]) parse!(uint[][uint]) (dt); @@ -142,12 +154,9 @@ } else if (id == QUEUE.AXIS) axis = cast(outQueue[uint]) parse!(uint[][uint]) (dt); else if (id == QUEUE.MOUSE) mouse = cast(outQueue[uint]) parse!(uint[][uint]) (dt); - else { - char[80] tmp; - logger.info (logger.format(tmp, "Unexpected tag encountered with ID {}", id)); - } + else logger.warn ("Unexpected tag encountered with ID " ~ cast(char[])id); } // FIXME: add support for name and inheritants. - else throw new MTUnknownTypeException ("Input Config: only uint[][uint] type supported"); + else throw new MT.MTUnknownTypeException ("Input Config: only uint[][uint] type supported"); } void writeAll (ItemDelg) { // FIXME
--- a/mde/input/core.d.old Wed Jan 16 12:48:07 2008 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,36 +0,0 @@ -/// This module contains the core (i.e. common part) of the input system. -module mde.input.core; - -import mde.input.config; - -typedef uint inputID; -struct RelPair { // for mouse/joystick ball motion - real x, y; - static RelPair opCall (real a, real b) { - RelPair ret; - ret.x = a; ret.y = b; - return ret; - } -} - -/* Note: We really want an array, not a stack. We cannot edit these lists, so we can either -* copy the stack or just iterate through it as an array. -*/ -typedef uint[] outQueue; /// This is the type for the out queue config data. -struct readOutQueue { /// A convenient structure for reading an outQueue item by item. - private outQueue _q; // the queue, stored by reference to the original - private uint p = 0; // current read position (start at beginning) - - static readOutQueue opCall (outQueue q) { /// Static constructor - readOutQueue ret; - ret._q = q; - return ret; - } - uint next () { /// Get the next element. Throws an exception if there isn't another. - if (p >= _q.length) - throw new InputException ("Input: Invalid configuration: incomplete config stack"); - uint ret = _q[p]; - ++p; - return ret; - } -}
--- a/mde/input/eventstream.d.old Wed Jan 16 12:48:07 2008 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,90 +0,0 @@ -/** This module contains functions called on an event, which may modify the event (adjuster - * functions), and finally output to one (or more) of the state tables (the event stream). - * - * Adjuster and other event functions should have a format to fit the ES_X_Func types, for X is B - * (button event), A (axis event) or M (mouse relative motion event or joystick ball event). - * Adjusters should call one of the xEventOut() functions with their output and the remainder of - * the readOutQueue. - * - * To control which adjusters get called and pass parameters, a stack is used. - */ -module mde.input.eventstream; - -// package imports -import mde.input.core; - -/// Module constructor fills es_*_fcts tables and rehashes them. -static this () { - es_b_fcts = [ ES_B_OUT : &es_b_out ]; -} - -/// These aliases are for pointers to the event functions. -alias void function (bool, readOutQueue) ES_B_Func; -alias void function (short, readOutQueue) ES_A_Func; /// ditto -alias void function (short, short, readOutQueue) ES_M_Func; /// ditto - -/// These are the tables for looking up which event function to call. -static ES_B_Func[uint] es_b_fcts; -static ES_A_Func[uint] es_a_fcts; /// ditto -static ES_M_Func[uint] es_m_fcts; /// ditto - -/// These are the codes allowing the config to specify event functions: -enum : uint { - ES_B_OUT = 0x0000_0100u, - ES_A_OUT = 0x0000_0200u, - ES_M_OUT = 0x0000_0300u, -} - -/** These functions pass an event to the appropriate event function (adjuster or output func). */ -void bEventOut (bool b, readOutQueue s) -{ - ES_B_Func* func = (s.next() in es_b_fcts); - if (func != null) (*func)(b,s); - else throw new InputException ("Input: Invalid configuration: bad event function code"); -} -void aEventOut (short x, readOutQueue s) /// ditto -{ - ES_A_Func* func = (s.next() in es_a_fcts); - if (func != null) (*func)(x,s); - else throw new InputException ("Input: Invalid configuration: bad event function code"); -} -void mEventOut (short x, short y, readOutQueue s) /// ditto -{ - ES_M_Func* func = (s.next() in es_m_fcts); - if (func != null) (*func)(x,y,s); - else throw new InputException ("Input: Invalid configuration: bad event function code"); -} - -/// Simple output function -void es_b_out (bool b, readOutQueue s) { - current.button[cast(inputID) s.next()] = b; -} -/// Adjuster to check modifier keys -void es_b_modifier (bool b, readOutQueue s); - -/** Simple output function - -Adds 1-2 items on the stack. -*/ -void es_a_out (short x, readOutQueue s) { - real y = x; - uint conf = s.next(); - enum : uint { - HALF_RANGE = 0x8000_0000u, - SENSITIVITY = 0x0080_0000u, - } - // Convert ranges into standard intervals (with or without reverse values) - if (conf & HALF_RANGE) y = (y + 32767.0) * 1.5259254737998596e-05; // range 0.0 - 1.0 - else y *= 3.0518509475997192e-05; // range -1.0 - 1.0 - real a; - if (conf & SENSITIVITY) a = s.next(); - /+ When a global sensitivity is available (possibly only use if it's enabled)... - else a = axis.sensitivity; - y = sign(y) * pow(abs(y), a); // sensitivity adjustment by a +/ - current.axis[cast(inputID) s.next()] = y; -} - -/// Simple output function -void es_m_out (short x, short y, readOutQueue s) { - current.axis_rel[cast(inputID) s.next()] = RelPair(x,y); -}
--- a/mde/input/exception.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/input/exception.d Fri Jan 25 18:17:38 2008 +0000 @@ -1,6 +1,6 @@ module mde.input.exception; -import mde.exception; +public import mde.exception; /// Base Input exception class. class inputException : mdeException {
--- a/mde/input/input.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/input/input.d Fri Jan 25 18:17:38 2008 +0000 @@ -6,11 +6,23 @@ // package imports import mde.input.config; -import mde.input.exception; +public import mde.input.exception; // sdl imports import derelict.sdl.events; +import tango.util.log.Log : Log, Logger; + +Input input; /// Global instance of input. +private Logger logger; + +static this() { + input = new Input; + + logger = Log.getLogger ("mde.input.config"); +} + +/// Class encapsulating all input functionality. class Input { /// Typedef for all indexes (type is uint). @@ -18,14 +30,6 @@ alias void delegate(inputID, bool) ButtonCallback; alias void delegate(inputID, real) AxisCallback; alias void delegate(inputID, real,real) MouseCallback; - - /// Stores a static instance of Input for most usage. - static Input instance () { - static Input instance; - - if (instance is null) instance = new Input; - return instance; - } /** Get key status at this ID. * @@ -103,9 +107,13 @@ switch (event.type) { case SDL_JOYBUTTONDOWN: case SDL_JOYBUTTONUP: + debug { + char tmp[128] = void; + logger.trace (logger.format (tmp, "Got a joystick button event: ({}, {}) - {}", event.jbutton.which, event.jbutton.button, event.jbutton.state)); + } outQueue* p = (Config.B.JOYBUTTON | (event.jbutton.which << 12) | event.jbutton.button) in config.button; if (p) bEventOut (event.jbutton.state == 0x1, readOutQueue(*p)); - break; + break; /+ case SDL_KEYDOWN: @@ -120,14 +128,15 @@ if (p) eventstream.mEventOut (event.motion.xrel, event.motion.yrel, readOutQueue(*p)); +/ default: - return false; + return false; } return true; } /** Resets relative movement of mice / joystick balls to zero. * - * Should probably be called once-per-frame if these are used. + * Should be called once-per-frame if these are used, but must be called after their state has + * been read (e.g. just before updating the input). */ void frameReset () { foreach (rp; axis_rel) { @@ -139,14 +148,15 @@ * * Returns: true if the requested config id wasn't found. */ - bool loadConfig (uint id) { + bool loadConfig () { Config.load("conf/input.mtt"); // FIXME: filename - Config* c_p = id in Config.configs; + Config* c_p = "Default" in Config.configs; if (c_p) { config = *c_p; return false; - logger.info ("Succesfully loaded config."); + debug logger.trace ("Succesfully loaded config."); } + debug logger.warn ("Config \"Default\" not found."); return true; }
--- a/mde/mde.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mde.d Fri Jan 25 18:17:38 2008 +0000 @@ -12,8 +12,6 @@ import mde.input.input; -import mde.mergetag.read; - // External library imports import tango.core.Thread; import tango.io.Stdout; @@ -36,14 +34,13 @@ return 1; } - Input input = Input.instance(); input.addButtonCallback (cast(Input.inputID) 3u, delegate void(Input.inputID i, bool b) { Stdout ("Event: ")(i)(" changed to: ")(b).newline; } ); bool oldb = false; - /+while (run)+/ - for (ulong t = 0; t < 100; ++t) { + while (run) + /+for (ulong t = 0; t < 100; ++t)+/ { Scheduler.run (Clock.now()); bool b = input.getButton (cast(Input.inputID) 3u);
--- a/mde/mergetag/dataset.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mergetag/dataset.d Fri Jan 25 18:17:38 2008 +0000 @@ -1,20 +1,14 @@ -/// This module contains a minimal definition of a MergeTag DataSet. +/// This module contains the mergetag DataSet class together with an interface for DataSections. module mde.mergetag.dataset; // package imports -import mde.mergetag.exception; +public import mde.mergetag.exception; +public import mde.mergetag.defaultdata; // used in DataSet so it should be publically imported -// other mde imports -import mde.text.util; -import mde.text.parse : parse; -import mde.text.format : format; - -// tango imports -import Util = tango.text.Util; -import tango.util.log.Log : Log, Logger; +//import tango.util.log.Log : Log, Logger; /** Typedef for data & section indexes (can be changed to ulong if necessary.) */ -typedef uint ID; +typedef char[] ID; package struct MTFormatVersion { enum VERS : ubyte { // convenient list of all known file format versions @@ -34,26 +28,17 @@ } } -private Logger logger; +/+private Logger logger; static this () { logger = Log.getLogger ("mde.mergetag.dataset"); -} - -struct TextTag { - TextTag opCall (char[] _tp, ID _id, char[] _dt) { - TextTag ret; - ret.tp = _tp; - ret.id = _id; - ret.dt = _dt; - return ret; - } - char[] tp, dt; - ID id; -} +}+/ /************************************************************************************************** * Data class; contains a DataSection class instance for each loaded section of a file. * + * Stored data is available for direct access via header and sec; all functions are just helper + * functions. + * * Any class implementing DataSection may be used to store data; by default a DefaultData class is * used when reading a file. Another class may be used by creating the sections before reading the * file or passing the reader a function to create the sections (see Reader.dataSecCreator). @@ -62,14 +47,9 @@ */ class DataSet { - DefaultData header; /// Header + DefaultData header; /// Header section. DataSection[ID] sec; /// Dynamic array of sections - /// Return a reference to the indexed item - DataSection opIndex(ID i) { - return sec[i]; - } - /// Template to return all sections of a child-class type. T[ID] getSections (T : DataSection) () { T[ID] ret; @@ -113,183 +93,6 @@ debug void debugFunc (); /// Run in debug builds after parseSection. } -/** - * Default DataSection class. - * - * Supports all the basic types currently supported and array versions of - * each (except no arrays of binary or string types; these are already arrays). - * Doesn't support custom types, but inheriting classes may add support. - */ -/* Note: I wrote this comment when the code looked rather worse. It's still partially applicable though. - * - * Due to a failure to use generic programming techniques for most of this (maybe because it's not - * possible or maybe just because I don't know how to use templates properly) a lot of this code is - * really horrible and has to refer to EVERY data member. - * Be really careful if you add any items to this class. - * - * I really don't like having to do things like this, but it provides a lot of benefits such as no - * need to store types and no need to check an argument's type for every access (this could be done - * by throwing errors, but then an incorrect (perhaps hand-edited) data file could cause a lot of - * errors to be thrown). - */ -class DefaultData : DataSection -{ - //BEGIN DATA - /** Data Members - * - * These names are available for direct access. - * - * An alternative access method is to use the provided templates: - * -------------------- - * template Arg(T) { - * alias Name Arg; - * } - * -------------------- - * - * Use with a mixin or directly: - * -------------------- - * mixin Arg!(type); - * auto x = Arg; - * - * type y = Arg!(type).Arg; - * -------------------- - * Note: trying to use Arg!(type) to implicitly refer to Arg!(type).Arg causes compiler errors due to - * --- alias Name Arg; --- - * actually being a mixin. - */ - - bool [ID] _bool; - byte [ID] _byte; /// ditto - short [ID] _short; /// ditto - int [ID] _int; /// ditto - long [ID] _long; /// ditto - ubyte [ID] _ubyte; /// ditto - ushort [ID] _ushort; /// ditto - uint [ID] _uint; /// ditto - ulong [ID] _ulong; /// ditto - - char [ID] _char; /// ditto - - float [ID] _float; /// ditto - double [ID] _double; /// ditto - real [ID] _real; /// ditto - - bool[] [ID] _boolA; /// ditto - byte[] [ID] _byteA; /// ditto - short[] [ID] _shortA; /// ditto - int[] [ID] _intA; /// ditto - long[] [ID] _longA; /// ditto - ubyte[] [ID] _ubyteA; /// ditto - ushort[] [ID] _ushortA; /// ditto - uint[] [ID] _uintA; /// ditto - ulong[] [ID] _ulongA; /// ditto - - char[] [ID] _charA; /// ditto - - float[] [ID] _floatA; /// ditto - double[] [ID] _doubleA; /// ditto - real[] [ID] _realA; /// ditto - - /** Alias names */ - alias _ubyteA _binary; - alias _charA _string; /// ditto - //END DATA - - void addTag (char[] tp, ID id, char[] dt) { /// Supports all standard types. - if (tp.length == 0) throw new MTUnknownTypeException; - // split list up a bit for performance: - if (tp[0] < 'l') { - if (tp[0] < 'd') { - mixin ( `if (tp == "binary") addTag_add!(ubyte[]) (id, dt);` - ~ addTag_elifIsType_add!(bool) - ~ addTag_elifIsType_add!(bool[]) - ~ addTag_elifIsType_add!(byte) - ~ addTag_elifIsType_add!(byte[]) - ~ addTag_elifIsType_add!(char) - ~ addTag_elifIsType_add!(char[]) - ~ `else throw new MTUnknownTypeException;` ); - } else { - mixin ( `if (tp == "double") addTag_add!(double) (id, dt);` - ~ addTag_elifIsType_add!(double[]) - ~ addTag_elifIsType_add!(float) - ~ addTag_elifIsType_add!(float[]) - ~ addTag_elifIsType_add!(int) - ~ addTag_elifIsType_add!(int[]) - ~ `else throw new MTUnknownTypeException;` ); - } - } else { - if (tp[0] < 'u') { - mixin ( `if (tp == "long") addTag_add!(long) (id, dt);` - ~ addTag_elifIsType_add!(long[]) - ~ addTag_elifIsType_add!(real) - ~ addTag_elifIsType_add!(real[]) - ~ addTag_elifIsType_add!(short) - ~ addTag_elifIsType_add!(short[]) - ~ `else if (tp == "string") addTag_add!(char[]) (id, dt);` - ~ `else throw new MTUnknownTypeException;` ); - } else { - mixin ( `if (tp == "ubyte") addTag_add!(ubyte) (id, dt);` - ~ addTag_elifIsType_add!(ubyte[]) - ~ addTag_elifIsType_add!(ushort) - ~ addTag_elifIsType_add!(ushort[]) - ~ addTag_elifIsType_add!(uint) - ~ addTag_elifIsType_add!(uint[]) - ~ addTag_elifIsType_add!(ulong) - ~ addTag_elifIsType_add!(ulong[]) - ~ `else throw new MTUnknownTypeException;` ); - } - } - // try-catch block removed (caught by read) - } - private template addTag_elifIsType_add(T) { - const addTag_elifIsType_add = - `else if (tp == "`~T.stringof~`")` - `addTag_add!(`~T.stringof~`) (id, dt);` ; - } - private void addTag_add(T) (ID id, char[] dt) { - Arg!(T).Arg[id] = parse!(T) (dt); - } - - void writeAll (ItemDelg itemdlg) { - foreach (id, dt; _charA) itemdlg ("char[]", id, format!(char[])(dt)); - } - debug void debugFunc () {} - - /* These make no attempt to check Arg is valid. - * But if the symbol doesn't exist the complier will throw an error anyway, e.g.: - * Error: identifier '_boolAA' is not defined - */ - template Arg(T : T[]) { - const ArgString = Arg!(T).ArgString ~ `A`; - mixin(`alias `~ArgString~` Arg;`); - } - template Arg(T) { - const ArgString = `_` ~ T.stringof; - mixin(`alias `~ArgString~` Arg;`); - } -} - -/+class DynamicData : DataSection -{ - void*[TypeInfo] data; - -}+/ - -/+ -class TemplateData : DataSection -{ - void addTag (char[] tp, ID id, char[] dt) { - // runtime deduction of tp and aliasing? - // CANNOT add data at runtime though. - } - // will this work? no idea. - // templates can't be used to add non-static elements, so use a static array at index: this - template Data(T,TemplateData* p) { - static T[ID][TemplateData*] Data; - } -} -+/ - unittest { // Only covers DataSet really. DataSet ds = new DataSet; ds.sec[1] = new DefaultData;
--- a/mde/mergetag/doc/file-format-text.txt Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mergetag/doc/file-format-text.txt Fri Jan 25 18:17:38 2008 +0000 @@ -11,13 +11,12 @@ IDs: -IDs are used for several purposes; they are always stored as a uint number (0-4294967295). They may -be given in the file as a base-10 or hex number or, where a lookup table is provided to the reader, -as a double-quoted string (with no escape sequences). +IDs are used for several purposes; they are UTF-8 strings. They are stored in text files as unquoted strings; escape sequences are not supported and the strings should not contain the following characters, although this is not checked: <|=>{} +All characters between the appropriate markers are consumed into the ID, hence whitespace is meaningful. Multiple section or data tags with the same ID are allowed; see the "Merging rules" section. -Outside of tags the only whitespace or valid tags is allowed. Whitespace is ignored. +Outside of tags only whitespace or valid tags is allowed. Whitespace is ignored. The following tags are valid (see below for details): tag purpose {...} section identifiers @@ -28,19 +27,14 @@ Section identifier tags: -Format: {ID} or {ID|ID} -In the {ID|ID} case, the first ID is the section type, and the second ID the section name. -In the {ID} case, the section type ID has been ommitted and the default type is used (0). -A section identifier marks the beginning of a new section, extending until the next section -identifier or the end of the file. When a section is read, a new +Format: {ID} +The ID is the section identifier/name. The ID type is DefaultData unless overriden by the code using the reader. +A section identifier marks the beginning of a new section, extending until the next section identifier or the end of the file. Data item tags: Format: <tp|ID=dt> -A data item with type tp, identifier ID and data dt. If the data does not fit the given type it is -an error and the tag is ignored. Once split into a type string, ID and data string, the contents -are passed to an addTag() function within the DataSection class which will parse tags of a -recognised format and either ignore or print a warning about other tags. +A data item with type tp, identifier ID and data dt. If the data does not fit the given type it is an error and the tag is ignored. Once split into a type string, ID and data string, the contents are passed to an addTag() function within the DataSection class which will parse tags of a recognised format and either ignore or print a warning about other tags. Data item tags: Type format: @@ -53,8 +47,7 @@ tp[] a dynamic list of sub-type tp t1[t2] an associative array with key-type t2 Possible future additions: - tp() a dynamic merging list of sub-type tp (only valid as the primary type, ie - <subtype()|...>, not a sub-type of a tuple or another dynamic list) + tp() a dynamic merging list of sub-type tp (only valid as the primary type, ie <subtype()|...>, not a sub-type of a tuple or another dynamic list) {t1,t2,...,tn} a tuple with sub-types t1, t2, ..., tn Basic types (only items with a + are currently supported, items with * are in DefaultData): @@ -107,11 +100,8 @@ zi an imaginary floating point number (z is a floating point number) y+zi, y-zi a complex number (4+0i may be written as 4, etc) (y, z are f.p.s) 0xz, -0xz a hexadecimal integer z (composed of chars 0-9,a-f,A-F) - 'c' a char/wchar/dchar character, depending on the type specified (c may be any - single character except ' or an escape sequence) - "string" equivalent to ['s','t','r','i','n','g'] (for a string/wstring/dstring type) - may contain escape sequences - Escape sequences are a subset of those supported by D: \" \' \\ \a \b \f \n \r \t \v + 'c' a char/wchar/dchar character, depending on the type specified (c may be any single character except ' or an escape sequence) + "string" equivalent to ['s','t','r','i','n','g'] --- may contain the following escape sequences as defined in D: \" \' \\ \a \b \f \n \r \t \v XX...XX Binary (ubyte[]); each pair of chars is read as a hex ubyte <void> void "data" has no symbols @@ -123,14 +113,10 @@ Comment tags (there are no line comments): Simple comment blocks: Format: !{...} -This is a simple comment block, and only curly braces ({,}) are treated specially. A {, whether or -not it is preceded by a !, starts an embedded comment block, and a } ends either an embedded block -or the actual comment block. Note: beware commenting out {...} tags with a string ID containing -curly braces which aren't in matching pairs. +This is a simple comment block, and only curly braces ({,}) are treated specially. A {, whether or not it is preceded by a !, starts an embedded comment block, and a } ends either an embedded block or the actual comment block. Note: beware commenting out anything containing curly braces which aren't in matching pairs. Commented data tags: Format: !<tp|ID=dt> -Basically a commented out data tag. Conformance to the above spec may not be checked as strictly as -normal, but the dt section is checked for strings so that a > within a string won't end the tag. +Basically a commented out data tag. Conformance to the above spec may not be checked as strictly as normal, but the dt section is checked for strings so that a > within a string won't end the tag. Merging rules: @@ -149,10 +135,8 @@ Header: -The header is a standard section which is mandatory and must be the first section. Its section -identifier must start at the beginning of the file with no whitespace, declared with: - {MTXY} where XY is a two digit CAPITAL HEX version number representing the - mergetag format version, e.g. {MT01} . +The header is a standard section which is mandatory and must be the first section. Its section identifier must start at the beginning of the file with no whitespace, declared with: + {MTXY} where XY is a two digit CAPITAL HEX version number representing the mergetag format version, e.g. {MT01} . If these are not the first 6 characters of the file the file will not be regarded as valid. This formatting is very strict to allow reliable low-level parsing. @@ -164,8 +148,7 @@ <string|"Program"="..."> (which program created/uses this?) <*|"Version"=...> (use any supported type) <string|"Date"="YYYYMMDD"> (reverse date format; optionally "YYYYMMDDhhmmss") - <{u16,u8,u8}|"Date"={YYYY,MM,DD}> (actually this type probably won't be supported by - a standard section) + <{u16,u8,u8}|"Date"={YYYY,MM,DD}> (actually this type probably won't be supported by a standard section) <string|"Copyright"=...>
--- a/mde/mergetag/doc/issues.txt Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mergetag/doc/issues.txt Fri Jan 25 18:17:38 2008 +0000 @@ -1,16 +1,15 @@ This is mostly just a list of potential minor issues noticed while coding but not seen worth throwing an error about. Overall: - Threading support? - Use string IDs instead of integers? Could actually be faster due to no conversion being necessary. read.d: Allow only partially loading a file. - parseSection(): - as mentioned at end of function + parseSection(): as mentioned at end of function formatting errors could be more informative; in particular say where the error is + No binary support. write.d: + Threading support? There is currently no way to specify the base in which numbers are written (in text form). parse.d:
--- a/mde/mergetag/exception.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mergetag/exception.d Fri Jan 25 18:17:38 2008 +0000 @@ -26,13 +26,6 @@ this () {} } -/** Thrown when a string ID is parsed but cannot be found in the lookup table, hence cannot be used - * as an ID. It should always be caught and handled gracefully (by ignoring the tag or section - * involved). */ -class MTStringIDException : MTException { - this () {} -} - /** Thrown by addTag (in classes implementing dataset.DataSection) when a tag is read with an * unrecognised type field. */ class MTUnknownTypeException : MTException { @@ -53,11 +46,6 @@ this () {} } -/// Thrown by TextWriter.indexTable (only in debug mode). -class MTBadIDStringException : MTException { - this () {} -} - /// Thrown by *Writer.write. class MTNoDataSetException : MTException { this (char[] msg) {
--- a/mde/mergetag/old-code/dataset.d.old Wed Jan 16 12:48:07 2008 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,149 +0,0 @@ -/// This file contains text removed from dataset.d, which might possibly still be useful. - -/** - Class for exceptions of type "Incorrect Type" thrown when trying to read an item. Identical use to class Exception, but provides an easy way to handle only exceptions of this type. -*/ -class IncorrectType : Exception { - this (char[] s) { - super(s); - } -} - -/** - Data class; contains a Data member for each loaded section of a file. - - Could be a struct, except structs are value types (not reference). -*/ -class DataSet -{ -/+ Data[SecID] sec; /// Dynamic array of section data +/ - void*[char[]][char[]] data; - - void*[char[]] opIndex(char[] i) { - return data[i]; - } - void*[char[]][char[]] opSlice() { - return data[]; - } - void*[char[]][char[]] opSlice(char[] i, char[] j) { - return data[i,j]; - } -} - -struct Item { - enum Type : ubyte { - _void = 0, // initial type - tuple, dynlist, - _bool, _byte, _ubyte, _short, _ushort, _int, _uint, _long, _ulong, _cent, _ucent, - _char, _wchar, _dchar; - _float, _double, _real, - _ifloat, _idouble, _ireal, - _cfloat, _cdouble, _creal - } - static char[][26] typeName = ["void","tuple",]; - Type type; - new union { - Tuple tuple; - DynList dynlist; -// DynMerge dynmerge; merging lists are stored as dynamic lists - bool _bool; - byte _byte; - ubyte _ubyte; - short _short; - ushort _ushort; - int _int; - uint _uint; - long _long; - ulong _ulong; - cent _cent; - ucent _ucent; - char _char; - wchar _wchar; - dchar _dchar; - float _float; - double _double; - real _real; - ifloat _ifloat; - idouble _idouble; - ireal _ireal; - cfloat _cfloat; - cdouble _cdouble; - creal _creal; - } - - /** Functions to get data - - Each function will, if the element is of the appropriate type, return the element; if the type - is incorrect it will throw an error. - */ - bool _bool () { - if (type != _bool) throw new IncorrectType("Incorrect type when trying to read: tried to read as bool when item had type " ~ typeName[type]); - return _bool; - } - int _int () { /// ditto - if (type != _int) throw new IncorrectType("Incorrect type when trying to read: tried to read as int"); - return _int; - } - uint _uint () { /// ditto - if (type != _uint) throw new IncorrectType("Incorrect type when trying to read: tried to read as uint"); - return _uint; - } -} - -struct DynList -{ - -} - -class Data -{ - // added & accessed soley by templates - private (uint,void*)[Index] _gd; // generic data -} - -// Externally, types are given as a string: -typedef char[] typeIDstr; - private: -// Internally, types are given by a uint for performance: -typedef uint typeID; -typeID[typeIDstr] typeIDTable; // used to look up type typeID - -// This (belongs in read.d) contains a table of reading functions for all supported types. Do similarly for writing. -(void function (Data, char[]))[typeID] genericReader; - -// Template function for creating a function to read a new type and adding it to genericReader: -/+ don't actually use this without specialization -void addSupport (T) () { - // create a function to read this type from a string and add it into Data.genericData as type void*; put a function pointer into generic Reader - // do same for write support - // create a reader function, accessible by the user of the library, for accessing elements/converting to the proper type -}+/ - -/** - Get data of the appropriate type. - - The function performs a check that the data is of the appropriate type and throws an exception if - not. - - Note: can be called as d.get!($(I type))(i). -*/ -get(T : int) (Data d, Index i) { - return cast(T) *d._gd[i]; -} - -// add support for basic types (for all basic types): -void addSupport (T : int) () { - T read_int (char[]); - void* get_voidp (T d) { - return cast(void*) &d; - } -} - -void addSupport (T : T[]) () { // for any array type - // reader: split input and call appropriate fct to convert sub-strings to type T - // writer: use appropriate fct to convert to substrings; concat into "[val1,val2,...,valn]" format - // access: store as void* to T[] and something like this: - T[] get(Data d, Index i) { // but cannot overload by return-type! - return cast(T[]) genericData[i] - } -}
--- a/mde/mergetag/old-code/typeSpec.d.old Wed Jan 16 12:48:07 2008 +0000 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,103 +0,0 @@ -/+ I wrote this and then realised it's not worth using, it being easier and faster to match type -strings without this. This module does enable spaces to be in appropriate places, but so what... - -module typeSpec; - -import mde.mergetag.exception; - -import Util = tango.text.Util; - -/** - * Struct to view a type specifier string. - */ -struct TypeView -{ - /// Type of stored type specifier (NULL, BASIC, ARRAY, ASSOCIATIVE_ARRAY). - enum SPEC_TYPE : ubyte { - NULL, BASIC, ARRAY, ASSOCIATIVE_ARRAY - } - - SPEC_TYPE spec_type; /// Current specifier type. - union { - char[] type_name; /// For BASIC types is the type name. - TypeView* type_array_value; /// For ARRAY and ASSOCIATIVE_ARRAY types, this is the value-type. - } - TypeView* type_array_key; /// For ASSOCIATIVE_ARRAY types, this is the key-type; - - /** Construct and return a TypeView from a source string. - * - * Such a string should have one of the following forms: - * TYPE_NAME: [_a-zA-Z][_a-zA-Z0-9]* (regexp) - * ARRAY: TYPE_NAME[] - * ASSOC_ARRAY: TYPE_NAME[TYPE_NAME] - * - * Throws MTBadTypeStringException on bad type specifier strings. - */ - static TypeSpec parse (char[] src) { - bool isNameChar (char c) { - return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c = '_'; - } - void throwErr () { - logger.warn ("Bad type string"); - throw new MTBadTypeStringException (); - } - - src = Util.trim(src); - if (!src.length) throwErr; - - TypeView ret; - if (src[$-1] == ']') { // ARRAY or ASSOCIATIVE_ARRAY type - if (src.length < 2) throwErr; - - if (src[$-2] == '[') { // ARRAY type - ret.spec_type = SPEC_TYPE.ARRAY; // this is an array of... - ret.type_array_value = parse (src[0..$-2]); // whatever this is - } - else { // ASSOCIATIVE_ARRAY type - // look backwards for matching '[' bracket: - uint i = src.length-2; - uint recursion_depth = 0; - while (true) { - char c = src[i]; - - if (c == ']') ++recursion_depth; - if (c == '[') { - if (recursion_depth) --recursion_depth; - else break; - } - if (i == 0) throwErr; // no matching bracket! - else --i; - } - - ret.spec_type = SPEC_TYPE.ASSOCIATIVE_ARRAY; - ret.type_array_value = parse (src[0..i]); - ret.type_array_key = parse (src[i+1..$-1]); - } - } - else { // BASIC type - char c = src[0]; - if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || c = '_')) - throwErr; - for (uint i=1; i < src.length; ++i) { - if (!isNameChar(src[i])) throwErr; - } - - ret.spec_type = SPEC_TYPE.BASIC; - ret.type_name = src; - } - return ret; - } -} - -unittest { - TypeView tv = TypeView.parse (` type1 [] [ Type2 [] ] [] `); - assert (tv.spec_type == SPEC_TYPE.ARRAY); - assert (tv.type_array_value.spec_type == SPEC_TYPE.ASSOCIATIVE_ARRAY); - assert (tv.type_array_value.type_array_value.spec_type == SPEC_TYPE.ARRAY); - assert (tv.type_array_value.type_array_value.type_array_value.spec_type == SPEC_TYPE.BASIC); - assert (tv.type_array_value.type_array_value.type_array_value.type_name == `type1`); - assert (tv.type_array_value.type_array_key.spec_type == SPEC_TYPE.ARRAY); - assert (tv.type_array_value.type_array_key.type_array_value.spec_type == SPEC_TYPE.BASIC); - assert (tv.type_array_value.type_array_key.type_array_value.type_name == `Type2`); -} -+/ \ No newline at end of file
--- a/mde/mergetag/read.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mergetag/read.d Fri Jan 25 18:17:38 2008 +0000 @@ -8,8 +8,8 @@ // package imports public import mde.mergetag.dataset; -public import mde.mergetag.exception; -import mde.text.exception : textParseException; + +import mde.text.exception; // tango imports import tango.io.UnicodeFile; @@ -22,9 +22,6 @@ // TODO: allow compressing with zlib for both binary and text? (.mtz, .mtt, .mtb extensions) -// For now, all section & data tag IDs are uints. -// TODO: allow a lookup table or function to find a uint ID from a string ID - /** * Class for reading a file. * @@ -42,9 +39,11 @@ * Only a child-class of MTException will ever be thrown, currently MTFileIOException if the file * could not be read or MTFileFormatException on any error when parsing the file. * - * Threading: Reader should be thread-safe provided access to the same dataset is synchronized; - * i.e. no two readers refering to the same dataset should run simultaneously. (The Reader class - * could be made thread-safe w.r.t. datasets, but performance-wise I doubt it would be worth it.) + * Threading: Separate instances of Reader should be thread-safe provided access to the same + * dataset is synchronized; i.e. no two readers refering to the same dataset should run + * simultaneously. (The Reader class could be made thread-safe w.r.t. datasets, but + * performance-wise I doubt it would be worth it.) + * Do not run a single instance of Reader in multiple threads simultaneously. */ class Reader { @@ -57,13 +56,6 @@ */ DataSet dataset; - /** A table, which if created, allows items in a text file to have a string ID. - * - * If a string ID is given for a section or tag identifier and that string is a key in this - * table, then the corresponding ID type is used (if the string is not found an error is thrown). - */ - ID[char[]] indexTable; // see setIndexLookupTable() doc for use. - /** A function for creating new DataSections within the dataset. * * Allows a user-made class to be used in the DataSet instead of DefaultData. @@ -75,8 +67,6 @@ DataSection function (ID) dataSecCreator = null; private: - typedef void delegate (TypeInfo,ID,char[]) readDelg; // Delegate for accepting tags. - static Logger logger; // Non-static symbols: @@ -180,12 +170,8 @@ if (fatal) return []; if (!secTable.length) for (uint pos = endOfHeader; pos < fbuf.length;) { - try { - ID id = fbufReadSecMarker (pos); - secTable[id] = SecMD(pos,false); // add to table - } catch (MTStringIDException) { - // ignore section; this happens anyway (but don't add to table) - } + ID id = fbufReadSecMarker (pos); + secTable[id] = SecMD(pos,false); // add to table pos = parseSection (pos, null); } return cast(uint[]) secTable.keys; @@ -246,17 +232,13 @@ } } else { for (uint pos = endOfHeader; pos < fbuf.length;) { - try { - ID id = fbufReadSecMarker (pos); - secTable[id] = SecMD(pos,false); // add to table - if (secSet.contains(id)) { - DataSection ds = getOrCreateSec (id); - pos = parseSection (pos, &ds); - debug ds.debugFunc (); - secTable[id].read = true; - } - } catch (MTStringIDException) { // don't do any of the stuff above - pos = parseSection (pos, null); // and skip the section + ID id = fbufReadSecMarker (pos); + secTable[id] = SecMD(pos,false); // add to table + if (secSet.contains(id)) { + DataSection ds = getOrCreateSec (id); + pos = parseSection (pos, &ds); + debug ds.debugFunc (); + secTable[id].read = true; } } } @@ -272,14 +254,10 @@ } } else { // this time we don't need to use secTable for (uint pos = endOfHeader; pos < fbuf.length;) { - try { - ID id = fbufReadSecMarker (pos); - DataSection ds = getOrCreateSec (id); - pos = parseSection (pos, &ds); - debug ds.debugFunc (); - } catch (MTStringIDException) { - pos = parseSection (pos, null); // just skip the section - } + ID id = fbufReadSecMarker (pos); + DataSection ds = getOrCreateSec (id); + pos = parseSection (pos, &ds); + debug ds.debugFunc (); } } allRead = true; @@ -291,7 +269,7 @@ /* Reads a section, starting from index pos, finishing at the next section marker (returning the position of the start of the marker). pos should start after the section marker. - After analysing tags, the function passes the type, ID (possibly converted) and data to addTag. + After analysing tags, the function passes the type, ID and data to addTag. NOTE: from performance tests on indexing char[]'s and dereferencing char*'s, the char*'s are slightly faster, but a tiny difference isn't worth the extra effort/risk of using char*'s. @@ -323,30 +301,31 @@ for (; pos < fbuf.length; ++pos) { if (Util.isSpace(fbuf[pos])) continue; // whitespace else if (fbuf[pos] == '<') { // data tag - const char[] ERR_DTAG = "Bad data tag format: not <type|id=data>"; - char[] type, data; - ID tagID; + char[] ErrDTAG = "Bad data tag format: not <type|id=data>" ~ ErrInFile; + + fbufIncrement (pos); // Type section of tag: - fbufIncrement (pos); uint pos_s = pos; fbufLocateDataTagChar (pos, false); // find end of type section - if (fbuf[pos] != '|') throwMTErr (ERR_DTAG ~ ErrInFile); - type = fbuf[pos_s..pos]; - // ID section of tag: + if (fbuf[pos] != '|') throwMTErr (ErrDTAG); + char[] type = fbuf[pos_s..pos]; + fbufIncrement (pos); - try { - tagID = fbufReadID (pos); // read the ID, put pos at whatever's next - } catch (MTStringIDException) { - comment = true; // easiest way to ignore this tag - } - if (fbuf[pos] != '=') throwMTErr (ERR_DTAG ~ ErrInFile); + + // ID section of tag: + pos_s = pos; + fbufLocateDataTagChar (pos, false); // find end of type section + if (fbuf[pos] != '=') throwMTErr (ErrDTAG); + ID tagID = cast(ID) fbuf[pos_s..pos]; + + fbufIncrement (pos); + // Data section of tag: - fbufIncrement (pos); pos_s = pos; fbufLocateDataTagChar (pos, true); // find end of data section - if (fbuf[pos] != '>') throwMTErr (ERR_DTAG ~ ErrInFile); - data = fbuf[pos_s..pos]; + if (fbuf[pos] != '>') throwMTErr (ErrDTAG); + char[] data = fbuf[pos_s..pos]; if (!comment && dsec != null) { type = Util.trim(type); @@ -399,36 +378,17 @@ // at this point pos is whatever a parseSection run returned // since we haven't hit EOF, fbuf[pos] MUST be '{' so no need to check fbufIncrement(pos); - ID id = fbufReadID (pos); + + uint start = pos; + for (; pos < fbuf.length; ++pos) + if (fbuf[pos] == '}' || fbuf[pos] == '{') break; + if (fbuf[pos] != '}') throwMTErr ("Bad section tag format: not {id}" ~ ErrInFile); + ID id = cast(ID) fbuf[start..pos]; fbufIncrement(pos); return id; } - /* Parses fbuf from pos to read an ID. - On return pos is the index of the character following the ID. - */ - private ID fbufReadID (inout uint pos) { - while (Util.isSpace(fbuf[pos])) fbufIncrement(pos); // skip any space - if (fbuf[pos] == '"') { - fbufIncrement(pos); - uint start = pos; - while (fbuf[pos] != '"') fbufIncrement(pos); - ID* i_p = fbuf[start..pos] in indexTable; - while (Util.isSpace(fbuf[pos])) fbufIncrement(pos); // skip any space - if (i_p != null) return *i_p; // looked-up value - // FIXME: log a warning - throw new MTStringIDException (); // string not in look-up table - } else { - uint ate; - long x = ConvInt.parse (fbuf[pos..$], 0, &ate); - if (x < 0L || x > 0xFFFF_FFFFL) throwMTErr ("Tag has invalid integer ID: not a valid uint value" ~ ErrInFile); - pos += ate; // this is where ConvInt.parse stopped - while (Util.isSpace(fbuf[pos])) fbufIncrement(pos); // skip any space - return cast(ID) x; - } - } - /* Increments pos and checks it hasn't hit fbuf.length . */ private void fbufIncrement(inout uint pos) { ++pos; @@ -442,20 +402,6 @@ } //END METHODS: PRIVATE - invariant { - // Check secTable is valid, but not if it's complete. - // This is something I really wouldn't expect to fail. - /+ um... this causes a lot of linker errors. Shouldn't be necessary anyway.. - foreach (ID id, SecMD smd; secTable) { - uint pos = smd.pos; - for (; true; --pos) { - assert (pos); // we should never reach 0 - if (fbuf[pos] == '{') break; - } - ++pos; - assert (fbufReadID(pos) == id); - }+/ - } /+ A unittest here is really not practical since a file must be read from. Suggestion: Involve + both reading and writing functions in a single unittest for the entire package mergetag. + This is just here to point anyone looking in the right direction...
--- a/mde/mergetag/write.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/mergetag/write.d Fri Jan 25 18:17:38 2008 +0000 @@ -15,7 +15,6 @@ // package imports import mde.mergetag.dataset; -import mde.mergetag.exception; // tango imports import tango.core.Exception; @@ -25,7 +24,7 @@ import convInt = tango.text.convert.Integer; import tango.util.log.Log : Log, Logger; -Logger logger; +private Logger logger; static this () { logger = Log.getLogger ("mde.mergetag.write"); } @@ -79,12 +78,6 @@ DataSet dataset (); void dataset (DataSet); /// ditto - /** Only used in a TextWriter; see TextWriter.indexTable(). - * - * Note that other implementors implement this as a blank function and will not throw an error. - */ - void indexTable (char[][ID]); - void write (); /// Writing method. } @@ -108,35 +101,6 @@ { dataset = ds; } - /** A table, which if created, allows items in a text file to be written with a string ID. - * - * If any ID (for a section or tag) to be written is found in this table, the corresponding - * string is written instead. - * - * In debug mode, logs a warning if any of the given strings are invalid (but never throws). - */ - void indexTable (char[][ID] iT) { - debug { // check validity (no unescaped " symbols or unfinished escape sequence) - foreach (str; iT) { - for (uint i = 0; i < str.length; ++i) { - auto ERR_STRING = "While preparing to write " ~ path.toUtf8() ~ ":"; - if (str[i] == '"') { - logger.warn (ERR_STRING); - logger.warn ("In given indexTable: unescaped \" char. Likely to cause errors when reading!"); - } - if (str[i] == '\\') { - ++i; - if (i == str.length) { - logger.warn (ERR_STRING); - logger.warn ("In given indexTable: trailing \\ (half-escape sequence). Likely to cause errors when reading!"); - } - } - } - } - } - _indexTable = iT; - } - private: // taken from tango.io.Console, mostly to make sure notepad can read our files: version (Win32) @@ -148,8 +112,6 @@ DataSet _dataset; PathView path; - - char[][ID] _indexTable; // see indexTable() doc for use. //END DATA //BEGIN CTOR / DTOR @@ -221,20 +183,12 @@ } private void writeSectionIdentifier (IBuffer buffer, ID id) { - buffer ("{"); - char[]* s = id in _indexTable; // look for a string ID - if (s) buffer ("\"" ~ *s ~ "\""); // write a string ID - else buffer (convInt.toUtf8(id)); // write a numeric ID - buffer ("}" ~ Eol); + buffer ("{" ~ id ~ "}" ~ Eol); } private void writeSection (IBuffer buffer, DataSection sec) { void writeItem (char[] tp, ID id, char[] dt) { // actually writes an item - buffer ("<" ~ tp ~ "|"); - char[]* s = id in _indexTable; - if (s) buffer ("\"" ~ *s ~ "\""); - else buffer (convInt.toUtf8(id)); - buffer ("=" ~ dt ~ ">" ~ Eol); + buffer ("<" ~ tp ~ "|" ~ id ~"=" ~ dt ~ ">" ~ Eol); } sec.writeAll (&writeItem);
--- a/mde/test.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/test.d Fri Jan 25 18:17:38 2008 +0000 @@ -1,7 +1,7 @@ /// Contains some functions for testing stuff. module mde.test; -import mde.mergetag.dataset; +import MT = mde.mergetag.dataset; import mde.text.parse; import tango.io.Stdout; @@ -55,7 +55,7 @@ } } void printLabel (ID i) { - Stdout (cast(uint) i)(":\t"); + Stdout (i)(":\t"); } class DataPrinter : DataSection
--- a/mde/text/format.d Wed Jan 16 12:48:07 2008 +0000 +++ b/mde/text/format.d Fri Jan 25 18:17:38 2008 +0000 @@ -22,7 +22,7 @@ // TODO: write unittests; check strings generate quotes. // package imports -import mde.text.exception; +public import mde.text.exception; // tango imports import cInt = tango.text.convert.Integer;