Refactored Lexer.scan(). Illegal characters are not ignored anymore. They are reported as errors. Added a new member 'ws' to Token. When a token is scanned the lexer sets ws to the leading whitespace or leaves it at null when no whitespace was found. Added Illegal to enum TOK and IllegalCharacter to enum MID. Added localized messages for MID.IllegalCharacter. Adapted code of cmd.Generate to make use of Token.ws.

--- a/trunk/src/dil/Messages.d	Wed Sep 12 18:18:29 2007 +0200
+++ b/trunk/src/dil/Messages.d	Wed Sep 12 21:03:41 2007 +0200
@@ -10,6 +10,7 @@
 enum MID
 {
   // Lexer messages:
+  IllegalCharacter,
   InvalidUnicodeCharacter,
   InvalidUTF8Sequence,
   // ''