Mercurial > projects > dil

diff trunk/src/dil/Token.d @ 383:6a5fc22cae34

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Implemented scanner for new string literals and applied some fixes. Added methods scanDelimitedStringLiteral() and scanTokenStringLiteral(). They are practically fully implemented. Error messages have to be added yet. Partially rewrote scanShebang() as it was buggy. Now it decodes Unicode characters and the code is simpler to read. Fixed scanRawStringLiteral(): Unicode characters weren't decoded. Added some safeguards to decodeUTF8().
author Aziz K?ksal <aziz.koeksal@gmail.com>
date Tue, 11 Sep 2007 18:15:02 +0200
parents eb08126cca56
children 4d36eea1bbc9
line wrap: on
line diff
--- a/trunk/src/dil/Token.d	Mon Sep 10 17:36:58 2007 +0200
+++ b/trunk/src/dil/Token.d	Tue Sep 11 18:15:02 2007 +0200
@@ -134,7 +134,11 @@
     struct
     {
       string str;
-      char pf;
+      char pf; /// Postfix 'c', 'w' or 'd'
+    version(D2)
+      Token* tok_str; /// Points to the contents of a token string stored as a
+                      /// doubly linked list. The last token is always '}' or
+                      /// EOF in case end of source text is "q{" EOF.
     }
     dchar  dchar_;
     long   long_;