Mercurial > projects > dil

annotate trunk/src/dil/LexerFuncs.d @ 490:47be6bfe39cd

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Refactored code and added new modules. Added ProtectionDeclaration and changed Parser to make use of it. Moved class Location to its own module. Moved some Lexer functions to new module LexerFuncs. Moved Lexer.getLocation() to struct Token. Added methods semantic() and error() to class Expression. Added method error() to class Scope.
author Aziz K?ksal <aziz.koeksal@gmail.com>
date Tue, 04 Dec 2007 23:31:20 +0100
parents
children 9a7ca8c56e59
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
490
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
1 /++
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
2 Author: Aziz Köksal
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
3 License: GPL3
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
4 +/
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
5 module dil.LexerFuncs;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
6
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
7 const char[3] LS = \u2028; /// Line separator.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
8 const char[3] PS = \u2029; /// Paragraph separator.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
9 const dchar LSd = 0x2028;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
10 const dchar PSd = 0x2029;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
11 static assert(LS[0] == PS[0] && LS[1] == PS[1]);
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
12
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
13 const uint _Z_ = 26; /// Control+Z
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
14
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
15 /// Returns true if d is a Unicode line or paragraph separator.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
16 bool isUnicodeNewlineChar(dchar d)
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
17 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
18 return d == LSd || d == PSd;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
19 }
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
20
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
21 /// Returns true if p points to a line or paragraph separator.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
22 bool isUnicodeNewline(char* p)
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
23 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
24 return *p == LS[0] && p[1] == LS[1] && (p[2] == LS[2] || p[2] == PS[2]);
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
25 }
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
26
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
27 /++
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
28 Returns true if p points to the start of a Newline.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
29 Newline: \n | \r | \r\n | LS | PS
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
30 +/
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
31 bool isNewline(char* p)
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
32 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
33 return *p == '\n' || *p == '\r' || isUnicodeNewline(p);
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
34 }
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
35
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
36 /++
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
37 Returns true if p points to the first character of an EndOfLine.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
38 EndOfLine: Newline | 0 | _Z_
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
39 +/
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
40 bool isEndOfLine(char* p)
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
41 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
42 return isNewline(p) || *p == 0 || *p == _Z_;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
43 }
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
44
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
45 /++
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
46 Scans a Newline and sets p one character past it.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
47 Returns '\n' if scanned or 0 otherwise.
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
48 +/
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
49 dchar scanNewline(ref char* p)
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
50 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
51 switch (*p)
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
52 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
53 case '\r':
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
54 if (p[1] == '\n')
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
55 ++p;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
56 case '\n':
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
57 ++p;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
58 return '\n';
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
59 default:
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
60 if (isUnicodeNewline(p))
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
61 {
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
62 ++p; ++p; ++p;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
63 return '\n';
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
64 }
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
65 }
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
66 return 0;
47be6bfe39cd Refactored code and added new modules.
Aziz K?ksal <aziz.koeksal@gmail.com>
parents:
diff changeset
67 }