Mercurial > projects > dang

annotate lexer/Token.d @ 189:75d0544ddc45

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Better error handling on unexpected EOF.
author Anders Johnsen <skabet@gmail.com>
date Fri, 25 Jul 2008 13:50:01 +0200
parents dc9bf56b7ace
children 08f68d684047
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
1 module lexer.Token;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
2
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
3 public
189
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
4 import basic.SourceLocation,
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
5 basic.SourceManager;
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
6
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
7 import Integer = tango.text.convert.Integer;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
8
42
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
9 /**
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
10 The Token struct will be used through the Lexer, Parser and other
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
11 modules as a location into source.
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
12
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
13 The Token should always be optimized for size to limit unnecessary
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
14 memory usage.
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
15 */
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
16 struct Token
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
17 {
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
18 Tok type;
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents: 74
diff changeset
19 SLoc location;
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
20 uint length;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
21
42
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
22 /**
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
23 Create a new token with a Tok type, Location in source and a
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
24 length of how many chars the Token span in the source
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
25 */
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents: 74
diff changeset
26 static Token opCall (Tok type, SLoc location, uint length)
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
27 {
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
28 Token t;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
29 t.type = type;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
30 t.location = location;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
31 t.length = length;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
32 return t;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
33 }
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
34
42
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
35 /**
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
36 Get the type of the Token as a string
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
37 */
189
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
38 char[] get (SourceManager sm)
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
39 {
189
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
40 if (isIdentifier)
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
41 return sm.getText(asRange);
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
42 return typeToString[this.type];
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
43 }
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
44
42
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
45 /**
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
46 A human readable dump of a Token
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
47 */
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
48 char[] toString ()
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
49 {
189
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
50 return typeToString[this.type];
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
51 }
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
52
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents: 74
diff changeset
53 /// Get the range of this token
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents: 74
diff changeset
54 SourceRange asRange() { return SourceRange(location, location + length); }
44
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
55
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
56 /**
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
57 Returns true if the type of this token is a basic type (int, float, ...).
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
58 Void is included, although a void in it self is not really a type.
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
59 */
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
60 bool isBasicType()
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
61 {
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
62 return type >= Tok.Byte && type <= Tok.Void;
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
63 }
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
64
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
65 /**
48
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
66 Returns true for all the various assignments (=, +=, *= ...)
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
67 */
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
68 bool isAssignment()
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
69 {
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
70 return type >= Tok.Assign && type <= Tok.PercentAssign;
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
71 }
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
72
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
73 /**
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
74 Returns true for all attributes( public, static, private...)
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
75 */
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
76 bool isAttribute()
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
77 {
140
927ae00bd9d2 Added support for extern keyword. Being ignored atm though. Also changed ast/Module, so that you can get a list of only vars, functions or structs.
Anders Johnsen <skabet@gmail.com>
parents: 136
diff changeset
78 return type >= Tok.Public && type <= Tok.Extern;
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
79 }
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
80
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
81 /**
136
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
82 Returns true for all attributes( public, static, private...)
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
83 */
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
84 bool isBaseClassProtection()
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
85 {
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
86 return type >= Tok.Public && type <= Tok.Export;
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
87 }
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
88
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
89 /**
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
90 just a shortcut to avoid `token.type == tok.Switch`.
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
91 */
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
92 bool isSwitch()
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
93 {
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
94 return type == Tok.Switch;
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
95 }
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
96
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
97 /**
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
98 just a shortcut to avoid `token.type == tok.While`.
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
99 */
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
100 bool isWhile()
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
101 {
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
102 return type == Tok.While;
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
103 }
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
104
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
105 /**
146
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
106 just a shortcut to avoid `token.type == tok.For`.
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
107 */
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
108 bool isFor()
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
109 {
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
110 return type == Tok.For;
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
111 }
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
112
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
113 /**
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
114 just a shortcut to avoid `token.type == tok.If`.
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
115 */
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
116 bool isIf()
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
117 {
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
118 return type == Tok.If;
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
119 }
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
120
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
121 /**
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
122 just a shortcut to avoid `token.type == tok.Return`.
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
123 */
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
124 bool isReturn()
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
125 {
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
126 return type == Tok.Return;
48
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
127 }
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
128
b6c1dc30ca4b Only tests that dont pass now are structs and switches
Anders Halager <halager@gmail.com>
parents: 44
diff changeset
129 /**
44
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
130 Just a shortcut to avoid `token.type == Tok.Identifier`.
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
131 */
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
132 bool isIdentifier()
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
133 {
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
134 return type == Tok.Identifier;
495188f9078e Big update - Moving towards a better, more seperated parser
Anders Halager <halager@gmail.com>
parents: 42
diff changeset
135 }
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
136 }
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
137
42
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
138 /**
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
139 Tok is short for TokenType. This enum list is to supply the Token
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
140 with a type.
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
141
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
142 This enum is used to switch over "many" places.
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
143 */
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
144 enum Tok : ushort
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
145 {
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
146 /* Non-code related tokens */
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
147 EOF,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
148
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
149 /* Basic types */
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
150 Identifier,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
151 Integer,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
152 String,
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
153
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
154 /* Basic operators */
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
155 Assign,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
156 PlusAssign,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
157 MinusAssign,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
158 StarAssign,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
159 SlashAssign,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
160 PercentAssign,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
161 Plus,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
162 Minus,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
163 Star,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
164 Slash,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
165 Percent,
123
6a5f745d351c Parsing <<, >> and >>>.
Anders Johnsen <skabet@gmail.com>
parents: 104
diff changeset
166 LeftShift, RightShift, UnsignedRightShift,
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
167 Comma,
176
dc9bf56b7ace Can now use & as a unary operator and take an AddressOf
Anders Johnsen <skabet@gmail.com>
parents: 172
diff changeset
168 And,
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
169
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
170 /* Symbols */
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
171 OpenParentheses,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
172 CloseParentheses,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
173 OpenBrace,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
174 CloseBrace,
72
628cb46ab13b First update on the way to Arrays! :)
Anders Johnsen <skabet@gmail.com>
parents: 68
diff changeset
175 OpenBracket,
628cb46ab13b First update on the way to Arrays! :)
Anders Johnsen <skabet@gmail.com>
parents: 68
diff changeset
176 CloseBracket,
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
177 Seperator,
36
ce17bea8e9bd Switch statements support
Anders Halager <halager@gmail.com>
parents: 28
diff changeset
178 Colon,
28
69464d465284 Now supporting structs - both read and write. Still a few errors though, so watch out.
Anders Johnsen <skabet@gmail.com>
parents: 22
diff changeset
179 Dot,
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
180
8
2e1069ee21af Added Ne, Lt, Le, Gt, Ge and Not in lexer
johnsen@johnsen-desktop
parents: 6
diff changeset
181 /* Comparator operators */
2e1069ee21af Added Ne, Lt, Le, Gt, Ge and Not in lexer
johnsen@johnsen-desktop
parents: 6
diff changeset
182 Eq, Ne,
2e1069ee21af Added Ne, Lt, Le, Gt, Ge and Not in lexer
johnsen@johnsen-desktop
parents: 6
diff changeset
183 Lt, Gt,
2e1069ee21af Added Ne, Lt, Le, Gt, Ge and Not in lexer
johnsen@johnsen-desktop
parents: 6
diff changeset
184 Le, Ge,
2e1069ee21af Added Ne, Lt, Le, Gt, Ge and Not in lexer
johnsen@johnsen-desktop
parents: 6
diff changeset
185
2e1069ee21af Added Ne, Lt, Le, Gt, Ge and Not in lexer
johnsen@johnsen-desktop
parents: 6
diff changeset
186 Not,
6
606a57c90a0b Now lexing == as Equals
johnsen@johnsen-desktop
parents: 5
diff changeset
187
176
dc9bf56b7ace Can now use & as a unary operator and take an AddressOf
Anders Johnsen <skabet@gmail.com>
parents: 172
diff changeset
188
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
189 /* Keywords */
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
190 Byte, Ubyte,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
191 Short, Ushort,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
192 Int, Uint,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
193 Long, Ulong,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
194
103
09b4d74cb3f5 Parsing char, wchar and dchar as basic types.
Anders Johnsen <skabet@gmail.com>
parents: 94
diff changeset
195 Char, Wchar, Dchar,
09b4d74cb3f5 Parsing char, wchar and dchar as basic types.
Anders Johnsen <skabet@gmail.com>
parents: 94
diff changeset
196
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
197 Float, Double,
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
198
5
2c5a8f4c254a Added very simple if support.
Anders Halager <halager@gmail.com>
parents: 1
diff changeset
199 Bool,
2c5a8f4c254a Added very simple if support.
Anders Halager <halager@gmail.com>
parents: 1
diff changeset
200
37
858b9805843d Bug-fixes
Anders Halager <halager@gmail.com>
parents: 36
diff changeset
201 Void,
858b9805843d Bug-fixes
Anders Halager <halager@gmail.com>
parents: 36
diff changeset
202
154
0ea5d2f3e96b Parsing "this" as constructor. Also removed regex from the test run program(seg fault - dmd???)
Anders Johnsen <skabet@gmail.com>
parents: 148
diff changeset
203 Struct, Function, Delegate, Class, This,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
204 Interface, Union, Typedef, Typeid,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
205 Typeof, Sizeof, Alias,
22
e331e4e816e4 now handling structs to some extend
johnsen@johnsen-laptop
parents: 21
diff changeset
206
11
642c6a998fd9 Support for while statements and fixed scope for if
Anders Halager <halager@gmail.com>
parents: 8
diff changeset
207 If, Else,
642c6a998fd9 Support for while statements and fixed scope for if
Anders Halager <halager@gmail.com>
parents: 8
diff changeset
208 While,
146
8c09fdaa724e Parsing for-loop.
Anders Johnsen <skabet@gmail.com>
parents: 140
diff changeset
209 For,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
210 Switch, Case, Default, Break,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
211 Return, Cast,
104
7ff4bc2accf2 Lexing more types of strings. Now all's left is to parse the string in the AST.
Anders Johnsen <skabet@gmail.com>
parents: 103
diff changeset
212
94
48bb2287c035 Added Modules. Right now it's very simple - will grow with time and need.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
213 Module, Import,
48bb2287c035 Added Modules. Right now it's very simple - will grow with time and need.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
214
158
57b0b4464a0b Parsing "new", putting it in AST and performs some tests on it. Eg. if the contructor exists and the params matches.
Anders Johnsen <skabet@gmail.com>
parents: 154
diff changeset
215 New,
57b0b4464a0b Parsing "new", putting it in AST and performs some tests on it. Eg. if the contructor exists and the params matches.
Anders Johnsen <skabet@gmail.com>
parents: 154
diff changeset
216
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
217 /* Attributes */
136
2be29b296081 Lots of changes:
johnsen@johnsen-laptop
parents: 126
diff changeset
218 Public, Private, Package, Export, Protected,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
219 Static,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
220 Final,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
221 Const,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
222 Abstract,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
223 Override,
148
6ec686d9c87d Fixed some for parsing, and removed a little ugly bug.
Anders Johnsen <skabet@gmail.com>
parents: 146
diff changeset
224 Deprecated,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
225 Auto,
140
927ae00bd9d2 Added support for extern keyword. Being ignored atm though. Also changed ast/Module, so that you can get a list of only vars, functions or structs.
Anders Johnsen <skabet@gmail.com>
parents: 136
diff changeset
226 Extern,
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
227
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
228 Align,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
229
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
230 Asm,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
231
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
232 In, Out, Body,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
233
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
234 Assert, Throw, Try, Catch, Finally,
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
235
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
236
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
237
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
238
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
239 }
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
240
42
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
241 /**
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
242 An associative array to supply a Tok to String function.
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
243
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
244 Keep always this list updated when adding a new Tok.
4e879f82dd64 Added some docs for the lexer - now you can understand _some_ of the madness going on here :)
Anders Johnsen <skabet@gmail.com>
parents: 37
diff changeset
245 */
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
246 public char[][Tok] typeToString;
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
247
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
248 static this()
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
249 {
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
250 typeToString =
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
251 [
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
252 Tok.EOF:"EOF"[],
189
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
253 Tok.Identifier:"identifier",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
254 Tok.Byte:"byte",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
255 Tok.Short:"short",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
256 Tok.Int:"int",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
257 Tok.Long:"long",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
258 Tok.Char:"char",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
259 Tok.Wchar:"wchar",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
260 Tok.Dchar:"dchar",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
261 Tok.Bool:"bool",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
262 Tok.Void:"void",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
263 Tok.Function:"function",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
264 Tok.Eq:"==",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
265 Tok.Ne:"!=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
266 Tok.Lt:"<",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
267 Tok.Le:"<=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
268 Tok.Gt:">",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
269 Tok.Ge:">=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
270 Tok.OpenParentheses:"(",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
271 Tok.CloseParentheses:")",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
272 Tok.OpenBrace:"{",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
273 Tok.CloseBrace:"}",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
274 Tok.OpenBracket:"[",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
275 Tok.CloseBracket:"]",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
276 Tok.Dot:"-",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
277 Tok.Assign:"=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
278 Tok.Plus:"+",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
279 Tok.PlusAssign:"+=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
280 Tok.Minus:"-",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
281 Tok.MinusAssign:"-=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
282 Tok.Star:"*",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
283 Tok.StarAssign:"*=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
284 Tok.Slash:"/",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
285 Tok.SlashAssign:"/=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
286 Tok.Percent:"%",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
287 Tok.PercentAssign:"%=",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
288 Tok.LeftShift:"<<",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
289 Tok.RightShift:">>",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
290 Tok.UnsignedRightShift:">>>",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
291 Tok.Integer:"int",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
292 Tok.If:"if",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
293 Tok.While:"while",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
294 Tok.For:"for",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
295 Tok.Switch:"switch",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
296 Tok.Case:"case",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
297 Tok.Default:"default",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
298 Tok.Comma:",",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
299 Tok.Return:"return",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
300 Tok.Struct:"struct",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
301 Tok.Class:"class",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
302 Tok.This:"this",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
303 Tok.Colon:":",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
304 Tok.Seperator:";",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
305 Tok.And:"&",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
306 Tok.Cast:"cast",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
307 Tok.Module:"module",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
308 Tok.Import:"import",
126
c3b24e7e8cf8 Carius changes to the parser. Parsing attributes, lexing many keywords(not all yet).
Anders Johnsen <skabet@gmail.com>
parents: 123
diff changeset
309 Tok.String:"String",
189
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
310 Tok.Public:"public",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
311 Tok.Private:"private",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
312 Tok.Protected:"protected",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
313 Tok.Package:"package",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
314 Tok.Export:"export",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
315 Tok.Static:"static",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
316 Tok.Final:"finale",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
317 Tok.Public:"public",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
318 Tok.Const:"const",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
319 Tok.Abstract:"abstract",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
320 Tok.Override:"override",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
321 Tok.Deprecated:"deprecated",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
322 Tok.Auto:"auto",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
323 Tok.Extern:"extern",
75d0544ddc45 Better error handling on unexpected EOF.
Anders Johnsen <skabet@gmail.com>
parents: 176
diff changeset
324 Tok.New:"new"
1
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
325 ];
2168f4cb73f1 First push
johnsen@johnsen-desktop
parents:
diff changeset
326 }