Mercurial > projects > dang

annotate basic/SourceManager.d @ 92:771ac63898e2 new_gen

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
A few better parser errors plus renaming most of the sema classes to match that they do now. Some have changes a lot.
author Anders Johnsen <skabet@gmail.com>
date Mon, 05 May 2008 18:44:20 +0200
parents 1a24e61eb104
children 48bb2287c035
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
1 module basic.SourceManager;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
2
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
3 import tango.core.Memory : GC;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
4 import tango.io.UnicodeFile;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
5 import tango.io.Stdout;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
6 import tango.text.convert.Layout;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
7
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
8 public import basic.SourceLocation;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
9
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
10 private alias char[] string;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
11
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
12 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
13 SourceManager is used to handle input files, by loading them in in chunks
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
14 that can be referenced elsewhere.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
15
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
16 It will also help extract the line/col of locations and convert between
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
17 real and virtual locations
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
18 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
19 class SourceManager
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
20 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
21 this()
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
22 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
23 layout = new Layout!(char);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
24 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
25
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
26 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
27 Will load in the file belonging to the filename
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
28
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
29 filename = The file to load. Theres some assumptions about this file.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
30 1. The file has a BOM or is valid utf-8
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
31 2. The file is not empty, unreadable, a folder etc.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
32 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
33 SourceLocation addFile(string filename)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
34 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
35 scope file = new UnicodeFile!(char)(filename, Encoding.UTF_8);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
36 auto file_data = file.read();
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
37 return createCheckpoints(file_data, filename);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
38 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
39
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
40 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
41 Returns a string slice containing the part of the file after loc (a
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
42 pointer might be better, it allows negative indexing)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
43 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
44 string getRawData(SourceLocation loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
45 {
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
46 CP cp = checkpoints[loc.fileID];
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
47 auto length = cp.data_end - cp.data.ptr;
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
48 return cp.data.ptr[loc.fileOffset .. length];
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
49 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
50
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
51 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
52 Extracts the line number of the given location
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
53 O("file size") if cache isn't built, O(log "lines in file") else
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
54 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
55 uint getLineNumber(SourceLocation loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
56 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
57 assert(loc.isValid, "Location is invalid");
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
58 assert(loc.isReal, "Virtual locations not supported yet");
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
59 assert(loc.fileID < checkpoints.length, "Non-existent location");
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
60
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
61 CP* cp = &checkpoints[loc.fileID];
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
62 auto cache = &linecache[cp.meta_index];
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
63 if (!cache.isCached)
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
64 cache.build(cp.data_start[0 .. cp.data_end - cp.data_start]);
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
65 return cache.lineOf(getFileOffset(loc));
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
66 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
67
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
68 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
69 Extracts the full byte offset into a file, at which a location
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
70 is pointing.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
71 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
72 uint getFileOffset(SourceLocation loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
73 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
74 return loc.fileOffset
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
75 + checkpoints[loc.fileID].part * loc.Bits.MaxFileOffset;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
76 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
77
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
78 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
79 Extracts a string containing the entire line loc appears in.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
80 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
81 string getLine(SourceLocation loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
82 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
83 // The line is extracted by getting two pointers to the exact location
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
84 // and decreasing one until the nearest newline while the other ptr is
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
85 // increased to the nearest newline.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
86 CP* cp = &checkpoints[loc.fileID];
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
87 char* ptr = cp.data.ptr + loc.fileOffset;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
88 char* ptr_lo = ptr;
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
89 while (cp.inRange(ptr_lo) && *ptr_lo != '\n' && *ptr_lo != '\r')
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
90 --ptr_lo;
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
91 while (cp.inRange(ptr) && *ptr != '\n' && *ptr != '\r')
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
92 ++ptr;
89
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
93 return ptr_lo[1 .. ptr - ptr_lo];
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
94 }
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
95
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
96 /**
92
771ac63898e2 A few better parser errors plus renaming most of the sema classes to match that they do now. Some have changes a lot.
Anders Johnsen <skabet@gmail.com>
parents: 91
diff changeset
97 Gets the column of where the loc appears.
89
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
98 **/
92
771ac63898e2 A few better parser errors plus renaming most of the sema classes to match that they do now. Some have changes a lot.
Anders Johnsen <skabet@gmail.com>
parents: 91
diff changeset
99 int getColumn(SourceLocation loc)
89
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
100 {
92
771ac63898e2 A few better parser errors plus renaming most of the sema classes to match that they do now. Some have changes a lot.
Anders Johnsen <skabet@gmail.com>
parents: 91
diff changeset
101 // Use same approach as getLine
771ac63898e2 A few better parser errors plus renaming most of the sema classes to match that they do now. Some have changes a lot.
Anders Johnsen <skabet@gmail.com>
parents: 91
diff changeset
102
89
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
103 CP* cp = &checkpoints[loc.fileID];
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
104 char* ptr = cp.data.ptr + loc.fileOffset;
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
105 char* ptr_lo = ptr;
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
106 while (cp.inRange(ptr_lo) && *ptr_lo != '\n' && *ptr_lo != '\r')
89
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
107 --ptr_lo;
a49bb982a7b0 Using the new SourceLocation system to handle errors. Also, this is the first push for making the errors don't throw, but continue to check the source.
Anders Johnsen <skabet@gmail.com>
parents: 88
diff changeset
108 return cast(int)ptr - cast(int)ptr_lo - 1;
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
109 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
110
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
111 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
112 Get the original source text of a SourceRange
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
113 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
114 string getText(SourceRange loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
115 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
116 assert(loc.isValid, "Range is invalid");
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
117 assert(loc.isReal, "Virtual locations not supported yet");
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
118 auto begin = getFileOffset(loc.begin);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
119 auto end = getFileOffset(loc.end);
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
120 return checkpoints[loc.begin.fileID].data_start[begin .. end];
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
121 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
122
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
123 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
124 Get the original source text
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
125 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
126 string getText(SourceLocation loc, size_t length)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
127 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
128 return getText(SourceRange(loc, loc + length));
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
129 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
130
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
131 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
132 Convert a location into a string. Something like "file(line)"
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
133 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
134 string getLocationAsString(SourceLocation loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
135 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
136 assert(loc.isValid, "Location is invalid");
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
137 return layout.convert("{}({})",
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
138 checkpoints[loc.fileID].filename,
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
139 getLineNumber(loc));
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
140 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
141 string getLocationAsString(SourceRange loc)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
142 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
143 return layout.convert("{}({}:{})",
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
144 checkpoints[loc.begin.fileID].filename,
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
145 getFileOffset(loc.begin),
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
146 getFileOffset(loc.end));
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
147 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
148
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
149 private:
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
150 synchronized
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
151 SourceLocation createCheckpoints(string data, string source_file)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
152 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
153 // The line-cache is added, but not built,
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
154 // getLineNumber makes sure it is called when needed.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
155 linecache ~= FileLineCache();
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
156 uint meta_index = linecache.length - 1;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
157
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
158 // SourceLocation's can only index relatively short buffers, therefore
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
159 // the file is split into several checkpoints.
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
160 uint checkpoint_counter = 0;
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
161 char* data_start = data.ptr;
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
162 char* data_end = data.ptr + data.length;
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
163 while (data.length > 0)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
164 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
165 uint to_take = min(data.length, SourceLocation.Bits.MaxFileOffset);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
166 checkpoints ~=
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
167 CP(source_file,
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
168 data_start,
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
169 data_end,
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
170 data[0 .. to_take],
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
171 checkpoint_counter++,
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
172 meta_index);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
173 data = data[to_take .. $];
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
174 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
175 checkpoint_counter = checkpoints.length - checkpoint_counter;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
176 return SourceLocation.fromFileID(checkpoint_counter);
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
177 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
178
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
179 /// Contains the read/generated data.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
180 CP[] checkpoints;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
181 /// Cache used to speed up finding of line-starts.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
182 FileLineCache[] linecache;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
183 /// Used for formatting locations as strings.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
184 Layout!(char) layout;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
185
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
186 // These really should be magically available everywhere and templated.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
187 int min(int a, int b) { return a < b? a : b; }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
188 int max(int a, int b) { return a >= b? a : b; }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
189
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
190 // A Check Point is used to store a file in multiple parts, to overcome
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
191 // the limitation of SourceLocation only having a rather limited amount of
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
192 // bits to index any one file.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
193 struct CP
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
194 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
195 // read-only
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
196 char[] filename;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
197 // ditto
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
198 char* data_start;
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
199 char* data_end;
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
200 // ditto
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
201 char[] data;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
202 // ditto
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
203 uint part = 0;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
204 // ditto
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
205 uint meta_index = 0;
91
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
206
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
207 bool inRange(char* p)
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
208 {
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
209 return p >= data_start && p < data_end;
1a24e61eb104 Fixed the SourceLocation/SourceManager for files stretching by more then one CheckPoints
johnsen@johnsen-laptop
parents: 89
diff changeset
210 }
88
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
211 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
212
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
213 struct FileLineCache
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
214 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
215 /// Contains the offset of the i'th line on index i
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
216 uint[] line_starts;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
217
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
218 /// Indicates weather the cache has been built or not
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
219 bool isCached = false;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
220
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
221 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
222 This method does a binary search to find the line that contains the
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
223 given offset.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
224 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
225 uint lineOf(uint offset)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
226 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
227 size_t beg = 0,
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
228 end = line_starts.length,
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
229 mid = end >> 1;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
230
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
231 while( beg < end )
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
232 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
233 if( line_starts[mid] <= offset )
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
234 beg = mid + 1;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
235 else
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
236 end = mid;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
237 mid = beg + ( end - beg ) / 2;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
238 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
239 return mid;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
240 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
241
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
242 /**
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
243 Builds the cache data - always make sure this has been called before
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
244 calling lineOf.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
245 **/
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
246 void build(char[] data)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
247 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
248 // j starts at 1, because we need an additional place in the array
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
249 // to indicate that line 1 starts at index 0.
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
250 size_t j = 1;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
251 char* it = data.ptr, end = data.ptr + data.length;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
252 for (; it != end; ++it)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
253 if (*it == '\n')
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
254 ++j;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
255 // Allocate without initialization. Saves a bit of time
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
256 line_starts.length = j;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
257 line_starts[0] = 0;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
258
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
259 // Go over the data again, writing the line starts in our new array
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
260 j = 1;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
261 for (size_t i = 0; i < data.length; i++)
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
262 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
263 if (data[i] == '\n')
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
264 line_starts[j++] = i;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
265 else if (data[i] == '\r')
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
266 {
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
267 line_starts[j++] = i;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
268 i += cast(size_t)(data[i+1] == '\n');
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
269 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
270 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
271
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
272 isCached = true;
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
273 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
274 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
275 }
eb5b2c719a39 Major change to locations, tokens and expressions.
Anders Halager <halager@gmail.com>
parents:
diff changeset
276