|
639
|
1 <html>
|
|
|
2 <head>
|
|
|
3 <meta http-equiv="content-type" content="text/html; charset=us-ascii" />
|
|
|
4 <meta http-equiv="content-language" content="en" />
|
|
|
5 <meta http-equiv="pics-label" content='(pics-1.1 "http://www.icra.org/ratingsv02.html" l gen true for "http://kuehne.cn")' />
|
|
|
6 <meta http-equiv="PICS-Label" content='(PICS-1.1 "http://www.classify.org/safesurf/" L gen true for "http://dstress.kuehne.cn" r (SS~~000 1))' />
|
|
|
7 <meta http-equiv="PICS-Label" content='(PICS-1.1 "http://vancouver-webpages.com/VWP1.0/" l gen true comment "VWP1.0" by "webmastro@kuehne.cn" on "2004.09.21T01:30-0700" for "dstress.kuehne.cn" r (P 0 S 0 V 0 Com 0 Tol 0 Env 0 SF 0 Edu 0 Can 0 MC 0 Gam -1 ))' />
|
|
|
8 <meta http-equiv="Content-Script-Type" content="text/javascript" />
|
|
|
9 <meta http-equiv="Content-Style-Type" content="text/css" />
|
|
|
10 <title>DStress - D compiler test suite</title>
|
|
|
11 <meta name="DC.Identifier" content="http://dstress.kuehne.cn/readme.html" />
|
|
|
12 <link rel="stylesheet" type="text/css" href="./www/formate.css" />
|
|
|
13 <meta name="distribution" content="Global" />
|
|
|
14 <meta name="DC.Format" content="text/html" />
|
|
|
15 <meta name="DC.Type" content="Software" />
|
|
|
16 <meta name="generator" content="Thomas Kuehne <webmastro@kuehne.cn>" />
|
|
|
17 <meta name="author" content="Thomas Kuehne <webmastro@kuehne.cn>" />
|
|
|
18 <meta name="DC.Creator" content="Thomas Kuehne <webmastro@kuehne.cn>" />
|
|
|
19 <meta name="DC.Publisher" content="Thomas Kuehne <webmastro@kuehne.cn>" />
|
|
|
20 <meta name="robots" content="index,follow" />
|
|
|
21 <meta name="DOCUMENT-STATE" content="STATIC" />
|
|
|
22 <meta name="DC.Language" content="en" />
|
|
|
23 <meta name="date" content="2005-08-14" />
|
|
|
24 <meta name="DC.Date" content="2005-08-14" />
|
|
|
25 <link rel="Report" href="./www/dstress.html" title="Report" />
|
|
|
26 <link rel="Results" href="./www/results.html"/>
|
|
|
27 <link rel="copyright" href="./license.txt"/>
|
|
|
28 <link rel="author" href="./authors.txt">
|
|
|
29 </head>
|
|
|
30 <body>
|
|
|
31
|
|
|
32 <center><h2><a id="top" name="top"></a>DStress - a D compiler test suite</h2><p>© 2004, 2005 Thomas Kühne <thomas [at] kuehne.cn></p></center>
|
|
|
33
|
|
|
34
|
|
|
35 <h3><a name="abstract" id="abstract" href="#top">^</a> Abstract</h3>
|
|
|
36 <blockquote>
|
|
|
37 DStress is a collection of bug tracking and language conformance test cases for the <a href="http://digitalmars.com/d/">D programming language</a>. This document describes the use of the testing framework. The test summary can be found at <a href="./www/dstress.html">http://dstress.kuehne.cn/www/dstress.html</a> and a complete listing at <a href="./www/results.html">http://dstress.kuehne.cn/www/results.html</a>.
|
|
|
38 </blockquote>
|
|
|
39
|
|
|
40 <h3><a name="index" id="index" href="#top">^</a> Index</h3>
|
|
|
41 <blockquote>
|
|
|
42 <ol>
|
|
|
43 <li><a href="#conditions">Conditions</a></li>
|
|
|
44 <li><a href="#symbols">Symbols</a></li>
|
|
|
45 <li><a href="#environment-settings">Environment Settings</a></li>
|
|
|
46 <li><a href="#testcase-settings">Testcase Settings</a></li>
|
|
|
47 <li><a href="#testing">Testing</a></li>
|
|
|
48 <li><a href="#report-generation">Report Generation</a></li>
|
|
|
49 <li><a href="#sourcecode-metadata">Sourcecode Metadata</a></li>
|
|
|
50 </ol>
|
|
|
51 </blockquote>
|
|
|
52
|
|
|
53 <h3><a name="conditions" id="conditions" href="#top">^</a> Conditions</h3>
|
|
|
54 <blockquote>
|
|
|
55 <dl>
|
|
|
56 <dt>Passed</dt>
|
|
|
57 <dd><ol>
|
|
|
58 <li>compiler return code zero</li>
|
|
|
59 <li>no assertion was thrown</li>
|
|
|
60 <li>return code of main is zero (applies only to the runtime tests)</li>
|
|
|
61 </ol></dd>
|
|
|
62 <dt>Failed</dt>
|
|
|
63 <dd><ol>
|
|
|
64 <li>compiler return code unequal zero</li>
|
|
|
65 <li>linker return code unequal zero</li>
|
|
|
66 <li>assertion was thrown</li>
|
|
|
67 <li>return code of main isn't zero (applies only to the runtime tests)</li>
|
|
|
68 </ol></dd>
|
|
|
69 <dt>Error</dt>
|
|
|
70 <dd><ol>
|
|
|
71 <li>compiler segfaulted</li>
|
|
|
72 <li>test case segfaulted</li>
|
|
|
73 <li>unexpected compiler failure without source indication</li>
|
|
|
74 </ol></dd>
|
|
|
75 </dl>
|
|
|
76 </blockquote>
|
|
|
77
|
|
|
78 <h3><a name="symbols" id="symbols" href="#top">^</a> Symbols</h3>
|
|
|
79 <blockquote>
|
|
|
80 The symbols are the same as those used by the <a href="http://www.gnu.org/software/dejagnu/">DejaGnu</a> test framework.
|
|
|
81 <dl>
|
|
|
82 <dt>PASS</dt>
|
|
|
83 <dd>test case was expected to pass, and it did</dd>
|
|
|
84 <dt>XPASS</dt>
|
|
|
85 <dd>test case was expected to fail, but passed</dd>
|
|
|
86 <dt>FAIL</dt>
|
|
|
87 <dd>test case was expected to pass, but failed</dd>
|
|
|
88 <dt>XFAIL</dt>
|
|
|
89 <dd>test case was expected to fail, and it did</dd>
|
|
|
90 <dt>ERROR</dt>
|
|
|
91 <dd>compiler, linker or test case segfaulted</dd>
|
|
|
92 </dl>
|
|
|
93 </blockquote>
|
|
|
94
|
|
|
95 <h3><a name="environment-settings" id="environment-settings" href="#top">^</a> Environment Settings</h3>
|
|
|
96 <blockquote>
|
|
|
97 <dl>
|
|
|
98 <dd>
|
|
|
99 <dt>DMD / dmd</dt>
|
|
|
100 <dd>the D compiler, e.g.: <i>c:\dmd\bin\dmd.exe</i></dd>
|
|
|
101 <dd>use the wrapper <i>gdmd</i> for the GDC compiler</dd>
|
|
|
102 <dd>default value: <i>dmd</i></dd>
|
|
|
103 <dt>GDB / gdb</dt>
|
|
|
104 <dd>a debugger with gdb-like commands, e.g.: <i>/usr/local/bin/gdb</i></dd>
|
|
|
105 <dd>default value: <i>gdb</i></dd>
|
|
|
106 </dd>
|
|
|
107 </dl>
|
|
|
108 </blockquote>
|
|
|
109
|
|
|
110 <h3><a name="testcase-settings" id="testcase-settings" href="#top">^</a> Testcase Settings</h3>
|
|
|
111 <blockquote>
|
|
|
112 The settings starts with the end of one of the markers listed below and continues until the end of the line.
|
|
|
113 <dl>
|
|
|
114 <dt>__DSTRESS_DFLAGS__</dt>
|
|
|
115 <dd>additional compiler arguments</dd>
|
|
|
116 <dd>default: <i>empty</i></dd>
|
|
|
117 <dt>__DSTRESS_EFILE__</dd>
|
|
|
118 <dd>expected failing source file</dd>
|
|
|
119 <dd>default: if __DSTRESS_ELINE__ isn't empty <i>current source file</i>, else <i>empty</i></dd>
|
|
|
120 <dt>__DSTRESS_ELINE__</dt>
|
|
|
121 <dd>expected failing source line</dd>
|
|
|
122 <dd>default: <i>empty</i></dd>
|
|
|
123 <dt>__GDB_PATTERN__</dd>
|
|
|
124 <dd>expected regular expression in GDB's output</dd>
|
|
|
125 <dd>default: <i>empty</i></dd>
|
|
|
126 <dt>__GDB_SCRIPT__</dd>
|
|
|
127 <dd>GDB command sequence</dd>
|
|
|
128 <dd>note: replace linebreaks by <i>\n</i></dd>
|
|
|
129 <dd>default: <i>empty</i></dd>
|
|
|
130 </dl>
|
|
|
131 </blockquote>
|
|
|
132
|
|
|
133 <h3><a name="testing" id="testing" href="#top">^</a> Testing</h3>
|
|
|
134 <blockquote>
|
|
|
135 Testcases that are newer than the result file are ignored.
|
|
|
136 <dl>
|
|
|
137 <dt>Posix Systems: AIX, Apple, BSD, Linux, Unix</dt>
|
|
|
138 <dd>
|
|
|
139 <dl>
|
|
|
140 <dt>Requirements</dt>
|
|
|
141 <dd>a C compiler</dd>
|
|
|
142 <dd><a href="./dstress.c">dstress.c</a></dd>
|
|
|
143 <dd><a href="./crashRun.c">crashRun.c</a></dd>
|
|
|
144 <dd>a D compiler</dd>
|
|
|
145 <dd>GnuMake version 3.80 or later (optional)</dd>
|
|
|
146 <dt>Step-by-Step (without GnuMake)</dt>
|
|
|
147 <dd>
|
|
|
148 <ol>
|
|
|
149 <li>compile <i>dstress.c</i><blockquote><pre><span class="cmd">gcc -o dstress dstress.c</span></pre></blockquote></li>
|
|
|
150 <li>compile <i>crashRun.c</i><blockquote><pre><span class="cmd">gcc -o crashRun crashRun.c</span></pre></blockquote></li>
|
|
|
151 <li>test D compiler:<blockquote><pre><span class="cmd">./dstress compile compile/const_12.d</span></pre></blockquote></li>
|
|
|
152 <li>test linker and shell:<blockquote><pre><span class="cmd">./dstress run run/main_02.d</span></pre></blockquote></li>
|
|
|
153 <li>start testing:<blockquote><pre><span class="cmd">./dstress [compile|nocompile|run|norun] testfile.d 2>> log.txt >> results.txt</span></pre></blockquote></li>
|
|
|
154 </ol>
|
|
|
155 </dd>
|
|
|
156 <dt>Step-by-Step (with GnuMake)</dt>
|
|
|
157 <dd>
|
|
|
158 <ol>
|
|
|
159 <li>clean-up:<blockquote><pre><span class="cmd">make distclean</span></pre></blockquote></li>
|
|
|
160 <li>start testing:<blockquote><pre><span class="cmd">make all > results</span></pre></blockquote></li>
|
|
|
161 </ol>
|
|
|
162 </dd>
|
|
|
163 </dl>
|
|
|
164 </dd>
|
|
|
165 <dt>Windows Systems</dt>
|
|
|
166 <dd>
|
|
|
167 <dl>
|
|
|
168 <dt>Requirements</dt>
|
|
|
169 <dd>a C compiler</dd>
|
|
|
170 <dd><a href="./dstress.c">dstress.c</a></dd>
|
|
|
171 <dd>a D compiler</dd>
|
|
|
172 <dt><a name="windows-restrictions" id="windows-restrictions"></a>Restrictions</dt>
|
|
|
173 <dd>no eternal-loop protection</dd>
|
|
|
174 <dd>no proper Access Violation detection</dd>
|
|
|
175 <dd>Access Violations might cause pop-ups requiring human interaction</dd>
|
|
|
176 <dt>Step-by-Step</dt>
|
|
|
177 <dd>
|
|
|
178 <ol>
|
|
|
179 <li>make sure you understand the <a href="#windows-restrictions"> restrictions</a></li>
|
|
|
180 <li>open dstress.c, search for <i>#error comment me out</i> and follow the instruction</li>
|
|
|
181 <li>compile <i>dstress.c</i><blockquote><pre><span class="cmd">dmc -o dstress dstress.c</span></pre></blockquote></li>
|
|
|
182 <li>test D compiler:<blockquote><pre><span class="cmd">dstress compile compile\const_12.d</span></pre></blockquote></li>
|
|
|
183 <li>test linker and shell:<blockquote><pre><span class="cmd">dstress run run\main_02.d</span></pre></blockquote></li>
|
|
|
184 <li>start testing:<blockquote><pre><span class="cmd">dstress [compile|nocompile|run|norun] testfile.d 2>> log.txt >> results.txt</span></pre></blockquote></li>
|
|
|
185 </ol>
|
|
|
186 </dd>
|
|
|
187 </dd>
|
|
|
188 </dl>
|
|
|
189 </blockquote>
|
|
|
190
|
|
|
191 <h3><a name="report-generation" id="report-generation" href="#top">^</a> Report Generation</h3>
|
|
|
192 <blockquote>
|
|
|
193 <dl>
|
|
|
194 <dt>Requirements</dt>
|
|
|
195 <dd>
|
|
|
196 <ul>
|
|
|
197 <li>a Java compiler</li>
|
|
|
198 <li><a href="./www/GenReport.java">GenReport.java</a></li>
|
|
|
199 <li>at least one logfile</li>
|
|
|
200 </ul>
|
|
|
201 </dd>
|
|
|
202 <dt>Step-by-Step</dt>
|
|
|
203 <dd>
|
|
|
204 <ol>
|
|
|
205 <li>compile the reporter class<blockquote><pre><span class="cmd">javac GenReporter.java</span></pre></blockquote></li>
|
|
|
206 <li>feed the logfile(s) to the reporter class<blockquote><pre><span class="cmd">java cn.kuehne.dmd.dstress.GenReport results1 results2 > report.raw</span></pre></blockquote></li>
|
|
|
207 <li>sort and insert <i>report.raw</i> with apropiate HTML tags into a HTML file</li>
|
|
|
208 </ol>
|
|
|
209 </dd>
|
|
|
210 <dt>Notes</dt>
|
|
|
211 <dd>
|
|
|
212 <ul>
|
|
|
213 <li>the output is unsorted</li>
|
|
|
214 <li>one or more result-files can be feed to the reporter class</li>
|
|
|
215 <li>lines containing statistic data are marked by <i><!-- summary --></i></li>
|
|
|
216 <li>use <i>...resultfile</i> instead of <i>resultfile</i> to mark all FAIL, XPASS, ERROR testcases of this logfile by <i><!-- P! --></i></li>
|
|
|
217 <li>more than one resultfile can be marked</li>
|
|
|
218 </ul>
|
|
|
219 </dd>
|
|
|
220 </dl>
|
|
|
221 </blockquote>
|
|
|
222
|
|
|
223 <h3><a name="sourcecode-metadata" id="sourcecode-metadata" href="#top">^</a> Sourcecode Metadata</h3>
|
|
|
224 <blockquote>
|
|
|
225 Metadata starts with one of the markers listed below and continues until the end of the line.
|
|
|
226 <dl>
|
|
|
227 <dt>@author@</dt>
|
|
|
228 <dd>the bug/testcase reporter</dd>
|
|
|
229 <dd>common format: <i>Name <email@host></i></dd>
|
|
|
230 <dt>@date@</dt>
|
|
|
231 <dd>first report date</dd>
|
|
|
232 <dd>common format: <i>year-month-day</i>
|
|
|
233 <dt>@uri@</dt>
|
|
|
234 <dd>the message id of the email/newsgroup message or the webpage</dd>
|
|
|
235 <dd>common format: <i>news:msg-id@host</i> or <i>http://host/page</i></dd>
|
|
|
236 <dt>@WARNING@</dt>
|
|
|
237 <dd>marks testcases with external dependencies or architecture/compiler specific data</dd>
|
|
|
238 <dd>common format: <i>descriptive text</i></dd>
|
|
|
239 </dl>
|
|
|
240 </blockquote>
|
|
|
241
|
|
|
242 <div><br /><br /><hr /><a href='http://dstress.kuehne.cn/www/dstress.html'>http://dstress.kuehne.cn/readme.html</a>; 2005-08-21; <a href="mailto:webmastro@kuehne.cn">Webmaster</a></div>
|
|
|
243 </body>
|
|
|
244 </html> |
