DStress - a D compiler test suite

© 2004, 2005, 2006 Thomas Kühne <thomas [at] kuehne.cn>

^ Abstract

DStress is a collection of bug tracking and language conformance test cases for the D programming language. This document describes the use of the testing framework. The test summary can be found at http://dstress.kuehne.cn/www/dstress.html.

^ Index

1. Conditions
2. Symbols
3. Environment Settings
4. Testcase Settings
5. Testing
6. Report Generation
7. Sourcecode Metadata

^ Conditions

Passed

1. compiler return code zero
2. no assertion was thrown
3. return code of main is zero (applies only to the runtime tests)

Failed

1. compiler return code unequal zero
2. linker return code unequal zero
3. assertion was thrown
4. return code of main isn't zero (applies only to the runtime tests)

Error

1. compiler segfaulted
2. test case segfaulted
3. unexpected compiler failure without source indication

^ Symbols

The symbols are the same as those used by the DejaGnu test framework.

PASS

test case was expected to pass, and it did

XPASS

test case was expected to fail, but passed

FAIL

test case was expected to pass, but failed

XFAIL

test case was expected to fail, and it did

ERROR

compiler, linker or test case segfaulted

^ Environment Settings

DMD

the D compiler, e.g.: c:\dmd\bin\dmd.exe

use the wrapper gdmd for the GDC compiler

default value: dmd

GDB

a debugger with gdb-like commands, e.g.: /usr/local/bin/gdb

default value: gdb

note: only used on POSIX operating systems (e.g. BSD, Linux, Unix)

^ Testcase Settings

The settings starts with the end of one of the markers listed below and continues until the end of the line.

__DSTRESS_DFLAGS__

additional compiler arguments

default: empty

__DSTRESS_EFILE__

expected failing source file

default: if __DSTRESS_ELINE__ isn't empty current source file, else empty

__DSTRESS_ELINE__

expected failing source line

default: empty

__DSTRESS_TORTURE_BLOCK__

don't execute those torture tests which contain this plain string

default: unset

__DSTRESS_TORTURE_REQUIRE__

execute only those torture tests which contain this plain string

default: unset

__GDB_PATTERN__

expected regular expression in GDB's output

default: empty

__GDB_SCRIPT__

GDB command sequence

note: replace linebreaks by \n

default: empty

^ Testing

Posix Systems: AIX, Apple, BSD, Linux, Unix

Requirements

a C compiler

dstress.c

crashRun.c

a D compiler

GnuMake version 3.80 or later (optional)

Step-by-Step (without GnuMake)

1. compile dstress.c

   gcc -o dstress dstress.c

2. compile crashRun.c

   gcc -o crashRun crashRun.c

3. test D compiler:

   ./dstress compile compile/const_12.d

4. test linker and shell:

   ./dstress run run/main_02.d

5. start testing:

   ./dstress [compile|nocompile|run|norun] testfile.d 2>> log.txt >> results.txt

Step-by-Step (with GnuMake)

1. clean-up:

   make distclean

2. start testing:

   make all > results

Windows Systems

Requirements

a C compiler

dstress.c

a D compiler

Restrictions

no eternal-loop protection

no proper Access Violation detection

Access Violations might cause pop-ups requiring human interaction

Step-by-Step

1. make sure you understand the restrictions
2. open dstress.c, search for #error comment me out and follow the instruction
3. compile dstress.c

   dmc -o dstress dstress.c

4. test D compiler:

   dstress compile compile\const_12.d

5. test linker and shell:

   dstress run run\main_02.d

6. start testing:

   dstress [compile|nocompile|run|norun] testfile.d 2>> log.txt >> results.txt

^ Sourcecode Metadata

Metadata starts with one of the markers listed below and continues until the end of the line.

@author@

the bug/testcase reporter

common format: Name <email@host>

@date@

first report date

common format: year-month-day

@uri@

the message id of the email/newsgroup message or the webpage

common format: news:msg-id@host or http://host/page

@WARNING@

marks testcases with external dependencies or architecture/compiler specific data

common format: descriptive text