# HG changeset patch # User Aziz K?ksal # Date 1205017939 -3600 # Node ID bcb74c9b895cff5e756ec4d703197c0dc634b8be # Parent a3fab8b74a7d588df31d1f28207072dd6020864c Moved out files in the trunk folder to the root. diff -r a3fab8b74a7d -r bcb74c9b895c AUTHORS --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/AUTHORS Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,13 @@ +Founder: + Aziz Köksal + "All rights to the code I've written will pass over to Jari-Matti Mäkelä, + in case I catch the bus or the bus catches me. + My death will probably not be made known anywhere on the internet, + therefore my testament will become effective if I don't + show any signs of life for 6 months on the internet. + Within this time limit I may always revoke or edit this testament, + by committing to the hg repository at http://hg.sharesource.org/dil/ + Only the latest revision of this file is to be considered valid." + - Aziz Köksal +Contributors: + Jari-Matti Mäkelä diff -r a3fab8b74a7d -r bcb74c9b895c COPYING --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/COPYING Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff -r a3fab8b74a7d -r bcb74c9b895c README --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/README Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,32 @@ +dil +=== +Copyright (c) 2008 by Aziz Köksal +This program is free software, licensed under the GPL3. +Please, read the license file, COPYING, for further information. + +Description +=========== +This software is a compiler written in D for the D programming language. + +How To Compile dil +================== +In order to compile dil you need to have: + *) DMD 1.028 (http://www.digitalmars.com/d/1.0/changelog.html) + *) Tango 0.99.5 (http://dsource.org/projects/tango/) + *) DSSS 0.71 (http://dsource.org/projects/dsss/) + +If you can't compile dil because you have a newer version of these programs +then please report the problem to me (see Bugs section.) + +Before you run dil, make sure that the executable can find config.d and +lang_en.d (which are located in trunk/src/.) +The language can be configured in config.d. + +Bugs And Patches +================ +"errare humanum est, ignoscere divinum" - to err is human, to forgive divine. + - Cicero + +Users can report problems with this software or submit patches by: + *) contacting me: aziz.koeksal@gmail.com + *) filing a bug report here: http://code.google.com/p/dil/issues/list diff -r a3fab8b74a7d -r bcb74c9b895c dsss.conf --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/dsss.conf Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,10 @@ +name = dil +version = 0.1 +[src/main.d] +type = binary +target = dil +version(GNU) { + buildflags = -Isrc/ -Ldsss_objs/G/cmd.DDoc.o -Ldsss_objs/G/cmd.DDocXML.o +} else { + buildflags = -Isrc/ +} diff -r a3fab8b74a7d -r bcb74c9b895c i18n/build.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/i18n/build.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +# -*- coding: utf-8 -*- +# Author: Aziz Köksal +# License: GPL2 diff -r a3fab8b74a7d -r bcb74c9b895c i18n/de.cat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/i18n/de.cat Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,86 @@ +Authors: +- {EMail: aziz.koeksal@gmail.com, Name: Aziz Köksal} +LangCode: de +License: GPL3 +Messages: +- {Annot: '', ID: 0, LastEd: 0, Text: 'illegales Zeichen gefunden: ''{0}'''} +- {Annot: '', ID: 1, LastEd: 0, Text: ungültiges Unicodezeichen.} +- {Annot: '', ID: 2, LastEd: 0, Text: ungültige UTF-8-Sequenz.} +- {Annot: '', ID: 3, LastEd: 0, Text: unterminiertes Zeichenliteral.} +- {Annot: '', ID: 4, LastEd: 0, Text: leeres Zeichenliteral.} +- {Annot: '', ID: 5, LastEd: 0, Text: erwartete 'line' nach '#'.} +- {Annot: '', ID: 6, LastEd: 0, Text: 'Ganzzahl nach #line erwartet.'} +- {Annot: '', ID: 7, LastEd: 0, Text: erwartete Dateispezifikation (z.B. "pfad\zur\datei".)} +- {Annot: '', ID: 8, LastEd: 0, Text: unterminierte Dateispezifikation (filespec.)} +- {Annot: '', ID: 9, LastEd: 0, Text: ein Special Token muss mit einem Zeilenumbruch + abgeschlossen werden.} +- {Annot: '', ID: 10, LastEd: 0, Text: unterminiertes Zeichenkettenliteral.} +- {Annot: '', ID: 11, LastEd: 0, Text: 'Nicht-Hexzeichen ''{0}'' in Hexzeichenkette + gefunden.'} +- {Annot: '', ID: 12, LastEd: 0, Text: ungerade Anzahl von Hexziffern in Hexzeichenkette.} +- {Annot: '', ID: 13, LastEd: 0, Text: unterminierte Hexzeichenkette.} +- {Annot: '', ID: 14, LastEd: 0, Text: unterminierter Blockkommentar (/* */).} +- {Annot: '', ID: 15, LastEd: 0, Text: unterminierter verschachtelter Kommentar (/+ + +/).} +- {Annot: '', ID: 16, LastEd: 0, Text: unterminierte rohe Zeichenkette.} +- {Annot: '', ID: 17, LastEd: 0, Text: unterminierte Backquote-Zeichenkette.} +- {Annot: '', ID: 18, LastEd: 0, Text: 'undefinierte Escapesequenz ''{0}'' gefunden.'} +- {Annot: '', ID: 19, LastEd: 0, Text: 'ungültige Unicode-Escapesequenz ''{0}'' gefunden.'} +- {Annot: '', ID: 20, LastEd: 0, Text: unzureichende Anzahl von Hexziffern in Escapesequenz.} +- {Annot: '', ID: 21, LastEd: 0, Text: 'undefinierte HTML-Entität ''{0}'''} +- {Annot: '', ID: 22, LastEd: 0, Text: 'unterminierte HTML-Entität ''{0}''.'} +- {Annot: '', ID: 23, LastEd: 0, Text: HTML-Entitäten müssen mit einem Buchstaben + beginnen.} +- {Annot: '', ID: 24, LastEd: 0, Text: Dezimalzahl überläuft im Vorzeichenbit.} +- {Annot: '', ID: 25, LastEd: 0, Text: Überlauf in Dezimalzahl.} +- {Annot: '', ID: 26, LastEd: 0, Text: Überlauf in Hexadezimalzahl.} +- {Annot: '', ID: 27, LastEd: 0, Text: Überlauf in Binärzahl.} +- {Annot: '', ID: 28, LastEd: 0, Text: Überlauf in Oktalzahl.} +- {Annot: '', ID: 29, LastEd: 0, Text: Überlauf in Fließkommazahl.} +- {Annot: '', ID: 30, LastEd: 0, Text: die Ziffern 8 und 9 sind in Oktalzahlen unzulässig.} +- {Annot: '', ID: 31, LastEd: 0, Text: ungültige Hexzahl; mindestens eine Hexziffer + erforderlich.} +- {Annot: '', ID: 32, LastEd: 0, Text: ungültige Binärzahl; mindestens eine Binärziffer + erforderlich.} +- {Annot: '', ID: 33, LastEd: 0, Text: der Exponent einer hexadezimalen Fließkommazahl + ist erforderlich.} +- {Annot: '', ID: 34, LastEd: 0, Text: Hexadezimal-Exponenten müssen mit einer Dezimalziffer + anfangen.} +- {Annot: '', ID: 35, LastEd: 0, Text: Exponenten müssen mit einer Dezimalziffer anfangen.} +- {Annot: '', ID: 36, LastEd: 0, Text: 'erwartete ''{0}'', fand aber ''{1}''.'} +- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' ist redundant.'} +- {Annot: '', ID: 38, LastEd: 0, Text: Template-Tupel-Parameter dürfen nur am Ende + auftreten.} +- {Annot: '', ID: 39, LastEd: 0, Text: der 'in'-Vertrag der Funktion wurde bereits + geparsed.} +- {Annot: '', ID: 40, LastEd: 0, Text: der 'out'-Vertrag der Funktion wurde bereits + geparsed.} +- {Annot: '', ID: 41, LastEd: 0, Text: es wurde kein Verbindungstyp angegeben.} +- {Annot: '', ID: 42, LastEd: 0, Text: 'unbekannter Verbindungstyp ''{0}''; gültig + sind C, C++, D, Windows, Pascal und System.'} +- {Annot: '', ID: 43, LastEd: 0, Text: 'erwartete eine oder mehrere Basisklassen, + nicht ''{0}''.'} +- {Annot: '', ID: 44, LastEd: 0, Text: Basisklassen sind in Vorwärtsdeklarationen + nicht erlaubt.} +- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} + + Copyright (c) 2007, Aziz Köksal. Lizensiert unter der GPL3. + + + Befehle: + + {1} + + Geben Sie ''dil help '' ein, um mehr Hilfe zu einem bestimmten Befehl + zu + + erhalten. + + + Kompiliert mit {2} v{3} am {4}.'} +- {Annot: '', ID: 46, LastEd: 0, Text: "Generiere ein XML- oder HTML-Dokument aus\ + \ einer D-Quelltextdatei.\nVerwendung:\n dil gen datei.d [Optionen]\n\nOptionen:\n\ + \ --syntax : generiere Elemente für den Syntaxbaum\n --xml \ + \ : verwende XML-Format (voreingestellt)\n --html : verwende HTML-Format\n\ + \nBeispiel:\n dil gen Parser.d --html --syntax > Parser.html"} +- {Annot: '', ID: 47, LastEd: 0, Text: ''} diff -r a3fab8b74a7d -r bcb74c9b895c i18n/dil.tproj --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/i18n/dil.tproj Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,54 @@ +BuildScript: build.py +CreationDate: '2007-10-12 09:41:17.868084' +LangFiles: [de.cat, tr.cat, fi.cat] +MsgIDs: +- {ID: 0, Name: IllegalCharacter, Order: 0} +- {ID: 1, Name: InvalidUnicodeCharacter, Order: 1} +- {ID: 2, Name: InvalidUTF8Sequence, Order: 2} +- {ID: 3, Name: UnterminatedCharacterLiteral, Order: 3} +- {ID: 4, Name: EmptyCharacterLiteral, Order: 4} +- {ID: 5, Name: ExpectedIdentifierSTLine, Order: 5} +- {ID: 6, Name: ExpectedIntegerAfterSTLine, Order: 6} +- {ID: 7, Name: ExpectedFilespec, Order: 7} +- {ID: 8, Name: UnterminatedFilespec, Order: 8} +- {ID: 9, Name: UnterminatedSpecialToken, Order: 9} +- {ID: 10, Name: UnterminatedString, Order: 10} +- {ID: 11, Name: NonHexCharInHexString, Order: 11} +- {ID: 12, Name: OddNumberOfDigitsInHexString, Order: 12} +- {ID: 13, Name: UnterminatedHexString, Order: 13} +- {ID: 14, Name: UnterminatedBlockComment, Order: 14} +- {ID: 15, Name: UnterminatedNestedComment, Order: 15} +- {ID: 16, Name: UnterminatedRawString, Order: 16} +- {ID: 17, Name: UnterminatedBackQuoteString, Order: 17} +- {ID: 18, Name: UndefinedEscapeSequence, Order: 18} +- {ID: 19, Name: InvalidUnicodeEscapeSequence, Order: 19} +- {ID: 20, Name: InsufficientHexDigits, Order: 20} +- {ID: 21, Name: UndefinedHTMLEntity, Order: 21} +- {ID: 22, Name: UnterminatedHTMLEntity, Order: 22} +- {ID: 23, Name: InvalidBeginHTMLEntity, Order: 23} +- {ID: 24, Name: OverflowDecimalSign, Order: 24} +- {ID: 25, Name: OverflowDecimalNumber, Order: 25} +- {ID: 26, Name: OverflowHexNumber, Order: 26} +- {ID: 27, Name: OverflowBinaryNumber, Order: 27} +- {ID: 28, Name: OverflowOctalNumber, Order: 28} +- {ID: 29, Name: OverflowFloatNumber, Order: 29} +- {ID: 30, Name: OctalNumberHasDecimals, Order: 30} +- {ID: 31, Name: NoDigitsInHexNumber, Order: 31} +- {ID: 32, Name: NoDigitsInBinNumber, Order: 32} +- {ID: 33, Name: HexFloatExponentRequired, Order: 33} +- {ID: 34, Name: HexFloatExpMustStartWithDigit, Order: 34} +- {ID: 35, Name: FloatExpMustStartWithDigit, Order: 35} +- {ID: 36, Name: ExpectedButFound, Order: 36} +- {ID: 37, Name: RedundantStorageClass, Order: 37} +- {ID: 38, Name: TemplateTupleParameter, Order: 38} +- {ID: 39, Name: InContract, Order: 39} +- {ID: 40, Name: OutContract, Order: 40} +- {ID: 41, Name: MissingLinkageType, Order: 41} +- {ID: 42, Name: UnrecognizedLinkageType, Order: 42} +- {ID: 43, Name: ExpectedBaseClasses, Order: 43} +- {ID: 44, Name: BaseClassInForwardDeclaration, Order: 44} +- {ID: 45, Name: HelpMain, Order: 45} +- {ID: 46, Name: HelpGenerate, Order: 46} +- {ID: 47, Name: HelpImportGraph, Order: 47} +Name: dil +SourceLangFile: en.cat diff -r a3fab8b74a7d -r bcb74c9b895c i18n/en.cat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/i18n/en.cat Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,90 @@ +Authors: +- {EMail: aziz.koeksal@gmail.com, Name: Aziz Köksal} +LangCode: en +License: GPL3 +Messages: +- {Annot: '', ID: 0, LastEd: 0, Text: 'illegal character found: ''{0}'''} +- {Annot: '', ID: 1, LastEd: 0, Text: invalid Unicode character.} +- {Annot: '', ID: 2, LastEd: 0, Text: invalid UTF-8 sequence.} +- {Annot: '', ID: 3, LastEd: 0, Text: unterminated character literal.} +- {Annot: '', ID: 4, LastEd: 0, Text: empty character literal.} +- {Annot: '', ID: 5, LastEd: 0, Text: expected 'line' after '#'.} +- {Annot: '', ID: 6, LastEd: 0, Text: 'integer expected after #line'} +- {Annot: '', ID: 7, LastEd: 0, Text: expected filespec string (e.g. "path\to\file".)} +- {Annot: '', ID: 8, LastEd: 0, Text: unterminated filespec string.} +- {Annot: '', ID: 9, LastEd: 0, Text: expected a terminating newline after special + token.} +- {Annot: '', ID: 10, LastEd: 0, Text: unterminated string literal.} +- {Annot: '', ID: 11, LastEd: 0, Text: 'non-hex character ''{0}'' found in hex string.'} +- {Annot: '', ID: 12, LastEd: 0, Text: odd number of hex digits in hex string.} +- {Annot: '', ID: 13, LastEd: 0, Text: unterminated hex string.} +- {Annot: '', ID: 14, LastEd: 0, Text: unterminated block comment (/* */).} +- {Annot: '', ID: 15, LastEd: 0, Text: unterminated nested comment (/+ +/).} +- {Annot: '', ID: 16, LastEd: 0, Text: unterminated raw string.} +- {Annot: '', ID: 17, LastEd: 0, Text: unterminated back quote string.} +- {Annot: '', ID: 18, LastEd: 0, Text: 'found undefined escape sequence ''{0}''.'} +- {Annot: '', ID: 19, LastEd: 0, Text: 'found invalid Unicode escape sequence ''{0}''.'} +- {Annot: '', ID: 20, LastEd: 0, Text: insufficient number of hex digits in escape + sequence.} +- {Annot: '', ID: 21, LastEd: 0, Text: 'undefined HTML entity ''{0}'''} +- {Annot: '', ID: 22, LastEd: 0, Text: 'unterminated HTML entity ''{0}''.'} +- {Annot: '', ID: 23, LastEd: 0, Text: HTML entities must begin with a letter.} +- {Annot: '', ID: 24, LastEd: 0, Text: decimal number overflows sign bit.} +- {Annot: '', ID: 25, LastEd: 0, Text: overflow in decimal number.} +- {Annot: '', ID: 26, LastEd: 0, Text: overflow in hexadecimal number.} +- {Annot: '', ID: 27, LastEd: 0, Text: overflow in binary number.} +- {Annot: '', ID: 28, LastEd: 0, Text: overflow in octal number.} +- {Annot: '', ID: 29, LastEd: 0, Text: overflow in float number.} +- {Annot: '', ID: 30, LastEd: 0, Text: digits 8 and 9 are not allowed in octal numbers.} +- {Annot: '', ID: 31, LastEd: 0, Text: invalid hex number; at least one hex digit + expected.} +- {Annot: '', ID: 32, LastEd: 0, Text: invalid binary number; at least one binary + digit expected.} +- {Annot: '', ID: 33, LastEd: 0, Text: the exponent of a hexadecimal float number + is required.} +- {Annot: '', ID: 34, LastEd: 0, Text: hexadecimal float exponents must start with + a digit.} +- {Annot: '', ID: 35, LastEd: 0, Text: exponents must start with a digit.} +- {Annot: '', ID: 36, LastEd: 0, Text: 'expected ''{0}'', but found ''{1}''.'} +- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' is redundant.'} +- {Annot: '', ID: 38, LastEd: 0, Text: template tuple parameters can only be last.} +- {Annot: '', ID: 39, LastEd: 0, Text: the functions 'in' contract was already parsed.} +- {Annot: '', ID: 40, LastEd: 0, Text: the functions 'out' contract was already parsed.} +- {Annot: '', ID: 41, LastEd: 0, Text: no linkage type was specified.} +- {Annot: '', ID: 42, LastEd: 0, Text: 'unrecognized linkage type ''{0}''; valid types + are C, C++, D, Windows, Pascal und System.'} +- {Annot: '', ID: 43, LastEd: 0, Text: 'expected one or more base classes, not ''{0}''.'} +- {Annot: '', ID: 44, LastEd: 0, Text: base classes are not allowed in forward declarations.} +- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} + + Copyright (c) 2007 by Aziz Köksal. Licensed under the GPL3. + + + Subcommands: + + {1} + + Type ''dil help '' for more help on a particular subcommand. + + + Compiled with {2} v{3} on {4}.'} +- {Annot: '', ID: 46, LastEd: 0, Text: "Generate an XML or HTML document from a D\ + \ source file.\nUsage:\n dil gen file.d [Options]\n\nOptions:\n --syntax \ + \ : generate tags for the syntax tree\n --xml : use XML format\ + \ (default)\n --html : use HTML format\n\nExample:\n dil gen Parser.d\ + \ --html --syntax > Parser.html"} +- {Annot: '', ID: 47, LastEd: 0, Text: "Parse a module and extract information from\ + \ the resulting module dependency graph.\nUsage:\n dil igraph file.d Format [Options]\n\ + \n The directory of file.d is implicitly added to the list of import paths.\n\ + \nFormat:\n --dot : generate a dot document\n Further options for\ + \ --dot:\n -gbp : Group modules by package names\n -gbf \ + \ : Group modules by full package name\n -hle : highlight cyclic\ + \ edges in the graph\n -hlv : highlight modules in cyclic relationship\n\ + \n --paths : print a list of paths to the modules imported by file.d\n\ + \ --list : print a list of the module names imported by file.d\n Options\ + \ common to --paths and --list:\n -lN : print N levels.\n -m \ + \ : mark modules in cyclic relationships with a star.\n\nOptions:\n\ + \ -Ipath : add 'path' to the list of import paths where modules are\n\ + \ looked for\n -rREGEXP : exclude modules whose names\ + \ match the regular expression\n REGEXP\n -i \ + \ : include unlocatable modules\n\nExample:\n dil igraph src/main.d"} diff -r a3fab8b74a7d -r bcb74c9b895c i18n/fi.cat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/i18n/fi.cat Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,76 @@ +Authors: +- {EMail: jmjm@iki.fi, Name: Jari-Matti Mäkelä} +LangCode: fi +License: GPL3 +Messages: +- {Annot: '', ID: 0, LastEd: 0, Text: ''} +- {Annot: '', ID: 1, LastEd: 0, Text: virheellinen Unicode-merkki.} +- {Annot: '', ID: 2, LastEd: 0, Text: virheellinen UTF-8-merkkijono.} +- {Annot: '', ID: 3, LastEd: 0, Text: päättämätön merkkiliteraali.} +- {Annot: '', ID: 4, LastEd: 0, Text: tyhjä merkkiliteraali.} +- {Annot: '', ID: 5, LastEd: 0, Text: 'odotettiin rivinumeroa ''#'':n jälkeen.'} +- {Annot: '', ID: 6, LastEd: 0, Text: 'odotettiin kokonaislukua #line:n jälkeen'} +- {Annot: '', ID: 7, LastEd: 0, Text: odotettiin tiedostomäärittelyn merkkijonoa (esim. + "polku\tiedostoon")} +- {Annot: '', ID: 8, LastEd: 0, Text: päättämätön tiedostomäärittely.} +- {Annot: '', ID: 9, LastEd: 0, Text: odotettiin päättävää rivinvaihtoa erikoismerkin + jälkeen.} +- {Annot: '', ID: 10, LastEd: 0, Text: päättämätön merkkijonoliteraali.} +- {Annot: '', ID: 11, LastEd: 0, Text: 'ei-heksamerkki ''{0}'' löytyi heksajonossa.'} +- {Annot: '', ID: 12, LastEd: 0, Text: pariton määrä heksanumeroita heksajonossa.} +- {Annot: '', ID: 13, LastEd: 0, Text: päättämätön heksajono.} +- {Annot: '', ID: 14, LastEd: 0, Text: päättämätön lohkokommentti (/* */).} +- {Annot: '', ID: 15, LastEd: 0, Text: päättämätön sisäkkäinen kommentti (/+ +/).} +- {Annot: '', ID: 16, LastEd: 0, Text: päättämätön raakamerkkijono.} +- {Annot: '', ID: 17, LastEd: 0, Text: päättämätön gravisaksenttimerkkijono.} +- {Annot: '', ID: 18, LastEd: 0, Text: löydettiin määrittelemätön escape-sekvenssi.} +- {Annot: '', ID: 19, LastEd: 0, Text: 'found invalid Unicode escape sequence ''{0}''.'} +- {Annot: '', ID: 20, LastEd: 0, Text: riittämätön määrä heksanumeroita escape-sekvenssissä.} +- {Annot: '', ID: 21, LastEd: 0, Text: 'määrittelemätön HTML-entiteetti ''{0}'''} +- {Annot: '', ID: 22, LastEd: 0, Text: päättämätön HTML-entiteetti.} +- {Annot: '', ID: 23, LastEd: 0, Text: HTML-entiteettien tulee alkaa kirjaimella.} +- {Annot: '', ID: 24, LastEd: 0, Text: desimaaliluku ylivuotaa etumerkin.} +- {Annot: '', ID: 25, LastEd: 0, Text: desimaaliluvun ylivuoto.} +- {Annot: '', ID: 26, LastEd: 0, Text: heksadesimaaliluvun ylivuoto.} +- {Annot: '', ID: 27, LastEd: 0, Text: binääriluvun ylivuoto.} +- {Annot: '', ID: 28, LastEd: 0, Text: oktaaliluvun ylivuoto.} +- {Annot: '', ID: 29, LastEd: 0, Text: liukuluvun ylivuoto.} +- {Annot: '', ID: 30, LastEd: 0, Text: numerot 8 ja 9 eivät ole sallittuja oktaaliluvuissa.} +- {Annot: '', ID: 31, LastEd: 0, Text: virheellinen heksaluku; odotettiin vähintään + yhtä heksanumeroa.} +- {Annot: '', ID: 32, LastEd: 0, Text: virheellinen binääriluku; odotettiin vähintään + yhtä binäärinumeroa.} +- {Annot: '', ID: 33, LastEd: 0, Text: heksadesimaalisen liukuluvun eksponentti vaaditaan.} +- {Annot: '', ID: 34, LastEd: 0, Text: heksadesimaalisen liukuluvun eksponentista + puuttui numeroita.} +- {Annot: '', ID: 35, LastEd: 0, Text: eksponenttien tulee alkaa numerolla.} +- {Annot: '', ID: 36, LastEd: 0, Text: 'odotettiin ''{0}'':a, mutta löydettiin ''{1}''.'} +- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' on redundantti.'} +- {Annot: '', ID: 38, LastEd: 0, Text: tupla voi esiintyä ainoastaan mallin viimeisenä + parametrina.} +- {Annot: '', ID: 39, LastEd: 0, Text: funktion alkuehto jäsennettiin jo.} +- {Annot: '', ID: 40, LastEd: 0, Text: funktion loppuehto jäsennettiin jo.} +- {Annot: '', ID: 41, LastEd: 0, Text: linkitystyyppiä ei määritelty.} +- {Annot: '', ID: 42, LastEd: 0, Text: 'tunnistamaton linkitystyyppi ''{0}''; sallittuja + tyyppejä ovat C, C++, D, Windows, Pascal ja System.'} +- {Annot: '', ID: 43, LastEd: 0, Text: 'expected one or more base classes, not ''{0}''.'} +- {Annot: '', ID: 44, LastEd: 0, Text: base classes are not allowed in forward declarations.} +- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} + + Copyright (c) 2007, Aziz Köksal. GPL3-lisensöity. + + + Alikomennot: + + {1} + + Lisäohjeita tietystä alitoiminnosta saa kirjoittamalla ''dil help ''. + + + Käännetty {2}:n versiolla {3} {4}.'} +- {Annot: '', ID: 46, LastEd: 0, Text: "Luo XML- tai HTML-dokumentti D-lähdekoodista.\n\ + Käyttö:\n dil gen tiedosto.d [Valinnat]\n\nValinnat:\n --syntax : luo\ + \ elementtejä syntaksipuun mukaisesti\n --xml : käytä XML-muotoa (oletus)\n\ + \ --html : käytä HTML-muotoa\n\nEsimerkki:\n dil gen Parser.d --html\ + \ --syntax > Parser.html"} +- {Annot: '', ID: 47, LastEd: 0, Text: ''} diff -r a3fab8b74a7d -r bcb74c9b895c i18n/tr.cat --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/i18n/tr.cat Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,73 @@ +Authors: +- {EMail: aziz.koeksal@gmail.com, Name: Aziz Köksal} +LangCode: tr +License: GPL3 +Messages: +- {Annot: '', ID: 0, LastEd: 0, Text: 'illegal karakter bulundu: ''{0}'''} +- {Annot: '', ID: 1, LastEd: 0, Text: geçersiz Unikod karakteri.} +- {Annot: '', ID: 2, LastEd: 0, Text: geçersiz UTF-8 serisi.} +- {Annot: '', ID: 3, LastEd: 0, Text: kapanmamış karakter sabiti.} +- {Annot: '', ID: 4, LastEd: 0, Text: boş karakter sabiti.} +- {Annot: '', ID: 5, LastEd: 0, Text: '''#'' karakter''den sonra ''line'' beklendi.'} +- {Annot: '', ID: 6, LastEd: 0, Text: '''#line''''den sonra rakam beklendi.'} +- {Annot: '', ID: 7, LastEd: 0, Text: filespec dizgisi beklendi (e.g. "yol\dosya".)} +- {Annot: '', ID: 8, LastEd: 0, Text: kapanmamış filespec dizgisi.} +- {Annot: '', ID: 9, LastEd: 0, Text: özel belirtici'den (special token) sonra yeni + bir satır beklendi.} +- {Annot: '', ID: 10, LastEd: 0, Text: kapanmamış çift tırnak dizgisi.} +- {Annot: '', ID: 11, LastEd: 0, Text: 'heks sayı olmayan karakter ''{0}'' heks dizgisi + içinde bulundu.'} +- {Annot: '', ID: 12, LastEd: 0, Text: heks dizginin içindeki sayılar çifter çifter + olmalıdır.} +- {Annot: '', ID: 13, LastEd: 0, Text: kapanmamış heks dizgisi.} +- {Annot: '', ID: 14, LastEd: 0, Text: kapanmamış blok açıklaması (/* */).} +- {Annot: '', ID: 15, LastEd: 0, Text: kapanmamış iç içe koyulabilen açıklaması (/+ + +/).} +- {Annot: '', ID: 16, LastEd: 0, Text: kapanmamış çiğ dizgisi.} +- {Annot: '', ID: 17, LastEd: 0, Text: kapanmamış ters tırnak dizgisi.} +- {Annot: '', ID: 18, LastEd: 0, Text: 'tanımlanmamış çıkış serisi ''{0}'' bulundu.'} +- {Annot: '', ID: 19, LastEd: 0, Text: 'geçersiz Unikod çıkış serisi ''{0}'' bulundu.'} +- {Annot: '', ID: 20, LastEd: 0, Text: heksadesimal çıkış serisi sayıları yeterli + değil.} +- {Annot: '', ID: 21, LastEd: 0, Text: 'tanımlanmamış HTML varlık ''{0}'''} +- {Annot: '', ID: 22, LastEd: 0, Text: 'kapanmamış HTML varlık ''{0}''.'} +- {Annot: '', ID: 23, LastEd: 0, Text: HTML varlık bir harf ile başlamalı.} +- {Annot: '', ID: 24, LastEd: 0, Text: desimal rakamın bit işareti taşdı.} +- {Annot: '', ID: 25, LastEd: 0, Text: desimal rakam taşması.} +- {Annot: '', ID: 26, LastEd: 0, Text: heksadesimal rakam taşması.} +- {Annot: '', ID: 27, LastEd: 0, Text: binari rakam taşması.} +- {Annot: '', ID: 28, LastEd: 0, Text: oktal rakam taşması.} +- {Annot: '', ID: 29, LastEd: 0, Text: float rakam taşması.} +- {Annot: '', ID: 30, LastEd: 0, Text: 8 ve 9 sayılar oktal rakamlar'da geçersizdir.} +- {Annot: '', ID: 31, LastEd: 0, Text: geçersiz heks rakam; minimum bir heks sayı + gereklidir.} +- {Annot: '', ID: 32, LastEd: 0, Text: geçersiz binari rakam; minimum bir binari sayı + gereklidir.} +- {Annot: '', ID: 33, LastEd: 0, Text: bir heksadesimal float rakamın üsü gereklidir.} +- {Annot: '', ID: 34, LastEd: 0, Text: heksadesimal float üsler desimal sayı ile başlamalı.} +- {Annot: '', ID: 35, LastEd: 0, Text: üsler desimal sayı ile başlamalı.} +- {Annot: '', ID: 36, LastEd: 0, Text: '''{0}'' beklendi, ama ''{1}'' bulundu.'} +- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' lüzumsuz.'} +- {Annot: '', ID: 38, LastEd: 0, Text: şablon tuple parametre son sırada olmalı.} +- {Annot: '', ID: 39, LastEd: 0, Text: fonksiyonun 'in' kontratı daha önceden ayrıştırılmış.} +- {Annot: '', ID: 40, LastEd: 0, Text: fonksiyonun 'out' kontratı daha önceden ayrıştırılmış.} +- {Annot: '', ID: 41, LastEd: 0, Text: bağlantı tüp (linkage type) belirtilmedi.} +- {Annot: '', ID: 42, LastEd: 0, Text: 'bilinmeyen bağlantı tüpü (linkage type) ''{0}''; + geçerli olanlar C, C++, D, Windows, Pascal ve System.'} +- {Annot: '', ID: 43, LastEd: 0, Text: ''} +- {Annot: '', ID: 44, LastEd: 0, Text: ''} +- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} + + Copyright (c) 2007, Aziz Köksal. Lisans GPL3. + + + Komutlar: {1} Belirli komut''a yardım edinmek için ''dil help '' yazınız. + + + Bu yazılım {2} v{3} ile {4} tarihinde derletilmiş.'} +- {Annot: '', ID: 46, LastEd: 0, Text: "Bir D kaynak kodundan XML veya HTML dosyası\ + \ oluştur.\nKullanım:\n dil gen dosya.d [Seçenekler]\n\nSeçenekler:\n --syntax\ + \ : söz dizimi için etiketler yazdır\n --xml : XML biçimi\ + \ kullan (varsayılır)\n --html : HTML biçimi kullan\n\nÖrnek:\n dil\ + \ gen Parser.d --html --syntax > Parser.html"} +- {Annot: '', ID: 47, LastEd: 0, Text: ''} diff -r a3fab8b74a7d -r bcb74c9b895c release.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/release.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,8 @@ +# -*- coding: utf-8 -*- +# Author: Aziz Köksal + +# TODO: port release.sh to Python + +# TODO: write subcommand that creates a Makefile +def writeMakefile(): + pass diff -r a3fab8b74a7d -r bcb74c9b895c release.sh --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/release.sh Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,96 @@ +#!/bin/bash + +if [[ $1 != [0-9].[0-9][0-9][0-9] ]]; then + echo Wrong version format. Expected: d.ddd + exit; +fi + +BUILD="./build" +DIR="dil.$1" +DEST="$BUILD/$DIR" +FRESH_REPOS="$BUILD/fresh_repos" + +# Create build directory if it doesn't exist. +[ ! -e $BUILD ] && mkdir $BUILD + +# Convert Unix newlines to Windows newlines +# function unix2win +# { +# sed {s/$/\\r/} $* +# } + +# We need dil to get a list of all modules to be compiled. +if [ ! -s ./dil ]; then + dsss build -full &> /dev/null +fi + +if [ ! -s ./dil ]; then + echo "Couldn't build DIL. Can't get list of modules to be built." + exit; +fi + +# Used by doc generation and winbuild function. +SRC_FILES=`./dil igraph src/main.d --paths` + +# Recreate destination directory. +rm -rf $DEST +mkdir -p $DEST/{bin,doc/htmlsrc,src} + +# Create documentation. +./dil ddoc $DEST/doc/ -v src/macros_dil.ddoc -version=DDoc src/config.d $SRC_FILES +# Generate syntax highlighted HTML files. +HTMLSRC="$DEST/doc/htmlsrc" +for filepath in $SRC_FILES; +do + htmlfile=`echo $filepath | sed -e 's@^src/@@' -e 's@/@.@g' -e 's@.d$@@'`.html + echo "FILE: $filepath > $HTMLSRC/$htmlfile"; + ./dil gen --lines --syntax --html $filepath > "$HTMLSRC/$htmlfile"; +done + +# Linux Debug +echo "***** Building Linux binaries *****" +dsss build -clean -full -version=D2 +cp dil $DEST/bin/dil2_d +dsss build -clean -full +cp dil $DEST/bin/dil_d +# Linux Release +dsss build -clean -full -release -O -inline -version=D2 +cp dil $DEST/bin/dil2 +dsss build -clean -full -release -O -inline +cp dil $DEST/bin/dil + +if [ -s ~/bin/dmd.exe ]; then + echo "***** Building Windows binaries *****" + function winbuild + { # obj dir is winobj. -op = don't strip paths from obj files. + wine ~/bin/dmd.exe -odwinobj -op -ofdil $* $SRC_FILES + } + # Windows Debug + winbuild -version=D2 + cp dil.exe $DEST/bin/dil2_d.exe + winbuild + cp dil.exe $DEST/bin/dil_d.exe + # Windows Release + winbuild -release -O -inline -version=D2 + cp dil.exe $DEST/bin/dil2.exe + winbuild -release -O -inline + cp dil.exe $DEST/bin/dil.exe +fi + +# Copy source and other files. +rm -rf $FRESH_REPOS +hg archive -r tip -t files $FRESH_REPOS +cp -r $FRESH_REPOS/trunk/* $DEST + +cp $FRESH_REPOS/trunk/src/config.d $DEST/bin/ +cp $FRESH_REPOS/trunk/src/lang_*.d $DEST/bin/ +cp $FRESH_REPOS/trunk/src/*_map.d $DEST/bin/ +cp $FRESH_REPOS/trunk/src/*.css $DEST/bin/ +cp $FRESH_REPOS/trunk/src/predefined.ddoc $DEST/bin/ +cp $FRESH_REPOS/trunk/src/html.css $HTMLSRC + +# Build archives +# tar.gz doesn't compress well +tar --owner root --group root -czf $DEST.tar.gz $DEST +tar --owner root --group root --bzip2 -cf $DEST.tar.bz2 $DEST +zip -q -9 -r $DEST.zip $DEST diff -r a3fab8b74a7d -r bcb74c9b895c src/Settings.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/Settings.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,29 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module Settings; +import common; + +/// Global application settings. +struct GlobalSettings +{ +static: + /// Predefined version identifiers. + string[] versionIds; + /// Path to the language file. + string langFile = "lang_en.d"; + /// Language code of loaded messages catalogue. + string langCode = "en"; + /// Table of localized compiler messages. + string[] messages; + /// Array of import paths to look for modules. + string[] importPaths; + /// Array of DDoc macro file paths. + string[] ddocFilePaths; + string xmlMapFile = "xml_map.d"; /// XML map file. + string htmlMapFile = "html_map.d"; /// HTML map file. + string lexerErrorFormat = "{0}({1},{2})L: {3}"; /// Lexer error. + string parserErrorFormat = "{0}({1},{2})P: {3}"; /// Parser error. + string semanticErrorFormat = "{0}({1},{2})S: {3}"; /// Semantic error. +} diff -r a3fab8b74a7d -r bcb74c9b895c src/SettingsLoader.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/SettingsLoader.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,252 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module SettingsLoader; + +import Settings; +import dil.Messages; +import dil.ast.Node, dil.ast.Declarations, dil.ast.Expressions; +import dil.semantic.Module; +import dil.semantic.Pass1; +import dil.semantic.Symbol; +import dil.semantic.Symbols; +import dil.Information; +import dil.Compilation; +import common; + +import tango.io.FilePath; + +/// Loads settings from a D module file. +class SettingsLoader +{ + InfoManager infoMan; /// Collects error messages. + Module mod; /// Current module. + + this(InfoManager infoMan) + { + this.infoMan = infoMan; + } + + static SettingsLoader opCall(InfoManager infoMan) + { + return new SettingsLoader(infoMan); + } + + /// Creates an error report. + /// Params: + /// token = where the error occurred. + /// formatMsg = error message. + void error(Token* token, char[] formatMsg, ...) + { + auto location = token.getErrorLocation(); + auto msg = Format(_arguments, _argptr, formatMsg); + infoMan ~= new SemanticError(location, msg); + } + + T getValue(T)(char[] name) + { + auto var = mod.lookup(name); + if (!var) // Returning T.init instead of null, because dmd gives an error. + return error(mod.firstToken, "variable '{}' is not defined", name), T.init; + auto t = var.node.begin; + if (!var.isVariable) + return error(t, "'{}' is not a variable declaration", name), T.init; + auto value = var.to!(Variable).value; + if (!value) + return error(t, "'{}' variable has no value set", name), T.init; + T val = value.Is!(T); // Try casting to T. + if (!val) + error(value.begin, "the value of '{}' is not of type {}", name, typeof(T).stringof); + return val; + } + + T castTo(T)(Node n) + { + char[] type; + is(T == StringExpression) && (type = "char[]"); + if (!n.Is!(T)) + error(n.begin, "expression is not of type {}", type); + return n.Is!(T); + } + + void load() + { + scope execPath = new FilePath(GetExecutableFilePath()); + execPath = new FilePath(execPath.folder()); + + // Load config.d + auto filePath = resolvePath(execPath, "config.d"); + mod = new Module(filePath, infoMan); + mod.parse(); + + if (mod.hasErrors) + return; + + auto context = new CompilationContext; + auto pass1 = new SemanticPass1(mod, context); + pass1.start(); + + if (auto array = getValue!(ArrayInitExpression)("version_ids")) + foreach (value; array.values) + if (auto str = castTo!(StringExpression)(value)) + GlobalSettings.versionIds ~= str.getString(); + if (auto val = getValue!(StringExpression)("langfile")) + GlobalSettings.langFile = val.getString(); + if (auto array = getValue!(ArrayInitExpression)("import_paths")) + foreach (value; array.values) + if (auto str = castTo!(StringExpression)(value)) + GlobalSettings.importPaths ~= str.getString(); + if (auto array = getValue!(ArrayInitExpression)("ddoc_files")) + foreach (value; array.values) + if (auto str = castTo!(StringExpression)(value)) + GlobalSettings.ddocFilePaths ~= resolvePath(execPath, str.getString()); + if (auto val = getValue!(StringExpression)("xml_map")) + GlobalSettings.xmlMapFile = val.getString(); + if (auto val = getValue!(StringExpression)("html_map")) + GlobalSettings.htmlMapFile = val.getString(); + if (auto val = getValue!(StringExpression)("lexer_error")) + GlobalSettings.lexerErrorFormat = val.getString(); + if (auto val = getValue!(StringExpression)("parser_error")) + GlobalSettings.parserErrorFormat = val.getString(); + if (auto val = getValue!(StringExpression)("semantic_error")) + GlobalSettings.semanticErrorFormat = val.getString(); + + // Load language file. + filePath = resolvePath(execPath, GlobalSettings.langFile); + mod = new Module(filePath); + mod.parse(); + + if (mod.hasErrors) + return; + + pass1 = new SemanticPass1(mod, context); + pass1.start(); + + if (auto array = getValue!(ArrayInitExpression)("messages")) + { + char[][] messages; + foreach (value; array.values) + if (auto str = castTo!(StringExpression)(value)) + messages ~= str.getString(); + if (messages.length != MID.max+1) + error(mod.firstToken, + "messages table in {} must exactly have {} entries, but not {}.", + filePath, MID.max+1, messages.length); + GlobalSettings.messages = messages; + dil.Messages.SetMessages(messages); + } + if (auto val = getValue!(StringExpression)("lang_code")) + GlobalSettings.langCode = val.getString(); + } +} + +/// Loads an associative array from a D module file. +class TagMapLoader : SettingsLoader +{ + this(InfoManager infoMan) + { + super(infoMan); + } + + static TagMapLoader opCall(InfoManager infoMan) + { + return new TagMapLoader(infoMan); + } + + string[string] load(string filePath) + { + mod = new Module(filePath, infoMan); + mod.parse(); + if (mod.hasErrors) + return null; + + auto context = new CompilationContext; + auto pass1 = new SemanticPass1(mod, context); + pass1.start(); + + string[string] map; + if (auto array = getValue!(ArrayInitExpression)("map")) + foreach (i, value; array.values) + { + auto key = array.keys[i]; + if (auto valExp = castTo!(StringExpression)(value)) + if (!key) + error(value.begin, "expected key : value"); + else if (auto keyExp = castTo!(StringExpression)(key)) + map[keyExp.getString()] = valExp.getString(); + } + return map; + } +} + +/// Resolves the path to a file from the executable's dir path +/// if it is relative. +/// Returns: filePath if it is absolute or execPath + filePath. +string resolvePath(FilePath execPath, string filePath) +{ + if ((new FilePath(filePath)).isAbsolute()) + return filePath; + return execPath.dup.append(filePath).toString(); +} + +version(DDoc) +{ + /// Returns the fully qualified path to this executable. + char[] GetExecutableFilePath(); +} +else version(Windows) +{ +private extern(Windows) uint GetModuleFileNameA(void*, char*, uint); + +char[] GetExecutableFilePath() +{ + alias GetModuleFileNameA GetModuleFileName; + char[] buffer = new char[256]; + uint count; + + while (1) + { + if (buffer is null) + return null; + + count = GetModuleFileName(null, buffer.ptr, buffer.length); + if (count == 0) + return null; + if (buffer.length != count && buffer[count] == 0) + break; + // Increase size of buffer + buffer.length = buffer.length * 2; + } + assert(buffer[count] == 0); + // Reduce buffer to the actual length of the string (excluding '\0'.) + if (count < buffer.length) + buffer.length = count; + return buffer; +} +} +else version(linux) +{ +private extern(C) size_t readlink(char* path, char* buf, size_t bufsize); + +char[] GetExecutableFilePath() +{ + char[] buffer = new char[256]; + size_t count; + + while (1) + { + // This won't work on very old Linux systems. + count = readlink("/proc/self/exe".ptr, buffer.ptr, buffer.length); + if (count == -1) + return null; + if (count < buffer.length) + break; + buffer.length = buffer.length * 2; + } + buffer.length = count; + return buffer; +} +} +else + static assert(0, "GetExecutableFilePath() is not implemented on this platform."); diff -r a3fab8b74a7d -r bcb74c9b895c src/TypeRules.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/TypeRules.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,129 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module TypeRules; + +import cmd.Generate : xml_escape; + +import TypeRulesData; +import common; + +static const string[] basicTypes = [ + "char"[], "wchar", "dchar", "bool", + "byte", "ubyte", "short", "ushort", + "int", "uint", "long", "ulong", + /+"cent", "ucent",+/ + "float", "double", "real", + "ifloat", "idouble", "ireal", + "cfloat", "cdouble", "creal"/+, "void"+/ +]; + +static const string[] unaryExpressions = [ + "!x", + "&x", + "~x", + "+x", + "-x", + "++x", + "--x", + "x++", + "x--", +]; + +static const string[] binaryExpressions = [ + "x!<>=y", + "x!<>y", + "x!<=y", + "x!=y", + "x!>y", + "x<>=y", + "x<>y", + + "x=y", "x==y", "x!=y", + "x<=y", "x=y", "x>y", + "x<<=y", "x<>=y","x>>y", + "x>>>=y", "x>>>y", + "x|=y", "x||y", "x|y", + "x&=y", "x&&y", "x&y", + "x+=y", "x+y", + "x-=y", "x-y", + "x/=y", "x/y", + "x*=y", "x*y", + "x%=y", "x%y", + "x^=y", "x^y", + "x~=y", + "x~y", + "x,y" +]; + +void genHTMLTypeRulesTables() +{ + Stdout( + ``\n + ``\n + ``\n + ` `\n + ` `\n + ` `\n + ``\n + ``\n + `

These tables show what the type results of certain expressions are.

`\n + ); + + Stdout.format("\n\n", unaryExpressions.length); + Stdout(""); + foreach (unaryExpression; unaryExpressions) + Stdout.format("", { + if (unaryExpression[0] == 'x') + return `x` ~ xml_escape(unaryExpression[1..$]); + else + return xml_escape(unaryExpression[0..$-1]) ~ `x`; + }()); + Stdout("\n"); + foreach (i, basicType; basicTypes) + { + Stdout.format("\n"``, basicType); + foreach (expResults; unaryExpsResults) + { + auto result = expResults[i]; + Stdout.format(``, result[0] == 'E' ? `Error`[] : result); + } + Stdout("\n\n"); + } + Stdout("
Unary Expressions
{}
{}{}
\n"); + + foreach (i, expResults; binaryExpsResults) + { + auto binaryExpression = binaryExpressions[i]; + binaryExpression = `x ` ~ + xml_escape(binaryExpression[1..$-1]) ~ + ` y`; + Stdout.format("\n\n", basicTypes.length, binaryExpression); + Stdout.format(""); + foreach (basicType; basicTypes) + Stdout.format(``, basicType); + Stdout("\n\n"); + foreach (j, results; expResults) + { + Stdout.format("\n"``, basicTypes[j]); + foreach (result; results) + Stdout.format(``, result[0] == 'E' ? `Error`[] : result); + Stdout("\n\n"); + } + Stdout("
{}
{}
{}{}
\n"); + } + + Stdout( + "\n" + "\n" + ); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/TypeRulesData.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/TypeRulesData.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +// Run TypeRulesGenerator.d diff -r a3fab8b74a7d -r bcb74c9b895c src/TypeRulesGenerator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/TypeRulesGenerator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,174 @@ +#! /usr/bin/rdmd +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module TypeRulesGenerator; + +import tango.io.File; +import tango.io.FilePath; + +alias char[] string; + +void main(char[][] args) +{ + char[] text = "// Generated by TypeRulesGenerator.d\n" + "module TypeRules.d;\n\n"; + text ~= "char[][][] unaryExpsResults = [\n"; + foreach (results; unaryExpsResults) + { + text ~= " ["; + foreach (result; results) + text ~= '"' ~ result ~ '"' ~ ", "; + text[$-2] = ']'; + text[$-1] = ','; + text ~= \n; + } + text[$-2] = '\n'; + text[$-1] = ']'; + text ~= ";\n\n"; + + text ~= "char[][][][] binaryExpsResults = [\n"; + foreach (expResults; binaryExpsResults) + { + text ~= " [\n"; + foreach (results; expResults) + { + text ~= " ["; + foreach (result; results) + text ~= '"' ~ result ~ '"' ~ ", "; + text[$-2] = ']'; + text[$-1] = ','; + text ~= \n; + } + text[$-2] = '\n'; + text[$-1] = ' '; + text ~= " ],\n"; + } + text[$-2] = '\n'; + text[$-1] = ']'; + text ~= ";\n"; + + // Write the text to a D module. + auto file = new File("TypeRulesData.d"); + file.write(text); +} + +template ExpressionType(char[] T1, char[] T2, char[] expression) +{ + mixin("alias "~T1~" X;"); + mixin("alias "~T2~" Y;"); + X x; + Y y; + static if(is(typeof(mixin(expression)) ResultType)) + const char[] result = ResultType.stringof; + else + const char[] result = "Error"; +} +alias ExpressionType EType; + +// pragma(msg, EType!("char", "int", "&x").result); + +static const string[] basicTypes = [ + "char"[], "wchar", "dchar", "bool", + "byte", "ubyte", "short", "ushort", + "int", "uint", "long", "ulong", + /+"cent", "ucent",+/ + "float", "double", "real", + "ifloat", "idouble", "ireal", + "cfloat", "cdouble", "creal"/+, "void"+/ +]; + +static const string[] unaryExpressions = [ + "!x", + "&x", + "~x", + "+x", + "-x", + "++x", + "--x", + "x++", + "x--", +]; + +static const string[] binaryExpressions = [ + "x!<>=y", + "x!<>y", + "x!<=y", + "x!=y", + "x!>y", + "x<>=y", + "x<>y", + + "x=y", "x==y", "x!=y", + "x<=y", "x=y", "x>y", + "x<<=y", "x<>=y","x>>y", + "x>>>=y", "x>>>y", + "x|=y", "x||y", "x|y", + "x&=y", "x&&y", "x&y", + "x+=y", "x+y", + "x-=y", "x-y", + "x/=y", "x/y", + "x*=y", "x*y", + "x%=y", "x%y", + "x^=y", "x^y", + "x~=y", + "x~y", + "x,y" +]; + +char[] genBinaryExpArray(char[] expression) +{ + char[] result = "[\n"; + foreach (t1; basicTypes) + { + result ~= "[\n"; + foreach (t2; basicTypes) + result ~= `EType!("`~t1~`", "`~t2~`", "`~expression~`").result,`\n; + result[result.length-2] = ']'; // Overwrite last comma. + result[result.length-1] = ','; // Overwrite last \n. + } + result[result.length-1] = ']'; // Overwrite last comma. + return result; +} +// pragma(msg, mixin(genBinaryExpArray("x%y")).stringof); + +char[] genBinaryExpsArray() +{ + char[] result = "[\n"; + foreach (expression; binaryExpressions) + { + result ~= genBinaryExpArray(expression); + result ~= ",\n"; + } + result[result.length-2] = ']'; + return result; +} + +// pragma(msg, mixin(genBinaryExpsArray()).stringof); + +char[] genUnaryExpArray(char[] expression) +{ + char[] result = "[\n"; + foreach (t1; basicTypes) + result ~= `EType!("`~t1~`", "int", "`~expression~`").result,`\n; + result[result.length-2] = ']'; // Overwrite last comma. + return result; +} + +char[] genUnaryExpsArray() +{ + char[] result = "[\n"; + foreach (expression; unaryExpressions) + result ~= genUnaryExpArray(expression) ~ ",\n"; + result[result.length-2] = ']'; + return result; +} + +// pragma(msg, mixin(genUnaryExpsArray()).stringof); + +auto unaryExpsResults = mixin(genUnaryExpsArray()); +auto binaryExpsResults = mixin(genBinaryExpsArray()); diff -r a3fab8b74a7d -r bcb74c9b895c src/cmd/ASTStats.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/cmd/ASTStats.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,33 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module cmd.ASTStats; + +import dil.ast.DefaultVisitor; +import dil.ast.Node, + dil.ast.Declaration, + dil.ast.Statement, + dil.ast.Expression, + dil.ast.Types; + +/// Counts the nodes in a syntax tree. +class ASTStats : DefaultVisitor +{ + uint[] table; /// Table for counting nodes. + + /// Starts counting. + uint[] count(Node root) + { + table = new uint[g_classNames.length]; + super.visitN(root); + return table; + } + + // Override dispatch function. + override Node dispatch(Node n) + { + table[n.kind]++; + return super.dispatch(n); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/cmd/DDoc.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/cmd/DDoc.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,849 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module cmd.DDoc; + +import cmd.DDocXML; +import cmd.Generate; +import dil.doc.Parser; +import dil.doc.Macro; +import dil.doc.Doc; +import dil.ast.Node; +import dil.ast.Declarations, + dil.ast.Statements, + dil.ast.Expression, + dil.ast.Parameters, + dil.ast.Types; +import dil.ast.DefaultVisitor; +import dil.lexer.Token; +import dil.lexer.Funcs; +import dil.semantic.Module; +import dil.semantic.Pass1; +import dil.semantic.Symbol; +import dil.semantic.Symbols; +import dil.Compilation; +import dil.Information; +import dil.Converter; +import dil.SourceText; +import dil.Enums; +import dil.Time; +import common; + +import tango.text.Ascii : toUpper; +import tango.io.File; +import tango.io.FilePath; + +/// Executes the doc generation command. +void execute(string[] filePaths, string destDir, string[] macroPaths, + bool writeXML, bool incUndoc, bool verbose, + CompilationContext context, InfoManager infoMan) +{ + // Parse macro files. + MacroTable mtable; + MacroParser mparser; + foreach (macroPath; macroPaths) + { + auto macros = mparser.parse(loadMacroFile(macroPath, infoMan)); + mtable = new MacroTable(mtable); + mtable.insert(macros); + } + +// foreach (k, v; mtable.table) +// Stdout(k)("=")(v.text); + + // For DDoc code sections. + auto tokenHL = new TokenHighlighter(infoMan, writeXML == false); + + // Process D files. + foreach (filePath; filePaths) + { + auto mod = new Module(filePath, infoMan); + // Parse the file. + mod.parse(); + if (mod.hasErrors) + continue; + + // Start semantic analysis. + auto pass1 = new SemanticPass1(mod, context); + pass1.start(); + + // Generate documentation. + auto dest = new FilePath(destDir); + dest.append(mod.getFQN() ~ (writeXML ? ".xml" : ".html")); + + InfoManager infoMan2; // Collects warnings from the macro expander. + if (verbose) + { + Stdout.formatln("{} > {}", mod.filePath, dest); + infoMan2 = new InfoManager(); + } + + writeDocFile(dest.toString(), mod, mtable, writeXML, incUndoc, tokenHL, infoMan2); + + if (infoMan2) + infoMan ~= infoMan2.info; + } +} + +void writeDocFile(string dest, Module mod, MacroTable mtable, + bool writeXML, bool incUndoc, + TokenHighlighter tokenHL, InfoManager infoMan) +{ + // Create a macro environment for this module. + mtable = new MacroTable(mtable); + // Define runtime macros. + // MODPATH is not in the specs. + mtable.insert("MODPATH", mod.getFQNPath() ~ "." ~ mod.fileExtension()); + mtable.insert("TITLE", mod.getFQN()); + mtable.insert("DOCFILENAME", mod.getFQN() ~ (writeXML ? ".xml" : ".html")); + auto timeStr = Time.toString(); + mtable.insert("DATETIME", timeStr); + mtable.insert("YEAR", Time.year(timeStr)); + + DDocEmitter docEmitter; + if (writeXML) + docEmitter = new DDocXMLEmitter(mod, mtable, incUndoc, tokenHL); + else + docEmitter = new DDocEmitter(mod, mtable, incUndoc, tokenHL); + docEmitter.emit(); + // Set BODY macro to the text produced by the DDocEmitter. + mtable.insert("BODY", docEmitter.text); + // Do the macro expansion pass. + auto fileText = MacroExpander.expand(mtable, "$(DDOC)", mod.filePath, infoMan); +// fileText ~= "\n
\n" ~ doc.text ~ "\n
"; + // Finally write the file out to the harddisk. + auto file = new File(dest); + file.write(fileText); +} + +/// Loads a macro file. Converts any Unicode encoding to UTF-8. +string loadMacroFile(string filePath, InfoManager infoMan) +{ + auto src = new SourceText(filePath); + src.load(infoMan); + auto text = src.data[0..$-1]; // Exclude '\0'. + return sanitizeText(text); +} + +/// Traverses the syntax tree and writes DDoc macros to a string buffer. +class DDocEmitter : DefaultVisitor +{ + char[] text; /// The buffer that is written to. + bool includeUndocumented; + MacroTable mtable; + Module modul; + TokenHighlighter tokenHL; + + /// Constructs a DDocEmitter object. + /// Params: + /// modul = the module to generate text for. + /// mtable = the macro table. + /// includeUndocumented = whether to include undocumented symbols. + /// tokenHL = used to highlight code sections. + this(Module modul, MacroTable mtable, bool includeUndocumented, + TokenHighlighter tokenHL) + { + this.mtable = mtable; + this.includeUndocumented = includeUndocumented; + this.modul = modul; + this.tokenHL = tokenHL; + } + + /// Entry method. + char[] emit() + { + if (auto d = modul.moduleDecl) + { + if (ddoc(d)) + { + if (auto copyright = cmnt.takeCopyright()) + mtable.insert(new Macro("COPYRIGHT", copyright.text)); + writeComment(); + } + } + MEMBERS("MODULE", { visitD(modul.root); }); + return text; + } + + char[] textSpan(Token* left, Token* right) + { + //assert(left && right && (left.end <= right.start || left is right)); + //char[] result; + //TODO: filter out whitespace tokens. + return Token.textSpan(left, right); + } + + TemplateParameters tparams; /// The template parameters of the current declaration. + + DDocComment cmnt; /// Current comment. + DDocComment prevCmnt; /// Previous comment in scope. + /// An empty comment. Used for undocumented symbols. + static const DDocComment emptyCmnt; + + /// Initializes the empty comment. + static this() + { + this.emptyCmnt = new DDocComment(null, null, null); + } + + /// Keeps track of previous comments in each scope. + scope class Scope + { + DDocComment saved_prevCmnt; + bool saved_cmntIsDitto; + uint saved_prevDeclOffset; + this() + { // Save the previous comment of the parent scope. + saved_prevCmnt = this.outer.prevCmnt; + saved_cmntIsDitto = this.outer.cmntIsDitto; + saved_prevDeclOffset = this.outer.prevDeclOffset; + // Entering a new scope. Clear variables. + this.outer.prevCmnt = null; + this.outer.cmntIsDitto = false; + this.outer.prevDeclOffset = 0; + } + + ~this() + { // Restore the previous comment of the parent scope. + this.outer.prevCmnt = saved_prevCmnt; + this.outer.cmntIsDitto = saved_cmntIsDitto; + this.outer.prevDeclOffset = saved_prevDeclOffset; + } + } + + bool cmntIsDitto; /// True if current comment is "ditto". + + /// Returns the DDocComment for node. + DDocComment ddoc(Node node) + { + auto c = getDDocComment(node); + this.cmnt = null; + if (c) + { + if (c.isDitto) + { + this.cmnt = this.prevCmnt; + this.cmntIsDitto = true; + } + else + { + this.cmntIsDitto = false; + this.cmnt = c; + this.prevCmnt = c; + } + } + else if (includeUndocumented) + this.cmnt = this.emptyCmnt; + return this.cmnt; + } + + /// List of predefined, special sections. + static char[][char[]] specialSections; + static this() + { + foreach (name; ["AUTHORS", "BUGS", "COPYRIGHT", "DATE", "DEPRECATED", + "EXAMPLES", "HISTORY", "LICENSE", "RETURNS", "SEE_ALSO", + "STANDARDS", "THROWS", "VERSION"]) + specialSections[name] = name; + } + + /// Writes the DDoc comment to the text buffer. + void writeComment() + { + auto c = this.cmnt; + assert(c !is null); + if (c.sections.length == 0) + return; + write("$(DDOC_SECTIONS "); + foreach (s; c.sections) + { + if (s is c.summary) + write("\n$(DDOC_SUMMARY "); + else if (s is c.description) + write("\n$(DDOC_DESCRIPTION "); + else if (auto name = toUpper(s.name.dup) in specialSections) + write("\n$(DDOC_" ~ *name ~ " "); + else if (s.Is("params")) + { // Process parameters section. + auto ps = new ParamsSection(s.name, s.text); + write("\n$(DDOC_PARAMS "); + foreach (i, paramName; ps.paramNames) + write("\n$(DDOC_PARAM_ROW ", + "$(DDOC_PARAM_ID $(DDOC_PARAM ", paramName, "))", + "$(DDOC_PARAM_DESC ", ps.paramDescs[i], ")", + ")"); + write(")"); + continue; + } + else if (s.Is("macros")) + { // Declare the macros in this section. + auto ms = new MacrosSection(s.name, s.text); + mtable.insert(ms.macroNames, ms.macroTexts); + continue; + } + else + write("\n$(DDOC_SECTION $(DDOC_SECTION_H " ~ s.name ~ ":)"); + write(scanCommentText(s.text), ")"); + } + write(")"); + } + + /// Scans the comment text and: + /// $(UL + /// $(LI skips and leaves macro invocations unchanged) + /// $(LI skips HTML tags) + /// $(LI escapes '(', ')', '<', '>' and '&') + /// $(LI inserts $(DDOC_BLANKLINE) in place of \n\n) + /// $(LI highlights code in code sections) + /// ) + char[] scanCommentText(char[] text) + { + char* p = text.ptr; + char* end = p + text.length; + char[] result = new char[text.length]; // Reserve space. + result.length = 0; + + while (p < end) + { + switch (*p) + { + case '$': + if (auto macroEnd = MacroParser.scanMacro(p, end)) + { + result ~= makeString(p, macroEnd); // Copy macro invocation as is. + p = macroEnd; + continue; + } + goto default; + case '<': + auto begin = p; + p++; + if (p+2 < end && *p == '!' && p[1] == '-' && p[2] == '-') // ". + while (++p < end) + if (p+2 < end && *p == '-' && p[1] == '-' && p[2] == '>') + { + p += 3; // Point one past '>'. + break; + } + result ~= makeString(begin, p); + } // or + else if (p < end && (isalpha(*p) || *p == '/')) + { + while (++p < end && *p != '>') // Skip to closing '>'. + {} + if (p == end) + { // No closing '>' found. + p = begin + 1; + result ~= "<"; + continue; + } + p++; // Skip '>'. + result ~= makeString(begin, p); + } + else + result ~= "<"; + continue; + case '(': result ~= "("; break; + case ')': result ~= ")"; break; + // case '\'': result ~= "'"; break; // ' + // case '"': result ~= """; break; + case '>': result ~= ">"; break; + case '&': + if (p+1 < end && (isalpha(p[1]) || p[1] == '#')) + goto default; + result ~= "&"; + break; + case '\n': + if (!(p+1 < end && p[1] == '\n')) + goto default; + ++p; + result ~= "$(DDOC_BLANKLINE)"; + break; + case '-': + if (p+2 < end && p[1] == '-' && p[2] == '-') + { + while (p < end && *p == '-') + p++; + auto codeBegin = p; + p--; + while (++p < end) + if (p+2 < end && *p == '-' && p[1] == '-' && p[2] == '-') + break; + auto codeText = makeString(codeBegin, p); + result ~= tokenHL.highlight(codeText, modul.filePath); + while (p < end && *p == '-') + p++; + continue; + } + //goto default; + default: + result ~= *p; + } + p++; + } + return result; + } + + /// Escapes '<', '>' and '&' with named HTML entities. + char[] escape(char[] text) + { + char[] result = new char[text.length]; // Reserve space. + result.length = 0; + foreach(c; text) + switch(c) + { + case '<': result ~= "<"; break; + case '>': result ~= ">"; break; + case '&': result ~= "&"; break; + default: result ~= c; + } + if (result.length != text.length) + return result; + // Nothing escaped. Return original text. + delete result; + return text; + } + + /// Writes an array of strings to the text buffer. + void write(char[][] strings...) + { + foreach (s; strings) + text ~= s; + } + + /// Writes params to the text buffer. + void writeParams(Parameters params) + { + if (!params.items.length) + return write("()"); + write("("); + auto lastParam = params.items[$-1]; + foreach (param; params.items) + { + if (param.isCVariadic) + write("..."); + else + { + assert(param.type); + // Write storage classes. + auto typeBegin = param.type.baseType.begin; + if (typeBegin !is param.begin) // Write storage classes. + write(textSpan(param.begin, typeBegin.prevNWS), " "); + write(escape(textSpan(typeBegin, param.type.end))); // Write type. + if (param.name) + write(" $(DDOC_PARAM ", param.name.str, ")"); + if (param.isDVariadic) + write("..."); + if (param.defValue) + write(" = ", escape(textSpan(param.defValue.begin, param.defValue.end))); + } + if (param !is lastParam) + write(", "); + } + write(")"); + } + + /// Writes the current template parameters to the text buffer. + void writeTemplateParams() + { + if (!tparams) + return; + write(escape(textSpan(tparams.begin, tparams.end))); + tparams = null; + } + + /// Writes bases to the text buffer. + void writeInheritanceList(BaseClassType[] bases) + { + if (bases.length == 0) + return; + auto basesBegin = bases[0].begin.prevNWS; + if (basesBegin.kind == TOK.Colon) + basesBegin = bases[0].begin; + write(" : ", escape(textSpan(basesBegin, bases[$-1].end))); + } + + /// Writes a symbol to the text buffer. E.g: $(SYMBOL Buffer, 123) + void SYMBOL(char[] name, Declaration d) + { + auto loc = d.begin.getRealLocation(); + auto str = Format("$(SYMBOL {}, {})", name, loc.lineNum); + write(str); + // write("$(DDOC_PSYMBOL ", name, ")"); + } + + /// Offset at which to insert a declaration which have a "ditto" comment. + uint prevDeclOffset; + + /// Writes a declaration to the text buffer. + void DECL(void delegate() dg, Declaration d, bool writeSemicolon = true) + { + if (cmntIsDitto) + { alias prevDeclOffset offs; + assert(offs != 0); + auto savedText = text; + text = ""; + write("\n$(DDOC_DECL "); + dg(); + writeSemicolon && write(";"); + writeAttributes(d); + write(")"); + // Insert text at offset. + auto len = text.length; + text = savedText[0..offs] ~ text ~ savedText[offs..$]; + offs += len; // Add length of the inserted text to the offset. + return; + } + write("\n$(DDOC_DECL "); + dg(); + writeSemicolon && write(";"); + writeAttributes(d); + write(")"); + prevDeclOffset = text.length; + } + + /// Wraps the DDOC_DECL_DD macro around the text written by dg(). + void DESC(void delegate() dg) + { + if (cmntIsDitto) + return; + write("\n$(DDOC_DECL_DD "); + dg(); + write(")"); + } + + /// Wraps the DDOC_kind_MEMBERS macro around the text written by dg(). + void MEMBERS(char[] kind, void delegate() dg) + { + write("\n$(DDOC_"~kind~"_MEMBERS "); + dg(); + write(")"); + } + + /// Writes a class or interface declaration. + void writeClassOrInterface(T)(T d) + { + if (!ddoc(d)) + return d; + DECL({ + write(d.begin.srcText, " "); + SYMBOL(d.name.str, d); + writeTemplateParams(); + writeInheritanceList(d.bases); + }, d); + DESC({ + writeComment(); + MEMBERS(is(T == ClassDeclaration) ? "CLASS" : "INTERFACE", { + scope s = new Scope(); + d.decls && super.visit(d.decls); + }); + }); + } + + // templated decls are not virtual so we need these: + + /// Writes a class declaration. + void writeClass(ClassDeclaration d) { + writeClassOrInterface(d); + } + + /// Writes an interface declaration. + void writeInterface(InterfaceDeclaration d) { + writeClassOrInterface(d); + } + + /// Writes a struct or union declaration. + void writeStructOrUnion(T)(T d) + { + if (!ddoc(d)) + return d; + DECL({ + write(d.begin.srcText, d.name ? " " : ""); + if (d.name) + SYMBOL(d.name.str, d); + writeTemplateParams(); + }, d); + DESC({ + writeComment(); + MEMBERS(is(T == StructDeclaration) ? "STRUCT" : "UNION", { + scope s = new Scope(); + d.decls && super.visit(d.decls); + }); + }); + } + + // templated decls are not virtual so we need these: + + /// Writes a struct declaration. + void writeStruct(StructDeclaration d) { + writeStructOrUnion(d); + } + + /// Writes an union declaration. + void writeUnion(UnionDeclaration d) { + writeStructOrUnion(d); + } + + /// Writes an alias or typedef declaration. + void writeAliasOrTypedef(T)(T d) + { + auto prefix = is(T == AliasDeclaration) ? "alias " : "typedef "; + if (auto vd = d.decl.Is!(VariablesDeclaration)) + { + auto type = textSpan(vd.typeNode.baseType.begin, vd.typeNode.end); + foreach (name; vd.names) + DECL({ write(prefix); write(escape(type), " "); SYMBOL(name.str, d); }, d); + } + else if (auto fd = d.decl.Is!(FunctionDeclaration)) + {} + // DECL({ write(textSpan(d.begin, d.end)); }, false); + DESC({ writeComment(); }); + } + + /// Writes the attributes of a declaration in brackets. + void writeAttributes(Declaration d) + { + char[][] attributes; + + if (d.prot != Protection.None) + attributes ~= "$(PROT " ~ .toString(d.prot) ~ ")"; + + auto stc = d.stc; + stc &= ~StorageClass.Auto; // Ignore auto. + foreach (stcStr; .toStrings(stc)) + attributes ~= "$(STC " ~ stcStr ~ ")"; + + LinkageType ltype; + if (auto vd = d.Is!(VariablesDeclaration)) + ltype = vd.linkageType; + else if (auto fd = d.Is!(FunctionDeclaration)) + ltype = fd.linkageType; + + if (ltype != LinkageType.None) + attributes ~= "$(LINKAGE extern(" ~ .toString(ltype) ~ "))"; + + if (!attributes.length) + return; + + write(" $(ATTRIBUTES "); + write(attributes[0]); + foreach (attribute; attributes[1..$]) + write(", ", attribute); + write(")"); + } + + alias Declaration D; + +override: + D visit(AliasDeclaration d) + { + if (!ddoc(d)) + return d; + writeAliasOrTypedef(d); + return d; + } + + D visit(TypedefDeclaration d) + { + if (!ddoc(d)) + return d; + writeAliasOrTypedef(d); + return d; + } + + D visit(EnumDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ + write("enum", d.name ? " " : ""); + d.name && SYMBOL(d.name.str, d); + }, d); + DESC({ + writeComment(); + MEMBERS("ENUM", { scope s = new Scope(); super.visit(d); }); + }); + return d; + } + + D visit(EnumMemberDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ SYMBOL(d.name.str, d); }, d, false); + DESC({ writeComment(); }); + return d; + } + + D visit(TemplateDeclaration d) + { + this.tparams = d.tparams; + if (d.begin.kind != TOK.Template) + { // This is a templatized class/interface/struct/union/function. + super.visit(d.decls); + this.tparams = null; + return d; + } + if (!ddoc(d)) + return d; + DECL({ + write("template "); + SYMBOL(d.name.str, d); + writeTemplateParams(); + }, d); + DESC({ + writeComment(); + MEMBERS("TEMPLATE", { + scope s = new Scope(); + super.visit(d.decls); + }); + }); + return d; + } + + D visit(ClassDeclaration d) + { + writeClass(d); + return d; + } + + D visit(InterfaceDeclaration d) + { + writeInterface(d); + return d; + } + + D visit(StructDeclaration d) + { + writeStruct(d); + return d; + } + + D visit(UnionDeclaration d) + { + writeUnion(d); + return d; + } + + D visit(ConstructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ SYMBOL("this", d); writeParams(d.params); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(StaticConstructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("static "); SYMBOL("this", d); write("()"); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(DestructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("~"); SYMBOL("this", d); write("()"); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(StaticDestructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("static ~"); SYMBOL("this", d); write("()"); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(FunctionDeclaration d) + { + if (!ddoc(d)) + return d; + auto type = textSpan(d.returnType.baseType.begin, d.returnType.end); + DECL({ + write(escape(type), " "); + SYMBOL(d.name.str, d); + writeTemplateParams(); + writeParams(d.params); + }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(NewDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ SYMBOL("new", d); writeParams(d.params); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(DeleteDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ SYMBOL("delete", d); writeParams(d.params); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(VariablesDeclaration d) + { + if (!ddoc(d)) + return d; + char[] type = "auto"; + if (d.typeNode) + type = textSpan(d.typeNode.baseType.begin, d.typeNode.end); + foreach (name; d.names) + DECL({ write(escape(type), " "); SYMBOL(name.str, d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(InvariantDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ SYMBOL("invariant", d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(UnittestDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ SYMBOL("unittest", d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(DebugDeclaration d) + { + d.compiledDecls && visitD(d.compiledDecls); + return d; + } + + D visit(VersionDeclaration d) + { + d.compiledDecls && visitD(d.compiledDecls); + return d; + } + + D visit(StaticIfDeclaration d) + { + d.ifDecls && visitD(d.ifDecls); + return d; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/cmd/DDocXML.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/cmd/DDocXML.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,406 @@ +/++ + Authors: Aziz Köksal & Jari-Matti Mäkelä + License: GPL3 ++/ +module cmd.DDocXML; + +import cmd.DDoc; +import cmd.Generate; +import dil.doc.Parser; +import dil.doc.Macro; +import dil.doc.Doc; +import dil.ast.Node; +import dil.ast.Declarations, + dil.ast.Statements, + dil.ast.Expression, + dil.ast.Parameters, + dil.ast.Types; +import dil.ast.DefaultVisitor; +import dil.lexer.Token; +import dil.lexer.Funcs; +import dil.semantic.Module; +import dil.semantic.Pass1; +import dil.semantic.Symbol; +import dil.semantic.Symbols; +import dil.Compilation; +import dil.Information; +import dil.Converter; +import dil.SourceText; +import dil.Enums; +import dil.Time; +import common; + +import tango.text.Ascii : toUpper; +import tango.io.File; +import tango.io.FilePath; + +/// Traverses the syntax tree and writes DDoc macros to a string buffer. +class DDocXMLEmitter : DDocEmitter +{ + this(Module modul, MacroTable mtable, bool includeUndocumented, + TokenHighlighter tokenHL) + { + super(modul, mtable, includeUndocumented, tokenHL); + } + + /// Writes params to the text buffer. + void writeParams(Parameters params) + { + if (!params.items.length) + return; + + write("$(PARAMS "); + auto lastParam = params.items[$-1]; + foreach (param; params.items) + { + if (param.isCVariadic) + write("..."); + else + { + assert(param.type); + // Write storage classes. + auto typeBegin = param.type.baseType.begin; + if (typeBegin !is param.begin) // Write storage classes. + write(textSpan(param.begin, typeBegin.prevNWS), " "); + write(escape(textSpan(typeBegin, param.type.end))); // Write type. + if (param.name) + write(" $(DDOC_PARAM ", param.name.str, ")"); + if (param.isDVariadic) + write("..."); + if (param.defValue) + write(" = ", escape(textSpan(param.defValue.begin, param.defValue.end))); + } + if (param !is lastParam) + write(", "); + } + write(")"); + } + + /// Writes the current template parameters to the text buffer. + void writeTemplateParams() + { + if (!tparams) + return; + write("$(TEMPLATE_PARAMS ", escape(textSpan(tparams.begin, tparams.end))[1..$-1], ")"); + tparams = null; + } + + /// Writes bases to the text buffer. + void writeInheritanceList(BaseClassType[] bases) + { + if (bases.length == 0) + return; + auto basesBegin = bases[0].begin.prevNWS; + if (basesBegin.kind == TOK.Colon) + basesBegin = bases[0].begin; + write("$(PARENTS ", escape(textSpan(basesBegin, bases[$-1].end)), ")"); + } + + /// Writes a symbol to the text buffer. E.g: $(SYMBOL Buffer, 123) + void SYMBOL(char[] name, Declaration d) + { + auto loc = d.begin.getRealLocation(); + auto str = Format("$(SYMBOL {}, {})", name, loc.lineNum); + write(str); + // write("$(DDOC_PSYMBOL ", name, ")"); + } + + /// Writes a declaration to the text buffer. + void DECL(void delegate() dg, Declaration d, bool writeSemicolon = true) + { + if (cmntIsDitto) + { alias prevDeclOffset offs; + assert(offs != 0); + auto savedText = text; + text = ""; + write("\n$(DDOC_DECL "); + dg(); + writeAttributes(d); + write(")"); + // Insert text at offset. + auto len = text.length; + text = savedText[0..offs] ~ text ~ savedText[offs..$]; + offs += len; // Add length of the inserted text to the offset. + return; + } + write("\n$(DDOC_DECL "); + dg(); + writeAttributes(d); + write(")"); + prevDeclOffset = text.length; + } + + + /// Writes a class or interface declaration. + void writeClassOrInterface(T)(T d) + { + if (!ddoc(d)) + return d; + DECL({ + write(d.begin.srcText, ", "); + SYMBOL(d.name.str, d); + writeTemplateParams(); + writeInheritanceList(d.bases); + }, d); + DESC({ + writeComment(); + MEMBERS(is(T == ClassDeclaration) ? "CLASS" : "INTERFACE", { + scope s = new Scope(); + d.decls && DefaultVisitor.visit(d.decls); + }); + }); + } + + // templated decls are not virtual so we need these: + + /// Writes a class declaration. + void writeClass(ClassDeclaration d) { + writeClassOrInterface(d); + } + + /// Writes an interface declaration. + void writeInterface(InterfaceDeclaration d) { + writeClassOrInterface(d); + } + + /// Writes a struct or union declaration. + void writeStructOrUnion(T)(T d) + { + if (!ddoc(d)) + return d; + DECL({ + write(d.begin.srcText, d.name ? ", " : ""); + if (d.name) + SYMBOL(d.name.str, d); + writeTemplateParams(); + }, d); + DESC({ + writeComment(); + MEMBERS(is(T == StructDeclaration) ? "STRUCT" : "UNION", { + scope s = new Scope(); + d.decls && DefaultVisitor.visit(d.decls); + }); + }); + } + + // templated decls are not virtual so we need these: + + /// Writes a struct declaration. + void writeStruct(StructDeclaration d) { + writeStructOrUnion(d); + } + + /// Writes an union declaration. + void writeUnion(UnionDeclaration d) { + writeStructOrUnion(d); + } + + /// Writes an alias or typedef declaration. + void writeAliasOrTypedef(T)(T d) + { + auto prefix = is(T == AliasDeclaration) ? "alias " : "typedef "; + if (auto vd = d.decl.Is!(VariablesDeclaration)) + { + auto type = textSpan(vd.typeNode.baseType.begin, vd.typeNode.end); + foreach (name; vd.names) + DECL({ write(prefix, ", "); write(escape(type), " "); SYMBOL(name.str, d); }, d); + } + else if (auto fd = d.decl.Is!(FunctionDeclaration)) + {} + // DECL({ write(textSpan(d.begin, d.end)); }, false); + DESC({ writeComment(); }); + } + + + /// Writes the attributes of a declaration in brackets. + void writeAttributes(Declaration d) + { + char[][] attributes; + + if (d.prot != Protection.None) + attributes ~= "$(PROT " ~ .toString(d.prot) ~ ")"; + + auto stc = d.stc; + stc &= ~StorageClass.Auto; // Ignore auto. + foreach (stcStr; .toStrings(stc)) + attributes ~= "$(STC " ~ stcStr ~ ")"; + + LinkageType ltype; + if (auto vd = d.Is!(VariablesDeclaration)) + ltype = vd.linkageType; + else if (auto fd = d.Is!(FunctionDeclaration)) + ltype = fd.linkageType; + + if (ltype != LinkageType.None) + attributes ~= "$(LINKAGE extern(" ~ .toString(ltype) ~ "))"; + + if (!attributes.length) + return; + + write("$(ATTRIBUTES "); + foreach (attribute; attributes) + write(attribute); + write(")"); + } + + alias Declaration D; + + alias DDocEmitter.visit visit; + + D visit(EnumDeclaration d) + { + /+ FIXME: broken, infinite recursion :/ + if (!ddoc(d)) + return d; + DECL({ + write("enum, ", d.name ? " " : ""); + d.name && SYMBOL(d.name.str, d); + }, d); + DESC({ + writeComment(); + Stdout("help\n"); +///*FIXME*/ MEMBERS("ENUM", { scope s = new Scope(); DDocEmitter.visit(d); }); + }); + +/ + return d; + } + + D visit(EnumMemberDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("member, "); SYMBOL(d.name.str, d); }, d, false); + DESC({ writeComment(); }); + return d; + } + + D visit(TemplateDeclaration d) + { + this.tparams = d.tparams; + if (d.begin.kind != TOK.Template) + { // This is a templatized class/interface/struct/union/function. + DefaultVisitor.visit(d.decls); + this.tparams = null; + return d; + } + if (!ddoc(d)) + return d; + DECL({ + write("template, "); + SYMBOL(d.name.str, d); + writeTemplateParams(); + }, d); + DESC({ + writeComment(); + MEMBERS("TEMPLATE", { + scope s = new Scope(); + DefaultVisitor.visit(d.decls); + }); + }); + return d; + } + + D visit(ConstructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("constructor, "); SYMBOL("this", d); writeParams(d.params); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(StaticConstructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("static constructor, "); SYMBOL("this", d); write("()"); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(DestructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("destructor, ~"); SYMBOL("this", d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(StaticDestructorDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("static destructor, ~"); SYMBOL("this", d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(FunctionDeclaration d) + { + if (!ddoc(d)) + return d; + auto type = textSpan(d.returnType.baseType.begin, d.returnType.end); + DECL({ + write("function, "); + write("$(TYPE "); + write("$(RETURNS ", escape(type), ")"); + writeTemplateParams(); + writeParams(d.params); + write(")"); + SYMBOL(d.name.str, d); + }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(NewDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("new, "); SYMBOL("new", d); writeParams(d.params); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(DeleteDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("delete, "); SYMBOL("delete", d); writeParams(d.params); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(VariablesDeclaration d) + { + if (!ddoc(d)) + return d; + char[] type = "auto"; + if (d.typeNode) + type = textSpan(d.typeNode.baseType.begin, d.typeNode.end); + foreach (name; d.names) + DECL({ write("variable, "); write("$(TYPE ", escape(type), ")"); SYMBOL(name.str, d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(InvariantDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("invariant, "); SYMBOL("invariant", d); }, d); + DESC({ writeComment(); }); + return d; + } + + D visit(UnittestDeclaration d) + { + if (!ddoc(d)) + return d; + DECL({ write("unittest, "); SYMBOL("unittest", d); }, d); + DESC({ writeComment(); }); + return d; + } + +} diff -r a3fab8b74a7d -r bcb74c9b895c src/cmd/Generate.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/cmd/Generate.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,489 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module cmd.Generate; + +import dil.ast.DefaultVisitor; +import dil.ast.Node, + dil.ast.Declaration, + dil.ast.Statement, + dil.ast.Expression, + dil.ast.Types; +import dil.lexer.Lexer; +import dil.parser.Parser; +import dil.semantic.Module; +import dil.SourceText; +import dil.Information; +import SettingsLoader; +import Settings; +import common; + +import tango.io.GrowBuffer; +import tango.io.Print; + +/// Options for the generate command. +enum GenOption +{ + Empty, + Tokens = 1, + Syntax = 1<<1, + HTML = 1<<2, + XML = 1<<3, + PrintLines = 1<<4 +} + +/// Executes the generate command. +void execute(string filePath, GenOption options, InfoManager infoMan) +{ + assert(options != GenOption.Empty); + auto mapFilePath = options & GenOption.HTML ? GlobalSettings.htmlMapFile + : GlobalSettings.xmlMapFile; + auto map = TagMapLoader(infoMan).load(mapFilePath); + auto tags = new TagMap(map); + + if (infoMan.hasInfo) + return; + + if (options & GenOption.Syntax) + highlightSyntax(filePath, tags, Stdout, options); + else + highlightTokens(filePath, tags, Stdout, options); +} + +/// Escapes the characters '<', '>' and '&' with named character entities. +char[] xml_escape(char[] text) +{ + char[] result; + foreach(c; text) + switch(c) + { + case '<': result ~= "<"; break; + case '>': result ~= ">"; break; + case '&': result ~= "&"; break; + default: result ~= c; + } + if (result.length != text.length) + return result; + // Nothing escaped. Return original text. + delete result; + return text; +} + +/// Maps tokens to (format) strings. +class TagMap +{ + string[string] table; + string[TOK.MAX] tokenTable; + + this(string[string] table) + { + this.table = table; + Identifier = this["Identifier", "{0}"]; + String = this["String", "{0}"]; + Char = this["Char", "{0}"]; + Number = this["Number", "{0}"]; + Keyword = this["Keyword", "{0}"]; + LineC = this["LineC", "{0}"]; + BlockC = this["BlockC", "{0}"]; + NestedC = this["NestedC", "{0}"]; + Shebang = this["Shebang", "{0}"]; + HLine = this["HLine", "{0}"]; + Filespec = this["Filespec", "{0}"]; + Illegal = this["Illegal", "{0}"]; + Newline = this["Newline", "{0}"]; + SpecialToken = this["SpecialToken", "{0}"]; + Declaration = this["Declaration", "d"]; + Statement = this["Statement", "s"]; + Expression = this["Expression", "e"]; + Type = this["Type", "t"]; + Other = this["Other", "o"]; + EOF = this["EOF", ""]; + + foreach (i, tokStr; tokToString) + if (auto pStr = tokStr in this.table) + tokenTable[i] = *pStr; + } + + /// Returns the value for str, or 'fallback' if str is not in the table. + string opIndex(string str, string fallback = "") + { + auto p = str in table; + if (p) + return *p; + return fallback; + } + + /// Returns the value for tok in O(1) time. + string opIndex(TOK tok) + { + return tokenTable[tok]; + } + + /// Shortcuts for quick access. + string Identifier, String, Char, Number, Keyword, LineC, BlockC, + NestedC, Shebang, HLine, Filespec, Illegal, Newline, SpecialToken, + Declaration, Statement, Expression, Type, Other, EOF; + + /// Returns the tag for the category 'nc'. + string getTag(NodeCategory nc) + { + string tag; + switch (nc) + { alias NodeCategory NC; + case NC.Declaration: tag = Declaration; break; + case NC.Statement: tag = Statement; break; + case NC.Expression: tag = Expression; break; + case NC.Type: tag = Type; break; + case NC.Other: tag = Other; break; + default: assert(0); + } + return tag; + } +} + +/// Find the last occurrence of object in subject. +/// Returns: the index if found, or -1 if not. +int rfind(char[] subject, char object) +{ + foreach_reverse(i, c; subject) + if (c == object) + return i; + return -1; +} + +/// Returns the short class name of a class descending from Node.$(BR) +/// E.g.: dil.ast.Declarations.ClassDeclaration -> Class +char[] getShortClassName(Node node) +{ + static char[][] name_table; + if (name_table is null) + name_table = new char[][NodeKind.max+1]; // Create a new table. + // Look up in table. + char[] name = name_table[node.kind]; + if (name !is null) + return name; // Return cached name. + + name = node.classinfo.name; // Get the fully qualified name of the class. + name = name[rfind(name, '.')+1 .. $]; // Remove package and module name. + + uint suffixLength; + switch (node.category) + { + alias NodeCategory NC; + case NC.Declaration: + suffixLength = "Declaration".length; + break; + case NC.Statement: + suffixLength = "Statement".length; + break; + case NC.Expression: + suffixLength = "Expression".length; + break; + case NC.Type: + suffixLength = "Type".length; + break; + case NC.Other: + break; + default: + assert(0); + } + // Remove common suffix. + name = name[0 .. $ - suffixLength]; + // Store the name in the table. + name_table[node.kind] = name; + return name; +} + +/// Extended token structure. +struct TokenEx +{ + Token* token; /// The lexer token. + Node[] beginNodes; /// beginNodes[n].begin == token + Node[] endNodes; /// endNodes[n].end == token +} + +/// Builds an array of TokenEx items. +class TokenExBuilder : DefaultVisitor +{ + private TokenEx*[Token*] tokenTable; + + TokenEx[] build(Node root, Token* first) + { + auto token = first; + + uint count; // Count tokens. + for (; token; token = token.next) + count++; + // Creat the exact number of TokenEx instances. + auto toks = new TokenEx[count]; + token = first; + foreach (ref tokEx; toks) + { + tokEx.token = token; + if (!token.isWhitespace) + tokenTable[token] = &tokEx; + token = token.next; + } + + super.visitN(root); + tokenTable = null; + return toks; + } + + TokenEx* getTokenEx()(Token* t) + { + auto p = t in tokenTable; + assert(p, t.srcText~" is not in tokenTable"); + return *p; + } + + // Override dispatch function. + override Node dispatch(Node n) + { + auto begin = n.begin; + if (begin) + { assert(n.end); + auto txbegin = getTokenEx(begin); + auto txend = getTokenEx(n.end); + txbegin.beginNodes ~= n; + txend.endNodes ~= n; + } + return super.dispatch(n); + } +} + +void printErrors(Lexer lx, TagMap tags, Print!(char) print) +{ + foreach (e; lx.errors) + print.format(tags["LexerError"], e.filePath, e.loc, e.col, xml_escape(e.getMsg)); +} + +void printErrors(Parser parser, TagMap tags, Print!(char) print) +{ + foreach (e; parser.errors) + print.format(tags["ParserError"], e.filePath, e.loc, e.col, xml_escape(e.getMsg)); +} + +void printLines(uint lines, TagMap tags, Print!(char) print) +{ + auto lineNumberFormat = tags["LineNumber"]; + for (auto lineNum = 1; lineNum <= lines; lineNum++) + print.format(lineNumberFormat, lineNum); +} + +// void printMultiline(Token* token, TagMap tags, Print!(char) print) +// { +// } + +/// Highlights the syntax in a source file. +void highlightSyntax(string filePath, TagMap tags, Print!(char) print, GenOption options) +{ + auto parser = new Parser(new SourceText(filePath, true)); + auto root = parser.start(); + auto lx = parser.lexer; + + auto builder = new TokenExBuilder(); + auto tokenExList = builder.build(root, lx.firstToken()); + + print(tags["DocHead"]); + if (lx.errors.length || parser.errors.length) + { // Output error messages. + print(tags["CompBegin"]); + printErrors(lx, tags, print); + printErrors(parser, tags, print); + print(tags["CompEnd"]); + } + + if (options & GenOption.PrintLines) + { + print(tags["LineNumberBegin"]); + printLines(lx.lineNum, tags, print); + print(tags["LineNumberEnd"]); + } + + print(tags["SourceBegin"]); + + auto tagNodeBegin = tags["NodeBegin"]; + auto tagNodeEnd = tags["NodeEnd"]; + + // Iterate over list of tokens. + foreach (ref tokenEx; tokenExList) + { + auto token = tokenEx.token; + + token.ws && print(token.wsChars); // Print preceding whitespace. + if (token.isWhitespace) { + printToken(token, tags, print); + continue; + } + // + foreach (node; tokenEx.beginNodes) + print.format(tagNodeBegin, tags.getTag(node.category), getShortClassName(node)); + // Token text. + printToken(token, tags, print); + // + if (options & GenOption.HTML) + foreach_reverse (node; tokenEx.endNodes) + print(tagNodeEnd); + else + foreach_reverse (node; tokenEx.endNodes) + print.format(tagNodeEnd, tags.getTag(node.category)); + } + print(tags["SourceEnd"]); + print(tags["DocEnd"]); +} + +/// Highlights all tokens of a source file. +void highlightTokens(string filePath, TagMap tags, Print!(char) print, GenOption options) +{ + auto lx = new Lexer(new SourceText(filePath, true)); + lx.scanAll(); + + print(tags["DocHead"]); + if (lx.errors.length) + { + print(tags["CompBegin"]); + printErrors(lx, tags, print); + print(tags["CompEnd"]); + } + + if (options & GenOption.PrintLines) + { + print(tags["LineNumberBegin"]); + printLines(lx.lineNum, tags, print); + print(tags["LineNumberEnd"]); + } + + print(tags["SourceBegin"]); + // Traverse linked list and print tokens. + for (auto token = lx.firstToken(); token; token = token.next) { + token.ws && print(token.wsChars); // Print preceding whitespace. + printToken(token, tags, print); + } + print(tags["SourceEnd"]); + print(tags["DocEnd"]); +} + +/// A token highlighter designed for DDoc. +class TokenHighlighter +{ + TagMap tags; + this(InfoManager infoMan, bool useHTML = true) + { + string filePath = GlobalSettings.htmlMapFile; + if (!useHTML) + filePath = GlobalSettings.xmlMapFile; + auto map = TagMapLoader(infoMan).load(filePath); + tags = new TagMap(map); + } + + /// Highlights tokens in a DDoc code section. + /// Returns: a string with the highlighted tokens (in HTML tags.) + string highlight(string text, string filePath) + { + auto buffer = new GrowBuffer(text.length); + auto print = new Print!(char)(Format, buffer); + + auto lx = new Lexer(new SourceText(filePath, text)); + lx.scanAll(); + + // Traverse linked list and print tokens. + print("$(D_CODE\n"); + if (lx.errors.length) + { // Output error messages. + print(tags["CompBegin"]); + printErrors(lx, tags, print); + print(tags["CompEnd"]); + } + // Traverse linked list and print tokens. + for (auto token = lx.firstToken(); token; token = token.next) { + token.ws && print(token.wsChars); // Print preceding whitespace. + printToken(token, tags, print); + } + print("\n)"); + return cast(char[])buffer.slice(); + } +} + +/// Prints a token to the stream print. +void printToken(Token* token, TagMap tags, Print!(char) print) +{ + switch(token.kind) + { + case TOK.Identifier: + print.format(tags.Identifier, token.srcText); + break; + case TOK.Comment: + string formatStr; + switch (token.start[1]) + { + case '/': formatStr = tags.LineC; break; + case '*': formatStr = tags.BlockC; break; + case '+': formatStr = tags.NestedC; break; + default: assert(0); + } + print.format(formatStr, xml_escape(token.srcText)); + break; + case TOK.String: + print.format(tags.String, xml_escape(token.srcText)); + break; + case TOK.CharLiteral: + print.format(tags.Char, xml_escape(token.srcText)); + break; + case TOK.Int32, TOK.Int64, TOK.Uint32, TOK.Uint64, + TOK.Float32, TOK.Float64, TOK.Float80, + TOK.Imaginary32, TOK.Imaginary64, TOK.Imaginary80: + print.format(tags.Number, token.srcText); + break; + case TOK.Shebang: + print.format(tags.Shebang, xml_escape(token.srcText)); + break; + case TOK.HashLine: + auto formatStr = tags.HLine; + // The text to be inserted into formatStr. + auto buffer = new GrowBuffer; + auto print2 = new Print!(char)(Format, buffer); + + void printWS(char* start, char* end) + { + start != end && print2(start[0 .. end - start]); + } + + auto num = token.tokLineNum; + if (num is null) + { // Malformed #line + print.format(formatStr, token.srcText); + break; + } + + // Print whitespace between #line and number. + printWS(token.start, num.start); // Prints "#line" as well. + printToken(num, tags, print2); // Print the number. + + if (auto filespec = token.tokLineFilespec) + { // Print whitespace between number and filespec. + printWS(num.end, filespec.start); + print2.format(tags.Filespec, xml_escape(filespec.srcText)); + } + // Finally print the whole token. + print.format(formatStr, cast(char[])buffer.slice()); + break; + case TOK.Illegal: + print.format(tags.Illegal, token.srcText()); + break; + case TOK.Newline: + print.format(tags.Newline, token.srcText()); + break; + case TOK.EOF: + print(tags.EOF); + break; + default: + if (token.isKeyword()) + print.format(tags.Keyword, token.srcText); + else if (token.isSpecialToken) + print.format(tags.SpecialToken, token.srcText); + else + print(tags[token.kind]); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/cmd/ImportGraph.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/cmd/ImportGraph.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,424 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module cmd.ImportGraph; + +import dil.ast.Node; +import dil.ast.Declarations; +import dil.semantic.Module; +import dil.parser.ImportParser; +import dil.SourceText; +import dil.Compilation; +import Settings; +import common; + +import tango.text.Regex : RegExp = Regex; +import tango.io.FilePath; +import tango.io.FileConst; +import tango.text.Util; + +alias FileConst.PathSeparatorChar dirSep; + +/// Options for the importgraph command. +enum IGraphOption +{ + None, + IncludeUnlocatableModules = 1, + PrintDot = 1<<1, + HighlightCyclicEdges = 1<<2, + HighlightCyclicVertices = 1<<3, + GroupByPackageNames = 1<<4, + GroupByFullPackageName = 1<<5, + PrintPaths = 1<<6, + PrintList = 1<<7, + MarkCyclicModules = 1<<8, +} + +/// Represents a module dependency graph. +class Graph +{ + Vertex[] vertices; /// The vertices or modules. + Edge[] edges; /// The edges or import statements. + + void addVertex(Vertex vertex) + { + vertex.id = vertices.length; + vertices ~= vertex; + } + + Edge addEdge(Vertex from, Vertex to) + { + auto edge = new Edge(from, to); + edges ~= edge; + from.outgoing ~= to; + to.incoming ~= from; + return edge; + } + + /// Walks the graph and marks cyclic vertices and edges. + void detectCycles() + { // Cycles could also be detected in the GraphBuilder, + // but having the code here makes things much clearer. + + // Commented out because this algorithm doesn't work. + // Returns true if the vertex is in status Visiting. + /+bool visit(Vertex vertex) + { + switch (vertex.status) + { + case Vertex.Status.Visiting: + vertex.isCyclic = true; + return true; + case Vertex.Status.None: + vertex.status = Vertex.Status.Visiting; // Flag as visiting. + foreach (outVertex; vertex.outgoing) // Visit successors. + vertex.isCyclic |= visit(outVertex); + vertex.status = Vertex.Status.Visited; // Flag as visited. + break; + case Vertex.Status.Visited: + break; + default: + assert(0, "unknown vertex status"); + } + return false; // return (vertex.status == Vertex.Status.Visiting); + } + // Start visiting vertices. + visit(vertices[0]);+/ + + //foreach (edge; edges) + // if (edge.from.isCyclic && edge.to.isCyclic) + // edge.isCyclic = true; + + // Use functioning algorithm. + analyzeGraph(vertices, edges); + } +} + +/// Represents a directed connection between two vertices. +class Edge +{ + Vertex from; /// Coming from vertex. + Vertex to; /// Going to vertex. + bool isCyclic; /// Edge connects cyclic vertices. + bool isPublic; /// Public import. + bool isStatic; /// Static import. + + this(Vertex from, Vertex to) + { + this.from = from; + this.to = to; + } +} + +/// Represents a module in the graph. +class Vertex +{ + Module modul; /// The module represented by this vertex. + uint id; /// The nth vertex in the graph. + Vertex[] incoming; /// Also called predecessors. + Vertex[] outgoing; /// Also called successors. + bool isCyclic; /// Whether this vertex is in a cyclic relationship with other vertices. + + enum Status : ubyte + { None, Visiting, Visited } + Status status; /// Used by the cycle detection algorithm. +} + +/// Searches for a module in the file system looking in importPaths. +/// Returns: the file path to the module, or null if it wasn't found. +string findModuleFilePath(string moduleFQNPath, string[] importPaths) +{ + auto filePath = new FilePath(); + foreach (importPath; importPaths) + { + filePath.set(importPath); + filePath.append(moduleFQNPath); + foreach (moduleSuffix; [".d", ".di"/*interface file*/]) + { + filePath.suffix(moduleSuffix); + if (filePath.exists()) + return filePath.toString(); + } + } + return null; +} + +/// Builds a module dependency graph. +class GraphBuilder +{ + Graph graph; + IGraphOption options; + string[] importPaths; /// Where to look for modules. + Vertex[string] loadedModulesTable; /// Maps FQN paths to modules. + bool delegate(string) filterPredicate; + + this() + { + this.graph = new Graph; + } + + /// Start building the graph and return that. + /// Params: + /// fileName = the file name of the root module. + Graph start(string fileName) + { + loadModule(fileName); + return graph; + } + + /// Loads all modules recursively and builds the graph at the same time. + /// Params: + /// moduleFQNPath = the path version of the module FQN.$(BR) + /// E.g.: FQN = dil.ast.Node -> FQNPath = dil/ast/Node + Vertex loadModule(string moduleFQNPath) + { + // Look up in table if the module is already loaded. + auto pVertex = moduleFQNPath in loadedModulesTable; + if (pVertex !is null) + return *pVertex; // Returns null for filtered or unlocatable modules. + + // Filter out modules. + if (filterPredicate && filterPredicate(moduleFQNPath)) + { // Store null for filtered modules. + loadedModulesTable[moduleFQNPath] = null; + return null; + } + + // Locate the module in the file system. + auto moduleFilePath = findModuleFilePath(moduleFQNPath, importPaths); + + Vertex vertex; + + if (moduleFilePath is null) + { // Module not found. + if (options & IGraphOption.IncludeUnlocatableModules) + { // Include module nevertheless. + vertex = new Vertex; + vertex.modul = new Module(""); + vertex.modul.setFQN(replace(moduleFQNPath, dirSep, '.')); + graph.addVertex(vertex); + } + // Store vertex in the table (vertex may be null.) + loadedModulesTable[moduleFQNPath] = vertex; + } + else + { + auto modul = new Module(moduleFilePath); + // Use lightweight ImportParser. + modul.setParser(new ImportParser(modul.sourceText)); + modul.parse(); + + vertex = new Vertex; + vertex.modul = modul; + + graph.addVertex(vertex); + loadedModulesTable[modul.getFQNPath()] = vertex; + + // Load the modules which this module depends on. + foreach (importDecl; modul.imports) + { + foreach (moduleFQNPath2; importDecl.getModuleFQNs(dirSep)) + { + auto loaded = loadModule(moduleFQNPath2); + if (loaded !is null) + { + auto edge = graph.addEdge(vertex, loaded); + edge.isPublic = importDecl.isPublic(); + edge.isStatic = importDecl.isStatic(); + } + } + } + } + return vertex; + } +} + +/// Executes the importgraph command. +void execute(string filePathString, CompilationContext context, string[] strRegexps, + uint levels, string siStyle, string piStyle, IGraphOption options) +{ + // Init regular expressions. + RegExp[] regexps; + foreach (strRegexp; strRegexps) + regexps ~= new RegExp(strRegexp); + + // Add the directory of the file to the import paths. + auto filePath = new FilePath(filePathString); + auto fileDir = filePath.folder(); + context.importPaths ~= fileDir; + + auto gbuilder = new GraphBuilder; + + gbuilder.importPaths = context.importPaths; + gbuilder.options = options; + gbuilder.filterPredicate = (string moduleFQNPath) { + foreach (rx; regexps) + // Replace slashes: dil/ast/Node -> dil.ast.Node + if (rx.test(replace(moduleFQNPath.dup, dirSep, '.'))) + return true; + return false; + }; + + auto graph = gbuilder.start(filePath.name()); + + if (options & (IGraphOption.PrintList | IGraphOption.PrintPaths)) + { + if (options & IGraphOption.MarkCyclicModules) + graph.detectCycles(); + + if (options & IGraphOption.PrintPaths) + printModulePaths(graph.vertices, levels+1, ""); + else + printModuleList(graph.vertices, levels+1, ""); + } + else + printDotDocument(graph, siStyle, piStyle, options); +} + +/// Prints the file paths to the modules. +void printModulePaths(Vertex[] vertices, uint level, char[] indent) +{ + if (level == 0) + return; + foreach (vertex; vertices) + { + Stdout(indent)((vertex.isCyclic?"*":"")~vertex.modul.filePath).newline; + if (vertex.outgoing.length) + printModulePaths(vertex.outgoing, level-1, indent~" "); + } +} + +/// Prints a list of module FQNs. +void printModuleList(Vertex[] vertices, uint level, char[] indent) +{ + if (level == 0) + return; + foreach (vertex; vertices) + { + Stdout(indent)((vertex.isCyclic?"*":"")~vertex.modul.getFQN()).newline; + if (vertex.outgoing.length) + printModuleList(vertex.outgoing, level-1, indent~" "); + } +} + +/// Prints the graph as a graphviz dot document. +void printDotDocument(Graph graph, string siStyle, string piStyle, + IGraphOption options) +{ + Vertex[][string] verticesByPckgName; + if (options & IGraphOption.GroupByFullPackageName) + foreach (vertex; graph.vertices) + verticesByPckgName[vertex.modul.packageName] ~= vertex; + + if (options & (IGraphOption.HighlightCyclicVertices | + IGraphOption.HighlightCyclicEdges)) + graph.detectCycles(); + + // Output header of the dot document. + Stdout("Digraph ImportGraph\n{\n"); + // Output nodes. + // 'i' and vertex.id should be the same. + foreach (i, vertex; graph.vertices) + Stdout.formatln(` n{} [label="{}"{}];`, i, vertex.modul.getFQN(), (vertex.isCyclic ? ",style=filled,fillcolor=tomato" : "")); + + // Output edges. + foreach (edge; graph.edges) + { + string edgeStyles = ""; + if (edge.isStatic || edge.isPublic) + { + edgeStyles = `[style="`; + edge.isStatic && (edgeStyles ~= siStyle ~ ","); + edge.isPublic && (edgeStyles ~= piStyle); + edgeStyles[$-1] == ',' && (edgeStyles = edgeStyles[0..$-1]); // Remove last comma. + edgeStyles ~= `"]`; + } + edge.isCyclic && (edgeStyles ~= "[color=red]"); + Stdout.formatln(` n{} -> n{} {};`, edge.from.id, edge.to.id, edgeStyles); + } + + if (options & IGraphOption.GroupByFullPackageName) + foreach (packageName, vertices; verticesByPckgName) + { // Output nodes in a cluster. + Stdout.format(` subgraph "cluster_{}" {`\n` label="{}";color=blue;`"\n ", packageName, packageName); + foreach (vertex; vertices) + Stdout.format(`n{};`, vertex.id); + Stdout("\n }\n"); + } + + Stdout("}\n"); +} + +// This is the old algorithm that was used to detect cycles in a directed graph. +void analyzeGraph(Vertex[] vertices_init, Edge[] edges) +{ + edges = edges.dup; + void recursive(Vertex[] vertices) + { + foreach (idx, vertex; vertices) + { + uint outgoing, incoming; + foreach (j, edge; edges) + { + if (edge.from is vertex) + outgoing++; + if (edge.to is vertex) + incoming++; + } + + if (outgoing == 0) + { + if (incoming != 0) + { + // Vertex is a sink. + alias outgoing i; // Reuse + alias incoming j; // Reuse + // Remove edges. + for (i=j=0; i < edges.length; i++) + if (edges[i].to !is vertex) + edges[j++] = edges[i]; + edges.length = j; + vertices = vertices[0..idx] ~ vertices[idx+1..$]; + recursive(vertices); + return; + } + else + { + // Edges to this vertex were removed previously. + // Only remove vertex now. + vertices = vertices[0..idx] ~ vertices[idx+1..$]; + recursive(vertices); + return; + } + } + else if (incoming == 0) + { + // Vertex is a source + alias outgoing i; // Reuse + alias incoming j; // Reuse + // Remove edges. + for (i=j=0; i < edges.length; i++) + if (edges[i].from !is vertex) + edges[j++] = edges[i]; + edges.length = j; + vertices = vertices[0..idx] ~ vertices[idx+1..$]; + recursive(vertices); + return; + } +// else +// { +// // source && sink +// // continue loop. +// } + } + + // When reaching this point it means only cylic edges and vertices are left. + foreach (vertex; vertices) + vertex.isCyclic = true; + foreach (edge; edges) + if (edge) + edge.isCyclic = true; + } + recursive(vertices_init); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/cmd/Statistics.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/cmd/Statistics.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,212 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module cmd.Statistics; + +import cmd.ASTStats; +import dil.lexer.Lexer; +import dil.lexer.Token; +import dil.parser.Parser; +import dil.ast.NodesEnum; +import dil.SourceText; +import common; + +/// A group of statistics variables. +struct Statistics +{ + uint whitespaceCount; /// Counter for whitespace characters. + uint wsTokenCount; /// Counter for all whitespace tokens. + uint keywordCount; /// Counter for keywords. + uint identCount; /// Counter for identifiers. + uint numberCount; /// Counter for number literals. + uint commentCount; /// Counter for comments. + uint tokenCount; /// Counter for all tokens produced by the Lexer. + uint linesOfCode; /// Number of lines. + uint[] tokensTable; /// Table of counters for all token kinds. + uint[] nodesTable; /// Table of counters for all node kinds. + + static Statistics opCall(bool allocateTokensTable, bool allocateNodesTable = false) + { + Statistics s; + if (allocateTokensTable) + s.tokensTable = new uint[TOK.MAX]; + if (allocateNodesTable) + s.nodesTable = new uint[g_classNames.length]; + return s; + } + + void opAddAssign(Statistics s) + { + this.whitespaceCount += s.whitespaceCount; + this.wsTokenCount += s.wsTokenCount; + this.keywordCount += s.keywordCount; + this.identCount += s.identCount; + this.numberCount += s.numberCount; + this.commentCount += s.commentCount; + this.tokenCount += s.tokenCount; + this.linesOfCode += s.linesOfCode; + foreach (i, count; s.tokensTable) + this.tokensTable[i] += count; + foreach (i, count; s.nodesTable) + this.nodesTable[i] += count; + } +} + +/// Executes the statistics command. +void execute(string[] filePaths, bool printTokensTable, bool printNodesTable) +{ + Statistics[] stats; + foreach (filePath; filePaths) + stats ~= getStatistics(filePath, printTokensTable, printNodesTable); + + auto total = Statistics(printTokensTable, printNodesTable); + + foreach (i, ref stat; stats) + { + total += stat; + Stdout.formatln( + "----\n" + "File: {}\n" + "Whitespace character count: {}\n" + "Whitespace token count: {}\n" + "Keyword count: {}\n" + "Identifier count: {}\n" + "Number count: {}\n" + "Comment count: {}\n" + "All tokens count: {}\n" + "Lines of code: {}", + filePaths[i], + stat.whitespaceCount, + stat.wsTokenCount, + stat.keywordCount, + stat.identCount, + stat.numberCount, + stat.commentCount, + stat.tokenCount, + stat.linesOfCode + ); + } + + if (filePaths.length > 1) + { + Stdout.formatln( + "--------------------------------------------------------------------------------\n" + "Total of {} files:\n" + "Whitespace character count: {}\n" + "Whitespace token count: {}\n" + "Keyword count: {}\n" + "Identifier count: {}\n" + "Number count: {}\n" + "Comment count: {}\n" + "All tokens count: {}\n" + "Lines of code: {}", + filePaths.length, + total.whitespaceCount, + total.wsTokenCount, + total.keywordCount, + total.identCount, + total.numberCount, + total.commentCount, + total.tokenCount, + total.linesOfCode + ); + } + + if (printTokensTable) + { + Stdout("Table of tokens:").newline; + Stdout.formatln(" {,10} | {}", "Count", "Token kind"); + Stdout("-----------------------------").newline; + foreach (i, count; total.tokensTable) + Stdout.formatln(" {,10} | {}", count, Token.toString(cast(TOK)i)); + Stdout("// End of tokens table.").newline; + } + + if(printNodesTable) + { + Stdout("Table of nodes:").newline; + Stdout.formatln(" {,10} | {}", "Count", "Node kind"); + Stdout("-----------------------------").newline; + foreach (i, count; total.nodesTable) + Stdout.formatln(" {,10} | {}", count, g_classNames[i]); + Stdout("// End of nodes table.").newline; + } +} + +/// Returns the statistics for a D source file. +Statistics getStatistics(string filePath, bool printTokensTable, bool printNodesTable) +{ + // Create a new record. + auto stats = Statistics(printTokensTable); + + auto sourceText = new SourceText(filePath, true); + Parser parser; + Lexer lx; + if (printNodesTable) + { + parser = new Parser(sourceText); + auto rootNode = parser.start(); + // Count nodes. + stats.nodesTable = (new ASTStats).count(rootNode); + lx = parser.lexer; + } + else + { + lx = new Lexer(sourceText); + lx.scanAll(); + } + + auto token = lx.firstToken(); + + // Count tokens. + // Lexer creates HEAD + Newline, which are not in the source text. + // No token left behind! + stats.tokenCount = 2; + stats.linesOfCode = lx.lineNum; + if (printTokensTable) + { + stats.tokensTable[TOK.HEAD] = 1; + stats.tokensTable[TOK.Newline] = 1; + } + // Traverse linked list. + while (1) + { + stats.tokenCount += 1; + + if (printTokensTable) + stats.tokensTable[token.kind] += 1; + + // Count whitespace characters + if (token.ws !is null) + stats.whitespaceCount += token.start - token.ws; + + switch (token.kind) + { + case TOK.Identifier: + stats.identCount++; + break; + case TOK.Comment: + stats.commentCount++; + break; + case TOK.Int32, TOK.Int64, TOK.Uint32, TOK.Uint64, + TOK.Float32, TOK.Float64, TOK.Float80, + TOK.Imaginary32, TOK.Imaginary64, TOK.Imaginary80: + stats.numberCount++; + break; + case TOK.Newline: + break; + default: + if (token.isKeyword) + stats.keywordCount++; + else if (token.isWhitespace) + stats.wsTokenCount++; + } + + if (token.next is null) + break; + token = token.next; + } + assert(token.kind == TOK.EOF); + return stats; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/common.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/common.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,20 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module common; + +public import tango.io.Stdout; +public import tango.text.convert.Layout; + +/// String aliases. +alias char[] string; +alias wchar[] wstring; /// ditto +alias dchar[] dstring; /// ditto + +/// Global formatter instance. +static Layout!(char) Format; +static this() +{ + Format = new typeof(Format); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/config.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/config.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,34 @@ +/// The configuration file of dil. +/// +/// Relative paths are resolved from the directory of the executable. +module config; + +/// Predefined version identifiers. +var version_ids = ["X86", "linux", "LittleEndian"]; +// "X86_64", "Windows", "Win32", "Win64", "BigEndian" + +/// Path to the language file. +var langfile = "lang_en.d"; + +/// An array of import paths to look for modules. +var import_paths = []; /// E.g.: ["src/", "import/"] + +/// DDoc macro file paths. +/// +/// Macro definitions in ddoc_files[n] override the ones in ddoc_files[n-1]. +var ddoc_files = ["predefined.ddoc"]; /// E.g.: ["src/mymacros.ddoc", "othermacros.ddoc"] + +var xml_map = "xml_map.d"; +var html_map = "html_map.d"; + +/// Customizable formats for error messages. +/// +///
    +///
  • 0: file path to the source text.
    • +///
  • 1: line number.
    • +///
  • 2: column number.
    • +///
  • 3: error message.
    • +///
+var lexer_error = "{0}({1},{2})L: {3}"; +var parser_error = "{0}({1},{2})P: {3}"; /// ditto +var semantic_error = "{0}({1},{2})S: {3}"; /// ditto diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Compilation.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Compilation.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,69 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Compilation; + +import common; + +/// A group of settings relevant to the compilation process. +class CompilationContext +{ + alias typeof(this) CC; + CC parent; + string[] importPaths; + uint debugLevel; + uint versionLevel; + bool[string] debugIds; + bool[string] versionIds; + bool releaseBuild; + uint structAlign = 4; + + this(CC parent = null) + { + this.parent = parent; + if (parent) + { + this.importPaths = parent.importPaths.dup; + this.debugLevel = parent.debugLevel; + this.versionLevel = parent.versionLevel; + this.releaseBuild = parent.releaseBuild; + this.structAlign = parent.structAlign; + } + } + + void addDebugId(string id) + { + debugIds[id] = true; + } + + void addVersionId(string id) + { + versionIds[id] = true; + } + + bool findDebugId(string id) + { + auto pId = id in debugIds; + if (pId) + return true; + if (!isRoot()) + return parent.findDebugId(id); + return false; + } + + bool findVersionId(string id) + { + auto pId = id in versionIds; + if (pId) + return true; + if (!isRoot()) + return parent.findVersionId(id); + return false; + } + + bool isRoot() + { + return parent is null; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/CompilerInfo.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/CompilerInfo.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,50 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.CompilerInfo; + +version(D2) + /// The major version number of this compiler. + const uint VERSION_MAJOR = 2; +else + /// The major version number of this compiler. + const uint VERSION_MAJOR = 1; + +/// The minor version number of this compiler. +const uint VERSION_MINOR = 0; + +private char[] toString(uint x) +{ + char[] str; + do + str = cast(char)('0' + (x % 10)) ~ str; + while (x /= 10) + return str; +} + +private char[] toString(uint x, uint pad) +{ + char[] str = toString(x); + if (pad < str.length) + return str; + for (uint i = pad-str.length; i; i--) + str = "0" ~ str; + return str; +} + +/// The compiler version formatted as a string. +const char[] VERSION = toString(VERSION_MAJOR)~"."~toString(VERSION_MINOR, 3); +/// The name of the compiler. +const char[] VENDOR = "dil"; + +/// The global, default alignment size for struct fields. +const uint DEFAULT_ALIGN_SIZE = 4; + +version(DDoc) + const uint PTR_SIZE = 0; /// The pointer size depending on the platform. +else +version(X86_64) + const uint PTR_SIZE = 8; // Pointer size on 64-bit platforms. +else + const uint PTR_SIZE = 4; // Pointer size on 32-bit platforms. diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Converter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Converter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,334 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Converter; + +import dil.Information; +import dil.Location; +import dil.Unicode; +import dil.FileBOM; +import dil.lexer.Funcs; +import dil.Messages; +import common; + +/// Converts various Unicode encoding formats to UTF-8. +struct Converter +{ + char[] filePath; /// For error messages. + InfoManager infoMan; + + static Converter opCall(char[] filePath, InfoManager infoMan) + { + Converter conv; + conv.filePath = filePath; + conv.infoMan = infoMan; + return conv; + } + + /// Byte-swaps c. + dchar swapBytes(dchar c) + { + return c = (c << 24) | + ((c >> 8) & 0xFF00) | + ((c << 8) & 0xFF0000) | + (c >> 24); + } + + /// Byte-swaps c. + wchar swapBytes(wchar c) + { + return (c << 8) | (c >> 8); + } + + /// Swaps the bytes of c on a little-endian machine. + dchar BEtoMachineDword(dchar c) + { + version(LittleEndian) + return swapBytes(c); + else + return c; + } + + /// Swaps the bytes of c on a big-endian machine. + dchar LEtoMachineDword(dchar c) + { + version(LittleEndian) + return c; + else + return swapBytes(c); + } + + /// Swaps the bytes of c on a little-endian machine. + wchar BEtoMachineWord(wchar c) + { + version(LittleEndian) + return swapBytes(c); + else + return c; + } + + /// Swaps the bytes of c on a big-endian machine. + wchar LEtoMachineWord(wchar c) + { + version(LittleEndian) + return c; + else + return swapBytes(c); + } + + /// Converts a UTF-32 text to UTF-8. + char[] UTF32toUTF8(bool isBigEndian)(ubyte[] data) + { + if (data.length == 0) + return null; + + char[] result; + uint lineNum = 1; + dchar[] text = cast(dchar[]) data[0 .. $-($%4)]; // Trim to multiple of 4. + foreach (dchar c; text) + { + static if (isBigEndian) + c = BEtoMachineDword(c); + else + c = LEtoMachineDword(c); + + if (!isValidChar(c)) + { + infoMan ~= new LexerError( + new Location(filePath, lineNum), + Format(MSG.InvalidUTF32Character, c) + ); + c = REPLACEMENT_CHAR; + } + + if (isNewline(c)) + ++lineNum; + dil.Unicode.encode(result, c); + } + + if (data.length % 4) + infoMan ~= new LexerError( + new Location(filePath, lineNum), + MSG.UTF32FileMustBeDivisibleBy4 + ); + + return result; + } + + alias UTF32toUTF8!(true) UTF32BEtoUTF8; /// Instantiation for UTF-32 BE. + alias UTF32toUTF8!(false) UTF32LEtoUTF8; /// Instantiation for UTF-32 LE. + + /// Converts a UTF-16 text to UTF-8. + char[] UTF16toUTF8(bool isBigEndian)(ubyte[] data) + { + if (data.length == 0) + return null; + + wchar[] text = cast(wchar[]) data[0 .. $-($%2)]; // Trim to multiple of two. + wchar* p = text.ptr, + end = text.ptr + text.length; + char[] result; + uint lineNum = 1; + + for (; p < end; p++) + { + dchar c = *p; + static if (isBigEndian) + c = BEtoMachineWord(c); + else + c = LEtoMachineWord(c); + + if (0xD800 > c || c > 0xDFFF) + {} + else if (c <= 0xDBFF && p+1 < end) + { // Decode surrogate pairs. + wchar c2 = p[1]; + static if (isBigEndian) + c2 = BEtoMachineWord(c2); + else + c2 = LEtoMachineWord(c2); + + if (0xDC00 <= c2 && c2 <= 0xDFFF) + { + c = (c - 0xD7C0) << 10; + c |= (c2 & 0x3FF); + ++p; + } + } + else + { + infoMan ~= new LexerError( + new Location(filePath, lineNum), + Format(MSG.InvalidUTF16Character, c) + ); + c = REPLACEMENT_CHAR; + } + + if (isNewline(c)) + ++lineNum; + dil.Unicode.encode(result, c); + } + + if (data.length % 2) + infoMan ~= new LexerError( + new Location(filePath, lineNum), + MSG.UTF16FileMustBeDivisibleBy2 + ); + return result; + } + + alias UTF16toUTF8!(true) UTF16BEtoUTF8; /// Instantiation for UTF-16 BE. + alias UTF16toUTF8!(false) UTF16LEtoUTF8; /// Instantiation for UTF-16 LE. + + /// Converts the text in data to UTF-8. + /// Leaves data unchanged if it is in UTF-8 already. + char[] data2UTF8(ubyte[] data) + { + if (data.length == 0) + return ""; + + char[] text; + BOM bom = tellBOM(data); + + switch (bom) + { + case BOM.None: + // No BOM found. According to the specs the first character + // must be an ASCII character. + if (data.length >= 4) + { + if (data[0..3] == cast(ubyte[3])x"00 00 00") + { + text = UTF32BEtoUTF8(data); // UTF-32BE: 00 00 00 XX + break; + } + else if (data[1..4] == cast(ubyte[3])x"00 00 00") + { + text = UTF32LEtoUTF8(data); // UTF-32LE: XX 00 00 00 + break; + } + } + if (data.length >= 2) + { + if (data[0] == 0) // UTF-16BE: 00 XX + { + text = UTF16BEtoUTF8(data); + break; + } + else if (data[1] == 0) // UTF-16LE: XX 00 + { + text = UTF16LEtoUTF8(data); + break; + } + } + text = cast(char[])data; // UTF-8 + break; + case BOM.UTF8: + text = cast(char[])data[3..$]; + break; + case BOM.UTF16BE: + text = UTF16BEtoUTF8(data[2..$]); + break; + case BOM.UTF16LE: + text = UTF16LEtoUTF8(data[2..$]); + break; + case BOM.UTF32BE: + text = UTF32BEtoUTF8(data[4..$]); + break; + case BOM.UTF32LE: + text = UTF32LEtoUTF8(data[4..$]); + break; + default: + assert(0); + } + return text; + } +} + +/// Replaces invalid UTF-8 sequences with U+FFFD (if there's enough space,) +/// and Newlines with '\n'. +string sanitizeText(string text) +{ + if (!text.length) + return null; + + char* p = text.ptr; + char* end = p + text.length; + char* q = p; + + for (; p < end; p++, q++) + { + assert(q <= p); + switch (*p) + { + case '\r': + if (p+1 < end && p[1] == '\n') + p++; + case '\n': + *q = '\n'; + continue; + default: + if (isascii(*p)) + break; + if (p+2 < end && isUnicodeNewline(p)) + { + p += 2; + goto case '\n'; + } + auto p2 = p; // Beginning of the UTF-8 sequence. + dchar c = decode(p, end); + if (c == ERROR_CHAR) + { // Skip to next ASCII character or valid UTF-8 sequence. + while (++p < end && isTrailByte(*p)) + {} + alias REPLACEMENT_STR R; + if (q+2 < p) // Copy replacement char if there is enough space. + (*q = R[0]), (*++q = R[1]), (*++q = R[2]); + p--; + } + else + { // Copy the valid UTF-8 sequence. + while (p2 <= p) // p points to the last trail byte. + *q++ = *p2++; // Copy code units. + q--; + } + continue; + } + assert(isascii(*p)); + *q = *p; + } + assert(p == end); + text.length = text.length - (p - q); + //text = text.ptr[0 .. q - text.ptr]; // Another way. + return text; +} + +unittest +{ + Stdout("Testing function Converter.\n"); + struct Data2Text + { + char[] text; + char[] expected = "source"; + ubyte[] data() + { return cast(ubyte[])text; } + } + const Data2Text[] map = [ + // Without BOM + {"source"}, + {"s\0o\0u\0r\0c\0e\0"}, + {"\0s\0o\0u\0r\0c\0e"}, + {"s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e\0\0\0"}, + {"\0\0\0s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e"}, + // With BOM + {"\xEF\xBB\xBFsource"}, + {"\xFE\xFF\0s\0o\0u\0r\0c\0e"}, + {"\xFF\xFEs\0o\0u\0r\0c\0e\0"}, + {"\x00\x00\xFE\xFF\0\0\0s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e"}, + {"\xFF\xFE\x00\x00s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e\0\0\0"}, + ]; + auto converter = Converter("", new InfoManager); + foreach (i, pair; map) + assert(converter.data2UTF8(pair.data) == pair.expected, Format("failed at item {}", i)); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Enums.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Enums.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,121 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Enums; + +import common; + +/// Enumeration of storage classes. +enum StorageClass +{ + None = 0, + Abstract = 1, + Auto = 1<<2, + Const = 1<<3, + Deprecated = 1<<4, + Extern = 1<<5, + Final = 1<<6, + Invariant = 1<<7, + Override = 1<<8, + Scope = 1<<9, + Static = 1<<10, + Synchronized = 1<<11, + In = 1<<12, + Out = 1<<13, + Ref = 1<<14, + Lazy = 1<<15, + Variadic = 1<<16, +} + +/// Enumeration of protection attributes. +enum Protection +{ + None, + Private/+ = 1+/, + Protected/+ = 1<<1+/, + Package/+ = 1<<2+/, + Public/+ = 1<<3+/, + Export/+ = 1<<4+/ +} + +/// Enumeration of linkage types. +enum LinkageType +{ + None, + C, + Cpp, + D, + Windows, + Pascal, + System +} + +/// Returns the string for prot. +string toString(Protection prot) +{ + switch (prot) + { alias Protection P; + case P.None: return ""; + case P.Private: return "private"; + case P.Protected: return "protected"; + case P.Package: return "package"; + case P.Public: return "public"; + case P.Export: return "export"; + default: + assert(0); + } +} + +/// Returns the string of a storage class. Only one bit may be set. +string toString(StorageClass stc) +{ + switch (stc) + { alias StorageClass SC; + case SC.Abstract: return "abstract"; + case SC.Auto: return "auto"; + case SC.Const: return "const"; + case SC.Deprecated: return "deprecated"; + case SC.Extern: return "extern"; + case SC.Final: return "final"; + case SC.Invariant: return "invariant"; + case SC.Override: return "override"; + case SC.Scope: return "scope"; + case SC.Static: return "static"; + case SC.Synchronized: return "synchronized"; + case SC.In: return "in"; + case SC.Out: return "out"; + case SC.Ref: return "ref"; + case SC.Lazy: return "lazy"; + case SC.Variadic: return "variadic"; + default: + assert(0); + } +} + +/// Returns the strings for stc. Any number of bits may be set. +string[] toStrings(StorageClass stc) +{ + string[] result; + for (auto i = StorageClass.max; i; i >>= 1) + if (stc & i) + result ~= toString(i); + return result; +} + +/// Returns the string for ltype. +string toString(LinkageType ltype) +{ + switch (ltype) + { alias LinkageType LT; + case LT.None: return ""; + case LT.C: return "C"; + case LT.Cpp: return "Cpp"; + case LT.D: return "D"; + case LT.Windows: return "Windows"; + case LT.Pascal: return "Pascal"; + case LT.System: return "System"; + default: + assert(0); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/FileBOM.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/FileBOM.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,86 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.FileBOM; +import common; + +/// Enumeration of byte order marks. +enum BOM +{ + None, /// No BOM + UTF8, /// UTF-8: EF BB BF + UTF16BE, /// UTF-16 Big Endian: FE FF + UTF16LE, /// UTF-16 Little Endian: FF FE + UTF32BE, /// UTF-32 Big Endian: 00 00 FE FF + UTF32LE /// UTF-32 Little Endian: FF FE 00 00 +} + +/// Looks at the first bytes of data and returns the corresponding BOM. +BOM tellBOM(ubyte[] data) +{ + BOM bom = BOM.None; + if (data.length < 2) + return bom; + + if (data[0..2] == cast(ubyte[2])x"FE FF") + { + bom = BOM.UTF16BE; // FE FF + } + else if (data[0..2] == cast(ubyte[2])x"FF FE") + { + if (data.length >= 4 && data[2..4] == cast(ubyte[2])x"00 00") + bom = BOM.UTF32LE; // FF FE 00 00 + else + bom = BOM.UTF16LE; // FF FE XX XX + } + else if (data[0..2] == cast(ubyte[2])x"00 00") + { + if (data.length >= 4 && data[2..4] == cast(ubyte[2])x"FE FF") + bom = BOM.UTF32BE; // 00 00 FE FF + } + else if (data[0..2] == cast(ubyte[2])x"EF BB") + { + if (data.length >= 3 && data[2] == '\xBF') + bom = BOM.UTF8; // EF BB BF + } + return bom; +} + +unittest +{ + Stdout("Testing function tellBOM().\n"); + + struct Data2BOM + { + ubyte[] data; + BOM bom; + } + alias ubyte[] ub; + const Data2BOM[] map = [ + {cast(ub)x"12", BOM.None}, + {cast(ub)x"12 34", BOM.None}, + {cast(ub)x"00 00 FF FE", BOM.None}, + {cast(ub)x"EF BB FF", BOM.None}, + + {cast(ub)x"EF", BOM.None}, + {cast(ub)x"EF BB", BOM.None}, + {cast(ub)x"FE", BOM.None}, + {cast(ub)x"FF", BOM.None}, + {cast(ub)x"00", BOM.None}, + {cast(ub)x"00 00", BOM.None}, + {cast(ub)x"00 00 FE", BOM.None}, + + {cast(ub)x"FE FF 00", BOM.UTF16BE}, + {cast(ub)x"FE FF 00 FF", BOM.UTF16BE}, + + {cast(ub)x"EF BB BF", BOM.UTF8}, + {cast(ub)x"FE FF", BOM.UTF16BE}, + {cast(ub)x"FF FE", BOM.UTF16LE}, + {cast(ub)x"00 00 FE FF", BOM.UTF32BE}, + {cast(ub)x"FF FE 00 00", BOM.UTF32LE} + ]; + + foreach (pair; map) + assert(tellBOM(pair.data) == pair.bom, Format("Failed at {0}", pair.data)); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/HtmlEntities.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/HtmlEntities.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,376 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.HtmlEntities; + +import common; + +/// A named HTML entity. +struct Entity +{ + char[] name; + dchar value; +} + +/// The table of named HTML entities. +static const Entity[] namedEntities = [ + {"Aacute", '\u00C1'}, + {"aacute", '\u00E1'}, + {"Acirc", '\u00C2'}, + {"acirc", '\u00E2'}, + {"acute", '\u00B4'}, + {"AElig", '\u00C6'}, + {"aelig", '\u00E6'}, + {"Agrave", '\u00C0'}, + {"agrave", '\u00E0'}, + {"alefsym", '\u2135'}, + {"Alpha", '\u0391'}, + {"alpha", '\u03B1'}, + {"amp", '\u0026'}, + {"and", '\u2227'}, + {"ang", '\u2220'}, + {"Aring", '\u00C5'}, + {"aring", '\u00E5'}, + {"asymp", '\u2248'}, + {"Atilde", '\u00C3'}, + {"atilde", '\u00E3'}, + {"Auml", '\u00C4'}, + {"auml", '\u00E4'}, + {"bdquo", '\u201E'}, + {"Beta", '\u0392'}, + {"beta", '\u03B2'}, + {"brvbar", '\u00A6'}, + {"bull", '\u2022'}, + {"cap", '\u2229'}, + {"Ccedil", '\u00C7'}, + {"ccedil", '\u00E7'}, + {"cedil", '\u00B8'}, + {"cent", '\u00A2'}, + {"Chi", '\u03A7'}, + {"chi", '\u03C7'}, + {"circ", '\u02C6'}, + {"clubs", '\u2663'}, + {"cong", '\u2245'}, + {"copy", '\u00A9'}, + {"crarr", '\u21B5'}, + {"cup", '\u222A'}, + {"curren", '\u00A4'}, + {"Dagger", '\u2021'}, + {"dagger", '\u2020'}, + {"dArr", '\u21D3'}, + {"darr", '\u2193'}, + {"deg", '\u00B0'}, + {"Delta", '\u0394'}, + {"delta", '\u03B4'}, + {"diams", '\u2666'}, + {"divide", '\u00F7'}, + {"Eacute", '\u00C9'}, + {"eacute", '\u00E9'}, + {"Ecirc", '\u00CA'}, + {"ecirc", '\u00EA'}, + {"Egrave", '\u00C8'}, + {"egrave", '\u00E8'}, + {"empty", '\u2205'}, + {"emsp", '\u2003'}, + {"ensp", '\u2002'}, + {"Epsilon", '\u0395'}, + {"epsilon", '\u03B5'}, + {"equiv", '\u2261'}, + {"Eta", '\u0397'}, + {"eta", '\u03B7'}, + {"ETH", '\u00D0'}, + {"eth", '\u00F0'}, + {"Euml", '\u00CB'}, + {"euml", '\u00EB'}, + {"euro", '\u20AC'}, + {"exist", '\u2203'}, + {"fnof", '\u0192'}, + {"forall", '\u2200'}, + {"frac12", '\u00BD'}, + {"frac14", '\u00BC'}, + {"frac34", '\u00BE'}, + {"frasl", '\u2044'}, + {"Gamma", '\u0393'}, + {"gamma", '\u03B3'}, + {"ge", '\u2265'}, + {"gt", '\u003E'}, + {"hArr", '\u21D4'}, + {"harr", '\u2194'}, + {"hearts", '\u2665'}, + {"hellip", '\u2026'}, + {"Iacute", '\u00CD'}, + {"iacute", '\u00ED'}, + {"Icirc", '\u00CE'}, + {"icirc", '\u00EE'}, + {"iexcl", '\u00A1'}, + {"Igrave", '\u00CC'}, + {"igrave", '\u00EC'}, + {"image", '\u2111'}, + {"infin", '\u221E'}, + {"int", '\u222B'}, + {"Iota", '\u0399'}, + {"iota", '\u03B9'}, + {"iquest", '\u00BF'}, + {"isin", '\u2208'}, + {"Iuml", '\u00CF'}, + {"iuml", '\u00EF'}, + {"Kappa", '\u039A'}, + {"kappa", '\u03BA'}, + {"Lambda", '\u039B'}, + {"lambda", '\u03BB'}, + {"lang", '\u2329'}, + {"laquo", '\u00AB'}, + {"lArr", '\u21D0'}, + {"larr", '\u2190'}, + {"lceil", '\u2308'}, + {"ldquo", '\u201C'}, + {"le", '\u2264'}, + {"lfloor", '\u230A'}, + {"lowast", '\u2217'}, + {"loz", '\u25CA'}, + {"lrm", '\u200E'}, + {"lsaquo", '\u2039'}, + {"lsquo", '\u2018'}, + {"lt", '\u003C'}, + {"macr", '\u00AF'}, + {"mdash", '\u2014'}, + {"micro", '\u00B5'}, + {"middot", '\u00B7'}, + {"minus", '\u2212'}, + {"Mu", '\u039C'}, + {"mu", '\u03BC'}, + {"nabla", '\u2207'}, + {"nbsp", '\u00A0'}, + {"ndash", '\u2013'}, + {"ne", '\u2260'}, + {"ni", '\u220B'}, + {"not", '\u00AC'}, + {"notin", '\u2209'}, + {"nsub", '\u2284'}, + {"Ntilde", '\u00D1'}, + {"ntilde", '\u00F1'}, + {"Nu", '\u039D'}, + {"nu", '\u03BD'}, + {"Oacute", '\u00D3'}, + {"oacute", '\u00F3'}, + {"Ocirc", '\u00D4'}, + {"ocirc", '\u00F4'}, + {"OElig", '\u0152'}, + {"oelig", '\u0153'}, + {"Ograve", '\u00D2'}, + {"ograve", '\u00F2'}, + {"oline", '\u203E'}, + {"Omega", '\u03A9'}, + {"omega", '\u03C9'}, + {"Omicron", '\u039F'}, + {"omicron", '\u03BF'}, + {"oplus", '\u2295'}, + {"or", '\u2228'}, + {"ordf", '\u00AA'}, + {"ordm", '\u00BA'}, + {"Oslash", '\u00D8'}, + {"oslash", '\u00F8'}, + {"Otilde", '\u00D5'}, + {"otilde", '\u00F5'}, + {"otimes", '\u2297'}, + {"Ouml", '\u00D6'}, + {"ouml", '\u00F6'}, + {"para", '\u00B6'}, + {"part", '\u2202'}, + {"permil", '\u2030'}, + {"perp", '\u22A5'}, + {"Phi", '\u03A6'}, + {"phi", '\u03C6'}, + {"Pi", '\u03A0'}, + {"pi", '\u03C0'}, + {"piv", '\u03D6'}, + {"plusmn", '\u00B1'}, + {"pound", '\u00A3'}, + {"Prime", '\u2033'}, + {"prime", '\u2032'}, + {"prod", '\u220F'}, + {"prop", '\u221D'}, + {"Psi", '\u03A8'}, + {"psi", '\u03C8'}, + {"quot", '\u0022'}, + {"radic", '\u221A'}, + {"rang", '\u232A'}, + {"raquo", '\u00BB'}, + {"rArr", '\u21D2'}, + {"rarr", '\u2192'}, + {"rceil", '\u2309'}, + {"rdquo", '\u201D'}, + {"real", '\u211C'}, + {"reg", '\u00AE'}, + {"rfloor", '\u230B'}, + {"Rho", '\u03A1'}, + {"rho", '\u03C1'}, + {"rlm", '\u200F'}, + {"rsaquo", '\u203A'}, + {"rsquo", '\u2019'}, + {"sbquo", '\u201A'}, + {"Scaron", '\u0160'}, + {"scaron", '\u0161'}, + {"sdot", '\u22C5'}, + {"sect", '\u00A7'}, + {"shy", '\u00AD'}, + {"Sigma", '\u03A3'}, + {"sigma", '\u03C3'}, + {"sigmaf", '\u03C2'}, + {"sim", '\u223C'}, + {"spades", '\u2660'}, + {"sub", '\u2282'}, + {"sube", '\u2286'}, + {"sum", '\u2211'}, + {"sup", '\u2283'}, + {"sup1", '\u00B9'}, + {"sup2", '\u00B2'}, + {"sup3", '\u00B3'}, + {"supe", '\u2287'}, + {"szlig", '\u00DF'}, + {"Tau", '\u03A4'}, + {"tau", '\u03C4'}, + {"there4", '\u2234'}, + {"Theta", '\u0398'}, + {"theta", '\u03B8'}, + {"thetasym", '\u03D1'}, + {"thinsp", '\u2009'}, + {"THORN", '\u00DE'}, + {"thorn", '\u00FE'}, + {"tilde", '\u02DC'}, + {"times", '\u00D7'}, + {"trade", '\u2122'}, + {"Uacute", '\u00DA'}, + {"uacute", '\u00FA'}, + {"uArr", '\u21D1'}, + {"uarr", '\u2191'}, + {"Ucirc", '\u00DB'}, + {"ucirc", '\u00FB'}, + {"Ugrave", '\u00D9'}, + {"ugrave", '\u00F9'}, + {"uml", '\u00A8'}, + {"upsih", '\u03D2'}, + {"Upsilon", '\u03A5'}, + {"upsilon", '\u03C5'}, + {"Uuml", '\u00DC'}, + {"uuml", '\u00FC'}, + {"weierp", '\u2118'}, + {"Xi", '\u039E'}, + {"xi", '\u03BE'}, + {"Yacute", '\u00DD'}, + {"yacute", '\u00FD'}, + {"yen", '\u00A5'}, + {"Yuml", '\u0178'}, + {"yuml", '\u00FF'}, + {"Zeta", '\u0396'}, + {"zeta", '\u03B6'}, + {"zwj", '\u200D'}, + {"zwnj", '\u200C'} +]; + +uint stringToHash(char[] str) +{ + uint hash; + foreach(c; str) { + hash *= 11; + hash += c; + } + return hash; +} + +char[] toString(uint x) +{ + char[] str; + do + str = cast(char)('0' + (x % 10)) ~ str; + while (x /= 10) + return str; +} + +char[] generateHashAndValueArrays() +{ + uint[] hashes; // String hashes. + dchar[] values; // Unicode codepoints. + // Build arrays: + foreach (entity; namedEntities) + { + auto hash = stringToHash(entity.name); + auto value = entity.value; + assert(hash != 0); + // Find insertion place. + uint i; + for (; i < hashes.length; ++i) + { + assert(hash != hashes[i], "bad hash function: conflicting hashes"); + if (hash < hashes[i]) + break; + } + // Insert hash and value into tables. + if (i == hashes.length) + { + hashes ~= hash; + values ~= value; + } + else + { + hashes = hashes[0..i] ~ hash ~ hashes[i..$]; // Insert before index. + values = values[0..i] ~ value ~ values[i..$]; // Insert before index. + } + assert(hashes[i] == hash && values[i] == value); + } + // Build source text: + char[] hashesText = "private static const uint[] hashes = [", + valuesText = "private static const dchar[] values = ["; + foreach (i, hash; hashes) + { + hashesText ~= toString(hash) ~ ","; + valuesText ~= toString(values[i]) ~ ","; + } + hashesText ~= "];"; + valuesText ~= "];"; + return hashesText ~"\n"~ valuesText; +} + +version(DDoc) +{ + /// Table of hash values of the entities' names. + private static const uint[] hashes; + /// Table of Unicode codepoints. + private static const dchar[] values; +} +else + mixin(generateHashAndValueArrays); +// pragma(msg, generateHashAndValueArrays()); + +/// Converts a named HTML entity into its equivalent Unicode codepoint. +/// Returns: the entity's value or 0xFFFF if it doesn't exist. +dchar entity2Unicode(char[] entity) +{ + auto hash = stringToHash(entity); + // Binary search: + size_t lower = void, index = void, upper = void; + lower = 0; + upper = hashes.length -1; + while (lower <= upper) + { + index = (lower + upper) / 2; + if (hash < hashes[index]) + upper = index - 1; + else if (hash > hashes[index]) + lower = index + 1; + else + return values[index]; // Return the Unicode codepoint. + } + return 0xFFFF; // Return error value. +} + +unittest +{ + Stdout("Testing entity2Unicode().").newline; + alias entity2Unicode f; + foreach (entity; namedEntities) + assert(f(entity.name) == entity.value, + Format("'&{};' == \\u{:X4}, not \\u{:X4}", entity.name, entity.value, cast(uint)f(entity.name)) + ); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Information.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Information.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,123 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Information; + +import dil.Messages; +import common; + +public import dil.Location; + +/// Information that can be displayed to the user. +class Information +{ + +} + +/// Collects information. +class InfoManager +{ + Information[] info; + + bool hasInfo() + { + return info.length != 0; + } + + void opCatAssign(Information info) + { + this.info ~= info; + } + + void opCatAssign(Information[] info) + { + this.info ~= info; + } +} + +/// For reporting a problem in the compilation process. +class Problem : Information +{ + Location location; + uint column; /// Cache variable for column. + string message; + + this(Location location, string message) + { + assert(location !is null); + this.location = location; + this.message = message; + } + + /// Returns the message. + string getMsg() + { + return this.message; + } + + /// Returns the line of code. + size_t loc() + { + return location.lineNum; + } + + /// Returns the column. + size_t col() + { + if (column == 0) + column = location.calculateColumn(); + return column; + } + + /// Returns the file path. + string filePath() + { + return location.filePath; + } +} + +/// For reporting warnings. +class Warning : Problem +{ + this(Location location, string message) + { + super(location, message); + } +} + +/// For reporting a compiler error. +class Error : Problem +{ + this(Location location, string message) + { + super(location, message); + } +} + +/// An error reported by the Lexer. +class LexerError : Error +{ + this(Location location, string message) + { + super(location, message); + } +} + +/// An error reported by the Parser. +class ParserError : Error +{ + this(Location location, string message) + { + super(location, message); + } +} + +/// An error reported by a semantic analyzer. +class SemanticError : Error +{ + this(Location location, string message) + { + super(location, message); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Location.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Location.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,85 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Location; + +import dil.lexer.Funcs; +import dil.Unicode; + +/// Represents a location in a source text. +final class Location +{ + char[] filePath; /// The file path. + size_t lineNum; /// The line number. + char* lineBegin, to; /// Used to calculate the column. + + /// Forwards the parameters to the second constructor. + this(char[] filePath, size_t lineNum) + { + set(filePath, lineNum); + } + + /// Constructs a Location object. + this(char[] filePath, size_t lineNum, char* lineBegin, char* to) + { + set(filePath, lineNum, lineBegin, to); + } + + void set(char[] filePath, size_t lineNum) + { + set(filePath, lineNum, null, null); + } + + void set(char[] filePath, size_t lineNum, char* lineBegin, char* to) + { + this.filePath = filePath; + set(lineNum, lineBegin, to); + } + + void set(size_t lineNum, char* lineBegin, char* to) + { + assert(lineBegin <= to); + this.lineNum = lineNum; + this.lineBegin = lineBegin; + this.to = to; + } + + void setFilePath(char[] filePath) + { + this.filePath = filePath; + } + + /// This is a primitive method to count the number of characters in a string. + /// Note: Unicode compound characters and other special characters are not + /// taken into account. The tabulator character is counted as one. + uint calculateColumn() + { + uint col; + auto p = lineBegin; + if (!p) + return 0; + for (; p <= to; ++p) + { + assert(delegate () + { + // Check that there is no newline between p and to. + // But 'to' may point to a newline. + if (p != to && isNewline(*p)) + return false; + if (to-p >= 2 && isUnicodeNewline(p)) + return false; + return true; + }() == true + ); + + // Skip this byte if it is a trail byte of a UTF-8 sequence. + if (isTrailByte(*p)) + continue; // *p == 0b10xx_xxxx + // Only count ASCII characters and the first byte of a UTF-8 sequence. + ++col; + } + return col; + } + alias calculateColumn colNum; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Messages.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Messages.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,164 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Messages; + +import common; + +/// Enumeration of indices into the table of compiler messages. +enum MID +{ + // Lexer messages: + IllegalCharacter, +// InvalidUnicodeCharacter, + InvalidUTF8Sequence, + // '' + UnterminatedCharacterLiteral, + EmptyCharacterLiteral, + // #line + ExpectedIdentifierSTLine, + ExpectedIntegerAfterSTLine, +// ExpectedFilespec, + UnterminatedFilespec, + UnterminatedSpecialToken, + // "" + UnterminatedString, + // x"" + NonHexCharInHexString, + OddNumberOfDigitsInHexString, + UnterminatedHexString, + // /* */ /+ +/ + UnterminatedBlockComment, + UnterminatedNestedComment, + // `` r"" + UnterminatedRawString, + UnterminatedBackQuoteString, + // \x \u \U + UndefinedEscapeSequence, + InvalidUnicodeEscapeSequence, + InsufficientHexDigits, + // \&[a-zA-Z][a-zA-Z0-9]+; + UndefinedHTMLEntity, + UnterminatedHTMLEntity, + InvalidBeginHTMLEntity, + // integer overflows + OverflowDecimalSign, + OverflowDecimalNumber, + OverflowHexNumber, + OverflowBinaryNumber, + OverflowOctalNumber, + OverflowFloatNumber, + OctalNumberHasDecimals, + NoDigitsInHexNumber, + NoDigitsInBinNumber, + HexFloatExponentRequired, + HexFloatExpMustStartWithDigit, + FloatExpMustStartWithDigit, + + // Parser messages: + ExpectedButFound, + RedundantStorageClass, + TemplateTupleParameter, + InContract, + OutContract, + MissingLinkageType, + UnrecognizedLinkageType, + ExpectedBaseClasses, + BaseClassInForwardDeclaration, + + // Help messages: + HelpMain, + HelpGenerate, + HelpImportGraph, +} + +/// The table of compiler messages. +private string[] g_compilerMessages; + +static this() +{ + g_compilerMessages = new string[MID.max+1]; +} + +/// Sets the compiler messages. +void SetMessages(string[] msgs) +{ + assert(MID.max+1 == msgs.length); + g_compilerMessages = msgs; +} + +/// Returns the compiler message for mid. +string GetMsg(MID mid) +{ + assert(mid < g_compilerMessages.length); + return g_compilerMessages[mid]; +} + +/// Returns a formatted string. +char[] FormatMsg(MID mid, ...) +{ + return Format(_arguments, _argptr, GetMsg(mid)); +} + +/// Collection of error messages with no MID yet. +struct MSG +{ +static: + // Converter: + auto InvalidUTF16Character = "invalid UTF-16 character '\\u{:X4}'."; + auto InvalidUTF32Character = "invalid UTF-32 character '\\U{:X8}'."; + auto UTF16FileMustBeDivisibleBy2 = "the byte length of a UTF-16 source file must be divisible by 2."; + auto UTF32FileMustBeDivisibleBy4 = "the byte length of a UTF-32 source file must be divisible by 4."; + // DDoc macros: + auto UndefinedDDocMacro = "DDoc macro '{}' is undefined"; + auto UnterminatedDDocMacro = "DDoc macro '{}' has no closing ')'"; + // Lexer messages: + auto InvalidOctalEscapeSequence = "value of octal escape sequence is greater than 0xFF: '{}'"; + // Parser messages: + auto InvalidUTF8SequenceInString = "invalid UTF-8 sequence in string literal: '{0}'"; + auto ModuleDeclarationNotFirst = "a module declaration is only allowed as the first declaration in a file"; + auto StringPostfixMismatch = "string literal has mistmatching postfix character"; + auto ExpectedIdAfterTypeDot = "expected identifier after '(Type).', not '{}'"; + auto ExpectedModuleIdentifier = "expected module identifier, not '{}'"; + auto IllegalDeclaration = "illegal declaration found: {}"; + auto ExpectedFunctionName = "expected function name, not '{}'"; + auto ExpectedVariableName = "expected variable name, not '{}'"; + auto ExpectedFunctionBody = "expected function body, not '{}'"; + auto RedundantLinkageType = "redundant linkage type: {}"; + auto ExpectedPragmaIdentifier = "expected pragma identifier, not '{}'"; + auto ExpectedAliasModuleName = "expected alias module name, not '{}'"; + auto ExpectedAliasImportName = "expected alias name, not '{}'"; + auto ExpectedImportName = "expected an identifier, not '{}'"; + auto ExpectedEnumMember = "expected enum member, not '{}'"; + auto ExpectedEnumBody = "expected enum body, not '{}'"; + auto ExpectedClassName = "expected class name, not '{}'"; + auto ExpectedClassBody = "expected class body, not '{}'"; + auto ExpectedInterfaceName = "expected interface name, not '{}'"; + auto ExpectedInterfaceBody = "expected interface body, not '{}'"; + auto ExpectedStructBody = "expected struct body, not '{}'"; + auto ExpectedUnionBody = "expected union body, not '{}'"; + auto ExpectedTemplateName = "expected template name, not '{}'"; + auto ExpectedAnIdentifier = "expected an identifier, not '{}'"; + auto IllegalStatement = "illegal statement found: {}"; + auto ExpectedNonEmptyStatement = "didn't expect ';', use {{ } instead"; + auto ExpectedScopeIdentifier = "expected 'exit', 'success' or 'failure', not '{}'"; + auto InvalidScopeIdentifier = "'exit', 'success', 'failure' are valid scope identifiers, but not '{}'"; + auto ExpectedIntegerAfterAlign = "expected an integer after align, not '{}'"; + auto IllegalAsmStatement = "illegal asm statement found: {}"; + auto ExpectedDeclaratorIdentifier = "expected declarator identifier, not '{}'"; + auto ExpectedTemplateParameters = "expected one or more template parameters, not ')'"; + auto ExpectedTypeOrExpression = "expected a type or and expression, not ')'"; + auto ExpectedAliasTemplateParam = "expected name for alias template parameter, not '{}'"; + auto ExpectedNameForThisTempParam = "expected name for 'this' template parameter, not '{}'"; + auto ExpectedIdentOrInt = "expected an identifier or an integer, not '{}'"; + auto MissingCatchOrFinally = "try statement is missing a catch or finally body."; + // Semantic analysis: + auto UndefinedIdentifier = "undefined identifier '{}'"; + auto DeclConflictsWithDecl = "declaration '{}' conflicts with declaration @{}"; + auto VariableConflictsWithDecl = "variable '{}' conflicts with declaration @{}"; + auto InterfaceCantHaveVariables = "an interface can't have member variables"; + auto MixinArgumentMustBeString = "the mixin argument must evaluate to a string"; + auto DebugSpecModuleLevel = "debug={} must be a module level"; + auto VersionSpecModuleLevel = "version={} must be a module level"; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/SourceText.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/SourceText.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,62 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.SourceText; + +import dil.Converter; +import dil.Information; +import common; + +import tango.io.File; + +/// Represents D source code. +/// +/// The source text may come from a file or from a memory buffer. +final class SourceText +{ + string filePath; /// The file path to the source text. Mainly used for error messages. + char[] data; /// The UTF-8, zero-terminated source text. + + /// Constructs a SourceText object. + /// Params: + /// filePath = file path to the source file. + /// loadFile = whether to load the file in the constructor. + this(string filePath, bool loadFile = false) + { + this.filePath = filePath; + loadFile && load(); + } + + /// Constructs a SourceText object. + /// Params: + /// filePath = file path for error messages. + /// data = memory buffer. + this(string filePath, char[] data) + { + this(filePath); + this.data = data; + addSentinelCharacter(); + } + + /// Loads the source text from a file. + void load(InfoManager infoMan = null) + { + if (!infoMan) + infoMan = new InfoManager; + assert(filePath.length); + // Read the file. + auto rawdata = cast(ubyte[]) (new File(filePath)).read(); + // Convert the data. + auto converter = Converter(filePath, infoMan); + data = converter.data2UTF8(rawdata); + addSentinelCharacter(); + } + + /// Adds '\0' to the text (if not already there.) + private void addSentinelCharacter() + { + if (data.length == 0 || data[$-1] != 0) + data ~= 0; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Time.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Time.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,41 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Time; + +import tango.stdc.time : time_t, time, ctime; +import tango.stdc.string : strlen; + +/// Some convenience functions for dealing with C's time functions. +struct Time +{ +static: + /// Returns the current date as a string. + char[] toString() + { + time_t time_val; + .time(&time_val); + char* str = ctime(&time_val); // ctime returns a pointer to a static array. + char[] timeStr = str[0 .. strlen(str)-1]; // -1 removes trailing '\n'. + return timeStr.dup; + } + + /// Returns the time of timeStr: hh:mm:ss + char[] time(char[] timeStr) + { + return timeStr[11..19]; + } + + /// Returns the month and day of timeStr: Mmm dd + char[] month_day(char[] timeStr) + { + return timeStr[4..10]; + } + + /// Returns the year of timeStr: yyyy + char[] year(char[] timeStr) + { + return timeStr[20..24]; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/Unicode.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/Unicode.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,345 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.Unicode; +public import util.uni : isUniAlpha; + +/// U+FFFD = �. Used to replace invalid Unicode characters. +const dchar REPLACEMENT_CHAR = '\uFFFD'; +const char[3] REPLACEMENT_STR = \uFFFD; /// Ditto +/// Invalid character, returned on errors. +const dchar ERROR_CHAR = 0xD800; + +/// Returns: true if this character is not a surrogate +/// code point and not higher than 0x10FFFF. +bool isValidChar(dchar d) +{ + return d < 0xD800 || d > 0xDFFF && d <= 0x10FFFF; +} + +/// There are a total of 66 noncharacters. +/// Returns: true if this is one of them. +/// See_also: Chapter 16.7 Noncharacters in Unicode 5.0 +bool isNoncharacter(dchar d) +{ + return 0xFDD0 <= d && d <= 0xFDEF || // 32 + d <= 0x10FFFF && (d & 0xFFFF) >= 0xFFFE; // 34 +} + +/// Returns: true if this is a trail byte of a UTF-8 sequence. +bool isTrailByte(ubyte b) +{ + return (b & 0xC0) == 0x80; // 10xx_xxxx +} + +/// Returns: true if this is a lead byte of a UTF-8 sequence. +bool isLeadByte(ubyte b) +{ + return (b & 0xC0) == 0xC0; // 11xx_xxxx +} + +/// Advances ref_p only if this is a valid Unicode alpha character. +bool isUnicodeAlpha(ref char* ref_p, char* end) +in { assert(ref_p && ref_p < end); } +body +{ + if (*ref_p < 0x80) + return false; + auto p = ref_p; + auto c = decode(p, end); + if (!isUniAlpha(c)) + return false; + ref_p = p; + return true; +} + +/// Decodes a character from str at index. +/// Params: +/// index = set to one past the ASCII char or one past the last trail byte +/// of the valid UTF-8 sequence. +dchar decode(char[] str, ref size_t index) +in { assert(str.length && index < str.length); } +out { assert(index <= str.length); } +body +{ + char* p = str.ptr + index; + char* end = str.ptr + str.length; + dchar c = decode(p, end); + if (c != ERROR_CHAR) + index = p - str.ptr; + return c; +} + +/// Decodes a character starting at ref_p. +/// Params: +/// ref_p = set to one past the ASCII char or one past the last trail byte +/// of the valid UTF-8 sequence. +dchar decode(ref char* ref_p, char* end) +in { assert(ref_p && ref_p < end); } +out(c) { assert(ref_p <= end && (isValidChar(c) || c == ERROR_CHAR)); } +body +{ + char* p = ref_p; + dchar c = *p; + + if (c < 0x80) + return ref_p++, c; + + p++; // Move to second byte. + if (!(p < end)) + return ERROR_CHAR; + + // Error if second byte is not a trail byte. + if (!isTrailByte(*p)) + return ERROR_CHAR; + + // Check for overlong sequences. + switch (c) + { + case 0xE0, // 11100000 100xxxxx + 0xF0, // 11110000 1000xxxx + 0xF8, // 11111000 10000xxx + 0xFC: // 11111100 100000xx + if ((*p & c) == 0x80) + return ERROR_CHAR; + default: + if ((c & 0xFE) == 0xC0) // 1100000x + return ERROR_CHAR; + } + + const char[] checkNextByte = "if (!(++p < end && isTrailByte(*p)))" + " return ERROR_CHAR;"; + const char[] appendSixBits = "c = (c << 6) | *p & 0b0011_1111;"; + + // Decode + if ((c & 0b1110_0000) == 0b1100_0000) + { + // 110xxxxx 10xxxxxx + c &= 0b0001_1111; + mixin(appendSixBits); + } + else if ((c & 0b1111_0000) == 0b1110_0000) + { + // 1110xxxx 10xxxxxx 10xxxxxx + c &= 0b0000_1111; + mixin(appendSixBits ~ + checkNextByte ~ appendSixBits); + } + else if ((c & 0b1111_1000) == 0b1111_0000) + { + // 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx + c &= 0b0000_0111; + mixin(appendSixBits ~ + checkNextByte ~ appendSixBits ~ + checkNextByte ~ appendSixBits); + } + else + // 5 and 6 byte UTF-8 sequences are not allowed yet. + // 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx + // 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx + return ERROR_CHAR; + + assert(isTrailByte(*p)); + + if (!isValidChar(c)) + return ERROR_CHAR; + ref_p = p+1; + return c; +} + +/// Encodes c and appends it to str. +void encode(ref char[] str, dchar c) +{ + assert(isValidChar(c), "check if character is valid before calling encode()."); + + char[6] b = void; + if (c < 0x80) + str ~= c; + else if (c < 0x800) + { + b[0] = 0xC0 | (c >> 6); + b[1] = 0x80 | (c & 0x3F); + str ~= b[0..2]; + } + else if (c < 0x10000) + { + b[0] = 0xE0 | (c >> 12); + b[1] = 0x80 | ((c >> 6) & 0x3F); + b[2] = 0x80 | (c & 0x3F); + str ~= b[0..3]; + } + else if (c < 0x200000) + { + b[0] = 0xF0 | (c >> 18); + b[1] = 0x80 | ((c >> 12) & 0x3F); + b[2] = 0x80 | ((c >> 6) & 0x3F); + b[3] = 0x80 | (c & 0x3F); + str ~= b[0..4]; + } + /+ // There are no 5 and 6 byte UTF-8 sequences yet. + else if (c < 0x4000000) + { + b[0] = 0xF8 | (c >> 24); + b[1] = 0x80 | ((c >> 18) & 0x3F); + b[2] = 0x80 | ((c >> 12) & 0x3F); + b[3] = 0x80 | ((c >> 6) & 0x3F); + b[4] = 0x80 | (c & 0x3F); + str ~= b[0..5]; + } + else if (c < 0x80000000) + { + b[0] = 0xFC | (c >> 30); + b[1] = 0x80 | ((c >> 24) & 0x3F); + b[2] = 0x80 | ((c >> 18) & 0x3F); + b[3] = 0x80 | ((c >> 12) & 0x3F); + b[4] = 0x80 | ((c >> 6) & 0x3F); + b[5] = 0x80 | (c & 0x3F); + str ~= b[0..6]; + } + +/ + else + assert(0); +} + +/// Encodes c and appends it to str. +void encode(ref wchar[] str, dchar c) +in { assert(isValidChar(c)); } +body +{ + if (c < 0x10000) + str ~= cast(wchar)c; + else + { // Encode with surrogate pair. + wchar[2] pair = void; + c -= 0x10000; // c' + // higher10bits(c') | 0b1101_10xx_xxxx_xxxx + pair[0] = (c >> 10) | 0xD800; + // lower10bits(c') | 0b1101_11yy_yyyy_yyyy + pair[1] = (c & 0x3FF) | 0xDC00; + str ~= pair; + } +} + +/// Decodes a character from a UTF-16 sequence. +/// Params: +/// str = the UTF-16 sequence. +/// index = where to start from. +/// Returns: ERROR_CHAR in case of an error in the sequence. +dchar decode(wchar[] str, ref size_t index) +{ + assert(str.length && index < str.length); + dchar c = str[index]; + if (0xD800 > c || c > 0xDFFF) + { + ++index; + return c; + } + if (c <= 0xDBFF && index+1 != str.length) + { + wchar c2 = str[index+1]; + if (0xDC00 <= c2 && c2 <= 0xDFFF) + { // Decode surrogate pair. + // (c - 0xD800) << 10 + 0x10000 -> + // (c - 0xD800 + 0x40) << 10 -> + c = (c - 0xD7C0) << 10; + c |= (c2 & 0x3FF); + index += 2; + return c; + } + } + return ERROR_CHAR; +} + +/// Decodes a character from a UTF-16 sequence. +/// Params: +/// p = start of the UTF-16 sequence. +/// end = one past the end of the sequence. +/// Returns: ERROR_CHAR in case of an error in the sequence. +dchar decode(ref wchar* p, wchar* end) +{ + assert(p && p < end); + dchar c = *p; + if (0xD800 > c || c > 0xDFFF) + { + ++p; + return c; + } + if (c <= 0xDBFF && p+1 != end) + { + wchar c2 = p[1]; + if (0xDC00 <= c2 && c2 <= 0xDFFF) + { + c = (c - 0xD7C0) << 10; + c |= (c2 & 0x3FF); + p += 2; + return c; + } + } + return ERROR_CHAR; +} + +/// Decodes a character from a zero-terminated UTF-16 string. +/// Params: +/// p = start of the UTF-16 sequence. +/// Returns: ERROR_CHAR in case of an error in the sequence. +dchar decode(ref wchar* p) +{ + assert(p); + dchar c = *p; + if (0xD800 > c || c > 0xDFFF) + { + ++p; + return c; + } + if (c <= 0xDBFF) + { + wchar c2 = p[1]; + if (0xDC00 <= c2 && c2 <= 0xDFFF) + { + c = (c - 0xD7C0) << 10; + c |= (c2 & 0x3FF); + p += 2; + return c; + } + } + return ERROR_CHAR; +} + +/// Converts a UTF-8 string to a UTF-16 string. +wchar[] toUTF16(char[] str) +{ + wchar[] result; + size_t idx; + while (idx < str.length) + { + auto c = decode(str, idx); + if (c == ERROR_CHAR) + { // Skip trail bytes. + while (++idx < str.length && isTrailByte(str[idx])) + {} + c = REPLACEMENT_CHAR; + } + encode(result, c); + } + return result; +} + +/// Converts a UTF-8 string to a UTF-32 string. +dchar[] toUTF32(char[] str) +{ + dchar[] result; + size_t idx; + while (idx < str.length) + { + auto c = decode(str, idx); + if (c == ERROR_CHAR) + { // Skip trail bytes. + while (++idx < str.length && isTrailByte(str[idx])) + {} + c = REPLACEMENT_CHAR; + } + result ~= c; + } + return result; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Declaration.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Declaration.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,44 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Declaration; + +import dil.ast.Node; +import dil.Enums; + +/// The root class of all declarations. +abstract class Declaration : Node +{ + bool hasBody; + this() + { + super(NodeCategory.Declaration); + } + + // Members relevant to semantic phase. + StorageClass stc; /// The storage classes of this declaration. + Protection prot; /// The protection attribute of this declaration. + + final bool isStatic() + { + return !!(stc & StorageClass.Static); + } + + final bool isPublic() + { + return !!(prot & Protection.Public); + } + + final void setStorageClass(StorageClass stc) + { + this.stc = stc; + } + + final void setProtection(Protection prot) + { + this.prot = prot; + } + + override abstract Declaration copy(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Declarations.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Declarations.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,720 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Declarations; + +public import dil.ast.Declaration; +import dil.ast.Node; +import dil.ast.Expression; +import dil.ast.Types; +import dil.ast.Statements; +import dil.ast.Parameters; +import dil.ast.NodeCopier; +import dil.lexer.IdTable; +import dil.semantic.Symbols; +import dil.Enums; +import common; + +class CompoundDeclaration : Declaration +{ + this() + { + hasBody = true; + mixin(set_kind); + } + + void opCatAssign(Declaration d) + { + addChild(d); + } + + void opCatAssign(CompoundDeclaration ds) + { + addChildren(ds.children); + } + + Declaration[] decls() + { + return cast(Declaration[])this.children; + } + + void decls(Declaration[] decls) + { + this.children = decls; + } + + mixin(copyMethod); +} + +/// Single semicolon. +class EmptyDeclaration : Declaration +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +/// Illegal declarations encompass all tokens that don't +/// start a DeclarationDefinition. +/// See_Also: dil.lexer.Token.isDeclDefStartToken() +class IllegalDeclaration : Declaration +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +/// FQN = fully qualified name +alias Identifier*[] ModuleFQN; // Identifier(.Identifier)* + +class ModuleDeclaration : Declaration +{ + Identifier* moduleName; + Identifier*[] packages; + this(ModuleFQN moduleFQN) + { + mixin(set_kind); + assert(moduleFQN.length != 0); + this.moduleName = moduleFQN[$-1]; + this.packages = moduleFQN[0..$-1]; + } + + char[] getFQN() + { + auto pname = getPackageName('.'); + if (pname.length) + return pname ~ "." ~ getName(); + else + return getName(); + } + + char[] getName() + { + if (moduleName) + return moduleName.str; + return null; + } + + char[] getPackageName(char separator) + { + char[] pname; + foreach (pckg; packages) + if (pckg) + pname ~= pckg.str ~ separator; + if (pname.length) + pname = pname[0..$-1]; // Remove last separator + return pname; + } + + mixin(copyMethod); +} + +class ImportDeclaration : Declaration +{ + private alias Identifier*[] Ids; + ModuleFQN[] moduleFQNs; + Ids moduleAliases; + Ids bindNames; + Ids bindAliases; + + this(ModuleFQN[] moduleFQNs, Ids moduleAliases, Ids bindNames, Ids bindAliases, bool isStatic) + { + mixin(set_kind); + this.moduleFQNs = moduleFQNs; + this.moduleAliases = moduleAliases; + this.bindNames = bindNames; + this.bindAliases = bindAliases; + if (isStatic) + this.stc |= StorageClass.Static; + } + + char[][] getModuleFQNs(char separator) + { + char[][] FQNs; + foreach (moduleFQN; moduleFQNs) + { + char[] FQN; + foreach (ident; moduleFQN) + if (ident) + FQN ~= ident.str ~ separator; + FQNs ~= FQN[0..$-1]; // Remove last separator + } + return FQNs; + } + + mixin(copyMethod); +} + +class AliasDeclaration : Declaration +{ + Declaration decl; + this(Declaration decl) + { + mixin(set_kind); + addChild(decl); + this.decl = decl; + } + mixin(copyMethod); +} + +class TypedefDeclaration : Declaration +{ + Declaration decl; + this(Declaration decl) + { + mixin(set_kind); + addChild(decl); + this.decl = decl; + } + mixin(copyMethod); +} + +class EnumDeclaration : Declaration +{ + Identifier* name; + TypeNode baseType; + EnumMemberDeclaration[] members; + this(Identifier* name, TypeNode baseType, EnumMemberDeclaration[] members, bool hasBody) + { + super.hasBody = hasBody; + mixin(set_kind); + addOptChild(baseType); + addOptChildren(members); + + this.name = name; + this.baseType = baseType; + this.members = members; + } + + Enum symbol; + + mixin(copyMethod); +} + +class EnumMemberDeclaration : Declaration +{ + Identifier* name; + Expression value; + this(Identifier* name, Expression value) + { + mixin(set_kind); + addOptChild(value); + + this.name = name; + this.value = value; + } + + EnumMember symbol; + + mixin(copyMethod); +} + +class TemplateDeclaration : Declaration +{ + Identifier* name; + TemplateParameters tparams; + CompoundDeclaration decls; + this(Identifier* name, TemplateParameters tparams, CompoundDeclaration decls) + { + super.hasBody = true; + mixin(set_kind); + addChild(tparams); + addChild(decls); + + this.name = name; + this.tparams = tparams; + this.decls = decls; + } + + Template symbol; /// The template symbol for this declaration. + + mixin(copyMethod); +} + +abstract class AggregateDeclaration : Declaration +{ + Identifier* name; +// TemplateParameters tparams; + CompoundDeclaration decls; + this(Identifier* name, /+TemplateParameters tparams, +/CompoundDeclaration decls) + { + super.hasBody = decls !is null; + this.name = name; +// this.tparams = tparams; + this.decls = decls; + } + mixin(copyMethod); +} + +class ClassDeclaration : AggregateDeclaration +{ + BaseClassType[] bases; + this(Identifier* name, /+TemplateParameters tparams, +/BaseClassType[] bases, CompoundDeclaration decls) + { + super(name, /+tparams, +/decls); + mixin(set_kind); +// addChild(tparams); + addOptChildren(bases); + addOptChild(decls); + + this.bases = bases; + } + + Class symbol; /// The class symbol for this declaration. + + mixin(copyMethod); +} + +class InterfaceDeclaration : AggregateDeclaration +{ + BaseClassType[] bases; + this(Identifier* name, /+TemplateParameters tparams, +/BaseClassType[] bases, CompoundDeclaration decls) + { + super(name, /+tparams, +/decls); + mixin(set_kind); +// addChild(tparams); + addOptChildren(bases); + addOptChild(decls); + + this.bases = bases; + } + + alias dil.semantic.Symbols.Interface Interface; + + Interface symbol; /// The interface symbol for this declaration. + + mixin(copyMethod); +} + +class StructDeclaration : AggregateDeclaration +{ + uint alignSize; + this(Identifier* name, /+TemplateParameters tparams, +/CompoundDeclaration decls) + { + super(name, /+tparams, +/decls); + mixin(set_kind); +// addChild(tparams); + addOptChild(decls); + } + + void setAlignSize(uint alignSize) + { + this.alignSize = alignSize; + } + + Struct symbol; /// The struct symbol for this declaration. + + mixin(copyMethod); +} + +class UnionDeclaration : AggregateDeclaration +{ + this(Identifier* name, /+TemplateParameters tparams, +/CompoundDeclaration decls) + { + super(name, /+tparams, +/decls); + mixin(set_kind); +// addChild(tparams); + addOptChild(decls); + } + + Union symbol; /// The union symbol for this declaration. + + mixin(copyMethod); +} + +class ConstructorDeclaration : Declaration +{ + Parameters params; + FuncBodyStatement funcBody; + this(Parameters params, FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(params); + addChild(funcBody); + + this.params = params; + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +class StaticConstructorDeclaration : Declaration +{ + FuncBodyStatement funcBody; + this(FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(funcBody); + + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +class DestructorDeclaration : Declaration +{ + FuncBodyStatement funcBody; + this(FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(funcBody); + + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +class StaticDestructorDeclaration : Declaration +{ + FuncBodyStatement funcBody; + this(FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(funcBody); + + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +class FunctionDeclaration : Declaration +{ + TypeNode returnType; + Identifier* name; +// TemplateParameters tparams; + Parameters params; + FuncBodyStatement funcBody; + LinkageType linkageType; + this(TypeNode returnType, Identifier* name,/+ TemplateParameters tparams,+/ + Parameters params, FuncBodyStatement funcBody) + { + super.hasBody = funcBody.funcBody !is null; + mixin(set_kind); + addChild(returnType); +// addChild(tparams); + addChild(params); + addChild(funcBody); + + this.returnType = returnType; + this.name = name; +// this.tparams = tparams; + this.params = params; + this.funcBody = funcBody; + } + + void setLinkageType(LinkageType linkageType) + { + this.linkageType = linkageType; + } + + bool isTemplatized() + { // E.g.: void func(T)(T t) + // ^ params.begin.prevNWS + return params.begin.prevNWS.kind == TOK.RParen; + } + + mixin(copyMethod); +} + +/// VariablesDeclaration := Type? Identifier ("=" Init)? ("," Identifier ("=" Init)?)* ";" +class VariablesDeclaration : Declaration +{ + TypeNode typeNode; + Identifier*[] names; + Expression[] inits; + this(TypeNode typeNode, Identifier*[] names, Expression[] inits) + { + // No empty arrays allowed. Both arrays must be of same size. + assert(names.length != 0 && names.length == inits.length); + // If no type (in case of AutoDeclaration), first value mustn't be null. + assert(typeNode ? 1 : inits[0] !is null); + mixin(set_kind); + addOptChild(typeNode); + foreach(init; inits) + addOptChild(init); + + this.typeNode = typeNode; + this.names = names; + this.inits = inits; + } + + LinkageType linkageType; + + void setLinkageType(LinkageType linkageType) + { + this.linkageType = linkageType; + } + + Variable[] variables; + + mixin(copyMethod); +} + +class InvariantDeclaration : Declaration +{ + FuncBodyStatement funcBody; + this(FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(funcBody); + + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +class UnittestDeclaration : Declaration +{ + FuncBodyStatement funcBody; + this(FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(funcBody); + + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +abstract class ConditionalCompilationDeclaration : Declaration +{ + Token* spec; + Token* cond; + Declaration decls, elseDecls; + + this(Token* spec, Token* cond, Declaration decls, Declaration elseDecls) + { + super.hasBody = decls !is null; + addOptChild(decls); + addOptChild(elseDecls); + + this.spec = spec; + this.cond = cond; + this.decls = decls; + this.elseDecls = elseDecls; + } + + bool isSpecification() + { + return decls is null; + } + + bool isCondition() + { + return decls !is null; + } + + /// The branch to be compiled in. + Declaration compiledDecls; +} + +class DebugDeclaration : ConditionalCompilationDeclaration +{ + this(Token* spec, Token* cond, Declaration decls, Declaration elseDecls) + { + super(spec, cond, decls, elseDecls); + mixin(set_kind); + } + mixin(copyMethod); +} + +class VersionDeclaration : ConditionalCompilationDeclaration +{ + this(Token* spec, Token* cond, Declaration decls, Declaration elseDecls) + { + super(spec, cond, decls, elseDecls); + mixin(set_kind); + } + mixin(copyMethod); +} + +class StaticIfDeclaration : Declaration +{ + Expression condition; + Declaration ifDecls, elseDecls; + this(Expression condition, Declaration ifDecls, Declaration elseDecls) + { + super.hasBody = true; + mixin(set_kind); + addChild(condition); + addChild(ifDecls); + addOptChild(elseDecls); + + this.condition = condition; + this.ifDecls = ifDecls; + this.elseDecls = elseDecls; + } + mixin(copyMethod); +} + +class StaticAssertDeclaration : Declaration +{ + Expression condition, message; + this(Expression condition, Expression message) + { + super.hasBody = true; + mixin(set_kind); + addChild(condition); + addOptChild(message); + + this.condition = condition; + this.message = message; + } + mixin(copyMethod); +} + +class NewDeclaration : Declaration +{ + Parameters params; + FuncBodyStatement funcBody; + this(Parameters params, FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(params); + addChild(funcBody); + + this.params = params; + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +class DeleteDeclaration : Declaration +{ + Parameters params; + FuncBodyStatement funcBody; + this(Parameters params, FuncBodyStatement funcBody) + { + super.hasBody = true; + mixin(set_kind); + addChild(params); + addChild(funcBody); + + this.params = params; + this.funcBody = funcBody; + } + mixin(copyMethod); +} + +abstract class AttributeDeclaration : Declaration +{ + Declaration decls; + this(Declaration decls) + { + super.hasBody = true; + addChild(decls); + this.decls = decls; + } +} + +class ProtectionDeclaration : AttributeDeclaration +{ + Protection prot; + this(Protection prot, Declaration decls) + { + super(decls); + mixin(set_kind); + this.prot = prot; + } + mixin(copyMethod); +} + +class StorageClassDeclaration : AttributeDeclaration +{ + StorageClass storageClass; + this(StorageClass storageClass, Declaration decl) + { + super(decl); + mixin(set_kind); + + this.storageClass = storageClass; + } + mixin(copyMethod); +} + +class LinkageDeclaration : AttributeDeclaration +{ + LinkageType linkageType; + this(LinkageType linkageType, Declaration decls) + { + super(decls); + mixin(set_kind); + + this.linkageType = linkageType; + } + mixin(copyMethod); +} + +class AlignDeclaration : AttributeDeclaration +{ + int size; + this(int size, Declaration decls) + { + super(decls); + mixin(set_kind); + this.size = size; + } + mixin(copyMethod); +} + +class PragmaDeclaration : AttributeDeclaration +{ + Identifier* ident; + Expression[] args; + this(Identifier* ident, Expression[] args, Declaration decls) + { + addOptChildren(args); // Add args before calling super(). + super(decls); + mixin(set_kind); + + this.ident = ident; + this.args = args; + } + mixin(copyMethod); +} + +class MixinDeclaration : Declaration +{ + /// IdExpression := IdentifierExpression | TemplateInstanceExpression + /// MixinTemplate := IdExpression ("." IdExpression)* + Expression templateExpr; + Identifier* mixinIdent; /// Optional mixin identifier. + Expression argument; /// "mixin" "(" AssignExpression ")" + Declaration decls; /// Initialized in the semantic phase. + + this(Expression templateExpr, Identifier* mixinIdent) + { + mixin(set_kind); + addChild(templateExpr); + + this.templateExpr = templateExpr; + this.mixinIdent = mixinIdent; + } + + this(Expression argument) + { + mixin(set_kind); + addChild(argument); + + this.argument = argument; + } + + bool isMixinExpression() + { + return argument !is null; + } + + mixin(copyMethod); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/DefaultVisitor.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/DefaultVisitor.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,375 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.DefaultVisitor; + +import dil.ast.Visitor; + +import dil.ast.Node; +import dil.ast.Declarations, + dil.ast.Expressions, + dil.ast.Statements, + dil.ast.Types, + dil.ast.Parameters; +import common; + +/// This huge template function, when instantiated for a certain node class, +/// generates a body of calls to visit() on the subnodes. +returnType!(T.stringof) visitDefault(T)(T t) +{ + assert(t !is null, "node passed to visitDefault() is null"); + //Stdout(t).newline; + + alias t d, s, e, n; // Variable aliases of t. + + static if (is(T : Declaration)) + { + alias T D; + static if (is(D == CompoundDeclaration)) + foreach (decl; d.decls) + visitD(decl); + //EmptyDeclaration, + //IllegalDeclaration, + //ModuleDeclaration have no subnodes. + static if (is(D == AliasDeclaration) || + is(D == TypedefDeclaration)) + visitD(d.decl); + static if (is(D == EnumDeclaration)) + { + d.baseType && visitT(d.baseType); + foreach (member; d.members) + visitD(member); + } + static if (is(D == EnumMemberDeclaration)) + d.value && visitE(d.value); + static if (is(D == ClassDeclaration) || is( D == InterfaceDeclaration)) + { +// visitN(d.tparams); + foreach (base; d.bases) + visitT(base); + d.decls && visitD(d.decls); + } + static if (is(D == StructDeclaration) || is(D == UnionDeclaration)) +// visitN(d.tparams), + d.decls && visitD(d.decls); + static if (is(D == ConstructorDeclaration)) + visitN(d.params), visitS(d.funcBody); + static if (is(D == StaticConstructorDeclaration) || + is(D == DestructorDeclaration) || + is(D == StaticDestructorDeclaration) || + is(D == InvariantDeclaration) || + is(D == UnittestDeclaration)) + visitS(d.funcBody); + static if (is(D == FunctionDeclaration)) + visitT(d.returnType), +// visitN(d.tparams), + visitN(d.params), + visitS(d.funcBody); + static if (is(D == VariablesDeclaration)) + { + d.typeNode && visitT(d.typeNode); + foreach(init; d.inits) + init && visitE(init); + } + static if (is(D == DebugDeclaration) || is(D == VersionDeclaration)) + d.decls && visitD(d.decls), + d.elseDecls && visitD(d.elseDecls); + static if (is(D == StaticIfDeclaration)) + visitE(d.condition), + visitD(d.ifDecls), + d.elseDecls && visitD(d.elseDecls); + static if (is(D == StaticAssertDeclaration)) + visitE(d.condition), + d.message && visitE(d.message); + static if (is(D == TemplateDeclaration)) + visitN(d.tparams), + visitD(d.decls); + static if (is(D == NewDeclaration) || is(D == DeleteDeclaration)) + visitN(d.params), + visitS(d.funcBody); + static if (is(D == ProtectionDeclaration) || + is(D == StorageClassDeclaration) || + is(D == LinkageDeclaration) || + is(D == AlignDeclaration)) + visitD(d.decls); + static if (is(D == PragmaDeclaration)) + { + foreach (arg; d.args) + visitE(arg); + visitD(d.decls); + } + static if (is(D == MixinDeclaration)) + d.templateExpr ? visitE(d.templateExpr) : visitE(d.argument); + } + else + static if (is(T : Expression)) + { + alias T E; + static if (is(E == IllegalExpression)) + {} + else + static if (is(E == CondExpression)) + visitE(e.condition), visitE(e.lhs), visitE(e.rhs); + else + static if (is(E : BinaryExpression)) + visitE(e.lhs), visitE(e.rhs); + else + static if (is(E : UnaryExpression)) + { + static if (is(E == CastExpression)) + visitT(e.type); + visitE(e.e); // Visit member in base class UnaryExpression. + static if (is(E == IndexExpression)) + foreach (arg; e.args) + visitE(arg); + static if (is(E == SliceExpression)) + e.left && (visitE(e.left), visitE(e.right)); + static if (is(E == AsmPostBracketExpression)) + visitE(e.e2); + } + else + { + static if (is(E == NewExpression)) + { + foreach (arg; e.newArgs) + visitE(arg); + visitT(e.type); + foreach (arg; e.ctorArgs) + visitE(arg); + } + static if (is(E == NewAnonClassExpression)) + { + foreach (arg; e.newArgs) + visitE(arg); + foreach (base; e.bases) + visitT(base); + foreach (arg; e.ctorArgs) + visitE(arg); + visitD(e.decls); + } + static if (is(E == AsmBracketExpression)) + visitE(e.e); + static if (is(E == TemplateInstanceExpression)) + e.targs && visitN(e.targs); + static if (is(E == ArrayLiteralExpression)) + foreach (value; e.values) + visitE(value); + static if (is(E == AArrayLiteralExpression)) + foreach (i, key; e.keys) + visitE(key), visitE(e.values[i]); + static if (is(E == AssertExpression)) + visitE(e.expr), e.msg && visitE(e.msg); + static if (is(E == MixinExpression) || + is(E == ImportExpression)) + visitE(e.expr); + static if (is(E == TypeofExpression) || + is(E == TypeDotIdExpression) || + is(E == TypeidExpression)) + visitT(e.type); + static if (is(E == IsExpression)) + visitT(e.type), e.specType && visitT(e.specType), + e.tparams && visitN(e.tparams); + static if (is(E == FunctionLiteralExpression)) + e.returnType && visitT(e.returnType), + e.params && visitN(e.params), + visitS(e.funcBody); + static if (is(E == ParenExpression)) + visitE(e.next); + static if (is(E == TraitsExpression)) + visitN(e.targs); + // VoidInitializer has no subnodes. + static if (is(E == ArrayInitExpression)) + foreach (i, key; e.keys) + key && visitE(key), visitE(e.values[i]); + static if (is(E == StructInitExpression)) + foreach (value; e.values) + visitE(value); + } + } + else + static if (is(T : Statement)) + { + alias T S; + static if (is(S == CompoundStatement)) + foreach (stmnt; s.stmnts) + visitS(stmnt); + //IllegalStatement has no subnodes. + static if (is(S == FuncBodyStatement)) + s.funcBody && visitS(s.funcBody), + s.inBody && visitS(s.inBody), + s.outBody && visitS(s.outBody); + static if (is(S == ScopeStatement) || is(S == LabeledStatement)) + visitS(s.s); + static if (is(S == ExpressionStatement)) + visitE(s.e); + static if (is(S == DeclarationStatement)) + visitD(s.decl); + static if (is(S == IfStatement)) + { + s.variable ? cast(Node)visitS(s.variable) : visitE(s.condition); + visitS(s.ifBody), s.elseBody && visitS(s.elseBody); + } + static if (is(S == WhileStatement)) + visitE(s.condition), visitS(s.whileBody); + static if (is(S == DoWhileStatement)) + visitS(s.doBody), visitE(s.condition); + static if (is(S == ForStatement)) + s.init && visitS(s.init), + s.condition && visitE(s.condition), + s.increment && visitE(s.increment), + visitS(s.forBody); + static if (is(S == ForeachStatement)) + visitN(s.params), visitE(s.aggregate), visitS(s.forBody); + static if (is(S == ForeachRangeStatement)) + visitN(s.params), visitE(s.lower), visitE(s.upper), visitS(s.forBody); + static if (is(S == SwitchStatement)) + visitE(s.condition), visitS(s.switchBody); + static if (is(S == CaseStatement)) + { + foreach (value; s.values) + visitE(value); + visitS(s.caseBody); + } + static if (is(S == DefaultStatement)) + visitS(s.defaultBody); + //ContinueStatement, + //BreakStatement have no subnodes. + static if (is(S == ReturnStatement)) + s.e && visitE(s.e); + static if (is(S == GotoStatement)) + s.caseExpr && visitE(s.caseExpr); + static if (is(S == WithStatement)) + visitE(s.e), visitS(s.withBody); + static if (is(S == SynchronizedStatement)) + s.e && visitE(s.e), visitS(s.syncBody); + static if (is(S == TryStatement)) + { + visitS(s.tryBody); + foreach (catchBody; s.catchBodies) + visitS(catchBody); + s.finallyBody && visitS(s.finallyBody); + } + static if (is(S == CatchStatement)) + s.param && visitN(s.param), visitS(s.catchBody); + static if (is(S == FinallyStatement)) + visitS(s.finallyBody); + static if (is(S == ScopeGuardStatement)) + visitS(s.scopeBody); + static if (is(S == ThrowStatement)) + visitE(s.e); + static if (is(S == VolatileStatement)) + s.volatileBody && visitS(s.volatileBody); + static if (is(S == AsmBlockStatement)) + visitS(s.statements); + static if (is(S == AsmStatement)) + foreach (op; s.operands) + visitE(op); + //AsmAlignStatement, + //IllegalAsmStatement have no subnodes. + static if (is(S == PragmaStatement)) + { + foreach (arg; s.args) + visitE(arg); + visitS(s.pragmaBody); + } + static if (is(S == MixinStatement)) + visitE(s.templateExpr); + static if (is(S == StaticIfStatement)) + visitE(s.condition), visitS(s.ifBody), s.elseBody && visitS(s.elseBody); + static if (is(S == StaticAssertStatement)) + visitE(s.condition), s.message && visitE(s.message); + static if (is(S == DebugStatement) || is(S == VersionStatement)) + visitS(s.mainBody), s.elseBody && visitS(s.elseBody); + } + else + static if (is(T : TypeNode)) + { + //IllegalType, + //IntegralType, + //ModuleScopeType, + //IdentifierType have no subnodes. + static if (is(T == QualifiedType)) + visitT(t.lhs), visitT(t.rhs); + static if (is(T == TypeofType)) + visitE(t.e); + static if (is(T == TemplateInstanceType)) + t.targs && visitN(t.targs); + static if (is(T == PointerType)) + visitT(t.next); + static if (is(T == ArrayType)) + { + visitT(t.next); + if (t.assocType) + visitT(t.assocType); + else if (t.e1) + visitE(t.e1), t.e2 && visitE(t.e2); + } + static if (is(T == FunctionType) || is(T == DelegateType)) + visitT(t.returnType), visitN(t.params); + static if (is(T == CFuncPointerType)) + visitT(t.next), t.params && visitN(t.params); + static if (is(T == BaseClassType) || + is(T == ConstType) || + is(T == InvariantType)) + visitT(t.next); + } + else + static if (is(T == Parameter)) + { + n.type && visitT(n.type); + n.defValue && visitE(n.defValue); + } + else + static if (is(T == Parameters) || + is(T == TemplateParameters) || + is(T == TemplateArguments)) + { + foreach (node; n.children) + visitN(node); + } + else + static if (is(T : TemplateParameter)) + { + static if (is(N == TemplateAliasParameter) || + is(N == TemplateTypeParameter) || + is(N == TemplateThisParameter)) + n.specType && visitN(n.specType), + n.defType && visitN(n.defType); + static if (is(N == TemplateValueParameter)) + visitT(n.valueType), + n.specValue && visitN(n.specValue), + n.defValue && visitN(n.defValue); + //TemplateTupleParameter has no subnodes. + } + else + static assert(0, "Missing default visit method for: "~typeof(t).stringof); + return t; +} + +/// Generates the default visit methods. +/// +/// E.g: +/// --- +/// private mixin .visitDefault!(ClassDeclaration) _ClassDeclaration; +/// override returnType!("ClassDeclaration") visit(ClassDeclaration node) +/// { return _ClassDeclaration.visitDefault(node); } +/// --- +char[] generateDefaultVisitMethods() +{ + char[] text; + foreach (className; g_classNames) + text ~= "private mixin .visitDefault!("~className~") _"~className~";\n" + "override returnType!(\""~className~"\") visit("~className~" node)" + "{return _"~className~".visitDefault(node);}\n"; + return text; +} +// pragma(msg, generateDefaultVisitMethods()); + +/// This class provides default methods for +/// traversing nodes and their sub-nodes. +class DefaultVisitor : Visitor +{ + // Comment out if too many errors are shown. + mixin(generateDefaultVisitMethods()); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Expression.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Expression.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,22 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Expression; + +import dil.ast.Node; +import dil.semantic.Types; +import common; + +/// The root class of all expressions. +abstract class Expression : Node +{ + Type type; /// The semantic type of this expression. + + this() + { + super(NodeCategory.Expression); + } + + override abstract Expression copy(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Expressions.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Expressions.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1118 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Expressions; + +public import dil.ast.Expression; +import dil.ast.Node; +import dil.ast.Types; +import dil.ast.Declarations; +import dil.ast.Statements; +import dil.ast.Parameters; +import dil.ast.NodeCopier; +import dil.lexer.Identifier; +import dil.semantic.Types; +import common; + +class IllegalExpression : Expression +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +abstract class BinaryExpression : Expression +{ + Expression lhs; /// Left-hand side expression. + Expression rhs; /// Right-hand side expression. + Token* tok; + this(Expression lhs, Expression rhs, Token* tok) + { + addChildren([lhs, rhs]); + this.lhs = lhs; + this.rhs = rhs; + this.tok = tok; + } + mixin(copyMethod); +} + +class CondExpression : BinaryExpression +{ + Expression condition; + this(Expression condition, Expression left, Expression right, Token* tok) + { + addChild(condition); + super(left, right, tok); + mixin(set_kind); + this.condition = condition; + } + mixin(copyMethod); +} + +class CommaExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class OrOrExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class AndAndExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class OrExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class XorExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class AndExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +abstract class CmpExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + } +} + +class EqualExpression : CmpExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +/// Expression "!"? "is" Expression +class IdentityExpression : CmpExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class RelExpression : CmpExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class InExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class LShiftExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class RShiftExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class URShiftExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class PlusExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class MinusExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class CatExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class MulExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class DivExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class ModExpression : BinaryExpression +{ + this(Expression left, Expression right, Token* tok) + { + super(left, right, tok); + mixin(set_kind); + } +} + +class AssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class LShiftAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class RShiftAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class URShiftAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class OrAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class AndAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class PlusAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class MinusAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class DivAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class MulAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class ModAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class XorAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} +class CatAssignExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} + +/// DotExpression := Expression '.' Expression +class DotExpression : BinaryExpression +{ + this(Expression left, Expression right) + { + super(left, right, null); + mixin(set_kind); + } +} + +/*++++++++++++++++++++ ++ Unary Expressions: + +++++++++++++++++++++*/ + +abstract class UnaryExpression : Expression +{ + Expression e; + this(Expression e) + { + addChild(e); + this.e = e; + } + mixin(copyMethod); +} + +class AddressExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class PreIncrExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class PreDecrExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class PostIncrExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class PostDecrExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class DerefExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class SignExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } + + bool isPos() + { + assert(begin !is null); + return begin.kind == TOK.Plus; + } + + bool isNeg() + { + assert(begin !is null); + return begin.kind == TOK.Minus; + } +} + +class NotExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class CompExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class CallExpression : UnaryExpression +{ + Expression[] args; + this(Expression e, Expression[] args) + { + super(e); + mixin(set_kind); + addOptChildren(args); + this.args = args; + } +} + +class NewExpression : /*Unary*/Expression +{ + Expression[] newArgs; + TypeNode type; + Expression[] ctorArgs; + this(/*Expression e, */Expression[] newArgs, TypeNode type, Expression[] ctorArgs) + { + /*super(e);*/ + mixin(set_kind); + addOptChildren(newArgs); + addChild(type); + addOptChildren(ctorArgs); + this.newArgs = newArgs; + this.type = type; + this.ctorArgs = ctorArgs; + } + mixin(copyMethod); +} + +class NewAnonClassExpression : /*Unary*/Expression +{ + Expression[] newArgs; + BaseClassType[] bases; + Expression[] ctorArgs; + CompoundDeclaration decls; + this(/*Expression e, */Expression[] newArgs, BaseClassType[] bases, Expression[] ctorArgs, CompoundDeclaration decls) + { + /*super(e);*/ + mixin(set_kind); + addOptChildren(newArgs); + addOptChildren(bases); + addOptChildren(ctorArgs); + addChild(decls); + + this.newArgs = newArgs; + this.bases = bases; + this.ctorArgs = ctorArgs; + this.decls = decls; + } + mixin(copyMethod); +} + +class DeleteExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class CastExpression : UnaryExpression +{ + TypeNode type; + this(Expression e, TypeNode type) + { + addChild(type); // Add type before super(). + super(e); + mixin(set_kind); + this.type = type; + } + mixin(copyMethod); +} + +class IndexExpression : UnaryExpression +{ + Expression[] args; + this(Expression e, Expression[] args) + { + super(e); + mixin(set_kind); + addChildren(args); + this.args = args; + } + mixin(copyMethod); +} + +class SliceExpression : UnaryExpression +{ + Expression left, right; + this(Expression e, Expression left, Expression right) + { + super(e); + mixin(set_kind); + assert(left ? (right !is null) : right is null); + if (left) + addChildren([left, right]); + + this.left = left; + this.right = right; + } + mixin(copyMethod); +} + +/// Module scope operator: '.' (IdentifierExpression|TemplateInstanceExpression) +class ModuleScopeExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + assert(e.kind == NodeKind.IdentifierExpression || + e.kind == NodeKind.TemplateInstanceExpression + ); + mixin(set_kind); + } +} + +/*++++++++++++++++++++++ ++ Primary Expressions: + +++++++++++++++++++++++*/ + +class IdentifierExpression : Expression +{ + Identifier* identifier; + this(Identifier* identifier) + { + mixin(set_kind); + this.identifier = identifier; + } + mixin(copyMethod); +} + +class SpecialTokenExpression : Expression +{ + Token* specialToken; + this(Token* specialToken) + { + mixin(set_kind); + this.specialToken = specialToken; + } + + Expression value; /// The expression created in the semantic phase. + + mixin(copyMethod); +} + +class TemplateInstanceExpression : Expression +{ + Identifier* ident; + TemplateArguments targs; + this(Identifier* ident, TemplateArguments targs) + { + mixin(set_kind); + addOptChild(targs); + this.ident = ident; + this.targs = targs; + } + mixin(copyMethod); +} + +class ThisExpression : Expression +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class SuperExpression : Expression +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class NullExpression : Expression +{ + this() + { + mixin(set_kind); + } + + this(Type type) + { + this(); + this.type = type; + } + + mixin(copyMethod); +} + +class DollarExpression : Expression +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class BoolExpression : Expression +{ + this() + { + mixin(set_kind); + } + + bool toBool() + { + assert(begin !is null); + return begin.kind == TOK.True ? true : false; + } + + Expression value; /// IntExpression of type int. + + mixin(copyMethod); +} + +class IntExpression : Expression +{ + ulong number; + + this(ulong number, Type type) + { + mixin(set_kind); + this.number = number; + this.type = type; + } + + this(Token* token) + { + auto type = Types.Int; // Should be most common case. + switch (token.kind) + { + // case TOK.Int32: + // type = Types.Int; break; + case TOK.Uint32: + type = Types.Uint; break; + case TOK.Int64: + type = Types.Long; break; + case TOK.Uint64: + type = Types.Ulong; break; + default: + assert(token.kind == TOK.Int32); + } + this(token.ulong_, type); + } + + mixin(copyMethod); +} + +class RealExpression : Expression +{ + real number; + + this(real number, Type type) + { + mixin(set_kind); + this.number = number; + this.type = type; + } + + this(Token* token) + { + auto type = Types.Double; // Most common case? + switch (token.kind) + { + case TOK.Float32: + type = Types.Float; break; + // case TOK.Float64: + // type = Types.Double; break; + case TOK.Float80: + type = Types.Real; break; + case TOK.Imaginary32: + type = Types.Ifloat; break; + case TOK.Imaginary64: + type = Types.Idouble; break; + case TOK.Imaginary80: + type = Types.Ireal; break; + default: + assert(token.kind == TOK.Float64); + } + this(token.real_, type); + } + + mixin(copyMethod); +} + + +/// This expression holds a complex number. +/// It is only created in the semantic phase. +class ComplexExpression : Expression +{ + creal number; + this(creal number, Type type) + { + mixin(set_kind); + this.number = number; + this.type = type; + } + mixin(copyMethod); +} + +class CharExpression : Expression +{ + dchar character; + this(dchar character) + { + mixin(set_kind); + this.character = character; + } + mixin(copyMethod); +} + +class StringExpression : Expression +{ + ubyte[] str; /// The string data. + Type charType; /// The character type of the string. + + this(ubyte[] str, Type charType) + { + mixin(set_kind); + this.str = str; + this.charType = charType; + type = new TypeSArray(charType, str.length); + } + + this(char[] str) + { + this(cast(ubyte[])str, Types.Char); + } + + this(wchar[] str) + { + this(cast(ubyte[])str, Types.Wchar); + } + + this(dchar[] str) + { + this(cast(ubyte[])str, Types.Dchar); + } + + /// Returns the string excluding the terminating 0. + char[] getString() + { + // TODO: convert to char[] if charType !is Types.Char. + return cast(char[])str[0..$-1]; + } + + mixin(copyMethod); +} + +class ArrayLiteralExpression : Expression +{ + Expression[] values; + this(Expression[] values) + { + mixin(set_kind); + addOptChildren(values); + this.values = values; + } + mixin(copyMethod); +} + +class AArrayLiteralExpression : Expression +{ + Expression[] keys, values; + this(Expression[] keys, Expression[] values) + { + assert(keys.length == values.length); + mixin(set_kind); + foreach (i, key; keys) + addChildren([key, values[i]]); + this.keys = keys; + this.values = values; + } + mixin(copyMethod); +} + +class AssertExpression : Expression +{ + Expression expr, msg; + this(Expression expr, Expression msg) + { + mixin(set_kind); + addChild(expr); + addOptChild(msg); + this.expr = expr; + this.msg = msg; + } + mixin(copyMethod); +} + +class MixinExpression : Expression +{ + Expression expr; + this(Expression expr) + { + mixin(set_kind); + addChild(expr); + this.expr = expr; + } + mixin(copyMethod); +} + +class ImportExpression : Expression +{ + Expression expr; + this(Expression expr) + { + mixin(set_kind); + addChild(expr); + this.expr = expr; + } + mixin(copyMethod); +} + +class TypeofExpression : Expression +{ + TypeNode type; + this(TypeNode type) + { + mixin(set_kind); + addChild(type); + this.type = type; + } + mixin(copyMethod); +} + +class TypeDotIdExpression : Expression +{ + TypeNode type; + Identifier* ident; + this(TypeNode type, Identifier* ident) + { + mixin(set_kind); + addChild(type); + this.type = type; + this.ident = ident; + } + mixin(copyMethod); +} + +class TypeidExpression : Expression +{ + TypeNode type; + this(TypeNode type) + { + mixin(set_kind); + addChild(type); + this.type = type; + } + mixin(copyMethod); +} + +class IsExpression : Expression +{ + TypeNode type; + Identifier* ident; + Token* opTok, specTok; + TypeNode specType; + TemplateParameters tparams; // D 2.0 + this(TypeNode type, Identifier* ident, Token* opTok, Token* specTok, + TypeNode specType, typeof(tparams) tparams) + { + mixin(set_kind); + addChild(type); + addOptChild(specType); + version(D2) + addOptChild(tparams); + this.type = type; + this.ident = ident; + this.opTok = opTok; + this.specTok = specTok; + this.specType = specType; + this.tparams = tparams; + } + mixin(copyMethod); +} + +class FunctionLiteralExpression : Expression +{ + TypeNode returnType; + Parameters params; + FuncBodyStatement funcBody; + + this() + { + mixin(set_kind); + addOptChild(returnType); + addOptChild(params); + addChild(funcBody); + } + + this(TypeNode returnType, Parameters params, FuncBodyStatement funcBody) + { + this.returnType = returnType; + this.params = params; + this.funcBody = funcBody; + this(); + } + + this(FuncBodyStatement funcBody) + { + this.funcBody = funcBody; + this(); + } + + mixin(copyMethod); +} + +/// ParenthesisExpression := "(" Expression ")" +class ParenExpression : Expression +{ + Expression next; + this(Expression next) + { + mixin(set_kind); + addChild(next); + this.next = next; + } + mixin(copyMethod); +} + +// version(D2) +// { +class TraitsExpression : Expression +{ + Identifier* ident; + TemplateArguments targs; + this(typeof(ident) ident, typeof(targs) targs) + { + mixin(set_kind); + addOptChild(targs); + this.ident = ident; + this.targs = targs; + } + mixin(copyMethod); +} +// } + +class VoidInitExpression : Expression +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class ArrayInitExpression : Expression +{ + Expression[] keys; + Expression[] values; + this(Expression[] keys, Expression[] values) + { + assert(keys.length == values.length); + mixin(set_kind); + foreach (i, key; keys) + { + addOptChild(key); // The key is optional in ArrayInitializers. + addChild(values[i]); + } + this.keys = keys; + this.values = values; + } + mixin(copyMethod); +} + +class StructInitExpression : Expression +{ + Identifier*[] idents; + Expression[] values; + this(Identifier*[] idents, Expression[] values) + { + mixin(set_kind); + addOptChildren(values); + this.idents = idents; + this.values = values; + } + mixin(copyMethod); +} + +class AsmTypeExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class AsmOffsetExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class AsmSegExpression : UnaryExpression +{ + this(Expression e) + { + super(e); + mixin(set_kind); + } +} + +class AsmPostBracketExpression : UnaryExpression +{ + Expression e2; /// Expression in brackets: e [ e2 ] + this(Expression e, Expression e2) + { + super(e); + mixin(set_kind); + addChild(e2); + this.e2 = e2; + } + mixin(copyMethod); +} + +class AsmBracketExpression : Expression +{ + Expression e; + this(Expression e) + { + mixin(set_kind); + addChild(e); + this.e = e; + } + mixin(copyMethod); +} + +class AsmLocalSizeExpression : Expression +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class AsmRegisterExpression : Expression +{ + Identifier* register; + int number; // ST(0) - ST(7) or FS:0, FS:4, FS:8 + this(Identifier* register, int number = -1) + { + mixin(set_kind); + this.register = register; + this.number = number; + } + mixin(copyMethod); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Node.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Node.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,98 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Node; + +import common; + +public import dil.lexer.Token; +public import dil.ast.NodesEnum; + +/// The root class of all D syntax tree elements. +abstract class Node +{ + NodeCategory category; /// The category of this node. + NodeKind kind; /// The kind of this node. + Node[] children; // Will be probably removed sometime. + Token* begin, end; /// The begin and end tokens of this node. + + /// Constructs a node object. + this(NodeCategory category) + { + assert(category != NodeCategory.Undefined); + this.category = category; + } + + void setTokens(Token* begin, Token* end) + { + this.begin = begin; + this.end = end; + } + + Class setToks(Class)(Class node) + { + node.setTokens(this.begin, this.end); + return node; + } + + void addChild(Node child) + { + assert(child !is null, "failed in " ~ this.classinfo.name); + this.children ~= child; + } + + void addOptChild(Node child) + { + child is null || addChild(child); + } + + void addChildren(Node[] children) + { + assert(children !is null && delegate{ + foreach (child; children) + if (child is null) + return false; + return true; }(), + "failed in " ~ this.classinfo.name + ); + this.children ~= children; + } + + void addOptChildren(Node[] children) + { + children is null || addChildren(children); + } + + /// Returns a reference to Class if this node can be cast to it. + Class Is(Class)() + { + if (kind == mixin("NodeKind." ~ typeof(Class).stringof)) + return cast(Class)cast(void*)this; + return null; + } + + /// Casts this node to Class. + Class to(Class)() + { + return cast(Class)cast(void*)this; + } + + /// Returns a deep copy of this node. + abstract Node copy(); + + /// Returns a shallow copy of this object. + final Node dup() + { + // Find out the size of this object. + alias typeof(this.classinfo.init[0]) byte_t; + size_t size = this.classinfo.init.length; + // Copy this object's data. + byte_t[] data = (cast(byte_t*)this)[0..size].dup; + return cast(Node)data.ptr; + } + + /// This string is mixed into the constructor of a class that inherits + /// from Node. It sets the member kind. + const string set_kind = `this.kind = mixin("NodeKind." ~ typeof(this).stringof);`; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/NodeCopier.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/NodeCopier.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,324 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.NodeCopier; + +import common; + +/// Mixed into the body of a class that inherits from Node. +const string copyMethod = ` + override typeof(this) copy() + { + mixin copyNode!(typeof(this)); + return copyNode(this); + }`; + +/// A helper function that generates code for copying subnodes. +string doCopy_(string obj, string[] members) +{ + char[] result; + foreach (member; members) + { + if (member.length > 2 && member[$-2..$] == "[]") // Array copy. + { + member = member[0..$-2]; + // obj.member = obj.member.dup; + // foreach (ref m_; obj.member) + // m_ = m_.copy(); + result ~= obj~"."~member~" = "~obj~"."~member~".dup;" + "foreach (ref m_; "~obj~"."~member~")" + "m_ = m_.copy();"; + } + else if (member[$-1] == '?') // Optional member copy. + { + member = member[0..$-1]; + // obj.member && (obj.member = obj.member.copy()); + result ~= obj~"."~member~" && ("~obj~"."~member~" = "~obj~"."~member~".copy());"; + } + else // Non-optional member copy. + // obj.member = obj.member.copy(); + result ~= obj~"."~member~" = "~obj~"."~member~".copy();"; + } + return result; +} + +string doCopy(string[] members) +{ + return doCopy_("x", members); +} + +string doCopy(string member) +{ + return doCopy_("x", [member]); +} + +// pragma(msg, doCopy("decls?")); + +/// Returns a deep copy of node. +T copyNode(T)(T node) +{ + assert(node !is null); + + // Firstly do a shallow copy. + T x = cast(T)cast(void*)node.dup; + + // Now copy the subnodes. + static if (is(Declaration) && is(T : Declaration)) + { + alias T D; + static if (is(D == CompoundDeclaration)) + mixin(doCopy("decls[]")); + //EmptyDeclaration, + //IllegalDeclaration, + //ModuleDeclaration have no subnodes. + static if (is(D == AliasDeclaration) || + is(D == TypedefDeclaration)) + mixin(doCopy("decl")); + static if (is(D == EnumDeclaration)) + mixin(doCopy(["baseType?", "members[]"])); + static if (is(D == EnumMemberDeclaration)) + mixin(doCopy("value")); + static if (is(D == ClassDeclaration) || is( D == InterfaceDeclaration)) + mixin(doCopy(["bases[]", "decls"])); + static if (is(D == StructDeclaration) || is(D == UnionDeclaration)) + mixin(doCopy("decls")); + static if (is(D == ConstructorDeclaration)) + mixin(doCopy(["params", "funcBody"])); + static if (is(D == StaticConstructorDeclaration) || + is(D == DestructorDeclaration) || + is(D == StaticDestructorDeclaration) || + is(D == InvariantDeclaration) || + is(D == UnittestDeclaration)) + mixin(doCopy("funcBody")); + static if (is(D == FunctionDeclaration)) + mixin(doCopy(["returnType", "params", "funcBody"])); + static if (is(D == VariablesDeclaration)) + { + mixin(doCopy("typeNode?")); + x.inits = x.inits.dup; + foreach(ref init; x.inits) + init && (init = init.copy()); + } + static if (is(D == DebugDeclaration) || is(D == VersionDeclaration)) + mixin(doCopy(["decls?","elseDecls?"])); + static if (is(D == StaticIfDeclaration)) + mixin(doCopy(["condition","ifDecls", "elseDecls?"])); + static if (is(D == StaticAssertDeclaration)) + mixin(doCopy(["condition","message?"])); + static if (is(D == TemplateDeclaration)) + mixin(doCopy(["tparams","decls"])); + static if (is(D == NewDeclaration) || is(D == DeleteDeclaration)) + mixin(doCopy(["params","funcBody"])); + static if (is(D == ProtectionDeclaration) || + is(D == StorageClassDeclaration) || + is(D == LinkageDeclaration) || + is(D == AlignDeclaration)) + mixin(doCopy("decls")); + static if (is(D == PragmaDeclaration)) + mixin(doCopy(["args[]","decls"])); + static if (is(D == MixinDeclaration)) + mixin(doCopy(["templateExpr?","argument?"])); + } + else + static if (is(Expression) && is(T : Expression)) + { + alias T E; + static if (is(E == IllegalExpression)) + {} + else + static if (is(E == CondExpression)) + mixin(doCopy(["condition", "lhs", "rhs"])); + else + static if (is(E : BinaryExpression)) + mixin(doCopy(["lhs", "rhs"])); + else + static if (is(E : UnaryExpression)) + { + static if (is(E == CastExpression)) + mixin(doCopy("type")); + mixin(doCopy("e")); // Copy member in base class UnaryExpression. + static if (is(E == IndexExpression)) + mixin(doCopy("args[]")); + static if (is(E == SliceExpression)) + mixin(doCopy(["left?", "right?"])); + static if (is(E == AsmPostBracketExpression)) + mixin(doCopy("e2")); + } + else + { + static if (is(E == NewExpression)) + mixin(doCopy(["newArgs[]", "type", "ctorArgs[]"])); + static if (is(E == NewAnonClassExpression)) + mixin(doCopy(["newArgs[]", "bases[]", "ctorArgs[]", "decls"])); + static if (is(E == AsmBracketExpression)) + mixin(doCopy("e")); + static if (is(E == TemplateInstanceExpression)) + mixin(doCopy("targs?")); + static if (is(E == ArrayLiteralExpression)) + mixin(doCopy("values[]")); + static if (is(E == AArrayLiteralExpression)) + mixin(doCopy(["keys[]", "values[]"])); + static if (is(E == AssertExpression)) + mixin(doCopy(["expr", "msg?"])); + static if (is(E == MixinExpression) || + is(E == ImportExpression)) + mixin(doCopy("expr")); + static if (is(E == TypeofExpression) || + is(E == TypeDotIdExpression) || + is(E == TypeidExpression)) + mixin(doCopy("type")); + static if (is(E == IsExpression)) + mixin(doCopy(["type", "specType?", "tparams?"])); + static if (is(E == FunctionLiteralExpression)) + mixin(doCopy(["returnType?", "params?", "funcBody"])); + static if (is(E == ParenExpression)) + mixin(doCopy("next")); + static if (is(E == TraitsExpression)) + mixin(doCopy("targs")); + // VoidInitializer has no subnodes. + static if (is(E == ArrayInitExpression)) + { + mixin(doCopy("values[]")); + x.keys = x.keys.dup; + foreach(ref key; x.keys) + key && (key = key.copy()); + } + static if (is(E == StructInitExpression)) + mixin(doCopy("values[]")); + static if (is(E == StringExpression)) + x.str = x.str.dup; + } + } + else + static if (is(Statement) && is(T : Statement)) + { + alias T S; + static if (is(S == CompoundStatement)) + mixin(doCopy("stmnts[]")); + //IllegalStatement, + //EmptyStatement have no subnodes. + static if (is(S == FuncBodyStatement)) + mixin(doCopy(["funcBody?", "inBody?", "outBody?"])); + static if (is(S == ScopeStatement) || is(S == LabeledStatement)) + mixin(doCopy("s")); + static if (is(S == ExpressionStatement)) + mixin(doCopy("e")); + static if (is(S == DeclarationStatement)) + mixin(doCopy("decl")); + static if (is(S == IfStatement)) + { + if (x.variable) + mixin(doCopy("variable")); + else + mixin(doCopy("condition")); + mixin(doCopy(["ifBody", "elseBody?"])); + } + static if (is(S == WhileStatement)) + mixin(doCopy(["condition", "whileBody"])); + static if (is(S == DoWhileStatement)) + mixin(doCopy(["doBody", "condition"])); + static if (is(S == ForStatement)) + mixin(doCopy(["init?", "condition?", "increment?", "forBody"])); + static if (is(S == ForeachStatement)) + mixin(doCopy(["params", "aggregate", "forBody"])); + static if (is(S == ForeachRangeStatement)) + mixin(doCopy(["params", "lower", "upper", "forBody"])); + static if (is(S == SwitchStatement)) + mixin(doCopy(["condition", "switchBody"])); + static if (is(S == CaseStatement)) + mixin(doCopy(["values[]", "caseBody"])); + static if (is(S == DefaultStatement)) + mixin(doCopy("defaultBody")); + //ContinueStatement, + //BreakStatement have no subnodes. + static if (is(S == ReturnStatement)) + mixin(doCopy("e?")); + static if (is(S == GotoStatement)) + mixin(doCopy("caseExpr?")); + static if (is(S == WithStatement)) + mixin(doCopy(["e", "withBody"])); + static if (is(S == SynchronizedStatement)) + mixin(doCopy(["e?", "syncBody"])); + static if (is(S == TryStatement)) + mixin(doCopy(["tryBody", "catchBodies[]", "finallyBody?"])); + static if (is(S == CatchStatement)) + mixin(doCopy(["param?", "catchBody"])); + static if (is(S == FinallyStatement)) + mixin(doCopy("finallyBody")); + static if (is(S == ScopeGuardStatement)) + mixin(doCopy("scopeBody")); + static if (is(S == ThrowStatement)) + mixin(doCopy("e")); + static if (is(S == VolatileStatement)) + mixin(doCopy("volatileBody?")); + static if (is(S == AsmBlockStatement)) + mixin(doCopy("statements")); + static if (is(S == AsmStatement)) + mixin(doCopy("operands[]")); + //AsmAlignStatement, + //IllegalAsmStatement have no subnodes. + static if (is(S == PragmaStatement)) + mixin(doCopy(["args[]", "pragmaBody"])); + static if (is(S == MixinStatement)) + mixin(doCopy("templateExpr")); + static if (is(S == StaticIfStatement)) + mixin(doCopy(["condition", "ifBody", "elseBody?"])); + static if (is(S == StaticAssertStatement)) + mixin(doCopy(["condition", "message?"])); + static if (is(S == DebugStatement) || is(S == VersionStatement)) + mixin(doCopy(["mainBody", "elseBody?"])); + } + else + static if (is(TypeNode) && is(T : TypeNode)) + { + //IllegalType, + //IntegralType, + //ModuleScopeType, + //IdentifierType have no subnodes. + static if (is(T == QualifiedType)) + mixin(doCopy(["lhs", "rhs"])); + static if (is(T == TypeofType)) + mixin(doCopy("e")); + static if (is(T == TemplateInstanceType)) + mixin(doCopy("targs?")); + static if (is(T == PointerType)) + mixin(doCopy("next")); + static if (is(T == ArrayType)) + mixin(doCopy(["next", "assocType?", "e1?", "e2?"])); + static if (is(T == FunctionType) || is(T == DelegateType)) + mixin(doCopy(["returnType", "params"])); + static if (is(T == CFuncPointerType)) + mixin(doCopy(["next", "params?"])); + static if (is(T == BaseClassType) || + is(T == ConstType) || + is(T == InvariantType)) + mixin(doCopy("next")); + } + else + static if (is(Parameter) && is(T == Parameter)) + { + mixin(doCopy(["type?", "defValue?"])); + } + else + static if (is(Parameter) && is(T == Parameters) || + is(TemplateParameter) && is(T == TemplateParameters) || + is(TemplateArguments) && is(T == TemplateArguments)) + { + mixin(doCopy("children[]")); + } + else + static if (is(TemplateParameter) && is(T : TemplateParameter)) + { + static if (is(N == TemplateAliasParameter) || + is(N == TemplateTypeParameter) || + is(N == TemplateThisParameter)) + mixin(doCopy(["specType", "defType"])); + static if (is(N == TemplateValueParameter)) + mixin(doCopy(["valueType", "specValue?", "defValue?"])); + //TemplateTupleParameter has no subnodes. + } + else + static assert(0, "copying of "~typeof(x).stringof~" is not handled"); + return x; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/NodesEnum.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/NodesEnum.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,234 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.NodesEnum; + +/// Enumerates the categories of a node. +enum NodeCategory : ushort +{ + Undefined, + Declaration, + Statement, + Expression, + Type, + Other +} + +/// A list of all class names that inherit from Node. +static const char[][] g_classNames = [ + // Declarations: + "CompoundDeclaration", + "EmptyDeclaration", + "IllegalDeclaration", + "ModuleDeclaration", + "ImportDeclaration", + "AliasDeclaration", + "TypedefDeclaration", + "EnumDeclaration", + "EnumMemberDeclaration", + "ClassDeclaration", + "InterfaceDeclaration", + "StructDeclaration", + "UnionDeclaration", + "ConstructorDeclaration", + "StaticConstructorDeclaration", + "DestructorDeclaration", + "StaticDestructorDeclaration", + "FunctionDeclaration", + "VariablesDeclaration", + "InvariantDeclaration", + "UnittestDeclaration", + "DebugDeclaration", + "VersionDeclaration", + "StaticIfDeclaration", + "StaticAssertDeclaration", + "TemplateDeclaration", + "NewDeclaration", + "DeleteDeclaration", + "ProtectionDeclaration", + "StorageClassDeclaration", + "LinkageDeclaration", + "AlignDeclaration", + "PragmaDeclaration", + "MixinDeclaration", + + // Statements: + "CompoundStatement", + "IllegalStatement", + "EmptyStatement", + "FuncBodyStatement", + "ScopeStatement", + "LabeledStatement", + "ExpressionStatement", + "DeclarationStatement", + "IfStatement", + "WhileStatement", + "DoWhileStatement", + "ForStatement", + "ForeachStatement", + "ForeachRangeStatement", // D2.0 + "SwitchStatement", + "CaseStatement", + "DefaultStatement", + "ContinueStatement", + "BreakStatement", + "ReturnStatement", + "GotoStatement", + "WithStatement", + "SynchronizedStatement", + "TryStatement", + "CatchStatement", + "FinallyStatement", + "ScopeGuardStatement", + "ThrowStatement", + "VolatileStatement", + "AsmBlockStatement", + "AsmStatement", + "AsmAlignStatement", + "IllegalAsmStatement", + "PragmaStatement", + "MixinStatement", + "StaticIfStatement", + "StaticAssertStatement", + "DebugStatement", + "VersionStatement", + + // Expressions: + "IllegalExpression", + "CondExpression", + "CommaExpression", + "OrOrExpression", + "AndAndExpression", + "OrExpression", + "XorExpression", + "AndExpression", + "EqualExpression", + "IdentityExpression", + "RelExpression", + "InExpression", + "LShiftExpression", + "RShiftExpression", + "URShiftExpression", + "PlusExpression", + "MinusExpression", + "CatExpression", + "MulExpression", + "DivExpression", + "ModExpression", + "AssignExpression", + "LShiftAssignExpression", + "RShiftAssignExpression", + "URShiftAssignExpression", + "OrAssignExpression", + "AndAssignExpression", + "PlusAssignExpression", + "MinusAssignExpression", + "DivAssignExpression", + "MulAssignExpression", + "ModAssignExpression", + "XorAssignExpression", + "CatAssignExpression", + "AddressExpression", + "PreIncrExpression", + "PreDecrExpression", + "PostIncrExpression", + "PostDecrExpression", + "DerefExpression", + "SignExpression", + "NotExpression", + "CompExpression", + "CallExpression", + "NewExpression", + "NewAnonClassExpression", + "DeleteExpression", + "CastExpression", + "IndexExpression", + "SliceExpression", + "ModuleScopeExpression", + "IdentifierExpression", + "SpecialTokenExpression", + "DotExpression", + "TemplateInstanceExpression", + "ThisExpression", + "SuperExpression", + "NullExpression", + "DollarExpression", + "BoolExpression", + "IntExpression", + "RealExpression", + "ComplexExpression", + "CharExpression", + "StringExpression", + "ArrayLiteralExpression", + "AArrayLiteralExpression", + "AssertExpression", + "MixinExpression", + "ImportExpression", + "TypeofExpression", + "TypeDotIdExpression", + "TypeidExpression", + "IsExpression", + "ParenExpression", + "FunctionLiteralExpression", + "TraitsExpression", // D2.0 + "VoidInitExpression", + "ArrayInitExpression", + "StructInitExpression", + "AsmTypeExpression", + "AsmOffsetExpression", + "AsmSegExpression", + "AsmPostBracketExpression", + "AsmBracketExpression", + "AsmLocalSizeExpression", + "AsmRegisterExpression", + + // Types: + "IllegalType", + "IntegralType", + "QualifiedType", + "ModuleScopeType", + "IdentifierType", + "TypeofType", + "TemplateInstanceType", + "PointerType", + "ArrayType", + "FunctionType", + "DelegateType", + "CFuncPointerType", + "BaseClassType", + "ConstType", // D2.0 + "InvariantType", // D2.0 + + // Other: + "Parameter", + "Parameters", + "TemplateAliasParameter", + "TemplateTypeParameter", + "TemplateThisParameter", // D2.0 + "TemplateValueParameter", + "TemplateTupleParameter", + "TemplateParameters", + "TemplateArguments", +]; + +/// Generates the members of enum NodeKind. +char[] generateNodeKindMembers() +{ + char[] text; + foreach (className; g_classNames) + text ~= className ~ ","; + return text; +} +// pragma(msg, generateNodeKindMembers()); + +version(DDoc) + /// The node kind identifies every class that inherits from Node. + enum NodeKind : ushort; +else +mixin( + "enum NodeKind : ushort" + "{" + ~ generateNodeKindMembers ~ + "}" +); diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Parameters.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Parameters.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,227 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Parameters; + +import dil.ast.Node; +import dil.ast.Type; +import dil.ast.Expression; +import dil.ast.NodeCopier; +import dil.lexer.Identifier; +import dil.Enums; + +/// A function or foreach parameter. +class Parameter : Node +{ + StorageClass stc; /// The storage classes of the parameter. + TypeNode type; /// The parameter's type. + Identifier* name; /// The name of the parameter. + Expression defValue; /// The default initialization value. + + this(StorageClass stc, TypeNode type, Identifier* name, Expression defValue) + { + super(NodeCategory.Other); + mixin(set_kind); + // type can be null when param in foreach statement + addOptChild(type); + addOptChild(defValue); + + this.stc = stc; + this.type = type; + this.name = name; + this.defValue = defValue; + } + + /// Returns true if this is a D-style variadic parameter. + /// E.g.: func(int[] values ...) + bool isDVariadic() + { + return isVariadic && !isCVariadic; + } + + /// Returns true if this is a C-style variadic parameter. + /// E.g.: func(...) + bool isCVariadic() + { + return stc == StorageClass.Variadic && + type is null && name is null; + } + + /// Returns true if this is a D- or C-style variadic parameter. + bool isVariadic() + { + return !!(stc & StorageClass.Variadic); + } + + mixin(copyMethod); +} + +/// Array of parameters. +class Parameters : Node +{ + this() + { + super(NodeCategory.Other); + mixin(set_kind); + } + + bool hasVariadic() + { + if (children.length != 0) + return items[$-1].isVariadic(); + return false; + } + + void opCatAssign(Parameter param) + { addChild(param); } + + Parameter[] items() + { return cast(Parameter[])children; } + + size_t length() + { return children.length; } + + mixin(copyMethod); +} + +/*~~~~~~~~~~~~~~~~~~~~~~ +~ Template parameters: ~ +~~~~~~~~~~~~~~~~~~~~~~*/ + +/// Abstract base class for all template parameters. +abstract class TemplateParameter : Node +{ + Identifier* ident; + this(Identifier* ident) + { + super(NodeCategory.Other); + this.ident = ident; + } + override abstract TemplateParameter copy(); +} + +/// E.g.: (alias T) +class TemplateAliasParameter : TemplateParameter +{ + TypeNode specType, defType; + this(Identifier* ident, TypeNode specType, TypeNode defType) + { + super(ident); + mixin(set_kind); + addOptChild(specType); + addOptChild(defType); + this.ident = ident; + this.specType = specType; + this.defType = defType; + } + mixin(copyMethod); +} + +/// E.g.: (T t) +class TemplateTypeParameter : TemplateParameter +{ + TypeNode specType, defType; + this(Identifier* ident, TypeNode specType, TypeNode defType) + { + super(ident); + mixin(set_kind); + addOptChild(specType); + addOptChild(defType); + this.ident = ident; + this.specType = specType; + this.defType = defType; + } + mixin(copyMethod); +} + +// version(D2) +// { +/// E.g.: (this T) +class TemplateThisParameter : TemplateParameter +{ + TypeNode specType, defType; + this(Identifier* ident, TypeNode specType, TypeNode defType) + { + super(ident); + mixin(set_kind); + addOptChild(specType); + addOptChild(defType); + this.ident = ident; + this.specType = specType; + this.defType = defType; + } + mixin(copyMethod); +} +// } + +/// E.g.: (T) +class TemplateValueParameter : TemplateParameter +{ + TypeNode valueType; + Expression specValue, defValue; + this(TypeNode valueType, Identifier* ident, Expression specValue, Expression defValue) + { + super(ident); + mixin(set_kind); + addChild(valueType); + addOptChild(specValue); + addOptChild(defValue); + this.valueType = valueType; + this.ident = ident; + this.specValue = specValue; + this.defValue = defValue; + } + mixin(copyMethod); +} + +/// E.g.: (T...) +class TemplateTupleParameter : TemplateParameter +{ + this(Identifier* ident) + { + super(ident); + mixin(set_kind); + this.ident = ident; + } + mixin(copyMethod); +} + +/// Array of template parameters. +class TemplateParameters : Node +{ + this() + { + super(NodeCategory.Other); + mixin(set_kind); + } + + void opCatAssign(TemplateParameter parameter) + { + addChild(parameter); + } + + TemplateParameter[] items() + { + return cast(TemplateParameter[])children; + } + + mixin(copyMethod); +} + +/// Array of template arguments. +class TemplateArguments : Node +{ + this() + { + super(NodeCategory.Other); + mixin(set_kind); + } + + void opCatAssign(Node argument) + { + addChild(argument); + } + + mixin(copyMethod); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Statement.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Statement.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,18 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Statement; + +import dil.ast.Node; + +/// The root class of all statements. +abstract class Statement : Node +{ + this() + { + super(NodeCategory.Statement); + } + + override abstract Statement copy(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Statements.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Statements.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,607 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Statements; + +public import dil.ast.Statement; +import dil.ast.Node; +import dil.ast.Expression; +import dil.ast.Declaration; +import dil.ast.Type; +import dil.ast.Parameters; +import dil.ast.NodeCopier; +import dil.lexer.IdTable; + +class CompoundStatement : Statement +{ + this() + { + mixin(set_kind); + } + + void opCatAssign(Statement s) + { + addChild(s); + } + + Statement[] stmnts() + { + return cast(Statement[])this.children; + } + + void stmnts(Statement[] stmnts) + { + this.children = stmnts; + } + + mixin(copyMethod); +} + +class IllegalStatement : Statement +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class EmptyStatement : Statement +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class FuncBodyStatement : Statement +{ + Statement funcBody, inBody, outBody; + Identifier* outIdent; + this() + { + mixin(set_kind); + } + + void finishConstruction() + { + addOptChild(funcBody); + addOptChild(inBody); + addOptChild(outBody); + } + + bool isEmpty() + { + return funcBody is null; + } + + mixin(copyMethod); +} + +class ScopeStatement : Statement +{ + Statement s; + this(Statement s) + { + mixin(set_kind); + addChild(s); + this.s = s; + } + mixin(copyMethod); +} + +class LabeledStatement : Statement +{ + Identifier* label; + Statement s; + this(Identifier* label, Statement s) + { + mixin(set_kind); + addChild(s); + this.label = label; + this.s = s; + } + mixin(copyMethod); +} + +class ExpressionStatement : Statement +{ + Expression e; + this(Expression e) + { + mixin(set_kind); + addChild(e); + this.e = e; + } + mixin(copyMethod); +} + +class DeclarationStatement : Statement +{ + Declaration decl; + this(Declaration decl) + { + mixin(set_kind); + addChild(decl); + this.decl = decl; + } + mixin(copyMethod); +} + +class IfStatement : Statement +{ + Statement variable; // AutoDeclaration or VariableDeclaration + Expression condition; + Statement ifBody; + Statement elseBody; + this(Statement variable, Expression condition, Statement ifBody, Statement elseBody) + { + mixin(set_kind); + if (variable) + addChild(variable); + else + addChild(condition); + addChild(ifBody); + addOptChild(elseBody); + + this.variable = variable; + this.condition = condition; + this.ifBody = ifBody; + this.elseBody = elseBody; + } + mixin(copyMethod); +} + +class WhileStatement : Statement +{ + Expression condition; + Statement whileBody; + this(Expression condition, Statement whileBody) + { + mixin(set_kind); + addChild(condition); + addChild(whileBody); + + this.condition = condition; + this.whileBody = whileBody; + } + mixin(copyMethod); +} + +class DoWhileStatement : Statement +{ + Statement doBody; + Expression condition; + this(Expression condition, Statement doBody) + { + mixin(set_kind); + addChild(doBody); + addChild(condition); + + this.condition = condition; + this.doBody = doBody; + } + mixin(copyMethod); +} + +class ForStatement : Statement +{ + Statement init; + Expression condition, increment; + Statement forBody; + + this(Statement init, Expression condition, Expression increment, Statement forBody) + { + mixin(set_kind); + addOptChild(init); + addOptChild(condition); + addOptChild(increment); + addChild(forBody); + + this.init = init; + this.condition = condition; + this.increment = increment; + this.forBody = forBody; + } + mixin(copyMethod); +} + +class ForeachStatement : Statement +{ + TOK tok; + Parameters params; + Expression aggregate; + Statement forBody; + + this(TOK tok, Parameters params, Expression aggregate, Statement forBody) + { + mixin(set_kind); + addChildren([cast(Node)params, aggregate, forBody]); + + this.tok = tok; + this.params = params; + this.aggregate = aggregate; + this.forBody = forBody; + } + mixin(copyMethod); +} + +// version(D2) +// { +class ForeachRangeStatement : Statement +{ + TOK tok; + Parameters params; + Expression lower, upper; + Statement forBody; + + this(TOK tok, Parameters params, Expression lower, Expression upper, Statement forBody) + { + mixin(set_kind); + addChildren([cast(Node)params, lower, upper, forBody]); + + this.tok = tok; + this.params = params; + this.lower = lower; + this.upper = upper; + this.forBody = forBody; + } + mixin(copyMethod); +} +// } + +class SwitchStatement : Statement +{ + Expression condition; + Statement switchBody; + + this(Expression condition, Statement switchBody) + { + mixin(set_kind); + addChild(condition); + addChild(switchBody); + + this.condition = condition; + this.switchBody = switchBody; + } + mixin(copyMethod); +} + +class CaseStatement : Statement +{ + Expression[] values; + Statement caseBody; + + this(Expression[] values, Statement caseBody) + { + mixin(set_kind); + addChildren(values); + addChild(caseBody); + + this.values = values; + this.caseBody = caseBody; + } + mixin(copyMethod); +} + +class DefaultStatement : Statement +{ + Statement defaultBody; + this(Statement defaultBody) + { + mixin(set_kind); + addChild(defaultBody); + + this.defaultBody = defaultBody; + } + mixin(copyMethod); +} + +class ContinueStatement : Statement +{ + Identifier* ident; + this(Identifier* ident) + { + mixin(set_kind); + this.ident = ident; + } + mixin(copyMethod); +} + +class BreakStatement : Statement +{ + Identifier* ident; + this(Identifier* ident) + { + mixin(set_kind); + this.ident = ident; + } + mixin(copyMethod); +} + +class ReturnStatement : Statement +{ + Expression e; + this(Expression e) + { + mixin(set_kind); + addOptChild(e); + this.e = e; + } + mixin(copyMethod); +} + +class GotoStatement : Statement +{ + Identifier* ident; + Expression caseExpr; + this(Identifier* ident, Expression caseExpr) + { + mixin(set_kind); + addOptChild(caseExpr); + this.ident = ident; + this.caseExpr = caseExpr; + } + mixin(copyMethod); +} + +class WithStatement : Statement +{ + Expression e; + Statement withBody; + this(Expression e, Statement withBody) + { + mixin(set_kind); + addChild(e); + addChild(withBody); + + this.e = e; + this.withBody = withBody; + } + mixin(copyMethod); +} + +class SynchronizedStatement : Statement +{ + Expression e; + Statement syncBody; + this(Expression e, Statement syncBody) + { + mixin(set_kind); + addOptChild(e); + addChild(syncBody); + + this.e = e; + this.syncBody = syncBody; + } + mixin(copyMethod); +} + +class TryStatement : Statement +{ + Statement tryBody; + CatchStatement[] catchBodies; + FinallyStatement finallyBody; + this(Statement tryBody, CatchStatement[] catchBodies, FinallyStatement finallyBody) + { + mixin(set_kind); + addChild(tryBody); + addOptChildren(catchBodies); + addOptChild(finallyBody); + + this.tryBody = tryBody; + this.catchBodies = catchBodies; + this.finallyBody = finallyBody; + } + mixin(copyMethod); +} + +class CatchStatement : Statement +{ + Parameter param; + Statement catchBody; + this(Parameter param, Statement catchBody) + { + mixin(set_kind); + addOptChild(param); + addChild(catchBody); + this.param = param; + this.catchBody = catchBody; + } + mixin(copyMethod); +} + +class FinallyStatement : Statement +{ + Statement finallyBody; + this(Statement finallyBody) + { + mixin(set_kind); + addChild(finallyBody); + this.finallyBody = finallyBody; + } + mixin(copyMethod); +} + +class ScopeGuardStatement : Statement +{ + Identifier* condition; + Statement scopeBody; + this(Identifier* condition, Statement scopeBody) + { + mixin(set_kind); + addChild(scopeBody); + this.condition = condition; + this.scopeBody = scopeBody; + } + mixin(copyMethod); +} + +class ThrowStatement : Statement +{ + Expression e; + this(Expression e) + { + mixin(set_kind); + addChild(e); + this.e = e; + } + mixin(copyMethod); +} + +class VolatileStatement : Statement +{ + Statement volatileBody; + this(Statement volatileBody) + { + mixin(set_kind); + addOptChild(volatileBody); + this.volatileBody = volatileBody; + } + mixin(copyMethod); +} + +class AsmBlockStatement : Statement +{ + CompoundStatement statements; + this(CompoundStatement statements) + { + mixin(set_kind); + addChild(statements); + this.statements = statements; + } + mixin(copyMethod); +} + +class AsmStatement : Statement +{ + Identifier* ident; + Expression[] operands; + this(Identifier* ident, Expression[] operands) + { + mixin(set_kind); + addOptChildren(operands); + this.ident = ident; + this.operands = operands; + } + mixin(copyMethod); +} + +class AsmAlignStatement : Statement +{ + int number; + this(int number) + { + mixin(set_kind); + this.number = number; + } + mixin(copyMethod); +} + +class IllegalAsmStatement : IllegalStatement +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +class PragmaStatement : Statement +{ + Identifier* ident; + Expression[] args; + Statement pragmaBody; + this(Identifier* ident, Expression[] args, Statement pragmaBody) + { + mixin(set_kind); + addOptChildren(args); + addChild(pragmaBody); + + this.ident = ident; + this.args = args; + this.pragmaBody = pragmaBody; + } + mixin(copyMethod); +} + +class MixinStatement : Statement +{ + Expression templateExpr; + Identifier* mixinIdent; + this(Expression templateExpr, Identifier* mixinIdent) + { + mixin(set_kind); + addChild(templateExpr); + this.templateExpr = templateExpr; + this.mixinIdent = mixinIdent; + } + mixin(copyMethod); +} + +class StaticIfStatement : Statement +{ + Expression condition; + Statement ifBody, elseBody; + this(Expression condition, Statement ifBody, Statement elseBody) + { + mixin(set_kind); + addChild(condition); + addChild(ifBody); + addOptChild(elseBody); + this.condition = condition; + this.ifBody = ifBody; + this.elseBody = elseBody; + } + mixin(copyMethod); +} + +class StaticAssertStatement : Statement +{ + Expression condition, message; + this(Expression condition, Expression message) + { + mixin(set_kind); + addChild(condition); + addOptChild(message); + this.condition = condition; + this.message = message; + } + mixin(copyMethod); +} + +abstract class ConditionalCompilationStatement : Statement +{ + Token* cond; + Statement mainBody, elseBody; + this(Token* cond, Statement mainBody, Statement elseBody) + { + addChild(mainBody); + addOptChild(elseBody); + this.cond = cond; + this.mainBody = mainBody; + this.elseBody = elseBody; + } +} + +class DebugStatement : ConditionalCompilationStatement +{ + this(Token* cond, Statement debugBody, Statement elseBody) + { + super(cond, debugBody, elseBody); + mixin(set_kind); + } + mixin(copyMethod); +} + +class VersionStatement : ConditionalCompilationStatement +{ + this(Token* cond, Statement versionBody, Statement elseBody) + { + super(cond, versionBody, elseBody); + mixin(set_kind); + } + mixin(copyMethod); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Type.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Type.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,38 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Type; + +import dil.ast.Node; +import dil.semantic.Types; + +/// The root class of all type nodes. +abstract class TypeNode : Node +{ + TypeNode next; /// The next type in the type chain. + Type type; /// The semantic type of this type node. + + this() + { + this(null); + } + + this(TypeNode next) + { + super(NodeCategory.Type); + addOptChild(next); + this.next = next; + } + + /// Returns the root type of the type chain. + TypeNode baseType() + { + auto type = this; + while (type.next) + type = type.next; + return type; + } + + override abstract TypeNode copy(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Types.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Types.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,262 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Types; + +public import dil.ast.Type; +import dil.ast.Node; +import dil.ast.Expression; +import dil.ast.Parameters; +import dil.ast.NodeCopier; +import dil.lexer.Identifier; +import dil.semantic.Types; +import dil.Enums; + +/// Syntax error. +class IllegalType : TypeNode +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +/// char, int, float etc. +class IntegralType : TypeNode +{ + TOK tok; + this(TOK tok) + { + mixin(set_kind); + this.tok = tok; + } + mixin(copyMethod); +} + +/// Identifier +class IdentifierType : TypeNode +{ + Identifier* ident; + this(Identifier* ident) + { + mixin(set_kind); + this.ident = ident; + } + mixin(copyMethod); +} + +/// Type "." Type +class QualifiedType : TypeNode +{ + alias next lhs; /// Left-hand side type. + TypeNode rhs; /// Right-hand side type. + this(TypeNode lhs, TypeNode rhs) + { + super(lhs); + mixin(set_kind); + addChild(rhs); + this.rhs = rhs; + } + mixin(copyMethod); +} + +/// "." Type +class ModuleScopeType : TypeNode +{ + this() + { + mixin(set_kind); + } + mixin(copyMethod); +} + +/// "typeof" "(" Expression ")" or$(BR) +/// "typeof" "(" "return" ")" (D2.0) +class TypeofType : TypeNode +{ + Expression e; + this(Expression e) + { + this(); + addChild(e); + this.e = e; + } + + // For D2.0: "typeof" "(" "return" ")" + this() + { + mixin(set_kind); + } + + bool isTypeofReturn() + { + return e is null; + } + + mixin(copyMethod); +} + +/// Identifier "!" "(" TemplateParameters? ")" +class TemplateInstanceType : TypeNode +{ + Identifier* ident; + TemplateArguments targs; + this(Identifier* ident, TemplateArguments targs) + { + mixin(set_kind); + addOptChild(targs); + this.ident = ident; + this.targs = targs; + } + mixin(copyMethod); +} + +/// Type * +class PointerType : TypeNode +{ + this(TypeNode next) + { + super(next); + mixin(set_kind); + } + mixin(copyMethod); +} + +/// Dynamic array: T[] or$(BR) +/// Static array: T[E] or$(BR) +/// Slice array (for tuples): T[E..E] or$(BR) +/// Associative array: T[T] +class ArrayType : TypeNode +{ + Expression e1, e2; + TypeNode assocType; + + this(TypeNode t) + { + super(t); + mixin(set_kind); + } + + this(TypeNode t, Expression e1, Expression e2) + { + this(t); + addChild(e1); + addOptChild(e2); + this.e1 = e1; + this.e2 = e2; + } + + this(TypeNode t, TypeNode assocType) + { + this(t); + addChild(assocType); + this.assocType = assocType; + } + + bool isDynamic() + { + return !assocType && !e1; + } + + bool isStatic() + { + return e1 && !e2; + } + + bool isSlice() + { + return e1 && e2; + } + + bool isAssociative() + { + return assocType !is null; + } + + mixin(copyMethod); +} + +/// ReturnType "function" "(" Parameters? ")" +class FunctionType : TypeNode +{ + alias next returnType; + Parameters params; + this(TypeNode returnType, Parameters params) + { + super(returnType); + mixin(set_kind); + addChild(params); + this.params = params; + } + mixin(copyMethod); +} + +/// ReturnType "delegate" "(" Parameters? ")" +class DelegateType : TypeNode +{ + alias next returnType; + Parameters params; + this(TypeNode returnType, Parameters params) + { + super(returnType); + mixin(set_kind); + addChild(params); + this.params = params; + } + mixin(copyMethod); +} + +/// Type "(" BasicType2 Identifier ")" "(" Parameters? ")" +class CFuncPointerType : TypeNode +{ + Parameters params; + this(TypeNode type, Parameters params) + { + super(type); + mixin(set_kind); + addOptChild(params); + } + mixin(copyMethod); +} + +/// "class" Identifier : BaseClasses +class BaseClassType : TypeNode +{ + Protection prot; + this(Protection prot, TypeNode type) + { + super(type); + mixin(set_kind); + this.prot = prot; + } + mixin(copyMethod); +} + +// version(D2) +// { +/// "const" "(" Type ")" +class ConstType : TypeNode +{ + this(TypeNode next) + { + // If t is null: cast(const) + super(next); + mixin(set_kind); + } + mixin(copyMethod); +} + +/// "invariant" "(" Type ")" +class InvariantType : TypeNode +{ + this(TypeNode next) + { + // If t is null: cast(invariant) + super(next); + mixin(set_kind); + } + mixin(copyMethod); +} +// } // version(D2) diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/ast/Visitor.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/ast/Visitor.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,155 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.ast.Visitor; + +import dil.ast.Node; +import dil.ast.Declarations, + dil.ast.Expressions, + dil.ast.Statements, + dil.ast.Types, + dil.ast.Parameters; + +/// Generate visit methods. +/// +/// E.g.: +/// --- +/// Declaration visit(ClassDeclaration){return null;}; +/// Expression visit(CommaExpression){return null;}; +/// --- +char[] generateVisitMethods() +{ + char[] text; + foreach (className; g_classNames) + text ~= "returnType!(\""~className~"\") visit("~className~" node){return node;}\n"; + return text; +} +// pragma(msg, generateAbstractVisitMethods()); + +/// Gets the appropriate return type for the provided class. +template returnType(char[] className) +{ + static if (is(typeof(mixin(className)) : Declaration)) + alias Declaration returnType; + else + static if (is(typeof(mixin(className)) : Statement)) + alias Statement returnType; + else + static if (is(typeof(mixin(className)) : Expression)) + alias Expression returnType; + else + static if (is(typeof(mixin(className)) : TypeNode)) + alias TypeNode returnType; + else + alias Node returnType; +} + +/// Generate functions which do the second dispatch. +/// +/// E.g.: +/// --- +/// Expression visitCommaExpression(Visitor visitor, CommaExpression c) +/// { visitor.visit(c); /* Second dispatch. */ } +/// --- +/// The equivalent in the traditional visitor pattern would be: +/// --- +/// class CommaExpression : Expression +/// { +/// void accept(Visitor visitor) +/// { visitor.visit(this); } +/// } +/// --- +char[] generateDispatchFunctions() +{ + char[] text; + foreach (className; g_classNames) + text ~= "returnType!(\""~className~"\") visit"~className~"(Visitor visitor, "~className~" c)\n" + "{ return visitor.visit(c); }\n"; + return text; +} +// pragma(msg, generateDispatchFunctions()); + +/++ + Generates an array of function pointers. + + --- + [ + cast(void*)&visitCommaExpression, + // etc. + ] + --- ++/ +char[] generateVTable() +{ + char[] text = "["; + foreach (className; g_classNames) + text ~= "cast(void*)&visit"~className~",\n"; + return text[0..$-2]~"]"; // slice away last ",\n" +} +// pragma(msg, generateVTable()); + +/// Implements a variation of the visitor pattern. +/// +/// Inherited by classes that need to traverse a D syntax tree +/// and do computations, transformations and other things on it. +abstract class Visitor +{ + mixin(generateVisitMethods()); + + static + mixin(generateDispatchFunctions()); + + /// The table holding function pointers to the second dispatch functions. + static const void*[] dispatch_vtable = mixin(generateVTable()); + static assert(dispatch_vtable.length == g_classNames.length, "vtable length doesn't match number of classes"); + + /// Looks up the second dispatch function for n and returns that. + Node function(Visitor, Node) getDispatchFunction()(Node n) + { + return cast(Node function(Visitor, Node))dispatch_vtable[n.kind]; + } + + /// The main and first dispatch function. + Node dispatch(Node n) + { // Second dispatch is done in the called function. + return getDispatchFunction(n)(this, n); + } + +final: + Declaration visit(Declaration n) + { return visitD(n); } + Statement visit(Statement n) + { return visitS(n); } + Expression visit(Expression n) + { return visitE(n); } + TypeNode visit(TypeNode n) + { return visitT(n); } + Node visit(Node n) + { return visitN(n); } + + Declaration visitD(Declaration n) + { + return cast(Declaration)cast(void*)dispatch(n); + } + + Statement visitS(Statement n) + { + return cast(Statement)cast(void*)dispatch(n); + } + + Expression visitE(Expression n) + { + return cast(Expression)cast(void*)dispatch(n); + } + + TypeNode visitT(TypeNode n) + { + return cast(TypeNode)cast(void*)dispatch(n); + } + + Node visitN(Node n) + { + return dispatch(n); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/doc/Doc.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/doc/Doc.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,463 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.doc.Doc; + +import dil.doc.Parser; +import dil.ast.Node; +import dil.lexer.Funcs; +import dil.Unicode; +import common; + +import tango.text.Ascii : icompare; + +/// Represents a sanitized and parsed DDoc comment. +class DDocComment +{ + Section[] sections; /// The sections of this comment. + Section summary; /// Optional summary section. + Section description; /// Optional description section. + + this(Section[] sections, Section summary, Section description) + { + this.sections = sections; + this.summary = summary; + this.description = description; + } + + /// Removes the first copyright section and returns it. + Section takeCopyright() + { + foreach (i, section; sections) + if (section.Is("copyright")) + { + sections = sections[0..i] ~ sections[i+1..$]; + return section; + } + return null; + } + + /// Returns true if "ditto" is the only text in this comment. + bool isDitto() + { + if (summary && sections.length == 1 && + icompare(strip(summary.text), "ditto") == 0) + return true; + return false; + } +} + +/// Returns a node's DDocComment. +DDocComment getDDocComment(Node node) +{ + DDocParser p; + auto docTokens = getDocTokens(node); + if (!docTokens.length) + return null; + p.parse(getDDocText(docTokens)); + return new DDocComment(p.sections, p.summary, p.description); +} + +/// Strips leading and trailing whitespace characters. +/// Whitespace: ' ', '\t', '\v', '\f' and '\n' +/// Returns: a slice into str. +char[] strip(char[] str) +{ + if (str.length == 0) + return null; + uint i; + for (; i < str.length; i++) + if (!isspace(str[i]) && str[i] != '\n') + break; + if (str.length == i) + return null; + str = str[i..$]; + assert(str.length); + for (i = str.length; i; i--) + if (!isspace(str[i-1]) && str[i-1] != '\n') + break; + return str[0..i]; +} + +/// Parses a DDoc comment string. +struct DDocParser +{ + char* p; /// Current character pointer. + char* textEnd; /// Points one character past the end of the text. + Section[] sections; /// Parsed sections. + Section summary; /// Optional summary section. + Section description; /// Optional description section. + + /// Parses the DDoc text into sections. + Section[] parse(string text) + { + if (!text.length) + return null; + p = text.ptr; + textEnd = p + text.length; + + char* summaryBegin; + string ident, nextIdent; + char* bodyBegin, nextBodyBegin; + + skipWhitespace(p); + summaryBegin = p; + + if (findNextIdColon(ident, bodyBegin)) + { // Check that this is not an explicit section. + if (summaryBegin != ident.ptr) + scanSummaryAndDescription(summaryBegin, ident.ptr); + } + else // There are no explicit sections. + { + scanSummaryAndDescription(summaryBegin, textEnd); + return sections; + } + + assert(ident.length); + // Continue parsing. + while (findNextIdColon(nextIdent, nextBodyBegin)) + { + sections ~= new Section(ident, textBody(bodyBegin, nextIdent.ptr)); + ident = nextIdent; + bodyBegin = nextBodyBegin; + } + // Add last section. + sections ~= new Section(ident, textBody(bodyBegin, textEnd)); + return sections; + } + + /// Returns the text body. Trailing whitespace characters are not included. + char[] textBody(char* begin, char* end) + { + // The body of A is empty, e.g.: + // A: + // B: some text + // ^- begin and end point to B (or to this.textEnd in the 2nd case.) + if (begin is end) + return ""; + // Remove trailing whitespace. + while (isspace(*--end) || *end == '\n') + {} + end++; + return makeString(begin, end); + } + + /// Separates the text between p and end + /// into a summary and description section. + void scanSummaryAndDescription(char* p, char* end) + { + assert(p <= end); + char* sectionBegin = p; + // Search for the end of the first paragraph. + end--; // Decrement end, so we can look ahead one character. + while (p < end && !(*p == '\n' && p[1] == '\n')) + { + if (isCodeSection(p, end)) + skipCodeSection(p, end); + p++; + } + end++; + if (p+1 >= end) + p = end; + assert(p == end || (*p == '\n' && p[1] == '\n')); + // The first paragraph is the summary. + summary = new Section("", makeString(sectionBegin, p)); + sections ~= summary; + // The rest is the description section. + if (p < end) + { + skipWhitespace(p); + sectionBegin = p; + if (p < end) + { + description = new Section("", makeString(sectionBegin, end)); + sections ~= description; + } + } + } + + /// Returns true if p points to "$(DDD)". + bool isCodeSection(char* p, char* end) + { + return p+2 < end && *p == '-' && p[1] == '-' && p[2] == '-'; + } + + /// Skips over a code section. + /// + /// Note that dmd apparently doesn't skip over code sections when + /// parsing DDoc sections. However, from experience it seems + /// to be a good idea to do that. + void skipCodeSection(ref char* p, char* end) + out { assert(p+1 == end || *p == '-'); } + body + { + assert(isCodeSection(p, end)); + + while (p < end && *p == '-') + p++; + p--; + while (++p < end) + if (p+2 < end && *p == '-' && p[1] == '-' && p[2] == '-') + break; + while (p < end && *p == '-') + p++; + p--; + } + + void skipWhitespace(ref char* p) + { + while (p < textEnd && (isspace(*p) || *p == '\n')) + p++; + } + + /// Find next "Identifier:". + /// Params: + /// ident = set to the Identifier. + /// bodyBegin = set to the beginning of the text body (whitespace skipped.) + /// Returns: true if found. + bool findNextIdColon(ref char[] ident, ref char* bodyBegin) + { + while (p < textEnd) + { + skipWhitespace(p); + if (p >= textEnd) + break; + if (isCodeSection(p, textEnd)) + { + skipCodeSection(p, textEnd); + p++; + continue; + } + assert(isascii(*p) || isLeadByte(*p)); + auto idBegin = p; + if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart + { + do // IdChar* + p++; + while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) + auto idEnd = p; + if (p < textEnd && *p == ':') // : + { + p++; + skipWhitespace(p); + bodyBegin = p; + ident = makeString(idBegin, idEnd); + return true; + } + } + // Skip this line. + while (p < textEnd && *p != '\n') + p++; + } + return false; + } +} + +/// Represents a DDoc section. +class Section +{ + string name; + string text; + this(string name, string text) + { + this.name = name; + this.text = text; + } + + /// Case-insensitively compares the section's name with name2. + bool Is(char[] name2) + { + return icompare(name, name2) == 0; + } +} + +class ParamsSection : Section +{ + string[] paramNames; /// Parameter names. + string[] paramDescs; /// Parameter descriptions. + this(string name, string text) + { + super(name, text); + IdentValueParser parser; + auto idvalues = parser.parse(text); + this.paramNames = new string[idvalues.length]; + this.paramDescs = new string[idvalues.length]; + foreach (i, idvalue; idvalues) + { + this.paramNames[i] = idvalue.ident; + this.paramDescs[i] = idvalue.value; + } + } +} + +class MacrosSection : Section +{ + string[] macroNames; /// Macro names. + string[] macroTexts; /// Macro texts. + this(string name, string text) + { + super(name, text); + IdentValueParser parser; + auto idvalues = parser.parse(text); + this.macroNames = new string[idvalues.length]; + this.macroTexts = new string[idvalues.length]; + foreach (i, idvalue; idvalues) + { + this.macroNames[i] = idvalue.ident; + this.macroTexts[i] = idvalue.value; + } + } +} + +/// Returns true if token is a Doxygen comment. +bool isDoxygenComment(Token* token) +{ // Doxygen: '/+!' '/*!' '//!' + return token.kind == TOK.Comment && token.start[2] == '!'; +} + +/// Returns true if token is a DDoc comment. +bool isDDocComment(Token* token) +{ // DDOC: '/++' '/**' '///' + return token.kind == TOK.Comment && token.start[1] == token.start[2]; +} + +/// Returns the surrounding documentation comment tokens. +/// Params: +/// node = the node to find doc comments for. +/// isDocComment = a function predicate that checks for doc comment tokens. +/// Note: this function works correctly only if +/// the source text is syntactically correct. +Token*[] getDocTokens(Node node, bool function(Token*) isDocComment = &isDDocComment) +{ + Token*[] comments; + auto isEnumMember = node.kind == NodeKind.EnumMemberDeclaration; + // Get preceding comments. + auto token = node.begin; + // Scan backwards until we hit another declaration. +Loop: + for (; token; token = token.prev) + { + if (token.kind == TOK.LBrace || + token.kind == TOK.RBrace || + token.kind == TOK.Semicolon || + /+token.kind == TOK.HEAD ||+/ + (isEnumMember && token.kind == TOK.Comma)) + break; + + if (token.kind == TOK.Comment) + { // Check that this comment doesn't belong to the previous declaration. + switch (token.prev.kind) + { + case TOK.Semicolon, TOK.RBrace, TOK.Comma: + break Loop; + default: + if (isDocComment(token)) + comments = [token] ~ comments; + } + } + } + // Get single comment to the right. + token = node.end.next; + if (token.kind == TOK.Comment && isDocComment(token)) + comments ~= token; + else if (isEnumMember) + { + token = node.end.nextNWS; + if (token.kind == TOK.Comma) + { + token = token.next; + if (token.kind == TOK.Comment && isDocComment(token)) + comments ~= token; + } + } + return comments; +} + +bool isLineComment(Token* t) +{ + assert(t.kind == TOK.Comment); + return t.start[1] == '/'; +} + +/// Extracts the text body of the comment tokens. +string getDDocText(Token*[] tokens) +{ + if (tokens.length == 0) + return null; + string result; + foreach (token; tokens) + { + auto n = isLineComment(token) ? 0 : 2; // 0 for "//", 2 for "+/" and "*/". + result ~= sanitize(token.srcText[3 .. $-n], token.start[1]); + assert(token.next); + if (token.next.kind == TOK.Newline) + result ~= \n; + else + result ~= ' '; + } +// Stdout.formatln("→{}←", result); + return result[0..$-1]; // Remove \n or ' ' +} + +/// Sanitizes a DDoc comment string. +/// +/// Leading "commentChar"s are removed from the lines. +/// The various newline types are converted to '\n'. +/// Params: +/// comment = the string to be sanitized. +/// commentChar = '/', '+', or '*' +string sanitize(string comment, char commentChar) +{ + alias comment result; + + bool newline = true; // True when at the beginning of a new line. + uint i, j; + auto len = result.length; + for (; i < len; i++, j++) + { + if (newline) + { // Ignore commentChars at the beginning of each new line. + newline = false; + auto begin = i; + while (i < len && isspace(result[i])) + i++; + if (i < len && result[i] == commentChar) + while (++i < len && result[i] == commentChar) + {} + else + i = begin; // Reset. No commentChar found. + if (i >= len) + break; + } + // Check for Newline. + switch (result[i]) + { + case '\r': + if (i+1 < len && result[i+1] == '\n') + i++; + case '\n': + result[j] = '\n'; // Copy Newline as '\n'. + newline = true; + continue; + default: + if (!isascii(result[i]) && i+2 < len && isUnicodeNewline(result.ptr + i)) + { + i += 2; + goto case '\n'; + } + } + // Copy character. + result[j] = result[i]; + } + result.length = j; // Adjust length. + // Lastly, strip trailing commentChars. + if (!result.length) + return null; + i = result.length; + for (; i && result[i-1] == commentChar; i--) + {} + result.length = i; + return result; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/doc/Macro.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/doc/Macro.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,348 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.doc.Macro; + +import dil.doc.Parser; +import dil.lexer.Funcs; +import dil.Unicode; +import dil.Information; +import dil.Messages; +import common; + +/// The DDoc macro class. +class Macro +{ + string name; /// The name of the macro. + string text; /// The substitution text. + uint callLevel; /// Recursive call level. + this (string name, string text) + { + this.name = name; + this.text = text; + } +} + +/// Maps macro names to Macro objects. +/// +/// MacroTables can be chained so that they build a linear hierarchy. +/// Macro definitions in the current table override the ones in the parent tables. +class MacroTable +{ + /// The parent in the hierarchy. Or null if this is the root. + MacroTable parent; + Macro[string] table; /// The associative array that holds the macro definitions. + + /// Constructs a MacroTable instance. + this(MacroTable parent = null) + { + this.parent = parent; + } + + /// Inserts the macro m into the table. + /// Overwrites the current macro if one exists. + void insert(Macro m) + { + table[m.name] = m; + } + + /// Inserts an array of macros into the table. + void insert(Macro[] macros) + { + foreach (m; macros) + insert(m); + } + + /// Creates a macro using name and text and inserts that into the table. + void insert(string name, string text) + { + insert(new Macro(name, text)); + } + + /// Creates a macro using name[n] and text[n] and inserts that into the table. + void insert(string[] names, string[] texts) + { + assert(names.length == texts.length); + foreach (i, name; names) + insert(name, texts[i]); + } + + /// Searches for a macro. + /// + /// If the macro isn't found in this table the search + /// continues upwards in the table hierarchy. + /// Returns: the macro if found, or null if not. + Macro search(string name) + { + auto pmacro = name in table; + if (pmacro) + return *pmacro; + if (!isRoot()) + return parent.search(name); + return null; + } + + /// Returns: true if this is the root of the hierarchy. + bool isRoot() + { return parent is null; } +} + +/// Parses a text with macro definitions. +struct MacroParser +{ + Macro[] parse(string text) + { + IdentValueParser parser; + auto idvalues = parser.parse(text); + auto macros = new Macro[idvalues.length]; + foreach (i, idvalue; idvalues) + macros[i] = new Macro(idvalue.ident, idvalue.value); + return macros; + } + + /// Scans for a macro invocation. E.g.: $(DDOC) + /// Returns: a pointer set to one char past the closing parenthesis, + /// or null if this isn't a macro invocation. + static char* scanMacro(char* p, char* textEnd) + { + assert(*p == '$'); + if (p+2 < textEnd && p[1] == '(') + { + p += 2; + if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart + { + do // IdChar* + p++; + while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) + MacroExpander.scanArguments(p, textEnd); + p != textEnd && p++; // Skip ')'. + return p; + } + } + return null; + } +} + +/// Expands DDoc macros in a text. +struct MacroExpander +{ + MacroTable mtable; /// Used to look up macros. + InfoManager infoMan; /// Collects warning messages. + char[] filePath; /// Used in warning messages. + + /// Starts expanding the macros. + static char[] expand(MacroTable mtable, char[] text, char[] filePath, + InfoManager infoMan = null) + { + MacroExpander me; + me.mtable = mtable; + me.infoMan = infoMan; + me.filePath = filePath; + return me.expandMacros(text); + } + + /// Reports a warning message. + void warning(char[] msg, char[] macroName) + { + msg = Format(msg, macroName); + if (infoMan) + infoMan ~= new Warning(new Location(filePath, 0), msg); + } + + /// Expands the macros from the table in the text. + char[] expandMacros(char[] text, char[] prevArg0 = null/+, uint depth = 1000+/) + { + // if (depth == 0) + // return text; + // depth--; + char[] result; + char* p = text.ptr; + char* textEnd = p + text.length; + char* macroEnd = p; + while (p+3 < textEnd) // minimum 4 chars: $(x) + { + if (*p == '$' && p[1] == '(') + { + // Copy string between macros. + if (macroEnd != p) + result ~= makeString(macroEnd, p); + p += 2; + auto idBegin = p; + if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart + { + do // IdChar* + p++; + while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) + // Create macro name. + auto macroName = makeString(idBegin, p); + // Get arguments. + auto macroArgs = scanArguments(p, textEnd); + if (p == textEnd) + { + warning(MSG.UnterminatedDDocMacro, macroName); + result ~= "$(" ~ macroName ~ " "; + } + else + p++; + macroEnd = p; // Point past ')'. + + auto macro_ = mtable.search(macroName); + if (macro_) + { // Ignore recursive macro if: + auto macroArg0 = macroArgs.length ? macroArgs[0] : null; + if (macro_.callLevel != 0 && + (macroArgs.length == 0/+ || // Macro has no arguments. + prevArg0 == macroArg0+/)) // macroArg0 equals previous arg0. + { continue; } + macro_.callLevel++; + // Expand the arguments in the macro text. + auto expandedText = expandArguments(macro_.text, macroArgs); + result ~= expandMacros(expandedText, macroArg0/+, depth+/); + macro_.callLevel--; + } + else + { + warning(MSG.UndefinedDDocMacro, macroName); + //result ~= makeString(macroName.ptr-2, macroEnd); + } + continue; + } + } + p++; + } + if (macroEnd == text.ptr) + return text; // No macros found. Return original text. + if (macroEnd < textEnd) + result ~= makeString(macroEnd, textEnd); + return result; + } + + /// Scans until the closing parenthesis is found. Sets p to one char past it. + /// Returns: [arg0, arg1, arg2 ...]. + static char[][] scanArguments(ref char* p, char* textEnd) + out(args) { assert(args.length != 1); } + body + { + // D specs: "The argument text can contain nested parentheses, + // "" or '' strings, comments, or tags." + uint level = 1; // Nesting level of the parentheses. + char[][] args; + + // Skip leading spaces. + while (p < textEnd && isspace(*p)) + p++; + + char* arg0Begin = p; // Whole argument list. + char* argBegin = p; + MainLoop: + while (p < textEnd) + { + switch (*p) + { + case ',': + if (level != 1) // Ignore comma if inside (). + break; + // Add a new argument. + args ~= makeString(argBegin, p); + while (++p < textEnd && isspace(*p)) // Skip spaces. + {} + argBegin = p; + continue; + case '(': + level++; + break; + case ')': + if (--level == 0) + break MainLoop; + break; + // Commented out: causes too many problems in the expansion pass. + // case '"', '\'': + // auto c = *p; + // while (++p < textEnd && *p != c) // Scan to next " or '. + // {} + // assert(*p == c || p == textEnd); + // if (p == textEnd) + // break MainLoop; + // break; + case '<': + p++; + if (p+2 < textEnd && *p == '!' && p[1] == '-' && p[2] == '-') // ". + while (++p < textEnd) + if (p+2 < textEnd && *p == '-' && p[1] == '-' && p[2] == '>') + p += 2; // Point to '>'. + } // or + else if (p < textEnd && (isalpha(*p) || *p == '/')) + while (++p < textEnd && *p != '>') // Skip to closing '>'. + {} + else + continue MainLoop; + if (p == textEnd) + break MainLoop; + assert(*p == '>'); + break; + default: + } + p++; + } + assert(*p == ')' && level == 0 || p == textEnd); + if (arg0Begin == p) + return null; + // arg0 spans the whole argument list. + auto arg0 = makeString(arg0Begin, p); + // Add last argument. + args ~= makeString(argBegin, p); + return arg0 ~ args; + } + + /// Expands "$+", "$0" - "$9" with args[n] in text. + /// Params: + /// text = the text to scan for argument placeholders. + /// args = the first element, args[0], is the whole argument string and + /// the following elements are slices into it.$(BR) + /// The array is empty if there are no arguments. + char[] expandArguments(char[] text, char[][] args) + in { assert(args.length != 1, "zero or more than 1 args expected"); } + body + { + char[] result; + char* p = text.ptr; + char* textEnd = p + text.length; + char* placeholderEnd = p; + + while (p+1 < textEnd) + { + if (*p == '$' && (*++p == '+' || isdigit(*p))) + { + // Copy string between argument placeholders. + if (placeholderEnd != p-1) + result ~= makeString(placeholderEnd, p-1); + placeholderEnd = p+1; // Set new placeholder end. + + if (args.length == 0) + continue; + + if (*p == '+') + { // $+ = $2 to $n + if (args.length > 2) + result ~= makeString(args[2].ptr, args[0].ptr + args[0].length); + } + else + { // 0 - 9 + uint nthArg = *p - '0'; + if (nthArg < args.length) + result ~= args[nthArg]; + } + } + p++; + } + if (placeholderEnd == text.ptr) + return text; // No placeholders found. Return original text. + if (placeholderEnd < textEnd) + result ~= makeString(placeholderEnd, textEnd); + return result; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/doc/Parser.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/doc/Parser.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,131 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.doc.Parser; + +import dil.lexer.Funcs; +import dil.Unicode; +import common; + +/// A pair of strings. +class IdentValue +{ + string ident; + string value; + this (string ident, string value) + { + this.ident = ident; + this.value = value; + } +} + +/// Parses text of the form: +/// 
+/// ident = value
+/// ident2 = value2
+///          more text
+///
+struct IdentValueParser +{ + char* p; /// Current pointer. + char* textEnd; + + IdentValue[] parse(string text) + { + if (!text.length) + return null; + + p = text.ptr; + textEnd = p + text.length; + + IdentValue[] idvalues; + + string ident, nextIdent; + char* bodyBegin = p, nextBodyBegin; + + // Init. + findNextIdent(ident, bodyBegin); + // Continue. + while (findNextIdent(nextIdent, nextBodyBegin)) + { + idvalues ~= new IdentValue(ident, textBody(bodyBegin, nextIdent.ptr)); + ident = nextIdent; + bodyBegin = nextBodyBegin; + } + // Add last ident value. + idvalues ~= new IdentValue(ident, textBody(bodyBegin, textEnd)); + return idvalues; + } + + /// Returns the text body. Trailing whitespace characters are not included. + char[] textBody(char* begin, char* end) + { + // The body of A is empty, e.g.: + // A = + // B = some text + // ^- begin and end point to B (or to this.textEnd in the 2nd case.) + if (begin is end) + return ""; + // Remove trailing whitespace. + while (isspace(*--end) || *end == '\n') + {} + end++; + return makeString(begin, end); + } + + /// Finds the next "Identifier =". + /// Params: + /// ident = set to Identifier. + /// bodyBegin = set to the beginning of the text body (whitespace skipped.) + /// Returns: true if found. + bool findNextIdent(ref string ident, ref char* bodyBegin) + { + while (p < textEnd) + { + skipWhitespace(); + if (p >= textEnd) + break; + auto idBegin = p; + if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart + { + do // IdChar* + p++; + while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) + auto idEnd = p; + + skipWhitespace(); + if (p < textEnd && *p == '=') + { + p++; + skipWhitespace(); + bodyBegin = p; + ident = makeString(idBegin, idEnd); + return true; + } + } + skipLine(); + } + return false; + } + + void skipWhitespace() + { + while (p < textEnd && (isspace(*p) || *p == '\n')) + p++; + } + + void skipLine() + { + while (p < textEnd && *p != '\n') + p++; + p++; + } +} + +/// Returns a string slice ranging from begin to end. +char[] makeString(char* begin, char* end) +{ + assert(begin && end && begin <= end); + return begin[0 .. end - begin]; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/Funcs.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/Funcs.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,174 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.Funcs; + +const char[3] LS = \u2028; /// Unicode line separator. +const dchar LSd = 0x2028; /// ditto +const char[3] PS = \u2029; /// Unicode paragraph separator. +const dchar PSd = 0x2029; /// ditto +static assert(LS[0] == PS[0] && LS[1] == PS[1]); + +const dchar _Z_ = 26; /// Control+Z. + +/// Returns: true if d is a Unicode line or paragraph separator. +bool isUnicodeNewlineChar(dchar d) +{ + return d == LSd || d == PSd; +} + +/// Returns: true if p points to a line or paragraph separator. +bool isUnicodeNewline(char* p) +{ + return *p == LS[0] && p[1] == LS[1] && (p[2] == LS[2] || p[2] == PS[2]); +} + +/// Returns: true if p points to the start of a Newline. +/// Newline: \n | \r | \r\n | LS | PS +bool isNewline(char* p) +{ + return *p == '\n' || *p == '\r' || isUnicodeNewline(p); +} + +/// Returns: true if c is a Newline character. +bool isNewline(dchar c) +{ + return c == '\n' || c == '\r' || isUnicodeNewlineChar(c); +} + +/// Returns: true if p points to an EOF character. +/// EOF: 0 | _Z_ +bool isEOF(dchar c) +{ + return c == 0 || c == _Z_; +} + +/// Returns: true if p points to the first character of an EndOfLine. +/// EndOfLine: Newline | EOF +bool isEndOfLine(char* p) +{ + return isNewline(p) || isEOF(*p); +} + +/// Scans a Newline and sets p one character past it. +/// Returns: '\n' if found or 0 otherwise. +dchar scanNewline(ref char* p) +{ + switch (*p) + { + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + ++p; + return '\n'; + default: + if (isUnicodeNewline(p)) + { + p += 3; + return '\n'; + } + } + return 0; +} + +/// ASCII character properties table. +static const int ptable[256] = [ + 0, 0, 0, 0, 0, 0, 0, 0, 0,32, 0,32,32, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, +32, 0, 0x2200, 0, 0, 0, 0, 0x2700, 0, 0, 0, 0, 0, 0, 0, 0, + 7, 7, 7, 7, 7, 7, 7, 7, 6, 6, 0, 0, 0, 0, 0, 0x3f00, + 0,12,12,12,12,12,12, 8, 8, 8, 8, 8, 8, 8, 8, 8, + 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 0, 0x5c00, 0, 0,16, + 0, 0x70c, 0x80c,12,12,12, 0xc0c, 8, 8, 8, 8, 8, 8, 8, 0xa08, 8, + 8, 8, 0xd08, 8, 0x908, 8, 0xb08, 8, 8, 8, 8, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 +]; + +/// Enumeration of character property flags. +enum CProperty +{ + Octal = 1, /// 0-7 + Digit = 1<<1, /// 0-9 + Hex = 1<<2, /// 0-9a-fA-F + Alpha = 1<<3, /// a-zA-Z + Underscore = 1<<4, /// _ + Whitespace = 1<<5 /// ' ' \t \v \f +} + +const uint EVMask = 0xFF00; // Bit mask for escape value. + +private alias CProperty CP; +/// Returns: true if c is an octal digit. +int isoctal(char c) { return ptable[c] & CP.Octal; } +/// Returns: true if c is a decimal digit. +int isdigit(char c) { return ptable[c] & CP.Digit; } +/// Returns: true if c is a hexadecimal digit. +int ishexad(char c) { return ptable[c] & CP.Hex; } +/// Returns: true if c is a letter. +int isalpha(char c) { return ptable[c] & CP.Alpha; } +/// Returns: true if c is an alphanumeric. +int isalnum(char c) { return ptable[c] & (CP.Alpha | CP.Digit); } +/// Returns: true if c is the beginning of a D identifier (only ASCII.) +int isidbeg(char c) { return ptable[c] & (CP.Alpha | CP.Underscore); } +/// Returns: true if c is a D identifier character (only ASCII.) +int isident(char c) { return ptable[c] & (CP.Alpha | CP.Underscore | CP.Digit); } +/// Returns: true if c is a whitespace character. +int isspace(char c) { return ptable[c] & CP.Whitespace; } +/// Returns: the escape value for c. +int char2ev(char c) { return ptable[c] >> 8; /*(ptable[c] & EVMask) >> 8;*/ } +/// Returns: true if c is an ASCII character. +int isascii(uint c) { return c < 128; } + +version(gen_ptable) +static this() +{ + alias ptable p; + assert(p.length == 256); + // Initialize character properties table. + for (int i; i < p.length; ++i) + { + p[i] = 0; // Reset + if ('0' <= i && i <= '7') + p[i] |= CP.Octal; + if ('0' <= i && i <= '9') + p[i] |= CP.Digit | CP.Hex; + if ('a' <= i && i <= 'f' || 'A' <= i && i <= 'F') + p[i] |= CP.Hex; + if ('a' <= i && i <= 'z' || 'A' <= i && i <= 'Z') + p[i] |= CP.Alpha; + if (i == '_') + p[i] |= CP.Underscore; + if (i == ' ' || i == '\t' || i == '\v' || i == '\f') + p[i] |= CP.Whitespace; + } + // Store escape sequence values in second byte. + assert(CProperty.max <= ubyte.max, "character property flags and escape value byte overlap."); + p['\''] |= 39 << 8; + p['"'] |= 34 << 8; + p['?'] |= 63 << 8; + p['\\'] |= 92 << 8; + p['a'] |= 7 << 8; + p['b'] |= 8 << 8; + p['f'] |= 12 << 8; + p['n'] |= 10 << 8; + p['r'] |= 13 << 8; + p['t'] |= 9 << 8; + p['v'] |= 11 << 8; + // Print a formatted array literal. + char[] array = "[\n"; + foreach (i, c; ptable) + { + array ~= Format((c>255?" 0x{0:x},":"{0,2},"), c) ~ (((i+1) % 16) ? "":"\n"); + } + array[$-2..$] = "\n]"; + Stdout(array).newline; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/IdTable.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/IdTable.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,162 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.IdTable; + +import dil.lexer.TokensEnum; +import dil.lexer.IdentsGenerator; +import dil.lexer.Keywords; +import common; + +public import dil.lexer.Identifier; +public import dil.lexer.IdentsEnum; + +/// A namespace for the predefined identifiers. +struct Ident +{ + const static + { + mixin(generateIdentMembers()); + } + + static Identifier*[] allIds() + { + return __allIds; + } +} + +/// Global table for hoarding and retrieving identifiers. +struct IdTable +{ +static: + /// A set of common, predefined identifiers for fast lookups. + private Identifier*[string] staticTable; + /// A table that grows with every newly found, unique identifier. + private Identifier*[string] growingTable; + + /// Loads keywords and predefined identifiers into the static table. + static this() + { + foreach (ref k; g_reservedIds) + staticTable[k.str] = &k; + foreach (id; Ident.allIds()) + staticTable[id.str] = id; + staticTable.rehash; + } + + /// Looks up idString in both tables. + Identifier* lookup(string idString) + { + auto id = inStatic(idString); + if (id) + return id; + return inGrowing(idString); + } + + /// Looks up idString in the static table. + Identifier* inStatic(string idString) + { + auto id = idString in staticTable; + return id ? *id : null; + } + + alias Identifier* function(string idString) LookupFunction; + /// Looks up idString in the growing table. + LookupFunction inGrowing = &_inGrowing_unsafe; // Default to unsafe function. + + /// Sets the thread safety mode of the growing table. + void setThreadsafe(bool b) + { + if (b) + inGrowing = &_inGrowing_safe; + else + inGrowing = &_inGrowing_unsafe; + } + + /// Returns true if access to the growing table is thread-safe. + bool isThreadsafe() + { + return inGrowing is &_inGrowing_safe; + } + + /// Looks up idString in the table. + /// + /// Adds idString to the table if not found. + private Identifier* _inGrowing_unsafe(string idString) + out(id) + { assert(id !is null); } + body + { + auto id = idString in growingTable; + if (id) + return *id; + auto newID = Identifier(idString, TOK.Identifier); + growingTable[idString] = newID; + return newID; + } + + /// Looks up idString in the table. + /// + /// Adds idString to the table if not found. + /// Access to the data structure is synchronized. + private Identifier* _inGrowing_safe(string idString) + { + synchronized + return _inGrowing_unsafe(idString); + } + + /+ + Identifier* addIdentifiers(char[][] idStrings) + { + auto ids = new Identifier*[idStrings.length]; + foreach (i, idString; idStrings) + { + Identifier** id = idString in tabulatedIds; + if (!id) + { + auto newID = Identifier(TOK.Identifier, idString); + tabulatedIds[idString] = newID; + id = &newID; + } + ids[i] = *id; + } + } + +/ + + static uint anonCount; /// Counter for anonymous identifiers. + + /// Generates an anonymous identifier. + /// + /// Concatenates prefix with anonCount. + /// The identifier is not inserted into the table. + Identifier* genAnonymousID(string prefix) + { + ++anonCount; + auto x = anonCount; + // Convert count to a string and append it to str. + char[] num; + do + num = cast(char)('0' + (x % 10)) ~ num; + while (x /= 10) + return Identifier(prefix ~ num, TOK.Identifier); + } + + /// Generates an identifier for an anonymous enum. + Identifier* genAnonEnumID() + { + return genAnonymousID("__anonenum"); + } +} + +unittest +{ + // TODO: write benchmark. + // Single table + + // Single table. synchronized + + // Two tables. + + // Two tables. synchronized +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/Identifier.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/Identifier.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,54 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.Identifier; + +import dil.lexer.TokensEnum; +import dil.lexer.IdentsEnum; +import common; + +/// Represents an identifier as defined in the D specs. +/// +///
+///  Identifier := IdStart IdChar*
+///  IdStart := "_" | Letter
+///  IdChar := IdStart | "0"-"9"
+///  Letter := UniAlpha
+///
+/// Unicode alphas are defined in Unicode 5.0.0. +align(1) +struct Identifier +{ + string str; /// The UTF-8 string of the identifier. + TOK kind; /// The token kind. + IDK idKind; /// Only for predefined identifiers. + + static Identifier* opCall(string str, TOK kind) + { + auto id = new Identifier; + id.str = str; + id.kind = kind; + return id; + } + + static Identifier* opCall(string str, TOK kind, IDK idKind) + { + auto id = new Identifier; + id.str = str; + id.kind = kind; + id.idKind = idKind; + return id; + } + + uint toHash() + { + uint hash; + foreach(c; str) { + hash *= 11; + hash += c; + } + return hash; + } +} +// pragma(msg, Identifier.sizeof.stringof); diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/IdentsEnum.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/IdentsEnum.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,18 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.IdentsEnum; + +import dil.lexer.IdentsGenerator; + +version(DDoc) + enum IDK : ushort; /// Enumeration of predefined identifier kinds. +else +mixin( + // Enumerates predefined identifiers. + "enum IDK : ushort {" + "Null," + ~ generateIDMembers ~ + "}" +); diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/IdentsGenerator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/IdentsGenerator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,117 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.IdentsGenerator; + +struct StrPair +{ +const: + char[] str; /// Identifier string in code. + char[] idStr; /// In table. +} + +/// Table of predefined identifiers. +static const StrPair[] identPairs = [ + // Predefined version identifiers: + {"DigitalMars"}, {"X86"}, {"X86_64"}, + /*{"Windows"}, */{"Win32"}, {"Win64"}, + {"linux"}, {"LittleEndian"}, {"BigEndian"}, + {"D_Coverage"}, {"D_InlineAsm_X86"}, {"D_Version2"}, + {"none"}, {"all"}, + // Variadic parameters: + {"_arguments"}, {"_argptr"}, + // scope: + {"exit"}, {"success"}, {"failure"}, + // pragma: + {"msg"}, {"lib"}, {"startaddress"}, + // Linkage: + {"C"}, {"D"}, {"Windows"}, {"Pascal"}, {"System"}, + // Con-/Destructor: + {"__ctor"}, {"__dtor"}, + // new() and delete() methods. + {"__new"}, {"__delete"}, + // Unittest and invariant. + {"__unittest"}, {"__invariant"}, + // Operator methods: + {"opNeg"}, + {"opPos"}, + {"opComp"}, + {"opAddAssign"}, + {"opSubAssign"}, + {"opPostInc"}, + {"opPostDec"}, + {"opCall"}, + {"opCast"}, + {"opIndex"}, + {"opSlice"}, + // ASM identifiers: + {"near"}, {"far"}, {"word"}, {"dword"}, {"qword"}, + {"ptr"}, {"offset"}, {"seg"}, {"__LOCAL_SIZE"}, + {"FS"}, {"ST"}, + {"AL"}, {"AH"}, {"AX"}, {"EAX"}, + {"BL"}, {"BH"}, {"BX"}, {"EBX"}, + {"CL"}, {"CH"}, {"CX"}, {"ECX"}, + {"DL"}, {"DH"}, {"DX"}, {"EDX"}, + {"BP"}, {"EBP"}, {"SP"}, {"ESP"}, + {"DI"}, {"EDI"}, {"SI"}, {"ESI"}, + {"ES"}, {"CS"}, {"SS"}, {"DS"}, {"GS"}, + {"CR0"}, {"CR2"}, {"CR3"}, {"CR4"}, + {"DR0"}, {"DR1"}, {"DR2"}, {"DR3"}, {"DR6"}, {"DR7"}, + {"TR3"}, {"TR4"}, {"TR5"}, {"TR6"}, {"TR7"}, + {"MM0"}, {"MM1"}, {"MM2"}, {"MM3"}, + {"MM4"}, {"MM5"}, {"MM6"}, {"MM7"}, + {"XMM0"}, {"XMM1"}, {"XMM2"}, {"XMM3"}, + {"XMM4"}, {"XMM5"}, {"XMM6"}, {"XMM7"}, +]; + +/++ + CTF for generating the members of the struct Ident. + + The resulting string looks like this: + --- + private struct Ids {static const: + Identifier _str = {"str", TOK.Identifier, IDK.str}; + // more ... + } + Identifier* str = &Ids._str; + // more ... + private Identifier*[] __allIds = [ + str, + // more ... + ] + --- ++/ +char[] generateIdentMembers() +{ + char[] private_members = "private struct Ids {static const:"; + + char[] public_members = ""; + char[] array = "private Identifier*[] __allIds = ["; + foreach (pair; identPairs) + { + // N.B.: Compiler cries for some reason when trying to access pair.idStr. + // Identifier _str = {"str", TOK.Identifier, ID.str}; + private_members ~= "Identifier _"~pair.str~` = {"`~pair.str~`", TOK.Identifier, IDK.`~pair.str~"};\n"; + // Identifier* str = &_str; + public_members ~= "Identifier* "~pair.str~" = &Ids._"~pair.str~";\n"; + array ~= pair.str~","; + } + + private_members ~= "}"; // Close private { + array ~= "];"; + + return private_members ~ public_members ~ array; +} + +/// CTF for generating the members of the enum IDK. +char[] generateIDMembers() +{ + char[] members; + foreach (pair; identPairs) + members ~= pair.str ~ ",\n"; + return members; +} + +// pragma(msg, generateIdentMembers()); +// pragma(msg, generateIDMembers()); diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/Keywords.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/Keywords.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,122 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.Keywords; + +import dil.lexer.Token; +import dil.lexer.Identifier; + +/// Table of reserved identifiers. +static const Identifier[] g_reservedIds = [ + {"abstract", TOK.Abstract}, + {"alias", TOK.Alias}, + {"align", TOK.Align}, + {"asm", TOK.Asm}, + {"assert", TOK.Assert}, + {"auto", TOK.Auto}, + {"body", TOK.Body}, + {"bool", TOK.Bool}, + {"break", TOK.Break}, + {"byte", TOK.Byte}, + {"case", TOK.Case}, + {"cast", TOK.Cast}, + {"catch", TOK.Catch}, + {"cdouble", TOK.Cdouble}, + {"cent", TOK.Cent}, + {"cfloat", TOK.Cfloat}, + {"char", TOK.Char}, + {"class", TOK.Class}, + {"const", TOK.Const}, + {"continue", TOK.Continue}, + {"creal", TOK.Creal}, + {"dchar", TOK.Dchar}, + {"debug", TOK.Debug}, + {"default", TOK.Default}, + {"delegate", TOK.Delegate}, + {"delete", TOK.Delete}, + {"deprecated", TOK.Deprecated}, + {"do", TOK.Do}, + {"double", TOK.Double}, + {"else", TOK.Else}, + {"enum", TOK.Enum}, + {"export", TOK.Export}, + {"extern", TOK.Extern}, + {"false", TOK.False}, + {"final", TOK.Final}, + {"finally", TOK.Finally}, + {"float", TOK.Float}, + {"for", TOK.For}, + {"foreach", TOK.Foreach}, + {"foreach_reverse", TOK.Foreach_reverse}, + {"function", TOK.Function}, + {"goto", TOK.Goto}, + {"idouble", TOK.Idouble}, + {"if", TOK.If}, + {"ifloat", TOK.Ifloat}, + {"import", TOK.Import}, + {"in", TOK.In}, + {"inout", TOK.Inout}, + {"int", TOK.Int}, + {"interface", TOK.Interface}, + {"invariant", TOK.Invariant}, + {"ireal", TOK.Ireal}, + {"is", TOK.Is}, + {"lazy", TOK.Lazy}, + {"long", TOK.Long}, + {"macro", TOK.Macro}, // D2.0 + {"mixin", TOK.Mixin}, + {"module", TOK.Module}, + {"new", TOK.New}, + {"nothrow", TOK.Nothrow}, // D2.0 + {"null", TOK.Null}, + {"out", TOK.Out}, + {"override", TOK.Override}, + {"package", TOK.Package}, + {"pragma", TOK.Pragma}, + {"private", TOK.Private}, + {"protected", TOK.Protected}, + {"public", TOK.Public}, + {"pure", TOK.Pure}, // D2.0 + {"real", TOK.Real}, + {"ref", TOK.Ref}, + {"return", TOK.Return}, + {"scope", TOK.Scope}, + {"short", TOK.Short}, + {"static", TOK.Static}, + {"struct", TOK.Struct}, + {"super", TOK.Super}, + {"switch", TOK.Switch}, + {"synchronized", TOK.Synchronized}, + {"template", TOK.Template}, + {"this", TOK.This}, + {"throw", TOK.Throw}, + {"__traits", TOK.Traits}, // D2.0 + {"true", TOK.True}, + {"try", TOK.Try}, + {"typedef", TOK.Typedef}, + {"typeid", TOK.Typeid}, + {"typeof", TOK.Typeof}, + {"ubyte", TOK.Ubyte}, + {"ucent", TOK.Ucent}, + {"uint", TOK.Uint}, + {"ulong", TOK.Ulong}, + {"union", TOK.Union}, + {"unittest", TOK.Unittest}, + {"ushort", TOK.Ushort}, + {"version", TOK.Version}, + {"void", TOK.Void}, + {"volatile", TOK.Volatile}, + {"wchar", TOK.Wchar}, + {"while", TOK.While}, + {"with", TOK.With}, + // Special tokens: + {"__FILE__", TOK.FILE}, + {"__LINE__", TOK.LINE}, + {"__DATE__", TOK.DATE}, + {"__TIME__", TOK.TIME}, + {"__TIMESTAMP__", TOK.TIMESTAMP}, + {"__VENDOR__", TOK.VENDOR}, + {"__VERSION__", TOK.VERSION}, + {"__EOF__", TOK.EOF}, // D2.0 +]; diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/Lexer.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/Lexer.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,2900 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.Lexer; + +import dil.lexer.Token; +import dil.lexer.Keywords; +import dil.lexer.Identifier; +import dil.lexer.IdTable; +import dil.Information; +import dil.Messages; +import dil.HtmlEntities; +import dil.CompilerInfo; +import dil.Unicode; +import dil.SourceText; +import dil.Time; +import common; + +import tango.stdc.stdlib : strtof, strtod, strtold; +import tango.stdc.errno : errno, ERANGE; + +public import dil.lexer.Funcs; + +/// The Lexer analyzes the characters of a source text and +/// produces a doubly-linked list of tokens. +class Lexer +{ + SourceText srcText; /// The source text. + char* p; /// Points to the current character in the source text. + char* end; /// Points one character past the end of the source text. + + Token* head; /// The head of the doubly linked token list. + Token* tail; /// The tail of the linked list. Set in scan(). + Token* token; /// Points to the current token in the token list. + + // Members used for error messages: + InfoManager infoMan; + LexerError[] errors; + /// Always points to the first character of the current line. + char* lineBegin; +// Token* newline; /// Current newline token. + uint lineNum = 1; /// Current, actual source text line number. + uint lineNum_hline; /// Line number set by #line. + uint inTokenString; /// > 0 if inside q{ } + /// Holds the original file path and the modified one (by #line.) + NewlineData.FilePaths* filePaths; + + /// Construct a Lexer object. + /// Params: + /// srcText = the UTF-8 source code. + /// infoMan = used for collecting error messages. + this(SourceText srcText, InfoManager infoMan = null) + { + this.srcText = srcText; + this.infoMan = infoMan; + + assert(text.length && text[$-1] == 0, "source text has no sentinel character"); + this.p = text.ptr; + this.end = this.p + text.length; + this.lineBegin = this.p; + + this.head = new Token; + this.head.kind = TOK.HEAD; + this.head.start = this.head.end = this.p; + this.token = this.head; + // Initialize this.filePaths. + newFilePath(this.srcText.filePath); + // Add a newline as the first token after the head. + auto newline = new Token; + newline.kind = TOK.Newline; + newline.setWhitespaceFlag(); + newline.start = newline.end = this.p; + newline.newline.filePaths = this.filePaths; + newline.newline.oriLineNum = 1; + newline.newline.setLineNum = 0; + // Link in. + this.token.next = newline; + newline.prev = this.token; + this.token = newline; +// this.newline = newline; + scanShebang(); + } + + /// The destructor deletes the doubly-linked token list. + ~this() + { + auto token = head.next; + while (token !is null) + { + assert(token.kind == TOK.EOF ? token == tail && token.next is null : 1); + delete token.prev; + token = token.next; + } + delete tail; + } + + char[] text() + { + return srcText.data; + } + + /// The "shebang" may optionally appear once at the beginning of a file. + /// Regexp: #![^\EndOfLine]* + void scanShebang() + { + if (*p == '#' && p[1] == '!') + { + auto t = new Token; + t.kind = TOK.Shebang; + t.setWhitespaceFlag(); + t.start = p; + ++p; + while (!isEndOfLine(++p)) + isascii(*p) || decodeUTF8(); + t.end = p; + this.token.next = t; + t.prev = this.token; + } + } + + /// Sets the value of the special token. + void finalizeSpecialToken(ref Token t) + { + assert(t.srcText[0..2] == "__"); + switch (t.kind) + { + case TOK.FILE: + t.str = this.filePaths.setPath; + break; + case TOK.LINE: + t.uint_ = this.errorLineNumber(this.lineNum); + break; + case TOK.DATE, + TOK.TIME, + TOK.TIMESTAMP: + auto time_str = Time.toString(); + switch (t.kind) + { + case TOK.DATE: + time_str = Time.month_day(time_str) ~ ' ' ~ Time.year(time_str); break; + case TOK.TIME: + time_str = Time.time(time_str); break; + case TOK.TIMESTAMP: + break; // time_str is the timestamp. + default: assert(0); + } + time_str ~= '\0'; // Terminate with a zero. + t.str = time_str; + break; + case TOK.VENDOR: + t.str = VENDOR; + break; + case TOK.VERSION: + t.uint_ = VERSION_MAJOR*1000 + VERSION_MINOR; + break; + default: + assert(0); + } + } + + /// Sets a new file path. + void newFilePath(char[] newPath) + { + auto paths = new NewlineData.FilePaths; + paths.oriPath = this.srcText.filePath; + paths.setPath = newPath; + this.filePaths = paths; + } + + private void setLineBegin(char* p) + { + // Check that we can look behind one character. + assert((p-1) >= text.ptr && p < end); + // Check that previous character is a newline. + assert(isNewlineEnd(p - 1)); + this.lineBegin = p; + } + + /// Scans the next token in the source text. + /// + /// Creates a new token if t.next is null and appends it to the list. + private void scanNext(ref Token* t) + { + assert(t !is null); + if (t.next) + { + t = t.next; +// if (t.kind == TOK.Newline) +// this.newline = t; + } + else if (t != this.tail) + { + Token* new_t = new Token; + scan(*new_t); + new_t.prev = t; + t.next = new_t; + t = new_t; + } + } + + /// Advance t one token forward. + void peek(ref Token* t) + { + scanNext(t); + } + + /// Advance to the next token in the source text. + TOK nextToken() + { + scanNext(this.token); + return this.token.kind; + } + + /// Returns true if p points to the last character of a Newline. + bool isNewlineEnd(char* p) + { + if (*p == '\n' || *p == '\r') + return true; + if (*p == LS[2] || *p == PS[2]) + if ((p-2) >= text.ptr) + if (p[-1] == LS[1] && p[-2] == LS[0]) + return true; + return false; + } + + /// The main method which recognizes the characters that make up a token. + /// + /// Complicated tokens are scanned in separate methods. + public void scan(ref Token t) + in + { + assert(text.ptr <= p && p < end); + } + out + { + assert(text.ptr <= t.start && t.start < end, Token.toString(t.kind)); + assert(text.ptr <= t.end && t.end <= end, Token.toString(t.kind)); + } + body + { + // Scan whitespace. + if (isspace(*p)) + { + t.ws = p; + while (isspace(*++p)) + {} + } + + // Scan a token. + uint c = *p; + { + t.start = p; + // Newline. + switch (*p) + { + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + ++p; + ++lineNum; + setLineBegin(p); +// this.newline = &t; + t.kind = TOK.Newline; + t.setWhitespaceFlag(); + t.newline.filePaths = this.filePaths; + t.newline.oriLineNum = lineNum; + t.newline.setLineNum = lineNum_hline; + t.end = p; + return; + default: + if (isUnicodeNewline(p)) + { + ++p; ++p; + goto case '\n'; + } + } + // Identifier or string literal. + if (isidbeg(c)) + { + if (c == 'r' && p[1] == '"' && ++p) + return scanRawStringLiteral(t); + if (c == 'x' && p[1] == '"') + return scanHexStringLiteral(t); + version(D2) + { + if (c == 'q' && p[1] == '"') + return scanDelimitedStringLiteral(t); + if (c == 'q' && p[1] == '{') + return scanTokenStringLiteral(t); + } + // Scan identifier. + Lidentifier: + do + { c = *++p; } + while (isident(c) || !isascii(c) && isUnicodeAlpha()) + + t.end = p; + + auto id = IdTable.lookup(t.srcText); + t.kind = id.kind; + t.ident = id; + + if (t.kind == TOK.Identifier || t.isKeyword) + return; + else if (t.isSpecialToken) + finalizeSpecialToken(t); + else if (t.kind == TOK.EOF) + { + tail = &t; + assert(t.srcText == "__EOF__"); + } + else + assert(0, "unexpected token type: " ~ Token.toString(t.kind)); + return; + } + + if (isdigit(c)) + return scanNumber(t); + + if (c == '/') + { + c = *++p; + switch(c) + { + case '=': + ++p; + t.kind = TOK.DivAssign; + t.end = p; + return; + case '+': + return scanNestedComment(t); + case '*': + return scanBlockComment(t); + case '/': + while (!isEndOfLine(++p)) + isascii(*p) || decodeUTF8(); + t.kind = TOK.Comment; + t.setWhitespaceFlag(); + t.end = p; + return; + default: + t.kind = TOK.Div; + t.end = p; + return; + } + } + + switch (c) + { + case '\'': + return scanCharacterLiteral(t); + case '`': + return scanRawStringLiteral(t); + case '"': + return scanNormalStringLiteral(t); + case '\\': + char[] buffer; + do + { + bool isBinary; + c = scanEscapeSequence(isBinary); + if (isascii(c) || isBinary) + buffer ~= c; + else + encodeUTF8(buffer, c); + } while (*p == '\\') + buffer ~= 0; + t.kind = TOK.String; + t.str = buffer; + t.end = p; + return; + case '>': /* > >= >> >>= >>> >>>= */ + c = *++p; + switch (c) + { + case '=': + t.kind = TOK.GreaterEqual; + goto Lcommon; + case '>': + if (p[1] == '>') + { + ++p; + if (p[1] == '=') + { ++p; + t.kind = TOK.URShiftAssign; + } + else + t.kind = TOK.URShift; + } + else if (p[1] == '=') + { + ++p; + t.kind = TOK.RShiftAssign; + } + else + t.kind = TOK.RShift; + goto Lcommon; + default: + t.kind = TOK.Greater; + goto Lcommon2; + } + assert(0); + case '<': /* < <= <> <>= << <<= */ + c = *++p; + switch (c) + { + case '=': + t.kind = TOK.LessEqual; + goto Lcommon; + case '<': + if (p[1] == '=') { + ++p; + t.kind = TOK.LShiftAssign; + } + else + t.kind = TOK.LShift; + goto Lcommon; + case '>': + if (p[1] == '=') { + ++p; + t.kind = TOK.LorEorG; + } + else + t.kind = TOK.LorG; + goto Lcommon; + default: + t.kind = TOK.Less; + goto Lcommon2; + } + assert(0); + case '!': /* ! !< !> !<= !>= !<> !<>= */ + c = *++p; + switch (c) + { + case '<': + c = *++p; + if (c == '>') + { + if (p[1] == '=') { + ++p; + t.kind = TOK.Unordered; + } + else + t.kind = TOK.UorE; + } + else if (c == '=') + { + t.kind = TOK.UorG; + } + else { + t.kind = TOK.UorGorE; + goto Lcommon2; + } + goto Lcommon; + case '>': + if (p[1] == '=') + { + ++p; + t.kind = TOK.UorL; + } + else + t.kind = TOK.UorLorE; + goto Lcommon; + case '=': + t.kind = TOK.NotEqual; + goto Lcommon; + default: + t.kind = TOK.Not; + goto Lcommon2; + } + assert(0); + case '.': /* . .[0-9] .. ... */ + if (p[1] == '.') + { + ++p; + if (p[1] == '.') { + ++p; + t.kind = TOK.Ellipses; + } + else + t.kind = TOK.Slice; + } + else if (isdigit(p[1])) + { + return scanReal(t); + } + else + t.kind = TOK.Dot; + goto Lcommon; + case '|': /* | || |= */ + c = *++p; + if (c == '=') + t.kind = TOK.OrAssign; + else if (c == '|') + t.kind = TOK.OrLogical; + else { + t.kind = TOK.OrBinary; + goto Lcommon2; + } + goto Lcommon; + case '&': /* & && &= */ + c = *++p; + if (c == '=') + t.kind = TOK.AndAssign; + else if (c == '&') + t.kind = TOK.AndLogical; + else { + t.kind = TOK.AndBinary; + goto Lcommon2; + } + goto Lcommon; + case '+': /* + ++ += */ + c = *++p; + if (c == '=') + t.kind = TOK.PlusAssign; + else if (c == '+') + t.kind = TOK.PlusPlus; + else { + t.kind = TOK.Plus; + goto Lcommon2; + } + goto Lcommon; + case '-': /* - -- -= */ + c = *++p; + if (c == '=') + t.kind = TOK.MinusAssign; + else if (c == '-') + t.kind = TOK.MinusMinus; + else { + t.kind = TOK.Minus; + goto Lcommon2; + } + goto Lcommon; + case '=': /* = == */ + if (p[1] == '=') { + ++p; + t.kind = TOK.Equal; + } + else + t.kind = TOK.Assign; + goto Lcommon; + case '~': /* ~ ~= */ + if (p[1] == '=') { + ++p; + t.kind = TOK.CatAssign; + } + else + t.kind = TOK.Tilde; + goto Lcommon; + case '*': /* * *= */ + if (p[1] == '=') { + ++p; + t.kind = TOK.MulAssign; + } + else + t.kind = TOK.Mul; + goto Lcommon; + case '^': /* ^ ^= */ + if (p[1] == '=') { + ++p; + t.kind = TOK.XorAssign; + } + else + t.kind = TOK.Xor; + goto Lcommon; + case '%': /* % %= */ + if (p[1] == '=') { + ++p; + t.kind = TOK.ModAssign; + } + else + t.kind = TOK.Mod; + goto Lcommon; + // Single character tokens: + case '(': + t.kind = TOK.LParen; + goto Lcommon; + case ')': + t.kind = TOK.RParen; + goto Lcommon; + case '[': + t.kind = TOK.LBracket; + goto Lcommon; + case ']': + t.kind = TOK.RBracket; + goto Lcommon; + case '{': + t.kind = TOK.LBrace; + goto Lcommon; + case '}': + t.kind = TOK.RBrace; + goto Lcommon; + case ':': + t.kind = TOK.Colon; + goto Lcommon; + case ';': + t.kind = TOK.Semicolon; + goto Lcommon; + case '?': + t.kind = TOK.Question; + goto Lcommon; + case ',': + t.kind = TOK.Comma; + goto Lcommon; + case '$': + t.kind = TOK.Dollar; + Lcommon: + ++p; + Lcommon2: + t.end = p; + return; + case '#': + return scanSpecialTokenSequence(t); + default: + } + + // Check for EOF + if (isEOF(c)) + { + assert(isEOF(*p), ""~*p); + t.kind = TOK.EOF; + t.end = p; + tail = &t; + assert(t.start == t.end); + return; + } + + if (!isascii(c)) + { + c = decodeUTF8(); + if (isUniAlpha(c)) + goto Lidentifier; + } + + error(t.start, MID.IllegalCharacter, cast(dchar)c); + + ++p; + t.kind = TOK.Illegal; + t.setWhitespaceFlag(); + t.dchar_ = c; + t.end = p; + return; + } + } + + /// Converts a string literal to an integer. + template toUint(char[] T) + { + static assert(0 < T.length && T.length <= 4); + static if (T.length == 1) + const uint toUint = T[0]; + else + const uint toUint = (T[0] << ((T.length-1)*8)) | toUint!(T[1..$]); + } + static assert(toUint!("\xAA\xBB\xCC\xDD") == 0xAABBCCDD); + + /// Constructs case statements. E.g.: + /// --- + //// // case_!("<", "Less", "Lcommon") -> + /// case 60u: + /// t.kind = TOK.Less; + /// goto Lcommon; + /// --- + /// Note:Can't use this yet due to a $(DMDBUG 1534, bug) in DMD. + template case_(char[] str, char[] kind, char[] label) + { + const char[] case_ = + `case `~toUint!(str).stringof~`:` + `t.kind = TOK.`~kind~`;` + `goto `~label~`;`; + } + //pragma(msg, case_!("<", "Less", "Lcommon")); + + template case_L4(char[] str, TOK kind) + { + const char[] case_L4 = case_!(str, kind, "Lcommon_4"); + } + + template case_L3(char[] str, TOK kind) + { + const char[] case_L3 = case_!(str, kind, "Lcommon_3"); + } + + template case_L2(char[] str, TOK kind) + { + const char[] case_L2 = case_!(str, kind, "Lcommon_2"); + } + + template case_L1(char[] str, TOK kind) + { + const char[] case_L3 = case_!(str, kind, "Lcommon"); + } + + /// An alternative scan method. + /// Profiling shows it's a bit slower. + public void scan_(ref Token t) + in + { + assert(text.ptr <= p && p < end); + } + out + { + assert(text.ptr <= t.start && t.start < end, Token.toString(t.kind)); + assert(text.ptr <= t.end && t.end <= end, Token.toString(t.kind)); + } + body + { + // Scan whitespace. + if (isspace(*p)) + { + t.ws = p; + while (isspace(*++p)) + {} + } + + // Scan a token. + t.start = p; + // Newline. + switch (*p) + { + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + ++p; + ++lineNum; + setLineBegin(p); +// this.newline = &t; + t.kind = TOK.Newline; + t.setWhitespaceFlag(); + t.newline.filePaths = this.filePaths; + t.newline.oriLineNum = lineNum; + t.newline.setLineNum = lineNum_hline; + t.end = p; + return; + default: + if (isUnicodeNewline(p)) + { + ++p; ++p; + goto case '\n'; + } + } + + uint c = *p; + assert(end - p != 0); + switch (end - p) + { + case 1: + goto L1character; + case 2: + c <<= 8; c |= p[1]; + goto L2characters; + case 3: + c <<= 8; c |= p[1]; c <<= 8; c |= p[2]; + goto L3characters; + default: + version(BigEndian) + c = *cast(uint*)p; + else + { + c <<= 8; c |= p[1]; c <<= 8; c |= p[2]; c <<= 8; c |= p[3]; + /+ + c = *cast(uint*)p; + asm + { + mov EDX, c; + bswap EDX; + mov c, EDX; + } + +/ + } + } + + // 4 character tokens. + switch (c) + { + case toUint!(">>>="): + t.kind = TOK.RShiftAssign; + goto Lcommon_4; + case toUint!("!<>="): + t.kind = TOK.Unordered; + Lcommon_4: + p += 4; + t.end = p; + return; + default: + } + + c >>>= 8; + L3characters: + assert(p == t.start); + // 3 character tokens. + switch (c) + { + case toUint!(">>="): + t.kind = TOK.RShiftAssign; + goto Lcommon_3; + case toUint!(">>>"): + t.kind = TOK.URShift; + goto Lcommon_3; + case toUint!("<>="): + t.kind = TOK.LorEorG; + goto Lcommon_3; + case toUint!("<<="): + t.kind = TOK.LShiftAssign; + goto Lcommon_3; + case toUint!("!<="): + t.kind = TOK.UorG; + goto Lcommon_3; + case toUint!("!>="): + t.kind = TOK.UorL; + goto Lcommon_3; + case toUint!("!<>"): + t.kind = TOK.UorE; + goto Lcommon_3; + case toUint!("..."): + t.kind = TOK.Ellipses; + Lcommon_3: + p += 3; + t.end = p; + return; + default: + } + + c >>>= 8; + L2characters: + assert(p == t.start); + // 2 character tokens. + switch (c) + { + case toUint!("/+"): + ++p; // Skip / + return scanNestedComment(t); + case toUint!("/*"): + ++p; // Skip / + return scanBlockComment(t); + case toUint!("//"): + ++p; // Skip / + assert(*p == '/'); + while (!isEndOfLine(++p)) + isascii(*p) || decodeUTF8(); + t.kind = TOK.Comment; + t.setWhitespaceFlag(); + t.end = p; + return; + case toUint!(">="): + t.kind = TOK.GreaterEqual; + goto Lcommon_2; + case toUint!(">>"): + t.kind = TOK.RShift; + goto Lcommon_2; + case toUint!("<<"): + t.kind = TOK.LShift; + goto Lcommon_2; + case toUint!("<="): + t.kind = TOK.LessEqual; + goto Lcommon_2; + case toUint!("<>"): + t.kind = TOK.LorG; + goto Lcommon_2; + case toUint!("!<"): + t.kind = TOK.UorGorE; + goto Lcommon_2; + case toUint!("!>"): + t.kind = TOK.UorLorE; + goto Lcommon_2; + case toUint!("!="): + t.kind = TOK.NotEqual; + goto Lcommon_2; + case toUint!(".."): + t.kind = TOK.Slice; + goto Lcommon_2; + case toUint!("&&"): + t.kind = TOK.AndLogical; + goto Lcommon_2; + case toUint!("&="): + t.kind = TOK.AndAssign; + goto Lcommon_2; + case toUint!("||"): + t.kind = TOK.OrLogical; + goto Lcommon_2; + case toUint!("|="): + t.kind = TOK.OrAssign; + goto Lcommon_2; + case toUint!("++"): + t.kind = TOK.PlusPlus; + goto Lcommon_2; + case toUint!("+="): + t.kind = TOK.PlusAssign; + goto Lcommon_2; + case toUint!("--"): + t.kind = TOK.MinusMinus; + goto Lcommon_2; + case toUint!("-="): + t.kind = TOK.MinusAssign; + goto Lcommon_2; + case toUint!("=="): + t.kind = TOK.Equal; + goto Lcommon_2; + case toUint!("~="): + t.kind = TOK.CatAssign; + goto Lcommon_2; + case toUint!("*="): + t.kind = TOK.MulAssign; + goto Lcommon_2; + case toUint!("/="): + t.kind = TOK.DivAssign; + goto Lcommon_2; + case toUint!("^="): + t.kind = TOK.XorAssign; + goto Lcommon_2; + case toUint!("%="): + t.kind = TOK.ModAssign; + Lcommon_2: + p += 2; + t.end = p; + return; + default: + } + + c >>>= 8; + L1character: + assert(p == t.start); + assert(*p == c, Format("p={0},c={1}", *p, cast(dchar)c)); + // 1 character tokens. + // TODO: consider storing the token type in ptable. + switch (c) + { + case '\'': + return scanCharacterLiteral(t); + case '`': + return scanRawStringLiteral(t); + case '"': + return scanNormalStringLiteral(t); + case '\\': + char[] buffer; + do + { + bool isBinary; + c = scanEscapeSequence(isBinary); + if (isascii(c) || isBinary) + buffer ~= c; + else + encodeUTF8(buffer, c); + } while (*p == '\\') + buffer ~= 0; + t.kind = TOK.String; + t.str = buffer; + t.end = p; + return; + case '<': + t.kind = TOK.Greater; + goto Lcommon; + case '>': + t.kind = TOK.Less; + goto Lcommon; + case '^': + t.kind = TOK.Xor; + goto Lcommon; + case '!': + t.kind = TOK.Not; + goto Lcommon; + case '.': + if (isdigit(p[1])) + return scanReal(t); + t.kind = TOK.Dot; + goto Lcommon; + case '&': + t.kind = TOK.AndBinary; + goto Lcommon; + case '|': + t.kind = TOK.OrBinary; + goto Lcommon; + case '+': + t.kind = TOK.Plus; + goto Lcommon; + case '-': + t.kind = TOK.Minus; + goto Lcommon; + case '=': + t.kind = TOK.Assign; + goto Lcommon; + case '~': + t.kind = TOK.Tilde; + goto Lcommon; + case '*': + t.kind = TOK.Mul; + goto Lcommon; + case '/': + t.kind = TOK.Div; + goto Lcommon; + case '%': + t.kind = TOK.Mod; + goto Lcommon; + case '(': + t.kind = TOK.LParen; + goto Lcommon; + case ')': + t.kind = TOK.RParen; + goto Lcommon; + case '[': + t.kind = TOK.LBracket; + goto Lcommon; + case ']': + t.kind = TOK.RBracket; + goto Lcommon; + case '{': + t.kind = TOK.LBrace; + goto Lcommon; + case '}': + t.kind = TOK.RBrace; + goto Lcommon; + case ':': + t.kind = TOK.Colon; + goto Lcommon; + case ';': + t.kind = TOK.Semicolon; + goto Lcommon; + case '?': + t.kind = TOK.Question; + goto Lcommon; + case ',': + t.kind = TOK.Comma; + goto Lcommon; + case '$': + t.kind = TOK.Dollar; + Lcommon: + ++p; + t.end = p; + return; + case '#': + return scanSpecialTokenSequence(t); + default: + } + + assert(p == t.start); + assert(*p == c); + + // TODO: consider moving isidbeg() and isdigit() up. + if (isidbeg(c)) + { + if (c == 'r' && p[1] == '"' && ++p) + return scanRawStringLiteral(t); + if (c == 'x' && p[1] == '"') + return scanHexStringLiteral(t); + version(D2) + { + if (c == 'q' && p[1] == '"') + return scanDelimitedStringLiteral(t); + if (c == 'q' && p[1] == '{') + return scanTokenStringLiteral(t); + } + // Scan identifier. + Lidentifier: + do + { c = *++p; } + while (isident(c) || !isascii(c) && isUnicodeAlpha()) + + t.end = p; + + auto id = IdTable.lookup(t.srcText); + t.kind = id.kind; + t.ident = id; + + if (t.kind == TOK.Identifier || t.isKeyword) + return; + else if (t.isSpecialToken) + finalizeSpecialToken(t); + else if (t.kind == TOK.EOF) + { + tail = &t; + assert(t.srcText == "__EOF__"); + } + else + assert(0, "unexpected token type: " ~ Token.toString(t.kind)); + return; + } + + if (isdigit(c)) + return scanNumber(t); + + // Check for EOF + if (isEOF(c)) + { + assert(isEOF(*p), *p~""); + t.kind = TOK.EOF; + t.end = p; + tail = &t; + assert(t.start == t.end); + return; + } + + if (!isascii(c)) + { + c = decodeUTF8(); + if (isUniAlpha(c)) + goto Lidentifier; + } + + error(t.start, MID.IllegalCharacter, cast(dchar)c); + + ++p; + t.kind = TOK.Illegal; + t.setWhitespaceFlag(); + t.dchar_ = c; + t.end = p; + return; + } + + /// Scans a block comment. + /// + /// BlockComment := "/*" AnyChar* "*/" + void scanBlockComment(ref Token t) + { + assert(p[-1] == '/' && *p == '*'); + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + Loop: + while (1) + { + switch (*++p) + { + case '*': + if (p[1] != '/') + continue; + p += 2; + break Loop; + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + ++lineNum; + setLineBegin(p+1); + break; + default: + if (!isascii(*p)) + { + if (isUnicodeNewlineChar(decodeUTF8())) + goto case '\n'; + } + else if (isEOF(*p)) + { + error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedBlockComment); + break Loop; + } + } + } + t.kind = TOK.Comment; + t.setWhitespaceFlag(); + t.end = p; + return; + } + + /// Scans a nested comment. + /// + /// NestedComment := "/+" (AnyChar* | NestedComment) "+/" + void scanNestedComment(ref Token t) + { + assert(p[-1] == '/' && *p == '+'); + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + uint level = 1; + Loop: + while (1) + { + switch (*++p) + { + case '/': + if (p[1] == '+') + ++p, ++level; + continue; + case '+': + if (p[1] != '/') + continue; + ++p; + if (--level != 0) + continue; + ++p; + break Loop; + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + ++lineNum; + setLineBegin(p+1); + continue; + default: + if (!isascii(*p)) + { + if (isUnicodeNewlineChar(decodeUTF8())) + goto case '\n'; + } + else if (isEOF(*p)) + { + error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedNestedComment); + break Loop; + } + } + } + t.kind = TOK.Comment; + t.setWhitespaceFlag(); + t.end = p; + return; + } + + /// Scans the postfix character of a string literal. + /// + /// PostfixChar := "c" | "w" | "d" + char scanPostfix() + { + assert(p[-1] == '"' || p[-1] == '`' || + { version(D2) return p[-1] == '}'; + else return 0; }() + ); + switch (*p) + { + case 'c': + case 'w': + case 'd': + return *p++; + default: + return 0; + } + assert(0); + } + + /// Scans a normal string literal. + /// + /// NormalStringLiteral := "\"" Char* "\"" + void scanNormalStringLiteral(ref Token t) + { + assert(*p == '"'); + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + t.kind = TOK.String; + char[] buffer; + uint c; + while (1) + { + c = *++p; + switch (c) + { + case '"': + ++p; + t.pf = scanPostfix(); + Lreturn: + t.str = buffer ~ '\0'; + t.end = p; + return; + case '\\': + bool isBinary; + c = scanEscapeSequence(isBinary); + --p; + if (isascii(c) || isBinary) + buffer ~= c; + else + encodeUTF8(buffer, c); + continue; + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + c = '\n'; // Convert Newline to \n. + ++lineNum; + setLineBegin(p+1); + break; + case 0, _Z_: + error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedString); + goto Lreturn; + default: + if (!isascii(c)) + { + c = decodeUTF8(); + if (isUnicodeNewlineChar(c)) + goto case '\n'; + encodeUTF8(buffer, c); + continue; + } + } + assert(isascii(c)); + buffer ~= c; + } + assert(0); + } + + /// Scans a character literal. + /// + /// CharLiteral := "'" Char "'" + void scanCharacterLiteral(ref Token t) + { + assert(*p == '\''); + ++p; + t.kind = TOK.CharLiteral; + switch (*p) + { + case '\\': + bool notused; + t.dchar_ = scanEscapeSequence(notused); + break; + case '\'': + error(t.start, MID.EmptyCharacterLiteral); + break; + default: + if (isEndOfLine(p)) + break; + uint c = *p; + if (!isascii(c)) + c = decodeUTF8(); + t.dchar_ = c; + ++p; + } + + if (*p == '\'') + ++p; + else + error(t.start, MID.UnterminatedCharacterLiteral); + t.end = p; + } + + /// Scans a raw string literal. + /// + /// RawStringLiteral := "r\"" AnyChar* "\"" | "`" AnyChar* "`" + void scanRawStringLiteral(ref Token t) + { + assert(*p == '`' || *p == '"' && p[-1] == 'r'); + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + t.kind = TOK.String; + uint delim = *p; + char[] buffer; + uint c; + while (1) + { + c = *++p; + switch (c) + { + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + c = '\n'; // Convert Newline to '\n'. + ++lineNum; + setLineBegin(p+1); + break; + case '`': + case '"': + if (c == delim) + { + ++p; + t.pf = scanPostfix(); + Lreturn: + t.str = buffer ~ '\0'; + t.end = p; + return; + } + break; + case 0, _Z_: + error(tokenLineNum, tokenLineBegin, t.start, + delim == 'r' ? MID.UnterminatedRawString : MID.UnterminatedBackQuoteString); + goto Lreturn; + default: + if (!isascii(c)) + { + c = decodeUTF8(); + if (isUnicodeNewlineChar(c)) + goto case '\n'; + encodeUTF8(buffer, c); + continue; + } + } + assert(isascii(c)); + buffer ~= c; + } + assert(0); + } + + /// Scans a hexadecimal string literal. + /// + /// HexStringLiteral := "x\"" (HexChar HexChar)* "\"" + void scanHexStringLiteral(ref Token t) + { + assert(p[0] == 'x' && p[1] == '"'); + t.kind = TOK.String; + + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + + uint c; + ubyte[] buffer; + ubyte h; // hex number + uint n; // number of hex digits + + ++p; + assert(*p == '"'); + while (1) + { + c = *++p; + switch (c) + { + case '"': + if (n & 1) + error(tokenLineNum, tokenLineBegin, t.start, MID.OddNumberOfDigitsInHexString); + ++p; + t.pf = scanPostfix(); + Lreturn: + t.str = cast(string) (buffer ~= 0); + t.end = p; + return; + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + ++lineNum; + setLineBegin(p+1); + continue; + default: + if (ishexad(c)) + { + if (c <= '9') + c -= '0'; + else if (c <= 'F') + c -= 'A' - 10; + else + c -= 'a' - 10; + + if (n & 1) + { + h <<= 4; + h |= c; + buffer ~= h; + } + else + h = cast(ubyte)c; + ++n; + continue; + } + else if (isspace(c)) + continue; // Skip spaces. + else if (isEOF(c)) + { + error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedHexString); + t.pf = 0; + goto Lreturn; + } + else + { + auto errorAt = p; + if (!isascii(c)) + { + c = decodeUTF8(); + if (isUnicodeNewlineChar(c)) + goto case '\n'; + } + error(errorAt, MID.NonHexCharInHexString, cast(dchar)c); + } + } + } + assert(0); + } + +version(DDoc) +{ + /// Scans a delimited string literal. + void scanDelimitedStringLiteral(ref Token t); + /// Scans a token string literal. + /// + /// TokenStringLiteral := "q{" Token* "}" + void scanTokenStringLiteral(ref Token t); +} +else +version(D2) +{ + void scanDelimitedStringLiteral(ref Token t) + { + assert(p[0] == 'q' && p[1] == '"'); + t.kind = TOK.String; + + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + + char[] buffer; + dchar opening_delim = 0, // 0 if no nested delimiter or '[', '(', '<', '{' + closing_delim; // Will be ']', ')', '>', '}, + // the first character of an identifier or + // any other Unicode/ASCII character. + char[] str_delim; // Identifier delimiter. + uint level = 1; // Counter for nestable delimiters. + + ++p; ++p; // Skip q" + uint c = *p; + switch (c) + { + case '(': + opening_delim = c; + closing_delim = ')'; // c + 1 + break; + case '[', '<', '{': + opening_delim = c; + closing_delim = c + 2; // Get to closing counterpart. Feature of ASCII table. + break; + default: + dchar scanNewline() + { + switch (*p) + { + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + ++p; + ++lineNum; + setLineBegin(p); + return '\n'; + default: + if (isUnicodeNewline(p)) + { + ++p; ++p; + goto case '\n'; + } + } + return 0; + } + // Skip leading newlines: + while (scanNewline() != 0) + {} + assert(!isNewline(p)); + + char* begin = p; + c = *p; + closing_delim = c; + // TODO: Check for non-printable characters? + if (!isascii(c)) + { + closing_delim = decodeUTF8(); + if (!isUniAlpha(closing_delim)) + break; // Not an identifier. + } + else if (!isidbeg(c)) + break; // Not an identifier. + + // Parse Identifier + EndOfLine + do + { c = *++p; } + while (isident(c) || !isascii(c) && isUnicodeAlpha()) + // Store identifier + str_delim = begin[0..p-begin]; + // Scan newline + if (scanNewline() == '\n') + --p; // Go back one because of "c = *++p;" in main loop. + else + { + // TODO: error(p, MID.ExpectedNewlineAfterIdentDelim); + } + } + + bool checkStringDelim(char* p) + { + assert(str_delim.length != 0); + if (buffer[$-1] == '\n' && // Last character copied to buffer must be '\n'. + end-p >= str_delim.length && // Check remaining length. + p[0..str_delim.length] == str_delim) // Compare. + return true; + return false; + } + + while (1) + { + c = *++p; + switch (c) + { + case '\r': + if (p[1] == '\n') + ++p; + case '\n': + assert(isNewlineEnd(p)); + c = '\n'; // Convert Newline to '\n'. + ++lineNum; + setLineBegin(p+1); + break; + case 0, _Z_: + // TODO: error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedDelimitedString); + goto Lreturn3; + default: + if (!isascii(c)) + { + auto begin = p; + c = decodeUTF8(); + if (isUnicodeNewlineChar(c)) + goto case '\n'; + if (c == closing_delim) + { + if (str_delim.length) + { + if (checkStringDelim(begin)) + { + p = begin + str_delim.length; + goto Lreturn2; + } + } + else + { + assert(level == 1); + --level; + goto Lreturn; + } + } + encodeUTF8(buffer, c); + continue; + } + else + { + if (c == opening_delim) + ++level; + else if (c == closing_delim) + { + if (str_delim.length) + { + if (checkStringDelim(p)) + { + p += str_delim.length; + goto Lreturn2; + } + } + else if (--level == 0) + goto Lreturn; + } + } + } + assert(isascii(c)); + buffer ~= c; + } + Lreturn: // Character delimiter. + assert(c == closing_delim); + assert(level == 0); + ++p; // Skip closing delimiter. + Lreturn2: // String delimiter. + if (*p == '"') + ++p; + else + { + // TODO: error(p, MID.ExpectedDblQuoteAfterDelim, str_delim.length ? str_delim : closing_delim~""); + } + + t.pf = scanPostfix(); + Lreturn3: // Error. + t.str = buffer ~ '\0'; + t.end = p; + } + + void scanTokenStringLiteral(ref Token t) + { + assert(p[0] == 'q' && p[1] == '{'); + t.kind = TOK.String; + + auto tokenLineNum = lineNum; + auto tokenLineBegin = lineBegin; + + // A guard against changes to particular members: + // this.lineNum_hline and this.errorPath + ++inTokenString; + + uint lineNum = this.lineNum; + uint level = 1; + + ++p; ++p; // Skip q{ + + auto prev_t = &t; + Token* token; + while (1) + { + token = new Token; + scan(*token); + // Save the tokens in a doubly linked list. + // Could be useful for various tools. + token.prev = prev_t; + prev_t.next = token; + prev_t = token; + switch (token.kind) + { + case TOK.LBrace: + ++level; + continue; + case TOK.RBrace: + if (--level == 0) + { + t.tok_str = t.next; + t.next = null; + break; + } + continue; + case TOK.EOF: + // TODO: error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedTokenString); + t.tok_str = t.next; + t.next = token; + break; + default: + continue; + } + break; // Exit loop. + } + + assert(token.kind == TOK.RBrace || token.kind == TOK.EOF); + assert(token.kind == TOK.RBrace && t.next is null || + token.kind == TOK.EOF && t.next !is null); + + char[] buffer; + // token points to } or EOF + if (token.kind == TOK.EOF) + { + t.end = token.start; + buffer = t.srcText[2..$].dup ~ '\0'; + } + else + { + // Assign to buffer before scanPostfix(). + t.end = p; + buffer = t.srcText[2..$-1].dup ~ '\0'; + t.pf = scanPostfix(); + t.end = p; // Assign again because of postfix. + } + // Convert newlines to '\n'. + if (lineNum != this.lineNum) + { + assert(buffer[$-1] == '\0'); + uint i, j; + for (; i < buffer.length; ++i) + switch (buffer[i]) + { + case '\r': + if (buffer[i+1] == '\n') + ++i; + case '\n': + assert(isNewlineEnd(buffer.ptr + i)); + buffer[j++] = '\n'; // Convert Newline to '\n'. + break; + default: + if (isUnicodeNewline(buffer.ptr + i)) + { + ++i; ++i; + goto case '\n'; + } + buffer[j++] = buffer[i]; // Copy. + } + buffer.length = j; // Adjust length. + } + assert(buffer[$-1] == '\0'); + t.str = buffer; + + --inTokenString; + } +} // version(D2) + + /// Scans an escape sequence. + /// + /// EscapeSequence := "\" (Octal{1,3} | ("x" Hex{2}) | + /// ("u" Hex{4}) | ("U" Hex{8}) | + /// "'" | "\"" | "\\" | "?" | "a" | + /// "b" | "f" | "n" | "r" | "t" | "v") + /// Params: + /// isBinary = set to true for octal and hexadecimal escapes. + /// Returns: the escape value. + dchar scanEscapeSequence(ref bool isBinary) + out(result) + { assert(isValidChar(result)); } + body + { + assert(*p == '\\'); + + auto sequenceStart = p; // Used for error reporting. + + ++p; + uint c = char2ev(*p); + if (c) + { + ++p; + return c; + } + + uint digits = 2; + + switch (*p) + { + case 'x': + isBinary = true; + case_Unicode: + assert(c == 0); + assert(digits == 2 || digits == 4 || digits == 8); + while (1) + { + ++p; + if (ishexad(*p)) + { + c *= 16; + if (*p <= '9') + c += *p - '0'; + else if (*p <= 'F') + c += *p - 'A' + 10; + else + c += *p - 'a' + 10; + + if (--digits == 0) + { + ++p; + if (isValidChar(c)) + return c; // Return valid escape value. + + error(sequenceStart, MID.InvalidUnicodeEscapeSequence, + sequenceStart[0..p-sequenceStart]); + break; + } + continue; + } + + error(sequenceStart, MID.InsufficientHexDigits, + sequenceStart[0..p-sequenceStart]); + break; + } + break; + case 'u': + digits = 4; + goto case_Unicode; + case 'U': + digits = 8; + goto case_Unicode; + default: + if (isoctal(*p)) + { + isBinary = true; + assert(c == 0); + c += *p - '0'; + ++p; + if (!isoctal(*p)) + return c; + c *= 8; + c += *p - '0'; + ++p; + if (!isoctal(*p)) + return c; + c *= 8; + c += *p - '0'; + ++p; + if (c > 0xFF) + error(sequenceStart, MSG.InvalidOctalEscapeSequence, + sequenceStart[0..p-sequenceStart]); + return c; // Return valid escape value. + } + else if(*p == '&') + { + if (isalpha(*++p)) + { + auto begin = p; + while (isalnum(*++p)) + {} + + if (*p == ';') + { + // Pass entity excluding '&' and ';'. + c = entity2Unicode(begin[0..p - begin]); + ++p; // Skip ; + if (c != 0xFFFF) + return c; // Return valid escape value. + else + error(sequenceStart, MID.UndefinedHTMLEntity, sequenceStart[0 .. p - sequenceStart]); + } + else + error(sequenceStart, MID.UnterminatedHTMLEntity, sequenceStart[0 .. p - sequenceStart]); + } + else + error(sequenceStart, MID.InvalidBeginHTMLEntity); + } + else if (isEndOfLine(p)) + error(sequenceStart, MID.UndefinedEscapeSequence, + isEOF(*p) ? `\EOF` : `\NewLine`); + else + { + char[] str = `\`; + if (isascii(c)) + str ~= *p; + else + encodeUTF8(str, decodeUTF8()); + ++p; + // TODO: check for unprintable character? + error(sequenceStart, MID.UndefinedEscapeSequence, str); + } + } + return REPLACEMENT_CHAR; // Error: return replacement character. + } + + /// Scans a number literal. + /// + /// $(PRE + /// IntegerLiteral := (Dec|Hex|Bin|Oct)Suffix? + /// Dec := (0|[1-9][0-9_]*) + /// Hex := 0[xX][_]*[0-9a-zA-Z][0-9a-zA-Z_]* + /// Bin := 0[bB][_]*[01][01_]* + /// Oct := 0[0-7_]* + /// Suffix := (L[uU]?|[uU]L?) + /// ) + /// Invalid: "0b_", "0x_", "._" etc. + void scanNumber(ref Token t) + { + ulong ulong_; + bool overflow; + bool isDecimal; + size_t digits; + + if (*p != '0') + goto LscanInteger; + ++p; // skip zero + // check for xX bB ... + switch (*p) + { + case 'x','X': + goto LscanHex; + case 'b','B': + goto LscanBinary; + case 'L': + if (p[1] == 'i') + goto LscanReal; // 0Li + break; // 0L + case '.': + if (p[1] == '.') + break; // 0.. + // 0. + case 'i','f','F', // Imaginary and float literal suffixes. + 'e', 'E': // Float exponent. + goto LscanReal; + default: + if (*p == '_') + goto LscanOctal; // 0_ + else if (isdigit(*p)) + { + if (*p == '8' || *p == '9') + goto Loctal_hasDecimalDigits; // 08 or 09 + else + goto Loctal_enter_loop; // 0[0-7] + } + } + + // Number 0 + assert(p[-1] == '0'); + assert(*p != '_' && !isdigit(*p)); + assert(ulong_ == 0); + isDecimal = true; + goto Lfinalize; + + LscanInteger: + assert(*p != 0 && isdigit(*p)); + isDecimal = true; + goto Lenter_loop_int; + while (1) + { + if (*++p == '_') + continue; + if (!isdigit(*p)) + break; + Lenter_loop_int: + if (ulong_ < ulong.max/10 || (ulong_ == ulong.max/10 && *p <= '5')) + { + ulong_ *= 10; + ulong_ += *p - '0'; + continue; + } + // Overflow: skip following digits. + overflow = true; + while (isdigit(*++p)) {} + break; + } + + // The number could be a float, so check overflow below. + switch (*p) + { + case '.': + if (p[1] != '.') + goto LscanReal; + break; + case 'L': + if (p[1] != 'i') + break; + case 'i', 'f', 'F', 'e', 'E': + goto LscanReal; + default: + } + + if (overflow) + error(t.start, MID.OverflowDecimalNumber); + + assert((isdigit(p[-1]) || p[-1] == '_') && !isdigit(*p) && *p != '_'); + goto Lfinalize; + + LscanHex: + assert(digits == 0); + assert(*p == 'x' || *p == 'X'); + while (1) + { + if (*++p == '_') + continue; + if (!ishexad(*p)) + break; + ++digits; + ulong_ *= 16; + if (*p <= '9') + ulong_ += *p - '0'; + else if (*p <= 'F') + ulong_ += *p - 'A' + 10; + else + ulong_ += *p - 'a' + 10; + } + + assert(ishexad(p[-1]) || p[-1] == '_' || p[-1] == 'x' || p[-1] == 'X'); + assert(!ishexad(*p) && *p != '_'); + + switch (*p) + { + case '.': + if (p[1] == '.') + break; + case 'p', 'P': + return scanHexReal(t); + default: + } + + if (digits == 0 || digits > 16) + error(t.start, digits == 0 ? MID.NoDigitsInHexNumber : MID.OverflowHexNumber); + + goto Lfinalize; + + LscanBinary: + assert(digits == 0); + assert(*p == 'b' || *p == 'B'); + while (1) + { + if (*++p == '0') + { + ++digits; + ulong_ *= 2; + } + else if (*p == '1') + { + ++digits; + ulong_ *= 2; + ulong_ += *p - '0'; + } + else if (*p == '_') + continue; + else + break; + } + + if (digits == 0 || digits > 64) + error(t.start, digits == 0 ? MID.NoDigitsInBinNumber : MID.OverflowBinaryNumber); + + assert(p[-1] == '0' || p[-1] == '1' || p[-1] == '_' || p[-1] == 'b' || p[-1] == 'B', p[-1] ~ ""); + assert( !(*p == '0' || *p == '1' || *p == '_') ); + goto Lfinalize; + + LscanOctal: + assert(*p == '_'); + while (1) + { + if (*++p == '_') + continue; + if (!isoctal(*p)) + break; + Loctal_enter_loop: + if (ulong_ < ulong.max/2 || (ulong_ == ulong.max/2 && *p <= '1')) + { + ulong_ *= 8; + ulong_ += *p - '0'; + continue; + } + // Overflow: skip following digits. + overflow = true; + while (isoctal(*++p)) {} + break; + } + + bool hasDecimalDigits; + if (isdigit(*p)) + { + Loctal_hasDecimalDigits: + hasDecimalDigits = true; + while (isdigit(*++p)) {} + } + + // The number could be a float, so check errors below. + switch (*p) + { + case '.': + if (p[1] != '.') + goto LscanReal; + break; + case 'L': + if (p[1] != 'i') + break; + case 'i', 'f', 'F', 'e', 'E': + goto LscanReal; + default: + } + + if (hasDecimalDigits) + error(t.start, MID.OctalNumberHasDecimals); + + if (overflow) + error(t.start, MID.OverflowOctalNumber); +// goto Lfinalize; + + Lfinalize: + enum Suffix + { + None = 0, + Unsigned = 1, + Long = 2 + } + + // Scan optional suffix: L, Lu, LU, u, uL, U or UL. + Suffix suffix; + while (1) + { + switch (*p) + { + case 'L': + if (suffix & Suffix.Long) + break; + suffix |= Suffix.Long; + ++p; + continue; + case 'u', 'U': + if (suffix & Suffix.Unsigned) + break; + suffix |= Suffix.Unsigned; + ++p; + continue; + default: + break; + } + break; + } + + // Determine type of Integer. + switch (suffix) + { + case Suffix.None: + if (ulong_ & 0x8000_0000_0000_0000) + { + if (isDecimal) + error(t.start, MID.OverflowDecimalSign); + t.kind = TOK.Uint64; + } + else if (ulong_ & 0xFFFF_FFFF_0000_0000) + t.kind = TOK.Int64; + else if (ulong_ & 0x8000_0000) + t.kind = isDecimal ? TOK.Int64 : TOK.Uint32; + else + t.kind = TOK.Int32; + break; + case Suffix.Unsigned: + if (ulong_ & 0xFFFF_FFFF_0000_0000) + t.kind = TOK.Uint64; + else + t.kind = TOK.Uint32; + break; + case Suffix.Long: + if (ulong_ & 0x8000_0000_0000_0000) + { + if (isDecimal) + error(t.start, MID.OverflowDecimalSign); + t.kind = TOK.Uint64; + } + else + t.kind = TOK.Int64; + break; + case Suffix.Unsigned | Suffix.Long: + t.kind = TOK.Uint64; + break; + default: + assert(0); + } + t.ulong_ = ulong_; + t.end = p; + return; + LscanReal: + scanReal(t); + return; + } + + /// Scans a floating point number literal. + /// + /// $(PRE + /// FloatLiteral := Float[fFL]?i? + /// Float := DecFloat | HexFloat + /// DecFloat := ([0-9][0-9_]*[.][0-9_]*DecExponent?) | + /// [.][0-9][0-9_]*DecExponent? | [0-9][0-9_]*DecExponent + /// DecExponent := [eE][+-]?[0-9][0-9_]* + /// HexFloat := 0[xX](HexDigits[.]HexDigits | + /// [.][0-9a-zA-Z]HexDigits? | + /// HexDigits)HexExponent + /// HexExponent := [pP][+-]?[0-9][0-9_]* + /// ) + void scanReal(ref Token t) + { + if (*p == '.') + { + assert(p[1] != '.'); + // This function was called by scan() or scanNumber(). + while (isdigit(*++p) || *p == '_') {} + } + else + // This function was called by scanNumber(). + assert(delegate () + { + switch (*p) + { + case 'L': + if (p[1] != 'i') + return false; + case 'i', 'f', 'F', 'e', 'E': + return true; + default: + } + return false; + }() + ); + + // Scan exponent. + if (*p == 'e' || *p == 'E') + { + ++p; + if (*p == '-' || *p == '+') + ++p; + if (isdigit(*p)) + while (isdigit(*++p) || *p == '_') {} + else + error(t.start, MID.FloatExpMustStartWithDigit); + } + + // Copy whole number and remove underscores from buffer. + char[] buffer = t.start[0..p-t.start].dup; + uint j; + foreach (c; buffer) + if (c != '_') + buffer[j++] = c; + buffer.length = j; // Adjust length. + buffer ~= 0; // Terminate for C functions. + + finalizeFloat(t, buffer); + } + + /// Scans a hexadecimal floating point number literal. + void scanHexReal(ref Token t) + { + assert(*p == '.' || *p == 'p' || *p == 'P'); + MID mid; + if (*p == '.') + while (ishexad(*++p) || *p == '_') + {} + // Decimal exponent is required. + if (*p != 'p' && *p != 'P') + { + mid = MID.HexFloatExponentRequired; + goto Lerr; + } + // Scan exponent + assert(*p == 'p' || *p == 'P'); + ++p; + if (*p == '+' || *p == '-') + ++p; + if (!isdigit(*p)) + { + mid = MID.HexFloatExpMustStartWithDigit; + goto Lerr; + } + while (isdigit(*++p) || *p == '_') + {} + // Copy whole number and remove underscores from buffer. + char[] buffer = t.start[0..p-t.start].dup; + uint j; + foreach (c; buffer) + if (c != '_') + buffer[j++] = c; + buffer.length = j; // Adjust length. + buffer ~= 0; // Terminate for C functions. + finalizeFloat(t, buffer); + return; + Lerr: + t.kind = TOK.Float32; + t.end = p; + error(t.start, mid); + } + + /// Sets the value of the token. + /// Params: + /// t = receives the value. + /// buffer = the well-formed float number. + void finalizeFloat(ref Token t, string buffer) + { + assert(buffer[$-1] == 0); + // Float number is well-formed. Check suffixes and do conversion. + switch (*p) + { + case 'f', 'F': + t.kind = TOK.Float32; + t.float_ = strtof(buffer.ptr, null); + ++p; + break; + case 'L': + t.kind = TOK.Float80; + t.real_ = strtold(buffer.ptr, null); + ++p; + break; + default: + t.kind = TOK.Float64; + t.double_ = strtod(buffer.ptr, null); + } + if (*p == 'i') + { + ++p; + t.kind += 3; // Switch to imaginary counterpart. + assert(t.kind == TOK.Imaginary32 || + t.kind == TOK.Imaginary64 || + t.kind == TOK.Imaginary80); + } + if (errno() == ERANGE) + error(t.start, MID.OverflowFloatNumber); + t.end = p; + } + + /// Scans a special token sequence. + /// + /// SpecialTokenSequence := "#line" Integer Filespec? EndOfLine + void scanSpecialTokenSequence(ref Token t) + { + assert(*p == '#'); + t.kind = TOK.HashLine; + t.setWhitespaceFlag(); + + MID mid; + char* errorAtColumn = p; + char* tokenEnd = ++p; + + if (!(p[0] == 'l' && p[1] == 'i' && p[2] == 'n' && p[3] == 'e')) + { + mid = MID.ExpectedIdentifierSTLine; + goto Lerr; + } + p += 3; + tokenEnd = p + 1; + + // TODO: #line58"path/file" is legal. Require spaces? + // State.Space could be used for that purpose. + enum State + { /+Space,+/ Integer, Filespec, End } + + State state = State.Integer; + + while (!isEndOfLine(++p)) + { + if (isspace(*p)) + continue; + if (state == State.Integer) + { + if (!isdigit(*p)) + { + errorAtColumn = p; + mid = MID.ExpectedIntegerAfterSTLine; + goto Lerr; + } + t.tokLineNum = new Token; + scan(*t.tokLineNum); + tokenEnd = p; + if (t.tokLineNum.kind != TOK.Int32 && t.tokLineNum.kind != TOK.Uint32) + { + errorAtColumn = t.tokLineNum.start; + mid = MID.ExpectedIntegerAfterSTLine; + goto Lerr; + } + --p; // Go one back because scan() advanced p past the integer. + state = State.Filespec; + } + else if (state == State.Filespec && *p == '"') + { // MID.ExpectedFilespec is deprecated. + // if (*p != '"') + // { + // errorAtColumn = p; + // mid = MID.ExpectedFilespec; + // goto Lerr; + // } + t.tokLineFilespec = new Token; + t.tokLineFilespec.start = p; + t.tokLineFilespec.kind = TOK.Filespec; + t.tokLineFilespec.setWhitespaceFlag(); + while (*++p != '"') + { + if (isEndOfLine(p)) + { + errorAtColumn = t.tokLineFilespec.start; + mid = MID.UnterminatedFilespec; + t.tokLineFilespec.end = p; + tokenEnd = p; + goto Lerr; + } + isascii(*p) || decodeUTF8(); + } + auto start = t.tokLineFilespec.start +1; // +1 skips '"' + t.tokLineFilespec.str = start[0 .. p - start]; + t.tokLineFilespec.end = p + 1; + tokenEnd = p + 1; + state = State.End; + } + else/+ if (state == State.End)+/ + { + mid = MID.UnterminatedSpecialToken; + goto Lerr; + } + } + assert(isEndOfLine(p)); + + if (state == State.Integer) + { + errorAtColumn = p; + mid = MID.ExpectedIntegerAfterSTLine; + goto Lerr; + } + + // Evaluate #line only when not in token string. + if (!inTokenString && t.tokLineNum) + { + this.lineNum_hline = this.lineNum - t.tokLineNum.uint_ + 1; + if (t.tokLineFilespec) + newFilePath(t.tokLineFilespec.str); + } + p = tokenEnd; + t.end = tokenEnd; + + return; + Lerr: + p = tokenEnd; + t.end = tokenEnd; + error(errorAtColumn, mid); + } + + /// Inserts an empty dummy token (TOK.Empty) before t. + /// + /// Useful in the parsing phase for representing a node in the AST + /// that doesn't consume an actual token from the source text. + Token* insertEmptyTokenBefore(Token* t) + { + assert(t !is null && t.prev !is null); + assert(text.ptr <= t.start && t.start < end, Token.toString(t.kind)); + assert(text.ptr <= t.end && t.end <= end, Token.toString(t.kind)); + + auto prev_t = t.prev; + auto new_t = new Token; + new_t.kind = TOK.Empty; + new_t.start = new_t.end = prev_t.end; + // Link in new token. + prev_t.next = new_t; + new_t.prev = prev_t; + new_t.next = t; + t.prev = new_t; + return new_t; + } + + /// Returns the error line number. + uint errorLineNumber(uint lineNum) + { + return lineNum - this.lineNum_hline; + } + + /// Forwards error parameters. + void error(char* columnPos, char[] msg, ...) + { + error_(this.lineNum, this.lineBegin, columnPos, msg, _arguments, _argptr); + } + + /// ditto + void error(char* columnPos, MID mid, ...) + { + error_(this.lineNum, this.lineBegin, columnPos, GetMsg(mid), _arguments, _argptr); + } + + /// ditto + void error(uint lineNum, char* lineBegin, char* columnPos, MID mid, ...) + { + error_(lineNum, lineBegin, columnPos, GetMsg(mid), _arguments, _argptr); + } + + /// Creates an error report and appends it to a list. + /// Params: + /// lineNum = the line number. + /// lineBegin = points to the first character of the current line. + /// columnPos = points to the character where the error is located. + /// msg = the message. + void error_(uint lineNum, char* lineBegin, char* columnPos, char[] msg, + TypeInfo[] _arguments, Arg _argptr) + { + lineNum = this.errorLineNumber(lineNum); + auto errorPath = this.filePaths.setPath; + auto location = new Location(errorPath, lineNum, lineBegin, columnPos); + msg = Format(_arguments, _argptr, msg); + auto error = new LexerError(location, msg); + errors ~= error; + if (infoMan !is null) + infoMan ~= error; + } + + /// Scans the whole source text until EOF is encountered. + void scanAll() + { + while (nextToken() != TOK.EOF) + {} + } + + /// Returns the first token of the source text. + /// This can be the EOF token. + /// Structure: HEAD -> Newline -> First Token + Token* firstToken() + { + return this.head.next.next; + } + + /// Returns true if str is a valid D identifier. + static bool isIdentifierString(char[] str) + { + if (str.length == 0 || isdigit(str[0])) + return false; + size_t idx; + do + { + auto c = dil.Unicode.decode(str, idx); + if (c == ERROR_CHAR || !(isident(c) || !isascii(c) && isUniAlpha(c))) + return false; + } while (idx < str.length) + return true; + } + + /// Returns true if str is a keyword or a special token (__FILE__, __LINE__ etc.) + static bool isReservedIdentifier(char[] str) + { + if (!isIdentifierString(str)) + return false; // str is not a valid identifier. + + auto id = IdTable.inStatic(str); + if (id is null || id.kind == TOK.Identifier) + return false; // str is not in the table or a normal identifier. + + return true; + } + + /// Returns true if the current character to be decoded is + /// a Unicode alpha character. + /// + /// The current pointer 'p' is not advanced if false is returned. + bool isUnicodeAlpha() + { + assert(!isascii(*p), "check for ASCII char before calling decodeUTF8()."); + char* p = this.p; + dchar d = *p; + ++p; // Move to second byte. + // Error if second byte is not a trail byte. + if (!isTrailByte(*p)) + return false; + // Check for overlong sequences. + switch (d) + { + case 0xE0, 0xF0, 0xF8, 0xFC: + if ((*p & d) == 0x80) + return false; + default: + if ((d & 0xFE) == 0xC0) // 1100000x + return false; + } + const char[] checkNextByte = "if (!isTrailByte(*++p))" + " return false;"; + const char[] appendSixBits = "d = (d << 6) | *p & 0b0011_1111;"; + // Decode + if ((d & 0b1110_0000) == 0b1100_0000) + { + d &= 0b0001_1111; + mixin(appendSixBits); + } + else if ((d & 0b1111_0000) == 0b1110_0000) + { + d &= 0b0000_1111; + mixin(appendSixBits ~ + checkNextByte ~ appendSixBits); + } + else if ((d & 0b1111_1000) == 0b1111_0000) + { + d &= 0b0000_0111; + mixin(appendSixBits ~ + checkNextByte ~ appendSixBits ~ + checkNextByte ~ appendSixBits); + } + else + return false; + + assert(isTrailByte(*p)); + if (!isValidChar(d) || !isUniAlpha(d)) + return false; + // Only advance pointer if this is a Unicode alpha character. + this.p = p; + return true; + } + + /// Decodes the next UTF-8 sequence. + dchar decodeUTF8() + { + assert(!isascii(*p), "check for ASCII char before calling decodeUTF8()."); + char* p = this.p; + dchar d = *p; + + ++p; // Move to second byte. + // Error if second byte is not a trail byte. + if (!isTrailByte(*p)) + goto Lerr2; + + // Check for overlong sequences. + switch (d) + { + case 0xE0, // 11100000 100xxxxx + 0xF0, // 11110000 1000xxxx + 0xF8, // 11111000 10000xxx + 0xFC: // 11111100 100000xx + if ((*p & d) == 0x80) + goto Lerr; + default: + if ((d & 0xFE) == 0xC0) // 1100000x + goto Lerr; + } + + const char[] checkNextByte = "if (!isTrailByte(*++p))" + " goto Lerr2;"; + const char[] appendSixBits = "d = (d << 6) | *p & 0b0011_1111;"; + + // Decode + if ((d & 0b1110_0000) == 0b1100_0000) + { // 110xxxxx 10xxxxxx + d &= 0b0001_1111; + mixin(appendSixBits); + } + else if ((d & 0b1111_0000) == 0b1110_0000) + { // 1110xxxx 10xxxxxx 10xxxxxx + d &= 0b0000_1111; + mixin(appendSixBits ~ + checkNextByte ~ appendSixBits); + } + else if ((d & 0b1111_1000) == 0b1111_0000) + { // 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx + d &= 0b0000_0111; + mixin(appendSixBits ~ + checkNextByte ~ appendSixBits ~ + checkNextByte ~ appendSixBits); + } + else + // 5 and 6 byte UTF-8 sequences are not allowed yet. + // 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx + // 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx + goto Lerr; + + assert(isTrailByte(*p)); + + if (!isValidChar(d)) + { + Lerr: + // Three cases: + // *) the UTF-8 sequence was successfully decoded but the resulting + // character is invalid. + // p points to last trail byte in the sequence. + // *) the UTF-8 sequence is overlong. + // p points to second byte in the sequence. + // *) the UTF-8 sequence has more than 4 bytes or starts with + // a trail byte. + // p points to second byte in the sequence. + assert(isTrailByte(*p)); + // Move to next ASCII character or lead byte of a UTF-8 sequence. + while (p < (end-1) && isTrailByte(*p)) + ++p; + --p; + assert(!isTrailByte(p[1])); + Lerr2: + d = REPLACEMENT_CHAR; + error(this.p, MID.InvalidUTF8Sequence, formatBytes(this.p, p)); + } + + this.p = p; + return d; + } + + /// Encodes the character d and appends it to str. + static void encodeUTF8(ref char[] str, dchar d) + { + assert(!isascii(d), "check for ASCII char before calling encodeUTF8()."); + assert(isValidChar(d), "check if character is valid before calling encodeUTF8()."); + + char[6] b = void; + if (d < 0x800) + { + b[0] = 0xC0 | (d >> 6); + b[1] = 0x80 | (d & 0x3F); + str ~= b[0..2]; + } + else if (d < 0x10000) + { + b[0] = 0xE0 | (d >> 12); + b[1] = 0x80 | ((d >> 6) & 0x3F); + b[2] = 0x80 | (d & 0x3F); + str ~= b[0..3]; + } + else if (d < 0x200000) + { + b[0] = 0xF0 | (d >> 18); + b[1] = 0x80 | ((d >> 12) & 0x3F); + b[2] = 0x80 | ((d >> 6) & 0x3F); + b[3] = 0x80 | (d & 0x3F); + str ~= b[0..4]; + } + /+ // There are no 5 and 6 byte UTF-8 sequences yet. + else if (d < 0x4000000) + { + b[0] = 0xF8 | (d >> 24); + b[1] = 0x80 | ((d >> 18) & 0x3F); + b[2] = 0x80 | ((d >> 12) & 0x3F); + b[3] = 0x80 | ((d >> 6) & 0x3F); + b[4] = 0x80 | (d & 0x3F); + str ~= b[0..5]; + } + else if (d < 0x80000000) + { + b[0] = 0xFC | (d >> 30); + b[1] = 0x80 | ((d >> 24) & 0x3F); + b[2] = 0x80 | ((d >> 18) & 0x3F); + b[3] = 0x80 | ((d >> 12) & 0x3F); + b[4] = 0x80 | ((d >> 6) & 0x3F); + b[5] = 0x80 | (d & 0x3F); + str ~= b[0..6]; + } + +/ + else + assert(0); + } + + /// Formats the bytes between start and end. + /// Returns: e.g.: abc -> \x61\x62\x63 + static char[] formatBytes(char* start, char* end) + { + auto strLen = end-start; + const formatLen = `\xXX`.length; + char[] result = new char[strLen*formatLen]; // Reserve space. + result.length = 0; + foreach (c; cast(ubyte[])start[0..strLen]) + result ~= Format("\\x{:X}", c); + return result; + } + + /// Searches for an invalid UTF-8 sequence in str. + /// Returns: a formatted string of the invalid sequence (e.g. \xC0\x80). + static string findInvalidUTF8Sequence(string str) + { + char* p = str.ptr, end = p + str.length; + while (p < end) + { + if (decode(p, end) == ERROR_CHAR) + { + auto begin = p; + // Skip trail-bytes. + while (++p < end && isTrailByte(*p)) + {} + return Lexer.formatBytes(begin, p); + } + } + assert(p == end); + return ""; + } +} + +/// Tests the lexer with a list of tokens. +unittest +{ + Stdout("Testing Lexer.\n"); + struct Pair + { + char[] tokenText; + TOK kind; + } + static Pair[] pairs = [ + {"#!äöüß", TOK.Shebang}, {"\n", TOK.Newline}, + {"//çay", TOK.Comment}, {"\n", TOK.Newline}, + {"&", TOK.AndBinary}, + {"/*çağ*/", TOK.Comment}, {"&&", TOK.AndLogical}, + {"/+çak+/", TOK.Comment}, {"&=", TOK.AndAssign}, + {">", TOK.Greater}, {"+", TOK.Plus}, + {">=", TOK.GreaterEqual}, {"++", TOK.PlusPlus}, + {">>", TOK.RShift}, {"+=", TOK.PlusAssign}, + {">>=", TOK.RShiftAssign}, {"-", TOK.Minus}, + {">>>", TOK.URShift}, {"--", TOK.MinusMinus}, + {">>>=", TOK.URShiftAssign}, {"-=", TOK.MinusAssign}, + {"<", TOK.Less}, {"=", TOK.Assign}, + {"<=", TOK.LessEqual}, {"==", TOK.Equal}, + {"<>", TOK.LorG}, {"~", TOK.Tilde}, + {"<>=", TOK.LorEorG}, {"~=", TOK.CatAssign}, + {"<<", TOK.LShift}, {"*", TOK.Mul}, + {"<<=", TOK.LShiftAssign}, {"*=", TOK.MulAssign}, + {"!", TOK.Not}, {"/", TOK.Div}, + {"!=", TOK.NotEqual}, {"/=", TOK.DivAssign}, + {"!<", TOK.UorGorE}, {"^", TOK.Xor}, + {"!>", TOK.UorLorE}, {"^=", TOK.XorAssign}, + {"!<=", TOK.UorG}, {"%", TOK.Mod}, + {"!>=", TOK.UorL}, {"%=", TOK.ModAssign}, + {"!<>", TOK.UorE}, {"(", TOK.LParen}, + {"!<>=", TOK.Unordered}, {")", TOK.RParen}, + {".", TOK.Dot}, {"[", TOK.LBracket}, + {"..", TOK.Slice}, {"]", TOK.RBracket}, + {"...", TOK.Ellipses}, {"{", TOK.LBrace}, + {"|", TOK.OrBinary}, {"}", TOK.RBrace}, + {"||", TOK.OrLogical}, {":", TOK.Colon}, + {"|=", TOK.OrAssign}, {";", TOK.Semicolon}, + {"?", TOK.Question}, {",", TOK.Comma}, + {"$", TOK.Dollar}, {"cam", TOK.Identifier}, + {"çay", TOK.Identifier}, {".0", TOK.Float64}, + {"0", TOK.Int32}, {"\n", TOK.Newline}, + {"\r", TOK.Newline}, {"\r\n", TOK.Newline}, + {"\u2028", TOK.Newline}, {"\u2029", TOK.Newline} + ]; + + char[] src; + + // Join all token texts into a single string. + foreach (i, pair; pairs) + if (pair.kind == TOK.Comment && pair.tokenText[1] == '/' || // Line comment. + pair.kind == TOK.Shebang) + { + assert(pairs[i+1].kind == TOK.Newline); // Must be followed by a newline. + src ~= pair.tokenText; + } + else + src ~= pair.tokenText ~ " "; + + auto lx = new Lexer(new SourceText("", src)); + auto token = lx.getTokens(); + + uint i; + assert(token == lx.head); + assert(token.next.kind == TOK.Newline); + token = token.next.next; + do + { + assert(i < pairs.length); + assert(token.srcText == pairs[i].tokenText, Format("Scanned '{0}' but expected '{1}'", token.srcText, pairs[i].tokenText)); + ++i; + token = token.next; + } while (token.kind != TOK.EOF) +} + +/// Tests the Lexer's peek() method. +unittest +{ + Stdout("Testing method Lexer.peek()\n"); + auto sourceText = new SourceText("", "unittest { }"); + auto lx = new Lexer(sourceText, null); + + auto next = lx.head; + lx.peek(next); + assert(next.kind == TOK.Newline); + lx.peek(next); + assert(next.kind == TOK.Unittest); + lx.peek(next); + assert(next.kind == TOK.LBrace); + lx.peek(next); + assert(next.kind == TOK.RBrace); + lx.peek(next); + assert(next.kind == TOK.EOF); + + lx = new Lexer(new SourceText("", "")); + next = lx.head; + lx.peek(next); + assert(next.kind == TOK.Newline); + lx.peek(next); + assert(next.kind == TOK.EOF); +} + +unittest +{ + // Numbers unittest + // 0L 0ULi 0_L 0_UL 0x0U 0x0p2 0_Fi 0_e2 0_F 0_i + // 0u 0U 0uL 0UL 0L 0LU 0Lu + // 0Li 0f 0F 0fi 0Fi 0i + // 0b_1_LU 0b1000u + // 0x232Lu +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/Token.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/Token.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,400 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.Token; + +import dil.lexer.Identifier; +import dil.lexer.Funcs; +import dil.Location; +import tango.stdc.stdlib : malloc, free; +import tango.core.Exception; +import common; + +public import dil.lexer.TokensEnum; + +/// A Token is a sequence of characters formed by the lexical analyzer. +struct Token +{ /// Flags set by the Lexer. + enum Flags : ushort + { + None, + Whitespace = 1, /// Tokens with this flag are ignored by the Parser. + } + + TOK kind; /// The token kind. + Flags flags; /// The flags of the token. + /// Pointers to the next and previous tokens (doubly-linked list.) + Token* next, prev; + + /// Start of whitespace characters before token. Null if no WS. + /// TODO: remove to save space; can be replaced by 'prev.end'. + char* ws; + char* start; /// Points to the first character of the token. + char* end; /// Points one character past the end of the token. + + /// Data associated with this token. + /// TODO: move data structures out; use only pointers here to keep Token.sizeof small. + union + { + /// For newline tokens. + NewlineData newline; + /// For #line tokens. + struct + { + Token* tokLineNum; /// #line number + Token* tokLineFilespec; /// #line number filespec + } + /// The value of a string token. + struct + { + string str; /// Zero-terminated string. (The zero is included in the length.) + char pf; /// Postfix 'c', 'w', 'd' or 0 for none. + version(D2) + Token* tok_str; /++ Points to the contents of a token string stored as a + doubly linked list. The last token is always '}' or + EOF in case end of source text is "q{" EOF. + +/ + } + Identifier* ident; /// For keywords and identifiers. + dchar dchar_; /// A character value. + long long_; /// A long integer value. + ulong ulong_; /// An unsigned long integer value. + int int_; /// An integer value. + uint uint_; /// An unsigned integer value. + float float_; /// A float value. + double double_; /// A double value. + real real_; /// A real value. + } + + /// Returns the text of the token. + string srcText() + { + assert(start && end); + return start[0 .. end - start]; + } + + /// Returns the preceding whitespace of the token. + string wsChars() + { + assert(ws && start); + return ws[0 .. start - ws]; + } + + /// Finds the next non-whitespace token. + /// Returns: 'this' token if the next token is TOK.EOF or null. + Token* nextNWS() + out(token) + { + assert(token !is null); + } + body + { + auto token = next; + while (token !is null && token.isWhitespace) + token = token.next; + if (token is null || token.kind == TOK.EOF) + return this; + return token; + } + + /// Finds the previous non-whitespace token. + /// Returns: 'this' token if the previous token is TOK.HEAD or null. + Token* prevNWS() + out(token) + { + assert(token !is null); + } + body + { + auto token = prev; + while (token !is null && token.isWhitespace) + token = token.prev; + if (token is null || token.kind == TOK.HEAD) + return this; + return token; + } + + /// Returns the string for a token kind. + static string toString(TOK kind) + { + return tokToString[kind]; + } + + /// Adds Flags.Whitespace to this.flags. + void setWhitespaceFlag() + { + this.flags |= Flags.Whitespace; + } + + /// Returns true if this is a token that can have newlines in it. + /// + /// These can be block and nested comments and any string literal + /// except for escape string literals. + bool isMultiline() + { + return kind == TOK.String && start[0] != '\\' || + kind == TOK.Comment && start[1] != '/'; + } + + /// Returns true if this is a keyword token. + bool isKeyword() + { + return KeywordsBegin <= kind && kind <= KeywordsEnd; + } + + /// Returns true if this is an integral type token. + bool isIntegralType() + { + return IntegralTypeBegin <= kind && kind <= IntegralTypeEnd; + } + + /// Returns true if this is a whitespace token. + bool isWhitespace() + { + return !!(flags & Flags.Whitespace); + } + + /// Returns true if this is a special token. + bool isSpecialToken() + { + return SpecialTokensBegin <= kind && kind <= SpecialTokensEnd; + } + +version(D2) +{ + /// Returns true if this is a token string literal. + bool isTokenStringLiteral() + { + return kind == TOK.String && tok_str !is null; + } +} + + /// Returns true if this token starts a DeclarationDefinition. + bool isDeclDefStart() + { + return isDeclDefStartToken(kind); + } + + /// Returns true if this token starts a Statement. + bool isStatementStart() + { + return isStatementStartToken(kind); + } + + /// Returns true if this token starts an AsmStatement. + bool isAsmStatementStart() + { + return isAsmStatementStartToken(kind); + } + + int opEquals(TOK kind2) + { + return kind == kind2; + } + + int opCmp(Token* rhs) + { + return start < rhs.start; + } + + /// Returns the Location of this token. + Location getLocation(bool realLocation)() + { + auto search_t = this.prev; + // Find previous newline token. + while (search_t.kind != TOK.Newline) + search_t = search_t.prev; + static if (realLocation) + { + auto filePath = search_t.newline.filePaths.oriPath; + auto lineNum = search_t.newline.oriLineNum; + } + else + { + auto filePath = search_t.newline.filePaths.setPath; + auto lineNum = search_t.newline.oriLineNum - search_t.newline.setLineNum; + } + auto lineBegin = search_t.end; + // Determine actual line begin and line number. + while (1) + { + search_t = search_t.next; + if (search_t == this) + break; + // Multiline tokens must be rescanned for newlines. + if (search_t.isMultiline) + { + auto p = search_t.start, end = search_t.end; + while (p != end) + { + if (scanNewline(p) == '\n') + { + lineBegin = p; + ++lineNum; + } + else + ++p; + } + } + } + return new Location(filePath, lineNum, lineBegin, this.start); + } + + alias getLocation!(true) getRealLocation; + alias getLocation!(false) getErrorLocation; + + uint lineCount() + { + uint count = 1; + if (this.isMultiline) + { + auto p = this.start, end = this.end; + while (p != end) + { + if (scanNewline(p) == '\n') + ++count; + else + ++p; + } + } + return count; + } + + /// Return the source text enclosed by the left and right token. + static char[] textSpan(Token* left, Token* right) + { + assert(left.end <= right.start || left is right ); + return left.start[0 .. right.end - left.start]; + } + + /// Uses malloc() to allocate memory for a token. + new(size_t size) + { + void* p = malloc(size); + if (p is null) + throw new OutOfMemoryException(__FILE__, __LINE__); + // TODO: Token.init should be all zeros. + // Maybe use calloc() to avoid this line? + *cast(Token*)p = Token.init; + return p; + } + + /// Deletes a token using free(). + delete(void* p) + { + auto token = cast(Token*)p; + if (token) + { + if(token.kind == TOK.HashLine) + token.destructHashLineToken(); + else + { + version(D2) + if (token.isTokenStringLiteral) + token.destructTokenStringLiteral(); + } + } + free(p); + } + + void destructHashLineToken() + { + assert(kind == TOK.HashLine); + delete tokLineNum; + delete tokLineFilespec; + } + +version(D2) +{ + void destructTokenStringLiteral() + { + assert(kind == TOK.String); + assert(start && *start == 'q' && start[1] == '{'); + assert(tok_str !is null); + auto tok_it = tok_str; + auto tok_del = tok_str; + while (tok_it && tok_it.kind != TOK.EOF) + { + tok_it = tok_it.next; + assert(tok_del && tok_del.kind != TOK.EOF); + delete tok_del; + tok_del = tok_it; + } + } +} +} + +/// Data associated with newline tokens. +struct NewlineData +{ + struct FilePaths + { + char[] oriPath; /// Original path to the source text. + char[] setPath; /// Path set by #line. + } + FilePaths* filePaths; + uint oriLineNum; /// Actual line number in the source text. + uint setLineNum; /// Delta line number set by #line. +} + +/// Returns true if this token starts a DeclarationDefinition. +bool isDeclDefStartToken(TOK tok) +{ + switch (tok) + { + alias TOK T; + case T.Align, T.Pragma, T.Export, T.Private, T.Package, T.Protected, + T.Public, T.Extern, T.Deprecated, T.Override, T.Abstract, + T.Synchronized, T.Static, T.Final, T.Const, T.Invariant/*D 2.0*/, + T.Auto, T.Scope, T.Alias, T.Typedef, T.Import, T.Enum, T.Class, + T.Interface, T.Struct, T.Union, T.This, T.Tilde, T.Unittest, T.Debug, + T.Version, T.Template, T.New, T.Delete, T.Mixin, T.Semicolon, + T.Identifier, T.Dot, T.Typeof: + return true; + default: + if (IntegralTypeBegin <= tok && tok <= IntegralTypeEnd) + return true; + } + return false; +} + +/// Returns true if this token starts a Statement. +bool isStatementStartToken(TOK tok) +{ + switch (tok) + { + alias TOK T; + case T.Align, T.Extern, T.Final, T.Const, T.Auto, T.Identifier, T.Dot, + T.Typeof, T.If, T.While, T.Do, T.For, T.Foreach, T.Foreach_reverse, + T.Switch, T.Case, T.Default, T.Continue, T.Break, T.Return, T.Goto, + T.With, T.Synchronized, T.Try, T.Throw, T.Scope, T.Volatile, T.Asm, + T.Pragma, T.Mixin, T.Static, T.Debug, T.Version, T.Alias, T.Semicolon, + T.Enum, T.Class, T.Interface, T.Struct, T.Union, T.LBrace, T.Typedef, + T.This, T.Super, T.Null, T.True, T.False, T.Int32, T.Int64, T.Uint32, + T.Uint64, T.Float32, T.Float64, T.Float80, T.Imaginary32, + T.Imaginary64, T.Imaginary80, T.CharLiteral, T.String, T.LBracket, + T.Function, T.Delegate, T.Assert, T.Import, T.Typeid, T.Is, T.LParen, + T.Traits/*D2.0*/, T.AndBinary, T.PlusPlus, T.MinusMinus, T.Mul, + T.Minus, T.Plus, T.Not, T.Tilde, T.New, T.Delete, T.Cast: + return true; + default: + if (IntegralTypeBegin <= tok && tok <= IntegralTypeEnd || + SpecialTokensBegin <= tok && tok <= SpecialTokensEnd) + return true; + } + return false; +} + +/// Returns true if this token starts an AsmStatement. +bool isAsmStatementStartToken(TOK tok) +{ + switch(tok) + { + alias TOK T; + case T.In, T.Int, T.Out, T.Identifier, T.Align, T.Semicolon: + return true; + default: + } + return false; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/lexer/TokensEnum.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/lexer/TokensEnum.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,219 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.lexer.TokensEnum; + +import common; + +/// Enumeration of token kinds. +enum TOK : ushort +{ + Invalid, + + Illegal, + Comment, + Shebang, + HashLine, + Filespec, + Newline, + Empty, + + Identifier, + String, + CharLiteral, + + // Special tokens + FILE, + LINE, + DATE, + TIME, + TIMESTAMP, + VENDOR, + VERSION, + + // Number literals + Int32, Int64, Uint32, Uint64, + // Floating point number scanner relies on this order. (FloatXY + 3 == ImaginaryXY) + Float32, Float64, Float80, + Imaginary32, Imaginary64, Imaginary80, + + + // Brackets + LParen, + RParen, + LBracket, + RBracket, + LBrace, + RBrace, + + Dot, Slice, Ellipses, + + // Floating point number operators + Unordered, + UorE, + UorG, + UorGorE, + UorL, + UorLorE, + LorEorG, + LorG, + + // Normal operators + Assign, Equal, NotEqual, Not, + LessEqual, Less, + GreaterEqual, Greater, + LShiftAssign, LShift, + RShiftAssign,RShift, + URShiftAssign, URShift, + OrAssign, OrLogical, OrBinary, + AndAssign, AndLogical, AndBinary, + PlusAssign, PlusPlus, Plus, + MinusAssign, MinusMinus, Minus, + DivAssign, Div, + MulAssign, Mul, + ModAssign, Mod, + XorAssign, Xor, + CatAssign, + Tilde, + + Colon, + Semicolon, + Question, + Comma, + Dollar, + + /* Keywords: + NB.: Token.isKeyword() depends on this list being contiguous. + */ + Abstract, Alias, Align, Asm, Assert, Auto, Body, + Break, Case, Cast, Catch, + Class, Const, Continue, + Debug, Default, Delegate, Delete, Deprecated, Do, + Else, Enum, Export, Extern, False, Final, + Finally, For, Foreach, Foreach_reverse, Function, Goto, + If, Import, In, Inout, + Interface, Invariant, Is, Lazy, Macro/+D2.0+/, + Mixin, Module, New, Nothrow/+D2.0+/, Null, Out, Override, Package, + Pragma, Private, Protected, Public, Pure/+D2.0+/, Ref, Return, + Scope, Static, Struct, Super, Switch, Synchronized, + Template, This, Throw, Traits/+D2.0+/, True, Try, Typedef, Typeid, + Typeof, Union, Unittest, + Version, Volatile, While, With, + // Integral types. + Char, Wchar, Dchar, Bool, + Byte, Ubyte, Short, Ushort, + Int, Uint, Long, Ulong, + Cent, Ucent, + Float, Double, Real, + Ifloat, Idouble, Ireal, + Cfloat, Cdouble, Creal, Void, + + HEAD, // start of linked list + EOF, + MAX +} + +alias TOK.Abstract KeywordsBegin; +alias TOK.Void KeywordsEnd; +alias TOK.Char IntegralTypeBegin; +alias TOK.Void IntegralTypeEnd; +alias TOK.FILE SpecialTokensBegin; +alias TOK.VERSION SpecialTokensEnd; + +/// A table that maps each token kind to a string. +const string[TOK.MAX] tokToString = [ + "Invalid", + + "Illegal", + "Comment", + "#! /shebang/", + "#line", + `"filespec"`, + "Newline", + "Empty", + + "Identifier", + "String", + "CharLiteral", + + "__FILE__", + "__LINE__", + "__DATE__", + "__TIME__", + "__TIMESTAMP__", + "__VENDOR__", + "__VERSION__", + + "Int32", "Int64", "Uint32", "Uint64", + "Float32", "Float64", "Float80", + "Imaginary32", "Imaginary64", "Imaginary80", + + "(", + ")", + "[", + "]", + "{", + "}", + + ".", "..", "...", + + "!<>=", // Unordered + "!<>", // UorE + "!<=", // UorG + "!<", // UorGorE + "!>=", // UorL + "!>", // UorLorE + "<>=", // LorEorG + "<>", // LorG + + "=", "==", "!=", "!", + "<=", "<", + ">=", ">", + "<<=", "<<", + ">>=",">>", + ">>>=", ">>>", + "|=", "||", "|", + "&=", "&&", "&", + "+=", "++", "+", + "-=", "--", "-", + "/=", "/", + "*=", "*", + "%=", "%", + "^=", "^", + "~=", + "~", + + ":", + ";", + "?", + ",", + "$", + + "abstract","alias","align","asm","assert","auto","body", + "break","case","cast","catch", + "class","const","continue", + "debug","default","delegate","delete","deprecated","do", + "else","enum","export","extern","false","final", + "finally","for","foreach","foreach_reverse","function","goto", + "if","import","in","inout", + "interface","invariant","is","lazy","macro", + "mixin","module","new","nothrow","null","out","override","package", + "pragma","private","protected","public","pure","ref","return", + "scope","static","struct","super","switch","synchronized", + "template","this","throw","__traits","true","try","typedef","typeid", + "typeof","union","unittest", + "version","volatile","while","with", + // Integral types. + "char", "wchar", "dchar", "bool", + "byte", "ubyte", "short", "ushort", + "int", "uint", "long", "ulong", + "cent", "ucent", + "float", "double", "real", + "ifloat", "idouble", "ireal", + "cfloat", "cdouble", "creal", "void", + + "HEAD", + "EOF" +]; +static assert(tokToString.length == TOK.EOF+1); diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/parser/ImportParser.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/parser/ImportParser.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,373 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.parser.ImportParser; + +import dil.parser.Parser; +import dil.ast.Node; +import dil.ast.Declarations; +import dil.ast.Statements; +import dil.SourceText; +import dil.Enums; +import common; + +private alias TOK T; + +/// A light-weight parser which looks only for import statements +/// in the source text. +class ImportParser : Parser +{ + this(SourceText srcText) + { + super(srcText); + } + + override CompoundDeclaration start() + { + auto decls = new CompoundDeclaration; + super.init(); + if (token.kind == T.Module) + decls ~= parseModuleDeclaration(); + while (token.kind != T.EOF) + parseDeclarationDefinition(Protection.None); + return decls; + } + + void parseDeclarationDefinitionsBlock(Protection prot) + { + skip(T.LBrace); + while (token.kind != T.RBrace && token.kind != T.EOF) + parseDeclarationDefinition(prot); + skip(T.RBrace); + } + + void parseDeclarationsBlock(Protection prot) + { + switch (token.kind) + { + case T.LBrace: + parseDeclarationDefinitionsBlock(prot); + break; + case T.Colon: + nT(); + while (token.kind != T.RBrace && token.kind != T.EOF) + parseDeclarationDefinition(prot); + break; + default: + parseDeclarationDefinition(prot); + } + } + + bool skipToClosing(T opening, T closing) + { + alias token next; + uint level = 1; + while (1) + { + lexer.peek(next); + if (next.kind == opening) + ++level; + else if (next.kind == closing && --level == 0) + return true; + else if (next.kind == T.EOF) + break; + } + return false; + } + + void skipToTokenAfterClosingParen() + { + skipToClosing(T.LParen, T.RParen); + nT(); + } + + void skipToTokenAfterClosingBrace() + { + skipToClosing(T.LBrace, T.RBrace); + nT(); + } + + void skip(TOK tok) + { + token.kind == tok && nT(); + } + + void parseProtectionAttribute() + { + Protection prot; + switch (token.kind) + { + case T.Private: + prot = Protection.Private; break; + case T.Package: + prot = Protection.Package; break; + case T.Protected: + prot = Protection.Protected; break; + case T.Public: + prot = Protection.Public; break; + case T.Export: + prot = Protection.Export; break; + default: + assert(0); + } + nT(); + parseDeclarationsBlock(prot); + } + + void parseDeclarationDefinition(Protection prot) + { + switch (token.kind) + { + case T.Align: + nT(); + if (token.kind == T.LParen) + nT(), nT(), nT(); // ( Integer ) + parseDeclarationsBlock(prot); + break; + case T.Pragma: + nT(); + skipToTokenAfterClosingParen(); + parseDeclarationsBlock(prot); + break; + case T.Export, + T.Private, + T.Package, + T.Protected, + T.Public: + parseProtectionAttribute(); + break; + // Storage classes + case T.Extern: + nT(); + token.kind == T.LParen && skipToTokenAfterClosingParen(); + parseDeclarationsBlock(prot); + break; + case T.Const: + version(D2) + { + if (peekNext() == T.LParen) + goto case_Declaration; + } + case T.Override, + T.Deprecated, + T.Abstract, + T.Synchronized, + // T.Static, + T.Final, + T.Auto, + T.Scope: + case_StaticAttribute: + case_InvariantAttribute: + nT(); + parseDeclarationsBlock(prot); + break; + // End of storage classes. + case T.Alias, T.Typedef: + nT(); + goto case_Declaration; + case T.Static: + switch (peekNext()) + { + case T.Import: + goto case_Import; + case T.This: + nT(), nT(); // static this + skipToTokenAfterClosingParen(); + parseFunctionBody(); + break; + case T.Tilde: + nT(), nT(), nT(), nT(), nT(); // static ~ this ( ) + parseFunctionBody(); + break; + case T.If: + nT(), nT(); + skipToTokenAfterClosingParen(); + parseDeclarationsBlock(prot); + if (token.kind == T.Else) + nT(), parseDeclarationsBlock(prot); + break; + case T.Assert: + nT(), nT(); // static assert + skipToTokenAfterClosingParen(); + skip(T.Semicolon); + break; + default: + goto case_StaticAttribute; + } + break; + case T.Import: + case_Import: + auto decl = parseImportDeclaration(); + decl.setProtection(prot); // Set the protection attribute. + imports ~= decl.to!(ImportDeclaration); + break; + case T.Enum: + nT(); + token.kind == T.Identifier && nT(); + if (token.kind == T.Colon) + { + nT(); + while (token.kind != T.LBrace && token.kind != T.EOF) + nT(); + } + if (token.kind == T.Semicolon) + nT(); + else + skipToTokenAfterClosingBrace(); + break; + case T.Class: + case T.Interface: + nT(), skip(T.Identifier); // class Identifier + token.kind == T.LParen && skipToTokenAfterClosingParen(); // Skip template params. + if (token.kind == T.Colon) + { // BaseClasses + nT(); + while (token.kind != T.LBrace && token.kind != T.EOF) + if (token.kind == T.LParen) // Skip ( tokens... ) + skipToTokenAfterClosingParen(); + else + nT(); + } + if (token.kind == T.Semicolon) + nT(); + else + parseDeclarationDefinitionsBlock(Protection.None); + break; + case T.Struct, T.Union: + nT(); skip(T.Identifier); + token.kind == T.LParen && skipToTokenAfterClosingParen(); + if (token.kind == T.Semicolon) + nT(); + else + parseDeclarationDefinitionsBlock(Protection.None); + break; + case T.Tilde: + nT(); // ~ + case T.This: + nT(); nT(); nT(); // this ( ) + parseFunctionBody(); + break; + case T.Invariant: + version(D2) + { + auto next = token; + if (peekAfter(next) == T.LParen) + { + if (peekAfter(next) != T.RParen) + goto case_Declaration; + } + else + goto case_InvariantAttribute; + } + nT(); + token.kind == T.LParen && skipToTokenAfterClosingParen(); + parseFunctionBody(); + break; + case T.Unittest: + nT(); + parseFunctionBody(); + break; + case T.Debug: + nT(); + if (token.kind == T.Assign) + { + nT(), nT(), nT(); // = Condition ; + break; + } + if (token.kind == T.LParen) + nT(), nT(), nT(); // ( Condition ) + parseDeclarationsBlock(prot); + if (token.kind == T.Else) + nT(), parseDeclarationsBlock(prot); + break; + case T.Version: + nT(); + if (token.kind == T.Assign) + { + nT(), nT(), nT(); // = Condition ; + break; + } + nT(), nT(), nT(); // ( Condition ) + parseDeclarationsBlock(prot); + if (token.kind == T.Else) + nT(), parseDeclarationsBlock(prot); + break; + case T.Template: + nT(); + skip(T.Identifier); + skipToTokenAfterClosingParen(); + parseDeclarationDefinitionsBlock(Protection.None); + break; + case T.New: + nT(); + skipToTokenAfterClosingParen(); + parseFunctionBody(); + break; + case T.Delete: + nT(); + skipToTokenAfterClosingParen(); + parseFunctionBody(); + break; + case T.Mixin: + while (token.kind != T.Semicolon && token.kind != T.EOF) + if (token.kind == T.LParen) + skipToTokenAfterClosingParen(); + else + nT(); + skip(T.Semicolon); + break; + case T.Semicolon: + nT(); + break; + // Declaration + case T.Identifier, T.Dot, T.Typeof: + case_Declaration: + while (token.kind != T.Semicolon && token.kind != T.EOF) + if (token.kind == T.LParen) + skipToTokenAfterClosingParen(); + else if (token.kind == T.LBrace) + skipToTokenAfterClosingBrace(); + else + nT(); + skip(T.Semicolon); + break; + default: + if (token.isIntegralType) + goto case_Declaration; + nT(); + } + } + + FuncBodyStatement parseFunctionBody() + { + while (1) + { + switch (token.kind) + { + case T.LBrace: + skipToTokenAfterClosingBrace(); + break; + case T.Semicolon: + nT(); + break; + case T.In: + nT(); + skipToTokenAfterClosingBrace(); + continue; + case T.Out: + nT(); + if (token.kind == T.LParen) + nT(), nT(), nT(); // ( Identifier ) + skipToTokenAfterClosingBrace(); + continue; + case T.Body: + nT(); + goto case T.LBrace; + default: + } + break; // Exit loop. + } + return null; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/parser/Parser.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/parser/Parser.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,4127 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.parser.Parser; + +import dil.lexer.Lexer; +import dil.ast.Node; +import dil.ast.Declarations; +import dil.ast.Statements; +import dil.ast.Expressions; +import dil.ast.Types; +import dil.ast.Parameters; +import dil.lexer.IdTable; +import dil.Messages; +import dil.Information; +import dil.Enums; +import dil.CompilerInfo; +import dil.SourceText; +import dil.Unicode; +import common; + +/// The Parser produces a full parse tree by examining +/// the list of tokens provided by the Lexer. +class Parser +{ + Lexer lexer; /// Used to lex the source code. + Token* token; /// Current non-whitespace token. + Token* prevToken; /// Previous non-whitespace token. + + InfoManager infoMan; + ParserError[] errors; + + ImportDeclaration[] imports; /// ImportDeclarations in the source text. + + /// Attributes are evaluated in the parsing phase. + /// TODO: will be removed. SemanticPass1 takes care of attributes. + LinkageType linkageType; + Protection protection; /// ditto + StorageClass storageClass; /// ditto + uint alignSize = DEFAULT_ALIGN_SIZE; /// ditto + + private alias TOK T; /// Used often in this class. + private alias TypeNode Type; + + /// Constructs a Parser object. + /// Params: + /// text = the UTF-8 source code. + /// infoMan = used for collecting error messages. + this(SourceText srcText, InfoManager infoMan = null) + { + this.infoMan = infoMan; + lexer = new Lexer(srcText, infoMan); + } + + /// Moves to the first token. + protected void init() + { + nT(); + prevToken = token; + } + + /// Moves to the next token. + void nT() + { + prevToken = token; + do + { + lexer.nextToken(); + token = lexer.token; + } while (token.isWhitespace) // Skip whitespace + } + + /// Start the parser and return the parsed Declarations. + CompoundDeclaration start() + { + init(); + auto begin = token; + auto decls = new CompoundDeclaration; + if (token.kind == T.Module) + decls ~= parseModuleDeclaration(); + decls.addOptChildren(parseDeclarationDefinitions()); + set(decls, begin); + return decls; + } + + /// Start the parser and return the parsed Expression. + Expression start2() + { + init(); + return parseExpression(); + } + + // Members related to the method try_(). + uint trying; /// Greater than 0 if Parser is in try_(). + uint errorCount; /// Used to track nr. of errors while being in try_(). + + /// This method executes the delegate parseMethod and when an error occurred + /// the state of the lexer and parser are restored. + /// Returns: the return value of parseMethod(). + ReturnType try_(ReturnType)(ReturnType delegate() parseMethod, out bool success) + { + // Save members. + auto oldToken = this.token; + auto oldPrevToken = this.prevToken; + auto oldCount = this.errorCount; + + ++trying; + auto result = parseMethod(); + --trying; + // Check if an error occurred. + if (errorCount != oldCount) + { // Restore members. + token = oldToken; + prevToken = oldPrevToken; + lexer.token = oldToken; + errorCount = oldCount; + success = false; + } + else + success = true; + return result; + } + + /// Sets the begin and end tokens of a syntax tree node. + Class set(Class)(Class node, Token* begin) + { + node.setTokens(begin, this.prevToken); + return node; + } + + /// Sets the begin and end tokens of a syntax tree node. + Class set(Class)(Class node, Token* begin, Token* end) + { + node.setTokens(begin, end); + return node; + } + + /// Returns true if set() has been called on a node. + static bool isNodeSet(Node node) + { + return node.begin !is null && node.end !is null; + } + + /// Returns the token kind of the next token. + TOK peekNext() + { + Token* next = token; + do + lexer.peek(next); + while (next.isWhitespace) // Skip whitespace + return next.kind; + } + + /// Returns the token kind of the token that comes after t. + TOK peekAfter(ref Token* t) + { + assert(t !is null); + do + lexer.peek(t); + while (t.isWhitespace) // Skip whitespace + return t.kind; + } + + /// Consumes the current token if its kind matches k and returns true. + bool consumed()(TOK k) // Templatized, so it's inlined. + { + return token.kind == k ? (nT(), true) : false; + } + + /// Asserts that the current token is of kind expectedKind, + /// and then moves to the next token. + void skip()(TOK expectedKind) + { + assert(token.kind == expectedKind /+|| *(int*).init+/, token.srcText()); + nT(); + } + + /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + | Declaration parsing methods | + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ + + Declaration parseModuleDeclaration() + { + skip(T.Module); + auto begin = token; + ModuleFQN moduleFQN; + do + moduleFQN ~= requireIdentifier(MSG.ExpectedModuleIdentifier); + while (consumed(T.Dot)) + require(T.Semicolon); + return set(new ModuleDeclaration(moduleFQN), begin); + } + + /// Parses DeclarationDefinitions until the end of file is hit. + /// $(PRE + /// DeclDefs := + /// DeclDef + /// DeclDefs + /// ) + Declaration[] parseDeclarationDefinitions() + { + Declaration[] decls; + while (token.kind != T.EOF) + decls ~= parseDeclarationDefinition(); + return decls; + } + + /// Parse the body of a template, class, interface, struct or union. + /// $(PRE + /// DeclDefsBlock := + /// { } + /// { DeclDefs } + /// ) + CompoundDeclaration parseDeclarationDefinitionsBody() + { + // Save attributes. + auto linkageType = this.linkageType; + auto protection = this.protection; + auto storageClass = this.storageClass; + // Clear attributes. + this.linkageType = LinkageType.None; + this.protection = Protection.None; + this.storageClass = StorageClass.None; + + // Parse body. + auto begin = token; + auto decls = new CompoundDeclaration; + require(T.LBrace); + while (token.kind != T.RBrace && token.kind != T.EOF) + decls ~= parseDeclarationDefinition(); + require(T.RBrace); + set(decls, begin); + + // Restore original values. + this.linkageType = linkageType; + this.protection = protection; + this.storageClass = storageClass; + + return decls; + } + + /// Parses a DeclarationDefinition. + Declaration parseDeclarationDefinition() + out(decl) + { assert(isNodeSet(decl)); } + body + { + auto begin = token; + Declaration decl; + switch (token.kind) + { + case T.Align, + T.Pragma, + // Protection attributes + T.Export, + T.Private, + T.Package, + T.Protected, + T.Public: + decl = parseAttributeSpecifier(); + break; + // Storage classes + case T.Extern, + T.Deprecated, + T.Override, + T.Abstract, + T.Synchronized, + //T.Static, + T.Final, + T.Const, + //T.Invariant, // D 2.0 + T.Auto, + T.Scope: + case_StaticAttribute: + case_InvariantAttribute: // D 2.0 + return parseStorageAttribute(); + case T.Alias: + nT(); + decl = new AliasDeclaration(parseVariableOrFunction()); + break; + case T.Typedef: + nT(); + decl = new TypedefDeclaration(parseVariableOrFunction()); + break; + case T.Static: + switch (peekNext()) + { + case T.Import: + goto case_Import; + case T.This: + decl = parseStaticConstructorDeclaration(); + break; + case T.Tilde: + decl = parseStaticDestructorDeclaration(); + break; + case T.If: + decl = parseStaticIfDeclaration(); + break; + case T.Assert: + decl = parseStaticAssertDeclaration(); + break; + default: + goto case_StaticAttribute; + } + break; + case T.Import: + case_Import: + decl = parseImportDeclaration(); + imports ~= decl.to!(ImportDeclaration); + // Handle specially. StorageClass mustn't be set. + decl.setProtection(this.protection); + return set(decl, begin); + case T.Enum: + decl = parseEnumDeclaration(); + break; + case T.Class: + decl = parseClassDeclaration(); + break; + case T.Interface: + decl = parseInterfaceDeclaration(); + break; + case T.Struct, T.Union: + decl = parseStructOrUnionDeclaration(); + break; + case T.This: + decl = parseConstructorDeclaration(); + break; + case T.Tilde: + decl = parseDestructorDeclaration(); + break; + case T.Invariant: + version(D2) + { + auto next = token; + if (peekAfter(next) == T.LParen) + { + if (peekAfter(next) != T.RParen) + goto case_Declaration; + } + else + goto case_InvariantAttribute; + } + decl = parseInvariantDeclaration(); + break; + case T.Unittest: + decl = parseUnittestDeclaration(); + break; + case T.Debug: + decl = parseDebugDeclaration(); + break; + case T.Version: + decl = parseVersionDeclaration(); + break; + case T.Template: + decl = parseTemplateDeclaration(); + break; + case T.New: + decl = parseNewDeclaration(); + break; + case T.Delete: + decl = parseDeleteDeclaration(); + break; + case T.Mixin: + decl = parseMixin!(MixinDeclaration)(); + break; + case T.Semicolon: + nT(); + decl = new EmptyDeclaration(); + break; + // Declaration + case T.Identifier, T.Dot, T.Typeof: + case_Declaration: + return parseVariableOrFunction(this.storageClass, this.protection, this.linkageType); + default: + if (token.isIntegralType) + goto case_Declaration; + else if (token.kind == T.Module) + { + decl = parseModuleDeclaration(); + error(begin, MSG.ModuleDeclarationNotFirst); + return decl; + } + + decl = new IllegalDeclaration(); + // Skip to next valid token. + do + nT(); + while (!token.isDeclDefStart && + token.kind != T.RBrace && + token.kind != T.EOF) + auto text = Token.textSpan(begin, this.prevToken); + error(begin, MSG.IllegalDeclaration, text); + } + decl.setProtection(this.protection); + decl.setStorageClass(this.storageClass); + assert(!isNodeSet(decl)); + set(decl, begin); + return decl; + } + + /// Parses a DeclarationsBlock. + /// $(PRE + /// DeclarationsBlock := + /// : DeclDefs + /// { } + /// { DeclDefs } + /// DeclDef + /// ) + Declaration parseDeclarationsBlock(/+bool noColon = false+/) + { + Declaration d; + switch (token.kind) + { + case T.LBrace: + auto begin = token; + nT(); + auto decls = new CompoundDeclaration; + while (token.kind != T.RBrace && token.kind != T.EOF) + decls ~= parseDeclarationDefinition(); + require(T.RBrace); + d = set(decls, begin); + break; + case T.Colon: + // if (noColon == true) + // goto default; + nT(); + auto begin = token; + auto decls = new CompoundDeclaration; + while (token.kind != T.RBrace && token.kind != T.EOF) + decls ~= parseDeclarationDefinition(); + d = set(decls, begin); + break; + default: + d = parseDeclarationDefinition(); + } + assert(isNodeSet(d)); + return d; + } + + // Declaration parseDeclarationsBlockNoColon() + // { + // return parseDeclarationsBlock(true); + // } + + /// Parses either a VariableDeclaration or a FunctionDeclaration. + /// Params: + /// stc = previously parsed storage classes + /// protection = previously parsed protection attribute + /// linkType = previously parsed linkage type + /// testAutoDeclaration = whether to check for an AutoDeclaration + /// optionalParameterList = a hint for how to parse C-style function pointers + Declaration parseVariableOrFunction(StorageClass stc = StorageClass.None, + Protection protection = Protection.None, + LinkageType linkType = LinkageType.None, + bool testAutoDeclaration = false, + bool optionalParameterList = true) + { + auto begin = token; + Type type; + Identifier* name; + + // Check for AutoDeclaration: StorageClasses Identifier = + if (testAutoDeclaration && + token.kind == T.Identifier && + peekNext() == T.Assign) + { + name = token.ident; + skip(T.Identifier); + } + else + { + type = parseType(); // VariableType or ReturnType + if (token.kind == T.LParen) + { + // C-style function pointers make the grammar ambiguous. + // We have to treat them specially at function scope. + // Example: + // void foo() { + // // A pointer to a function taking an integer and returning 'some_type'. + // some_type (*p_func)(int); + // // In the following case precedence is given to a CallExpression. + // something(*p); // 'something' may be a function/method or an object having opCall overloaded. + // } + // // A pointer to a function taking no parameters and returning 'something'. + // something(*p); + type = parseCFunctionPointerType(type, name, optionalParameterList); + } + else if (peekNext() == T.LParen) + { // Type FunctionName ( ParameterList ) FunctionBody + name = requireIdentifier(MSG.ExpectedFunctionName); + name || nT(); // Skip non-identifier token. + assert(token.kind == T.LParen); + // It's a function declaration + TemplateParameters tparams; + if (tokenAfterParenIs(T.LParen)) + // ( TemplateParameterList ) ( ParameterList ) + tparams = parseTemplateParameterList(); + + auto params = parseParameterList(); + version(D2) + { + switch (token.kind) + { + case T.Const: + stc |= StorageClass.Const; + nT(); + break; + case T.Invariant: + stc |= StorageClass.Invariant; + nT(); + break; + default: + } + } + // ReturnType FunctionName ( ParameterList ) + auto funcBody = parseFunctionBody(); + auto fd = new FunctionDeclaration(type, name,/+ tparams,+/ params, funcBody); + fd.setStorageClass(stc); + fd.setLinkageType(linkType); + fd.setProtection(protection); + if (tparams) + { + auto d = putInsideTemplateDeclaration(begin, name, fd, tparams); + d.setStorageClass(stc); + d.setProtection(protection); + return set(d, begin); + } + return set(fd, begin); + } + else + { // Type VariableName DeclaratorSuffix + name = requireIdentifier(MSG.ExpectedVariableName); + type = parseDeclaratorSuffix(type); + } + } + + // It's a variables declaration. + Identifier*[] names = [name]; // One identifier has been parsed already. + Expression[] values; + goto LenterLoop; // Enter the loop and check for an initializer. + while (consumed(T.Comma)) + { + names ~= requireIdentifier(MSG.ExpectedVariableName); + LenterLoop: + if (consumed(T.Assign)) + values ~= parseInitializer(); + else + values ~= null; + } + require(T.Semicolon); + auto d = new VariablesDeclaration(type, names, values); + d.setStorageClass(stc); + d.setLinkageType(linkType); + d.setProtection(protection); + return set(d, begin); + } + + /// Parses a variable initializer. + Expression parseInitializer() + { + if (token.kind == T.Void) + { + auto begin = token; + auto next = peekNext(); + if (next == T.Comma || next == T.Semicolon) + { + skip(T.Void); + return set(new VoidInitExpression(), begin); + } + } + return parseNonVoidInitializer(); + } + + Expression parseNonVoidInitializer() + { + auto begin = token; + Expression init; + switch (token.kind) + { + case T.LBracket: + // ArrayInitializer: + // [ ] + // [ ArrayMemberInitializations ] + Expression[] keys; + Expression[] values; + + skip(T.LBracket); + while (token.kind != T.RBracket) + { + auto e = parseNonVoidInitializer(); + if (consumed(T.Colon)) + { + keys ~= e; + values ~= parseNonVoidInitializer(); + } + else + { + keys ~= null; + values ~= e; + } + + if (!consumed(T.Comma)) + break; + } + require(T.RBracket); + init = new ArrayInitExpression(keys, values); + break; + case T.LBrace: + // StructInitializer: + // { } + // { StructMemberInitializers } + Expression parseStructInitializer() + { + Identifier*[] idents; + Expression[] values; + + skip(T.LBrace); + while (token.kind != T.RBrace) + { + if (token.kind == T.Identifier && + // Peek for colon to see if this is a member identifier. + peekNext() == T.Colon) + { + idents ~= token.ident; + skip(T.Identifier), skip(T.Colon); + } + else + idents ~= null; + + // NonVoidInitializer + values ~= parseNonVoidInitializer(); + + if (!consumed(T.Comma)) + break; + } + require(T.RBrace); + return new StructInitExpression(idents, values); + } + + bool success; + auto si = try_(&parseStructInitializer, success); + if (success) + { + init = si; + break; + } + assert(token.kind == T.LBrace); + //goto default; + default: + init = parseAssignExpression(); + } + set(init, begin); + return init; + } + + FuncBodyStatement parseFunctionBody() + { + auto begin = token; + auto func = new FuncBodyStatement; + while (1) + { + switch (token.kind) + { + case T.LBrace: + func.funcBody = parseStatements(); + break; + case T.Semicolon: + nT(); + break; + case T.In: + if (func.inBody) + error(MID.InContract); + nT(); + func.inBody = parseStatements(); + continue; + case T.Out: + if (func.outBody) + error(MID.OutContract); + nT(); + if (consumed(T.LParen)) + { + func.outIdent = requireIdentifier(MSG.ExpectedAnIdentifier); + require(T.RParen); + } + func.outBody = parseStatements(); + continue; + case T.Body: + nT(); + goto case T.LBrace; + default: + error(token, MSG.ExpectedFunctionBody, token.srcText); + } + break; // Exit loop. + } + set(func, begin); + func.finishConstruction(); + return func; + } + + LinkageType parseLinkageType() + { + LinkageType linkageType; + + if (!consumed(T.LParen)) + return linkageType; + + if (consumed(T.RParen)) + { // extern() + error(MID.MissingLinkageType); + return linkageType; + } + + auto identTok = requireId(); + + IDK idKind = identTok ? identTok.ident.idKind : IDK.Null; + + switch (idKind) + { + case IDK.C: + if (consumed(T.PlusPlus)) + { + linkageType = LinkageType.Cpp; + break; + } + linkageType = LinkageType.C; + break; + case IDK.D: + linkageType = LinkageType.D; + break; + case IDK.Windows: + linkageType = LinkageType.Windows; + break; + case IDK.Pascal: + linkageType = LinkageType.Pascal; + break; + case IDK.System: + linkageType = LinkageType.System; + break; + default: + error(MID.UnrecognizedLinkageType, token.srcText); + } + require(T.RParen); + return linkageType; + } + + void checkLinkageType(ref LinkageType prev_lt, LinkageType lt, Token* begin) + { + if (prev_lt == LinkageType.None) + prev_lt = lt; + else + error(begin, MSG.RedundantLinkageType, Token.textSpan(begin, this.prevToken)); + } + + Declaration parseStorageAttribute() + { + StorageClass stc, stc_tmp; + LinkageType prev_linkageType; + + auto saved_storageClass = this.storageClass; // Save. + // Nested function. + Declaration parse() + { + Declaration decl; + auto begin = token; + switch (token.kind) + { + case T.Extern: + if (peekNext() != T.LParen) + { + stc_tmp = StorageClass.Extern; + goto Lcommon; + } + + nT(); + auto linkageType = parseLinkageType(); + checkLinkageType(prev_linkageType, linkageType, begin); + + auto saved = this.linkageType; // Save. + this.linkageType = linkageType; // Set. + decl = new LinkageDeclaration(linkageType, parse()); + set(decl, begin); + this.linkageType = saved; // Restore. + break; + case T.Override: + stc_tmp = StorageClass.Override; + goto Lcommon; + case T.Deprecated: + stc_tmp = StorageClass.Deprecated; + goto Lcommon; + case T.Abstract: + stc_tmp = StorageClass.Abstract; + goto Lcommon; + case T.Synchronized: + stc_tmp = StorageClass.Synchronized; + goto Lcommon; + case T.Static: + stc_tmp = StorageClass.Static; + goto Lcommon; + case T.Final: + stc_tmp = StorageClass.Final; + goto Lcommon; + case T.Const: + version(D2) + { + if (peekNext() == T.LParen) + goto case_Declaration; + } + stc_tmp = StorageClass.Const; + goto Lcommon; + version(D2) + { + case T.Invariant: // D 2.0 + auto next = token; + if (peekAfter(next) == T.LParen) + { + if (peekAfter(next) != T.RParen) + goto case_Declaration; // invariant ( Type ) + decl = parseDeclarationDefinition(); // invariant ( ) + decl.setStorageClass(stc); + break; + } + // invariant as StorageClass. + stc_tmp = StorageClass.Invariant; + goto Lcommon; + } + case T.Auto: + stc_tmp = StorageClass.Auto; + goto Lcommon; + case T.Scope: + stc_tmp = StorageClass.Scope; + goto Lcommon; + Lcommon: + // Issue error if redundant. + if (stc & stc_tmp) + error(MID.RedundantStorageClass, token.srcText); + else + stc |= stc_tmp; + + nT(); + decl = new StorageClassDeclaration(stc_tmp, parse()); + set(decl, begin); + break; + case T.Identifier: + case_Declaration: + // This could be a normal Declaration or an AutoDeclaration + decl = parseVariableOrFunction(stc, this.protection, prev_linkageType, true); + break; + default: + this.storageClass = stc; // Set. + decl = parseDeclarationsBlock(); + this.storageClass = saved_storageClass; // Reset. + } + assert(isNodeSet(decl)); + return decl; + } + return parse(); + } + + uint parseAlignAttribute() + { + skip(T.Align); + uint size = DEFAULT_ALIGN_SIZE; // Global default. + if (consumed(T.LParen)) + { + if (token.kind == T.Int32) + (size = token.int_), skip(T.Int32); + else + expected(T.Int32); + require(T.RParen); + } + return size; + } + + Declaration parseAttributeSpecifier() + { + Declaration decl; + + switch (token.kind) + { + case T.Align: + uint alignSize = parseAlignAttribute(); + auto saved = this.alignSize; // Save. + this.alignSize = alignSize; // Set. + decl = new AlignDeclaration(alignSize, parseDeclarationsBlock()); + this.alignSize = saved; // Restore. + break; + case T.Pragma: + // Pragma: + // pragma ( Identifier ) + // pragma ( Identifier , ExpressionList ) + nT(); + Identifier* ident; + Expression[] args; + + require(T.LParen); + ident = requireIdentifier(MSG.ExpectedPragmaIdentifier); + + if (consumed(T.Comma)) + args = parseExpressionList(); + require(T.RParen); + + decl = new PragmaDeclaration(ident, args, parseDeclarationsBlock()); + break; + default: + // Protection attributes + Protection prot; + switch (token.kind) + { + case T.Private: + prot = Protection.Private; break; + case T.Package: + prot = Protection.Package; break; + case T.Protected: + prot = Protection.Protected; break; + case T.Public: + prot = Protection.Public; break; + case T.Export: + prot = Protection.Export; break; + default: + assert(0); + } + nT(); + auto saved = this.protection; // Save. + this.protection = prot; // Set. + decl = new ProtectionDeclaration(prot, parseDeclarationsBlock()); + this.protection = saved; // Restore. + } + return decl; + } + + Declaration parseImportDeclaration() + { + bool isStatic = consumed(T.Static); + skip(T.Import); + + ModuleFQN[] moduleFQNs; + Identifier*[] moduleAliases; + Identifier*[] bindNames; + Identifier*[] bindAliases; + + do + { + ModuleFQN moduleFQN; + Identifier* moduleAlias; + // AliasName = ModuleName + if (peekNext() == T.Assign) + { + moduleAlias = requireIdentifier(MSG.ExpectedAliasModuleName); + skip(T.Assign); + } + // Identifier ("." Identifier)* + do + moduleFQN ~= requireIdentifier(MSG.ExpectedModuleIdentifier); + while (consumed(T.Dot)) + // Push identifiers. + moduleFQNs ~= moduleFQN; + moduleAliases ~= moduleAlias; + } while (consumed(T.Comma)) + + if (consumed(T.Colon)) + { // BindAlias "=" BindName ("," BindAlias "=" BindName)*; + // BindName ("," BindName)*; + do + { + Identifier* bindAlias; + // BindAlias = BindName + if (peekNext() == T.Assign) + { + bindAlias = requireIdentifier(MSG.ExpectedAliasImportName); + skip(T.Assign); + } + // Push identifiers. + bindNames ~= requireIdentifier(MSG.ExpectedImportName); + bindAliases ~= bindAlias; + } while (consumed(T.Comma)) + } + require(T.Semicolon); + + return new ImportDeclaration(moduleFQNs, moduleAliases, bindNames, bindAliases, isStatic); + } + + Declaration parseEnumDeclaration() + { + skip(T.Enum); + + Identifier* enumName; + Type baseType; + EnumMemberDeclaration[] members; + bool hasBody; + + enumName = optionalIdentifier(); + + if (consumed(T.Colon)) + baseType = parseBasicType(); + + if (enumName && consumed(T.Semicolon)) + {} + else if (consumed(T.LBrace)) + { + hasBody = true; + while (token.kind != T.RBrace) + { + auto begin = token; + auto name = requireIdentifier(MSG.ExpectedEnumMember); + Expression value; + + if (consumed(T.Assign)) + value = parseAssignExpression(); + else + value = null; + + members ~= set(new EnumMemberDeclaration(name, value), begin); + + if (!consumed(T.Comma)) + break; + } + require(T.RBrace); + } + else + error(token, MSG.ExpectedEnumBody, token.srcText); + + return new EnumDeclaration(enumName, baseType, members, hasBody); + } + + /// Wraps a declaration inside a template declaration. + /// Params: + /// begin = begin token of decl. + /// name = name of decl. + /// decl = the declaration to be wrapped. + /// tparams = the template parameters. + TemplateDeclaration putInsideTemplateDeclaration(Token* begin, + Identifier* name, + Declaration decl, + TemplateParameters tparams) + { + set(decl, begin); + auto cd = new CompoundDeclaration; + cd ~= decl; + set(cd, begin); + return new TemplateDeclaration(name, tparams, cd); + } + + Declaration parseClassDeclaration() + { + auto begin = token; + skip(T.Class); + + Identifier* className; + TemplateParameters tparams; + BaseClassType[] bases; + CompoundDeclaration decls; + + className = requireIdentifier(MSG.ExpectedClassName); + + if (token.kind == T.LParen) + tparams = parseTemplateParameterList(); + + if (token.kind == T.Colon) + bases = parseBaseClasses(); + + if (bases.length == 0 && consumed(T.Semicolon)) + {} + else if (token.kind == T.LBrace) + decls = parseDeclarationDefinitionsBody(); + else + error(token, MSG.ExpectedClassBody, token.srcText); + + Declaration d = new ClassDeclaration(className, /+tparams, +/bases, decls); + if (tparams) + d = putInsideTemplateDeclaration(begin, className, d, tparams); + return d; + } + + BaseClassType[] parseBaseClasses(bool colonLeadsOff = true) + { + colonLeadsOff && skip(T.Colon); + + BaseClassType[] bases; + do + { + Protection prot = Protection.Public; + switch (token.kind) + { + case T.Identifier, T.Dot, T.Typeof: goto LparseBasicType; + case T.Private: prot = Protection.Private; break; + case T.Protected: prot = Protection.Protected; break; + case T.Package: prot = Protection.Package; break; + case T.Public: /*prot = Protection.Public;*/ break; + default: + error(MID.ExpectedBaseClasses, token.srcText); + return bases; + } + nT(); // Skip protection attribute. + LparseBasicType: + auto begin = token; + auto type = parseBasicType(); + bases ~= set(new BaseClassType(prot, type), begin); + } while (consumed(T.Comma)) + return bases; + } + + Declaration parseInterfaceDeclaration() + { + auto begin = token; + skip(T.Interface); + + Identifier* name; + TemplateParameters tparams; + BaseClassType[] bases; + CompoundDeclaration decls; + + name = requireIdentifier(MSG.ExpectedInterfaceName); + + if (token.kind == T.LParen) + tparams = parseTemplateParameterList(); + + if (token.kind == T.Colon) + bases = parseBaseClasses(); + + if (bases.length == 0 && consumed(T.Semicolon)) + {} + else if (token.kind == T.LBrace) + decls = parseDeclarationDefinitionsBody(); + else + error(token, MSG.ExpectedInterfaceBody, token.srcText); + + Declaration d = new InterfaceDeclaration(name, /+tparams, +/bases, decls); + if (tparams) + d = putInsideTemplateDeclaration(begin, name, d, tparams); + return d; + } + + Declaration parseStructOrUnionDeclaration() + { + assert(token.kind == T.Struct || token.kind == T.Union); + auto begin = token; + skip(token.kind); + + Identifier* name; + TemplateParameters tparams; + CompoundDeclaration decls; + + name = optionalIdentifier(); + + if (name && token.kind == T.LParen) + tparams = parseTemplateParameterList(); + + if (name && consumed(T.Semicolon)) + {} + else if (token.kind == T.LBrace) + decls = parseDeclarationDefinitionsBody(); + else + error(token, begin.kind == T.Struct ? + MSG.ExpectedStructBody : + MSG.ExpectedUnionBody, token.srcText); + + Declaration d; + if (begin.kind == T.Struct) + { + auto sd = new StructDeclaration(name, /+tparams, +/decls); + sd.setAlignSize(this.alignSize); + d = sd; + } + else + d = new UnionDeclaration(name, /+tparams, +/decls); + + if (tparams) + d = putInsideTemplateDeclaration(begin, name, d, tparams); + return d; + } + + Declaration parseConstructorDeclaration() + { + skip(T.This); + auto parameters = parseParameterList(); + auto funcBody = parseFunctionBody(); + return new ConstructorDeclaration(parameters, funcBody); + } + + Declaration parseDestructorDeclaration() + { + skip(T.Tilde); + require(T.This); + require(T.LParen); + require(T.RParen); + auto funcBody = parseFunctionBody(); + return new DestructorDeclaration(funcBody); + } + + Declaration parseStaticConstructorDeclaration() + { + skip(T.Static); + skip(T.This); + require(T.LParen); + require(T.RParen); + auto funcBody = parseFunctionBody(); + return new StaticConstructorDeclaration(funcBody); + } + + Declaration parseStaticDestructorDeclaration() + { + skip(T.Static); + skip(T.Tilde); + require(T.This); + require(T.LParen); + require(T.RParen); + auto funcBody = parseFunctionBody(); + return new StaticDestructorDeclaration(funcBody); + } + + Declaration parseInvariantDeclaration() + { + skip(T.Invariant); + // Optional () for getting ready porting to D 2.0 + if (consumed(T.LParen)) + require(T.RParen); + auto funcBody = parseFunctionBody(); + return new InvariantDeclaration(funcBody); + } + + Declaration parseUnittestDeclaration() + { + skip(T.Unittest); + auto funcBody = parseFunctionBody(); + return new UnittestDeclaration(funcBody); + } + + Token* parseIdentOrInt() + { + if (consumed(T.Int32) || consumed(T.Identifier)) + return this.prevToken; + error(token, MSG.ExpectedIdentOrInt, token.srcText); + return null; + } + + Declaration parseDebugDeclaration() + { + skip(T.Debug); + + Token* spec; + Token* cond; + Declaration decls, elseDecls; + + if (consumed(T.Assign)) + { // debug = Integer ; + // debug = Identifier ; + spec = parseIdentOrInt(); + require(T.Semicolon); + } + else + { // ( Condition ) + if (consumed(T.LParen)) + { + cond = parseIdentOrInt(); + require(T.RParen); + } + // debug DeclarationsBlock + // debug ( Condition ) DeclarationsBlock + decls = parseDeclarationsBlock(); + // else DeclarationsBlock + if (consumed(T.Else)) + elseDecls = parseDeclarationsBlock(); + } + + return new DebugDeclaration(spec, cond, decls, elseDecls); + } + + Declaration parseVersionDeclaration() + { + skip(T.Version); + + Token* spec; + Token* cond; + Declaration decls, elseDecls; + + if (consumed(T.Assign)) + { // version = Integer ; + // version = Identifier ; + spec = parseIdentOrInt(); + require(T.Semicolon); + } + else + { // ( Condition ) + require(T.LParen); + cond = parseIdentOrInt(); + require(T.RParen); + // version ( Condition ) DeclarationsBlock + decls = parseDeclarationsBlock(); + // else DeclarationsBlock + if (consumed(T.Else)) + elseDecls = parseDeclarationsBlock(); + } + + return new VersionDeclaration(spec, cond, decls, elseDecls); + } + + Declaration parseStaticIfDeclaration() + { + skip(T.Static); + skip(T.If); + + Expression condition; + Declaration ifDecls, elseDecls; + + require(T.LParen); + condition = parseAssignExpression(); + require(T.RParen); + + ifDecls = parseDeclarationsBlock(); + + if (consumed(T.Else)) + elseDecls = parseDeclarationsBlock(); + + return new StaticIfDeclaration(condition, ifDecls, elseDecls); + } + + Declaration parseStaticAssertDeclaration() + { + skip(T.Static); + skip(T.Assert); + Expression condition, message; + require(T.LParen); + condition = parseAssignExpression(); + if (consumed(T.Comma)) + message = parseAssignExpression(); + require(T.RParen); + require(T.Semicolon); + return new StaticAssertDeclaration(condition, message); + } + + Declaration parseTemplateDeclaration() + { + skip(T.Template); + auto templateName = requireIdentifier(MSG.ExpectedTemplateName); + auto templateParams = parseTemplateParameterList(); + auto decls = parseDeclarationDefinitionsBody(); + return new TemplateDeclaration(templateName, templateParams, decls); + } + + Declaration parseNewDeclaration() + { + skip(T.New); + auto parameters = parseParameterList(); + auto funcBody = parseFunctionBody(); + return new NewDeclaration(parameters, funcBody); + } + + Declaration parseDeleteDeclaration() + { + skip(T.Delete); + auto parameters = parseParameterList(); + auto funcBody = parseFunctionBody(); + return new DeleteDeclaration(parameters, funcBody); + } + + Type parseTypeofType() + { + auto begin = token; + skip(T.Typeof); + require(T.LParen); + Type type; + switch (token.kind) + { + version(D2) + { + case T.Return: + nT(); + type = new TypeofType(); + break; + } + default: + type = new TypeofType(parseExpression()); + } + require(T.RParen); + set(type, begin); + return type; + } + + /// Parses a MixinDeclaration or MixinStatement. + /// $(PRE + /// TemplateMixin := + /// mixin ( AssignExpression ) ; + /// mixin TemplateIdentifier ; + /// mixin TemplateIdentifier MixinIdentifier ; + /// mixin TemplateIdentifier !( TemplateArguments ) ; + /// mixin TemplateIdentifier !( TemplateArguments ) MixinIdentifier ; + /// ) + Class parseMixin(Class)() + { + static assert(is(Class == MixinDeclaration) || is(Class == MixinStatement)); + skip(T.Mixin); + + static if (is(Class == MixinDeclaration)) + { + if (consumed(T.LParen)) + { + auto e = parseAssignExpression(); + require(T.RParen); + require(T.Semicolon); + return new MixinDeclaration(e); + } + } + + auto begin = token; + Expression e; + Identifier* mixinIdent; + + if (consumed(T.Dot)) + e = set(new ModuleScopeExpression(parseIdentifierExpression()), begin); + else + e = parseIdentifierExpression(); + + while (consumed(T.Dot)) + e = set(new DotExpression(e, parseIdentifierExpression()), begin); + + mixinIdent = optionalIdentifier(); + require(T.Semicolon); + + return new Class(e, mixinIdent); + } + + /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + | Statement parsing methods | + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ + + CompoundStatement parseStatements() + { + auto begin = token; + require(T.LBrace); + auto statements = new CompoundStatement(); + while (token.kind != T.RBrace && token.kind != T.EOF) + statements ~= parseStatement(); + require(T.RBrace); + return set(statements, begin); + } + + /// Parses a Statement. + Statement parseStatement() + { + auto begin = token; + Statement s; + Declaration d; + + if (token.isIntegralType) + { + d = parseVariableOrFunction(); + goto LreturnDeclarationStatement; + } + + switch (token.kind) + { + case T.Align: + uint size = parseAlignAttribute(); + // Restrict align attribute to structs in parsing phase. + StructDeclaration structDecl; + if (token.kind == T.Struct) + { + auto begin2 = token; + structDecl = parseStructOrUnionDeclaration().to!(StructDeclaration); + structDecl.setAlignSize(size); + set(structDecl, begin2); + } + else + expected(T.Struct); + + d = new AlignDeclaration(size, structDecl ? cast(Declaration)structDecl : new CompoundDeclaration); + goto LreturnDeclarationStatement; + /+ Not applicable for statements. + T.Private, T.Package, T.Protected, T.Public, T.Export, + T.Deprecated, T.Override, T.Abstract,+/ + case T.Extern, + T.Final, + T.Const, + T.Auto: + //T.Scope + //T.Static + case_parseAttribute: + s = parseAttributeStatement(); + return s; + case T.Identifier: + if (peekNext() == T.Colon) + { + auto ident = token.ident; + skip(T.Identifier); skip(T.Colon); + s = new LabeledStatement(ident, parseNoScopeOrEmptyStatement()); + break; + } + goto case T.Dot; + case T.Dot, T.Typeof: + bool success; + d = try_(delegate { + return parseVariableOrFunction(StorageClass.None, + Protection.None, + LinkageType.None, false, false); + }, success + ); + if (success) + goto LreturnDeclarationStatement; // Declaration + else + goto case_parseExpressionStatement; // Expression + + case T.If: + s = parseIfStatement(); + break; + case T.While: + s = parseWhileStatement(); + break; + case T.Do: + s = parseDoWhileStatement(); + break; + case T.For: + s = parseForStatement(); + break; + case T.Foreach, T.Foreach_reverse: + s = parseForeachStatement(); + break; + case T.Switch: + s = parseSwitchStatement(); + break; + case T.Case: + s = parseCaseStatement(); + break; + case T.Default: + s = parseDefaultStatement(); + break; + case T.Continue: + s = parseContinueStatement(); + break; + case T.Break: + s = parseBreakStatement(); + break; + case T.Return: + s = parseReturnStatement(); + break; + case T.Goto: + s = parseGotoStatement(); + break; + case T.With: + s = parseWithStatement(); + break; + case T.Synchronized: + s = parseSynchronizedStatement(); + break; + case T.Try: + s = parseTryStatement(); + break; + case T.Throw: + s = parseThrowStatement(); + break; + case T.Scope: + if (peekNext() != T.LParen) + goto case_parseAttribute; + s = parseScopeGuardStatement(); + break; + case T.Volatile: + s = parseVolatileStatement(); + break; + case T.Asm: + s = parseAsmBlockStatement(); + break; + case T.Pragma: + s = parsePragmaStatement(); + break; + case T.Mixin: + if (peekNext() == T.LParen) + goto case_parseExpressionStatement; // Parse as expression. + s = parseMixin!(MixinStatement)(); + break; + case T.Static: + switch (peekNext()) + { + case T.If: + s = parseStaticIfStatement(); + break; + case T.Assert: + s = parseStaticAssertStatement(); + break; + default: + goto case_parseAttribute; + } + break; + case T.Debug: + s = parseDebugStatement(); + break; + case T.Version: + s = parseVersionStatement(); + break; + // DeclDef + case T.Alias, T.Typedef: + d = parseDeclarationDefinition(); + goto LreturnDeclarationStatement; + case T.Enum: + d = parseEnumDeclaration(); + goto LreturnDeclarationStatement; + case T.Class: + d = parseClassDeclaration(); + goto LreturnDeclarationStatement; + case T.Interface: + d = parseInterfaceDeclaration(); + goto LreturnDeclarationStatement; + case T.Struct, T.Union: + d = parseStructOrUnionDeclaration(); + // goto LreturnDeclarationStatement; + LreturnDeclarationStatement: + set(d, begin); + s = new DeclarationStatement(d); + break; + case T.LBrace: + s = parseScopeStatement(); + break; + case T.Semicolon: + nT(); + s = new EmptyStatement(); + break; + // Parse an ExpressionStatement: + // Tokens that start a PrimaryExpression. + // case T.Identifier, T.Dot, T.Typeof: + case T.This: + case T.Super: + case T.Null: + case T.True, T.False: + // case T.Dollar: + case T.Int32, T.Int64, T.Uint32, T.Uint64: + case T.Float32, T.Float64, T.Float80, + T.Imaginary32, T.Imaginary64, T.Imaginary80: + case T.CharLiteral: + case T.String: + case T.LBracket: + // case T.LBrace: + case T.Function, T.Delegate: + case T.Assert: + // case T.Mixin: + case T.Import: + case T.Typeid: + case T.Is: + case T.LParen: + case T.Traits: // D2.0 + // Tokens that can start a UnaryExpression: + case T.AndBinary, T.PlusPlus, T.MinusMinus, T.Mul, T.Minus, + T.Plus, T.Not, T.Tilde, T.New, T.Delete, T.Cast: + case_parseExpressionStatement: + s = new ExpressionStatement(parseExpression()); + require(T.Semicolon); + break; + default: + if (token.isSpecialToken) + goto case_parseExpressionStatement; + + if (token.kind != T.Dollar) + // Assert that this isn't a valid expression. + assert(delegate bool(){ + bool success; + auto expression = try_(&parseExpression, success); + return success; + }() == false, "Didn't expect valid expression." + ); + + // Report error: it's an illegal statement. + s = new IllegalStatement(); + // Skip to next valid token. + do + nT(); + while (!token.isStatementStart && + token.kind != T.RBrace && + token.kind != T.EOF) + auto text = Token.textSpan(begin, this.prevToken); + error(begin, MSG.IllegalStatement, text); + } + assert(s !is null); + set(s, begin); + return s; + } + + /// $(PRE + /// Parses a ScopeStatement. + /// ScopeStatement := + /// NoScopeStatement + /// ) + Statement parseScopeStatement() + { + return new ScopeStatement(parseNoScopeStatement()); + } + + /// $(PRE + /// NoScopeStatement := + /// NonEmptyStatement + /// BlockStatement + /// BlockStatement := + /// { } + /// { StatementList } + /// ) + Statement parseNoScopeStatement() + { + auto begin = token; + Statement s; + if (consumed(T.LBrace)) + { + auto ss = new CompoundStatement(); + while (token.kind != T.RBrace && token.kind != T.EOF) + ss ~= parseStatement(); + require(T.RBrace); + s = set(ss, begin); + } + else if (token.kind == T.Semicolon) + { + error(token, MSG.ExpectedNonEmptyStatement); + nT(); + s = set(new EmptyStatement(), begin); + } + else + s = parseStatement(); + return s; + } + + /// $(PRE + /// NoScopeOrEmptyStatement := + /// ; + /// NoScopeStatement + /// ) + Statement parseNoScopeOrEmptyStatement() + { + if (consumed(T.Semicolon)) + return set(new EmptyStatement(), this.prevToken); + else + return parseNoScopeStatement(); + } + + Statement parseAttributeStatement() + { + StorageClass stc, stc_tmp; + LinkageType prev_linkageType; + + Declaration parse() // Nested function. + { + auto begin = token; + Declaration d; + switch (token.kind) + { + case T.Extern: + if (peekNext() != T.LParen) + { + stc_tmp = StorageClass.Extern; + goto Lcommon; + } + + nT(); + auto linkageType = parseLinkageType(); + checkLinkageType(prev_linkageType, linkageType, begin); + + d = new LinkageDeclaration(linkageType, parse()); + break; + case T.Static: + stc_tmp = StorageClass.Static; + goto Lcommon; + case T.Final: + stc_tmp = StorageClass.Final; + goto Lcommon; + case T.Const: + version(D2) + { + if (peekNext() == T.LParen) + goto case_Declaration; + } + stc_tmp = StorageClass.Const; + goto Lcommon; + version(D2) + { + case T.Invariant: // D 2.0 + if (peekNext() == T.LParen) + goto case_Declaration; + stc_tmp = StorageClass.Invariant; + goto Lcommon; + } + case T.Auto: + stc_tmp = StorageClass.Auto; + goto Lcommon; + case T.Scope: + stc_tmp = StorageClass.Scope; + goto Lcommon; + Lcommon: + // Issue error if redundant. + if (stc & stc_tmp) + error(MID.RedundantStorageClass, token.srcText); + else + stc |= stc_tmp; + + nT(); + d = new StorageClassDeclaration(stc_tmp, parse()); + break; + // TODO: allow "scope class", "abstract scope class" in function bodies? + //case T.Class: + default: + case_Declaration: + return parseVariableOrFunction(stc, Protection.None, prev_linkageType, true); + } + return set(d, begin); + } + return new DeclarationStatement(parse()); + } + + Statement parseIfStatement() + { + skip(T.If); + + Statement variable; + Expression condition; + Statement ifBody, elseBody; + + require(T.LParen); + + Identifier* ident; + auto begin = token; // For start of AutoDeclaration or normal Declaration. + // auto Identifier = Expression + if (consumed(T.Auto)) + { + ident = requireIdentifier(MSG.ExpectedVariableName); + require(T.Assign); + auto init = parseExpression(); + auto v = new VariablesDeclaration(null, [ident], [init]); + set(v, begin.nextNWS); + auto d = new StorageClassDeclaration(StorageClass.Auto, v); + set(d, begin); + variable = new DeclarationStatement(d); + set(variable, begin); + } + else + { // Declarator = Expression + Type parseDeclaratorAssign() + { + auto type = parseDeclarator(ident); + require(T.Assign); + return type; + } + bool success; + auto type = try_(&parseDeclaratorAssign, success); + if (success) + { + auto init = parseExpression(); + auto v = new VariablesDeclaration(type, [ident], [init]); + set(v, begin); + variable = new DeclarationStatement(v); + set(variable, begin); + } + else + condition = parseExpression(); + } + require(T.RParen); + ifBody = parseScopeStatement(); + if (consumed(T.Else)) + elseBody = parseScopeStatement(); + return new IfStatement(variable, condition, ifBody, elseBody); + } + + Statement parseWhileStatement() + { + skip(T.While); + require(T.LParen); + auto condition = parseExpression(); + require(T.RParen); + return new WhileStatement(condition, parseScopeStatement()); + } + + Statement parseDoWhileStatement() + { + skip(T.Do); + auto doBody = parseScopeStatement(); + require(T.While); + require(T.LParen); + auto condition = parseExpression(); + require(T.RParen); + return new DoWhileStatement(condition, doBody); + } + + Statement parseForStatement() + { + skip(T.For); + + Statement init, forBody; + Expression condition, increment; + + require(T.LParen); + if (!consumed(T.Semicolon)) + init = parseNoScopeStatement(); + if (token.kind != T.Semicolon) + condition = parseExpression(); + require(T.Semicolon); + if (token.kind != T.RParen) + increment = parseExpression(); + require(T.RParen); + forBody = parseScopeStatement(); + return new ForStatement(init, condition, increment, forBody); + } + + Statement parseForeachStatement() + { + assert(token.kind == T.Foreach || token.kind == T.Foreach_reverse); + TOK tok = token.kind; + nT(); + + auto params = new Parameters; + Expression e; // Aggregate or LwrExpression + + require(T.LParen); + auto paramsBegin = token; + do + { + auto paramBegin = token; + StorageClass stc; + Type type; + Identifier* ident; + + switch (token.kind) + { + case T.Ref, T.Inout: + stc = StorageClass.Ref; + nT(); + // fall through + case T.Identifier: + auto next = peekNext(); + if (next == T.Comma || next == T.Semicolon || next == T.RParen) + { + ident = requireIdentifier(MSG.ExpectedVariableName); + break; + } + // fall through + default: + type = parseDeclarator(ident); + } + + params ~= set(new Parameter(stc, type, ident, null), paramBegin); + } while (consumed(T.Comma)) + set(params, paramsBegin); + require(T.Semicolon); + e = parseExpression(); + version(D2) + { //Foreach (ForeachType; LwrExpression .. UprExpression ) ScopeStatement + if (consumed(T.Slice)) + { + // if (params.length != 1) + // error(MID.XYZ); // TODO: issue error msg + auto upper = parseExpression(); + require(T.RParen); + auto forBody = parseScopeStatement(); + return new ForeachRangeStatement(tok, params, e, upper, forBody); + } + } + // Foreach (ForeachTypeList; Aggregate) ScopeStatement + require(T.RParen); + auto forBody = parseScopeStatement(); + return new ForeachStatement(tok, params, e, forBody); + } + + Statement parseSwitchStatement() + { + skip(T.Switch); + require(T.LParen); + auto condition = parseExpression(); + require(T.RParen); + auto switchBody = parseScopeStatement(); + return new SwitchStatement(condition, switchBody); + } + + /// Helper function for parsing the body of a default or case statement. + Statement parseCaseOrDefaultBody() + { + // This function is similar to parseNoScopeStatement() + auto begin = token; + auto s = new CompoundStatement(); + while (token.kind != T.Case && + token.kind != T.Default && + token.kind != T.RBrace && + token.kind != T.EOF) + s ~= parseStatement(); + set(s, begin); + return set(new ScopeStatement(s), begin); + } + + Statement parseCaseStatement() + { + skip(T.Case); + auto values = parseExpressionList(); + require(T.Colon); + auto caseBody = parseCaseOrDefaultBody(); + return new CaseStatement(values, caseBody); + } + + Statement parseDefaultStatement() + { + skip(T.Default); + require(T.Colon); + auto defaultBody = parseCaseOrDefaultBody(); + return new DefaultStatement(defaultBody); + } + + Statement parseContinueStatement() + { + skip(T.Continue); + auto ident = optionalIdentifier(); + require(T.Semicolon); + return new ContinueStatement(ident); + } + + Statement parseBreakStatement() + { + skip(T.Break); + auto ident = optionalIdentifier(); + require(T.Semicolon); + return new BreakStatement(ident); + } + + Statement parseReturnStatement() + { + skip(T.Return); + Expression expr; + if (token.kind != T.Semicolon) + expr = parseExpression(); + require(T.Semicolon); + return new ReturnStatement(expr); + } + + Statement parseGotoStatement() + { + skip(T.Goto); + Identifier* ident; + Expression caseExpr; + switch (token.kind) + { + case T.Case: + ident = token.ident; + nT(); + if (token.kind == T.Semicolon) + break; + caseExpr = parseExpression(); + break; + case T.Default: + ident = token.ident; + nT(); + break; + default: + ident = requireIdentifier(MSG.ExpectedAnIdentifier); + } + require(T.Semicolon); + return new GotoStatement(ident, caseExpr); + } + + Statement parseWithStatement() + { + skip(T.With); + require(T.LParen); + auto expr = parseExpression(); + require(T.RParen); + return new WithStatement(expr, parseScopeStatement()); + } + + Statement parseSynchronizedStatement() + { + skip(T.Synchronized); + Expression expr; + if (consumed(T.LParen)) + { + expr = parseExpression(); + require(T.RParen); + } + return new SynchronizedStatement(expr, parseScopeStatement()); + } + + Statement parseTryStatement() + { + auto begin = token; + skip(T.Try); + + auto tryBody = parseScopeStatement(); + CatchStatement[] catchBodies; + FinallyStatement finBody; + + while (consumed(T.Catch)) + { + Parameter param; + if (consumed(T.LParen)) + { + auto begin2 = token; + Identifier* ident; + auto type = parseDeclarator(ident, true); + param = new Parameter(StorageClass.None, type, ident, null); + set(param, begin2); + require(T.RParen); + } + catchBodies ~= set(new CatchStatement(param, parseNoScopeStatement()), begin); + if (param is null) + break; // This is a LastCatch + begin = token; + } + + if (consumed(T.Finally)) + finBody = set(new FinallyStatement(parseNoScopeStatement()), prevToken); + + if (catchBodies.length == 0 && finBody is null) + assert(begin.kind == T.Try), error(begin, MSG.MissingCatchOrFinally); + + return new TryStatement(tryBody, catchBodies, finBody); + } + + Statement parseThrowStatement() + { + skip(T.Throw); + auto expr = parseExpression(); + require(T.Semicolon); + return new ThrowStatement(expr); + } + + Statement parseScopeGuardStatement() + { + skip(T.Scope); + skip(T.LParen); + auto condition = requireIdentifier(MSG.ExpectedScopeIdentifier); + if (condition) + switch (condition.idKind) + { + case IDK.exit, IDK.success, IDK.failure: + break; + default: + error(this.prevToken, MSG.InvalidScopeIdentifier, this.prevToken.srcText); + } + require(T.RParen); + Statement scopeBody; + if (token.kind == T.LBrace) + scopeBody = parseScopeStatement(); + else + scopeBody = parseNoScopeStatement(); + return new ScopeGuardStatement(condition, scopeBody); + } + + Statement parseVolatileStatement() + { + skip(T.Volatile); + Statement volatileBody; + if (token.kind == T.Semicolon) + nT(); + else if (token.kind == T.LBrace) + volatileBody = parseScopeStatement(); + else + volatileBody = parseStatement(); + return new VolatileStatement(volatileBody); + } + + Statement parsePragmaStatement() + { + skip(T.Pragma); + + Identifier* ident; + Expression[] args; + Statement pragmaBody; + + require(T.LParen); + ident = requireIdentifier(MSG.ExpectedPragmaIdentifier); + + if (consumed(T.Comma)) + args = parseExpressionList(); + require(T.RParen); + + pragmaBody = parseNoScopeOrEmptyStatement(); + + return new PragmaStatement(ident, args, pragmaBody); + } + + Statement parseStaticIfStatement() + { + skip(T.Static); + skip(T.If); + Expression condition; + Statement ifBody, elseBody; + + require(T.LParen); + condition = parseExpression(); + require(T.RParen); + ifBody = parseNoScopeStatement(); + if (consumed(T.Else)) + elseBody = parseNoScopeStatement(); + return new StaticIfStatement(condition, ifBody, elseBody); + } + + Statement parseStaticAssertStatement() + { + skip(T.Static); + skip(T.Assert); + Expression condition, message; + + require(T.LParen); + condition = parseAssignExpression(); // Condition. + if (consumed(T.Comma)) + message = parseAssignExpression(); // Error message. + require(T.RParen); + require(T.Semicolon); + return new StaticAssertStatement(condition, message); + } + + Statement parseDebugStatement() + { + skip(T.Debug); + Token* cond; + Statement debugBody, elseBody; + + // ( Condition ) + if (consumed(T.LParen)) + { + cond = parseIdentOrInt(); + require(T.RParen); + } + // debug Statement + // debug ( Condition ) Statement + debugBody = parseNoScopeStatement(); + // else Statement + if (consumed(T.Else)) + elseBody = parseNoScopeStatement(); + + return new DebugStatement(cond, debugBody, elseBody); + } + + Statement parseVersionStatement() + { + skip(T.Version); + Token* cond; + Statement versionBody, elseBody; + + // ( Condition ) + require(T.LParen); + cond = parseIdentOrInt(); + require(T.RParen); + // version ( Condition ) Statement + versionBody = parseNoScopeStatement(); + // else Statement + if (consumed(T.Else)) + elseBody = parseNoScopeStatement(); + + return new VersionStatement(cond, versionBody, elseBody); + } + + /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + | Assembler parsing methods | + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ + + /// Parses an AsmBlockStatement. + Statement parseAsmBlockStatement() + { + skip(T.Asm); + require(T.LBrace); + auto ss = new CompoundStatement; + while (token.kind != T.RBrace && token.kind != T.EOF) + ss ~= parseAsmStatement(); + require(T.RBrace); + return new AsmBlockStatement(ss); + } + + Statement parseAsmStatement() + { + auto begin = token; + Statement s; + Identifier* ident; + switch (token.kind) + { + // Keywords that are valid opcodes. + case T.In, T.Int, T.Out: + ident = token.ident; + nT(); + goto LOpcode; + case T.Identifier: + ident = token.ident; + nT(); + if (consumed(T.Colon)) + { // Identifier : AsmStatement + s = new LabeledStatement(ident, parseAsmStatement()); + break; + } + + LOpcode: + // Opcode ; + // Opcode Operands ; + // Opcode + // Identifier + Expression[] es; + if (token.kind != T.Semicolon) + do + es ~= parseAsmExpression(); + while (consumed(T.Comma)) + require(T.Semicolon); + s = new AsmStatement(ident, es); + break; + case T.Align: + // align Integer; + nT(); + int number = -1; + if (token.kind == T.Int32) + (number = token.int_), skip(T.Int32); + else + error(token, MSG.ExpectedIntegerAfterAlign, token.srcText); + require(T.Semicolon); + s = new AsmAlignStatement(number); + break; + case T.Semicolon: + s = new EmptyStatement(); + nT(); + break; + default: + s = new IllegalAsmStatement(); + // Skip to next valid token. + do + nT(); + while (!token.isAsmStatementStart && + token.kind != T.RBrace && + token.kind != T.EOF) + auto text = Token.textSpan(begin, this.prevToken); + error(begin, MSG.IllegalAsmStatement, text); + } + set(s, begin); + return s; + } + + Expression parseAsmExpression() + { + auto begin = token; + auto e = parseAsmOrOrExpression(); + if (consumed(T.Question)) + { + auto tok = this.prevToken; + auto iftrue = parseAsmExpression(); + require(T.Colon); + auto iffalse = parseAsmExpression(); + e = new CondExpression(e, iftrue, iffalse, tok); + set(e, begin); + } + // TODO: create AsmExpression that contains e? + return e; + } + + Expression parseAsmOrOrExpression() + { + alias parseAsmAndAndExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.OrLogical) + { + auto tok = token; + nT(); + e = new OrOrExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAsmAndAndExpression() + { + alias parseAsmOrExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.AndLogical) + { + auto tok = token; + nT(); + e = new AndAndExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAsmOrExpression() + { + alias parseAsmXorExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.OrBinary) + { + auto tok = token; + nT(); + e = new OrExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAsmXorExpression() + { + alias parseAsmAndExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.Xor) + { + auto tok = token; + nT(); + e = new XorExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAsmAndExpression() + { + alias parseAsmCmpExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.AndBinary) + { + auto tok = token; + nT(); + e = new AndExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAsmCmpExpression() + { + alias parseAsmShiftExpression parseNext; + auto begin = token; + auto e = parseNext(); + + auto operator = token; + switch (operator.kind) + { + case T.Equal, T.NotEqual: + nT(); + e = new EqualExpression(e, parseNext(), operator); + break; + case T.LessEqual, T.Less, T.GreaterEqual, T.Greater: + nT(); + e = new RelExpression(e, parseNext(), operator); + break; + default: + return e; + } + set(e, begin); + return e; + } + + Expression parseAsmShiftExpression() + { + alias parseAsmAddExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (1) + { + auto operator = token; + switch (operator.kind) + { + case T.LShift: nT(); e = new LShiftExpression(e, parseNext(), operator); break; + case T.RShift: nT(); e = new RShiftExpression(e, parseNext(), operator); break; + case T.URShift: nT(); e = new URShiftExpression(e, parseNext(), operator); break; + default: + return e; + } + set(e, begin); + } + assert(0); + } + + Expression parseAsmAddExpression() + { + alias parseAsmMulExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (1) + { + auto operator = token; + switch (operator.kind) + { + case T.Plus: nT(); e = new PlusExpression(e, parseNext(), operator); break; + case T.Minus: nT(); e = new MinusExpression(e, parseNext(), operator); break; + // Not allowed in asm + //case T.Tilde: nT(); e = new CatExpression(e, parseNext(), operator); break; + default: + return e; + } + set(e, begin); + } + assert(0); + } + + Expression parseAsmMulExpression() + { + alias parseAsmPostExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (1) + { + auto operator = token; + switch (operator.kind) + { + case T.Mul: nT(); e = new MulExpression(e, parseNext(), operator); break; + case T.Div: nT(); e = new DivExpression(e, parseNext(), operator); break; + case T.Mod: nT(); e = new ModExpression(e, parseNext(), operator); break; + default: + return e; + } + set(e, begin); + } + assert(0); + } + + Expression parseAsmPostExpression() + { + auto begin = token; + auto e = parseAsmUnaryExpression(); + while (consumed(T.LBracket)) + { + e = new AsmPostBracketExpression(e, parseAsmExpression()); + require(T.RBracket); + set(e, begin); + } + return e; + } + + Expression parseAsmUnaryExpression() + { + auto begin = token; + Expression e; + switch (token.kind) + { + case T.Byte, T.Short, T.Int, + T.Float, T.Double, T.Real: + goto LAsmTypePrefix; + case T.Identifier: + switch (token.ident.idKind) + { + case IDK.near, IDK.far,/* "byte", "short", "int",*/ + IDK.word, IDK.dword, IDK.qword/*, "float", "double", "real"*/: + LAsmTypePrefix: + nT(); + if (token.kind == T.Identifier && token.ident is Ident.ptr) + skip(T.Identifier); + else + error(MID.ExpectedButFound, "ptr", token.srcText); + e = new AsmTypeExpression(parseAsmExpression()); + break; + case IDK.offset: + nT(); + e = new AsmOffsetExpression(parseAsmExpression()); + break; + case IDK.seg: + nT(); + e = new AsmSegExpression(parseAsmExpression()); + break; + default: + goto LparseAsmPrimaryExpression; + } + break; + case T.Minus: + case T.Plus: + nT(); + e = new SignExpression(parseAsmUnaryExpression()); + break; + case T.Not: + nT(); + e = new NotExpression(parseAsmUnaryExpression()); + break; + case T.Tilde: + nT(); + e = new CompExpression(parseAsmUnaryExpression()); + break; + case T.Dot: + nT(); + e = new ModuleScopeExpression(parseIdentifierExpression()); + while (consumed(TOK.Dot)) + { + e = new DotExpression(e, parseIdentifierExpression()); + set(e, begin); + } + break; + default: + LparseAsmPrimaryExpression: + e = parseAsmPrimaryExpression(); + return e; + } + set(e, begin); + return e; + } + + Expression parseAsmPrimaryExpression() + { + auto begin = token; + Expression e; + switch (token.kind) + { + case T.Int32, T.Int64, T.Uint32, T.Uint64: + e = new IntExpression(token); + nT(); + break; + case T.Float32, T.Float64, T.Float80, + T.Imaginary32, T.Imaginary64, T.Imaginary80: + e = new RealExpression(token); + nT(); + break; + case T.Dollar: + e = new DollarExpression(); + nT(); + break; + case T.LBracket: + // [ AsmExpression ] + nT(); + e = parseAsmExpression(); + require(T.RBracket); + e = new AsmBracketExpression(e); + break; + case T.Identifier: + auto register = token.ident; + switch (register.idKind) + { + // __LOCAL_SIZE + case IDK.__LOCAL_SIZE: + nT(); + e = new AsmLocalSizeExpression(); + break; + // Register + case IDK.ST: + nT(); + // (1) - (7) + int number = -1; + if (consumed(T.LParen)) + { + if (token.kind == T.Int32) + (number = token.int_), skip(T.Int32); + else + expected(T.Int32); + require(T.RParen); + } + e = new AsmRegisterExpression(register, number); + break; + case IDK.FS: + nT(); + // TODO: is the colon-number part optional? + int number = -1; + if (consumed(T.Colon)) + { + // :0, :4, :8 + if (token.kind == T.Int32) + (number = token.int_), skip(T.Int32); + if (number != 0 && number != 4 && number != 8) + error(MID.ExpectedButFound, "0, 4 or 8", token.srcText); + } + e = new AsmRegisterExpression(register, number); + break; + case IDK.AL, IDK.AH, IDK.AX, IDK.EAX, + IDK.BL, IDK.BH, IDK.BX, IDK.EBX, + IDK.CL, IDK.CH, IDK.CX, IDK.ECX, + IDK.DL, IDK.DH, IDK.DX, IDK.EDX, + IDK.BP, IDK.EBP, IDK.SP, IDK.ESP, + IDK.DI, IDK.EDI, IDK.SI, IDK.ESI, + IDK.ES, IDK.CS, IDK.SS, IDK.DS, IDK.GS, + IDK.CR0, IDK.CR2, IDK.CR3, IDK.CR4, + IDK.DR0, IDK.DR1, IDK.DR2, IDK.DR3, IDK.DR6, IDK.DR7, + IDK.TR3, IDK.TR4, IDK.TR5, IDK.TR6, IDK.TR7, + IDK.MM0, IDK.MM1, IDK.MM2, IDK.MM3, + IDK.MM4, IDK.MM5, IDK.MM6, IDK.MM7, + IDK.XMM0, IDK.XMM1, IDK.XMM2, IDK.XMM3, + IDK.XMM4, IDK.XMM5, IDK.XMM6, IDK.XMM7: + nT(); + e = new AsmRegisterExpression(register); + break; + default: + e = parseIdentifierExpression(); + while (consumed(TOK.Dot)) + { + e = new DotExpression(e, parseIdentifierExpression()); + set(e, begin); + } + } // end of switch + break; + default: + error(MID.ExpectedButFound, "Expression", token.srcText); + e = new IllegalExpression(); + if (!trying) + { // Insert a dummy token and don't consume current one. + begin = lexer.insertEmptyTokenBefore(token); + this.prevToken = begin; + } + } + set(e, begin); + return e; + } + + /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + | Expression parsing methods | + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ + + /// Parses an Expression. + Expression parseExpression() + { + alias parseAssignExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.Comma) + { + auto comma = token; + nT(); + e = new CommaExpression(e, parseNext(), comma); + set(e, begin); + } + return e; + } + + Expression parseAssignExpression() + { + alias parseAssignExpression parseNext; + auto begin = token; + auto e = parseCondExpression(); + switch (token.kind) + { + case T.Assign: + nT(); e = new AssignExpression(e, parseNext()); break; + case T.LShiftAssign: + nT(); e = new LShiftAssignExpression(e, parseNext()); break; + case T.RShiftAssign: + nT(); e = new RShiftAssignExpression(e, parseNext()); break; + case T.URShiftAssign: + nT(); e = new URShiftAssignExpression(e, parseNext()); break; + case T.OrAssign: + nT(); e = new OrAssignExpression(e, parseNext()); break; + case T.AndAssign: + nT(); e = new AndAssignExpression(e, parseNext()); break; + case T.PlusAssign: + nT(); e = new PlusAssignExpression(e, parseNext()); break; + case T.MinusAssign: + nT(); e = new MinusAssignExpression(e, parseNext()); break; + case T.DivAssign: + nT(); e = new DivAssignExpression(e, parseNext()); break; + case T.MulAssign: + nT(); e = new MulAssignExpression(e, parseNext()); break; + case T.ModAssign: + nT(); e = new ModAssignExpression(e, parseNext()); break; + case T.XorAssign: + nT(); e = new XorAssignExpression(e, parseNext()); break; + case T.CatAssign: + nT(); e = new CatAssignExpression(e, parseNext()); break; + default: + return e; + } + set(e, begin); + return e; + } + + Expression parseCondExpression() + { + auto begin = token; + auto e = parseOrOrExpression(); + if (token.kind == T.Question) + { + auto tok = token; + nT(); + auto iftrue = parseExpression(); + require(T.Colon); + auto iffalse = parseCondExpression(); + e = new CondExpression(e, iftrue, iffalse, tok); + set(e, begin); + } + return e; + } + + Expression parseOrOrExpression() + { + alias parseAndAndExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.OrLogical) + { + auto tok = token; + nT(); + e = new OrOrExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAndAndExpression() + { + alias parseOrExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.AndLogical) + { + auto tok = token; + nT(); + e = new AndAndExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseOrExpression() + { + alias parseXorExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.OrBinary) + { + auto tok = token; + nT(); + e = new OrExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseXorExpression() + { + alias parseAndExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.Xor) + { + auto tok = token; + nT(); + e = new XorExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseAndExpression() + { + alias parseCmpExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (token.kind == T.AndBinary) + { + auto tok = token; + nT(); + e = new AndExpression(e, parseNext(), tok); + set(e, begin); + } + return e; + } + + Expression parseCmpExpression() + { + alias parseShiftExpression parseNext; + auto begin = token; + auto e = parseShiftExpression(); + + auto operator = token; + switch (operator.kind) + { + case T.Equal, T.NotEqual: + nT(); + e = new EqualExpression(e, parseNext(), operator); + break; + case T.Not: + if (peekNext() != T.Is) + break; + nT(); + // fall through + case T.Is: + nT(); + e = new IdentityExpression(e, parseNext(), operator); + break; + case T.LessEqual, T.Less, T.GreaterEqual, T.Greater, + T.Unordered, T.UorE, T.UorG, T.UorGorE, + T.UorL, T.UorLorE, T.LorEorG, T.LorG: + nT(); + e = new RelExpression(e, parseNext(), operator); + break; + case T.In: + nT(); + e = new InExpression(e, parseNext(), operator); + break; + default: + return e; + } + set(e, begin); + return e; + } + + Expression parseShiftExpression() + { + alias parseAddExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (1) + { + auto operator = token; + switch (operator.kind) + { + case T.LShift: nT(); e = new LShiftExpression(e, parseNext(), operator); break; + case T.RShift: nT(); e = new RShiftExpression(e, parseNext(), operator); break; + case T.URShift: nT(); e = new URShiftExpression(e, parseNext(), operator); break; + default: + return e; + } + set(e, begin); + } + assert(0); + } + + Expression parseAddExpression() + { + alias parseMulExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (1) + { + auto operator = token; + switch (operator.kind) + { + case T.Plus: nT(); e = new PlusExpression(e, parseNext(), operator); break; + case T.Minus: nT(); e = new MinusExpression(e, parseNext(), operator); break; + case T.Tilde: nT(); e = new CatExpression(e, parseNext(), operator); break; + default: + return e; + } + set(e, begin); + } + assert(0); + } + + Expression parseMulExpression() + { + alias parsePostExpression parseNext; + auto begin = token; + auto e = parseNext(); + while (1) + { + auto operator = token; + switch (operator.kind) + { + case T.Mul: nT(); e = new MulExpression(e, parseNext(), operator); break; + case T.Div: nT(); e = new DivExpression(e, parseNext(), operator); break; + case T.Mod: nT(); e = new ModExpression(e, parseNext(), operator); break; + default: + return e; + } + set(e, begin); + } + assert(0); + } + + Expression parsePostExpression() + { + auto begin = token; + auto e = parseUnaryExpression(); + while (1) + { + while (consumed(T.Dot)) + { + e = new DotExpression(e, parseNewOrIdentifierExpression()); + set(e, begin); + } + + switch (token.kind) + { + case T.PlusPlus: + e = new PostIncrExpression(e); + break; + case T.MinusMinus: + e = new PostDecrExpression(e); + break; + case T.LParen: + e = new CallExpression(e, parseArguments()); + goto Lset; + case T.LBracket: + // parse Slice- and IndexExpression + nT(); + // [] is a SliceExpression + if (token.kind == T.RBracket) + { + e = new SliceExpression(e, null, null); + break; + } + + Expression[] es = [parseAssignExpression()]; + + // [ AssignExpression .. AssignExpression ] + if (consumed(T.Slice)) + { + e = new SliceExpression(e, es[0], parseAssignExpression()); + require(T.RBracket); + goto Lset; + } + + // [ ExpressionList ] + if (consumed(T.Comma)) + es ~= parseExpressionList(); + require(T.RBracket); + + e = new IndexExpression(e, es); + goto Lset; + default: + return e; + } + nT(); + Lset: // Jumped here to skip nT(). + set(e, begin); + } + assert(0); + } + + Expression parseUnaryExpression() + { + auto begin = token; + Expression e; + switch (token.kind) + { + case T.AndBinary: + nT(); + e = new AddressExpression(parseUnaryExpression()); + break; + case T.PlusPlus: + nT(); + e = new PreIncrExpression(parseUnaryExpression()); + break; + case T.MinusMinus: + nT(); + e = new PreDecrExpression(parseUnaryExpression()); + break; + case T.Mul: + nT(); + e = new DerefExpression(parseUnaryExpression()); + break; + case T.Minus: + case T.Plus: + nT(); + e = new SignExpression(parseUnaryExpression()); + break; + case T.Not: + nT(); + e = new NotExpression(parseUnaryExpression()); + break; + case T.Tilde: + nT(); + e = new CompExpression(parseUnaryExpression()); + break; + case T.New: + e = parseNewExpression(); + return e; + case T.Delete: + nT(); + e = new DeleteExpression(parseUnaryExpression()); + break; + case T.Cast: + requireNext(T.LParen); + Type type; + switch (token.kind) + { + version(D2) + { + auto begin2 = token; + case T.Const: + type = new ConstType(null); + goto case_break; + case T.Invariant: + type = new InvariantType(null); + case_break: + nT(); + set(type, begin2); + break; + } + default: + type = parseType(); + } + require(T.RParen); + e = new CastExpression(parseUnaryExpression(), type); + break; + case T.LParen: + // ( Type ) . Identifier + Type parseType_() + { + skip(T.LParen); + auto type = parseType(); + require(T.RParen); + require(T.Dot); + return type; + } + bool success; + auto type = try_(&parseType_, success); + if (success) + { + auto ident = requireIdentifier(MSG.ExpectedIdAfterTypeDot); + e = new TypeDotIdExpression(type, ident); + break; + } + goto default; + case T.Dot: + nT(); + e = new ModuleScopeExpression(parseIdentifierExpression()); + break; + default: + e = parsePrimaryExpression(); + return e; + } + assert(e !is null); + set(e, begin); + return e; + } + + /// $(PRE + /// IdentifierExpression := + /// Identifier + /// TemplateInstance + /// TemplateInstance := + /// Identifier !( TemplateArguments ) + /// ) + Expression parseIdentifierExpression() + { + auto begin = token; + auto ident = requireIdentifier(MSG.ExpectedAnIdentifier); + Expression e; + // Peek for '(' to avoid matching: id !is id + if (token.kind == T.Not && peekNext() == T.LParen) + { // Identifier !( TemplateArguments ) + skip(T.Not); + auto tparams = parseTemplateArguments(); + e = new TemplateInstanceExpression(ident, tparams); + } + else // Identifier + e = new IdentifierExpression(ident); + return set(e, begin); + } + + Expression parseNewOrIdentifierExpression() + { + return token.kind == T.New ? parseNewExpression() : parseIdentifierExpression(); + } + + Expression parsePrimaryExpression() + { + auto begin = token; + Expression e; + switch (token.kind) + { + case T.Identifier: + e = parseIdentifierExpression(); + return e; + case T.Typeof: + e = new TypeofExpression(parseTypeofType()); + break; + case T.This: + nT(); + e = new ThisExpression(); + break; + case T.Super: + nT(); + e = new SuperExpression(); + break; + case T.Null: + nT(); + e = new NullExpression(); + break; + case T.True, T.False: + nT(); + e = new BoolExpression(); + break; + case T.Dollar: + nT(); + e = new DollarExpression(); + break; + case T.Int32, T.Int64, T.Uint32, T.Uint64: + e = new IntExpression(token); + nT(); + break; + case T.Float32, T.Float64, T.Float80, + T.Imaginary32, T.Imaginary64, T.Imaginary80: + e = new RealExpression(token); + nT(); + break; + case T.CharLiteral: + e = new CharExpression(token.dchar_); + nT(); + break; + case T.String: + char[] str = token.str; + char postfix = token.pf; + nT(); + while (token.kind == T.String) + { + /+if (postfix == 0) + postfix = token.pf; + else+/ + if (token.pf && token.pf != postfix) + error(token, MSG.StringPostfixMismatch); + str.length = str.length - 1; // Exclude '\0'. + str ~= token.str; + nT(); + } + switch (postfix) + { + case 'w': + if (checkString(begin, str)) + goto default; + e = new StringExpression(dil.Unicode.toUTF16(str)); break; + case 'd': + if (checkString(begin, str)) + goto default; + e = new StringExpression(dil.Unicode.toUTF32(str)); break; + case 'c': + default: + // No checking done to allow for binary data. + e = new StringExpression(str); break; + } + break; + case T.LBracket: + Expression[] values; + + nT(); + if (!consumed(T.RBracket)) + { + e = parseAssignExpression(); + if (consumed(T.Colon)) + goto LparseAssocArray; + if (consumed(T.Comma)) + values = [e] ~ parseExpressionList(); + require(T.RBracket); + } + + e = new ArrayLiteralExpression(values); + break; + + LparseAssocArray: + Expression[] keys = [e]; + + goto LenterLoop; + do + { + keys ~= parseAssignExpression(); + require(T.Colon); + LenterLoop: + values ~= parseAssignExpression(); + } while (consumed(T.Comma)) + require(T.RBracket); + e = new AArrayLiteralExpression(keys, values); + break; + case T.LBrace: + // DelegateLiteral := { Statements } + auto funcBody = parseFunctionBody(); + e = new FunctionLiteralExpression(funcBody); + break; + case T.Function, T.Delegate: + // FunctionLiteral := ("function"|"delegate") Type? "(" ArgumentList ")" FunctionBody + nT(); // Skip function or delegate keyword. + Type returnType; + Parameters parameters; + if (token.kind != T.LBrace) + { + if (token.kind != T.LParen) // Optional return type + returnType = parseType(); + parameters = parseParameterList(); + } + auto funcBody = parseFunctionBody(); + e = new FunctionLiteralExpression(returnType, parameters, funcBody); + break; + case T.Assert: + Expression msg; + requireNext(T.LParen); + e = parseAssignExpression(); + if (consumed(T.Comma)) + msg = parseAssignExpression(); + require(T.RParen); + e = new AssertExpression(e, msg); + break; + case T.Mixin: + requireNext(T.LParen); + e = parseAssignExpression(); + require(T.RParen); + e = new MixinExpression(e); + break; + case T.Import: + requireNext(T.LParen); + e = parseAssignExpression(); + require(T.RParen); + e = new ImportExpression(e); + break; + case T.Typeid: + requireNext(T.LParen); + auto type = parseType(); + require(T.RParen); + e = new TypeidExpression(type); + break; + case T.Is: + requireNext(T.LParen); + + Type type, specType; + Identifier* ident; // optional Identifier + Token* opTok, specTok; + + type = parseDeclarator(ident, true); + + switch (token.kind) + { + case T.Colon, T.Equal: + opTok = token; + nT(); + switch (token.kind) + { + case T.Typedef, + T.Struct, + T.Union, + T.Class, + T.Interface, + T.Enum, + T.Function, + T.Delegate, + T.Super, + T.Return: + case_Const_Invariant: + specTok = token; + nT(); + break; + case T.Const, T.Invariant: + if (peekNext() != T.LParen) + goto case_Const_Invariant; + // Fall through. It's a type. + default: + specType = parseType(); + } + default: + } + + TemplateParameters tparams; + version(D2) + { + // is ( Type Identifier : TypeSpecialization , TemplateParameterList ) + // is ( Type Identifier == TypeSpecialization , TemplateParameterList ) + if (ident && specType && token.kind == T.Comma) + tparams = parseTemplateParameterList2(); + } + require(T.RParen); + e = new IsExpression(type, ident, opTok, specTok, specType, tparams); + break; + case T.LParen: + if (tokenAfterParenIs(T.LBrace)) // Check for "(...) {" + { // ( ParameterList ) FunctionBody + auto parameters = parseParameterList(); + auto funcBody = parseFunctionBody(); + e = new FunctionLiteralExpression(null, parameters, funcBody); + } + else + { // ( Expression ) + skip(T.LParen); + e = parseExpression(); + require(T.RParen); + e = new ParenExpression(e); + } + break; + version(D2) + { + case T.Traits: + requireNext(T.LParen); + auto id = requireIdentifier(MSG.ExpectedAnIdentifier); + TemplateArguments args; + if (token.kind == T.Comma) + args = parseTemplateArguments2(); + else + require(T.RParen); + e = new TraitsExpression(id, args); + break; + } + default: + if (token.isIntegralType) + { // IntegralType . Identifier + auto type = new IntegralType(token.kind); + nT(); + set(type, begin); + require(T.Dot); + auto ident = requireIdentifier(MSG.ExpectedIdAfterTypeDot); + e = new TypeDotIdExpression(type, ident); + } + else if (token.isSpecialToken) + { + e = new SpecialTokenExpression(token); + nT(); + } + else + { + error(MID.ExpectedButFound, "Expression", token.srcText); + e = new IllegalExpression(); + if (!trying) + { // Insert a dummy token and don't consume current one. + begin = lexer.insertEmptyTokenBefore(token); + this.prevToken = begin; + } + } + } + set(e, begin); + return e; + } + + Expression parseNewExpression(/*Expression e*/) + { + auto begin = token; + skip(T.New); + + Expression[] newArguments; + Expression[] ctorArguments; + + if (token.kind == T.LParen) + newArguments = parseArguments(); + + // NewAnonClassExpression: + // new (ArgumentList)opt class (ArgumentList)opt SuperClassopt InterfaceClassesopt ClassBody + if (consumed(T.Class)) + { + if (token.kind == T.LParen) + ctorArguments = parseArguments(); + + BaseClassType[] bases = token.kind != T.LBrace ? parseBaseClasses(false) : null ; + + auto decls = parseDeclarationDefinitionsBody(); + return set(new NewAnonClassExpression(/*e, */newArguments, bases, ctorArguments, decls), begin); + } + + // NewExpression: + // NewArguments Type [ AssignExpression ] + // NewArguments Type ( ArgumentList ) + // NewArguments Type + auto type = parseType(); + + if (token.kind == T.LParen) + ctorArguments = parseArguments(); + + return set(new NewExpression(/*e, */newArguments, type, ctorArguments), begin); + } + + /// Parses a Type. + Type parseType() + { + return parseBasicType2(parseBasicType()); + } + + Type parseIdentifierType() + { + auto begin = token; + auto ident = requireIdentifier(MSG.ExpectedAnIdentifier); + Type t; + if (consumed(T.Not)) // Identifier !( TemplateArguments ) + t = new TemplateInstanceType(ident, parseTemplateArguments()); + else // Identifier + t = new IdentifierType(ident); + return set(t, begin); + } + + Type parseQualifiedType() + { + auto begin = token; + Type type; + if (token.kind == T.Dot) + type = set(new ModuleScopeType(), begin, begin); + else if (token.kind == T.Typeof) + type = parseTypeofType(); + else + type = parseIdentifierType(); + + while (consumed(T.Dot)) + type = set(new QualifiedType(type, parseIdentifierType()), begin); + return type; + } + + Type parseBasicType() + { + auto begin = token; + Type t; + + if (token.isIntegralType) + { + t = new IntegralType(token.kind); + nT(); + } + else + switch (token.kind) + { + case T.Identifier, T.Typeof, T.Dot: + t = parseQualifiedType(); + return t; + version(D2) + { + case T.Const: + // const ( Type ) + requireNext(T.LParen); + t = parseType(); + require(T.RParen); + t = new ConstType(t); + break; + case T.Invariant: + // invariant ( Type ) + requireNext(T.LParen); + t = parseType(); + require(T.RParen); + t = new InvariantType(t); + break; + } // version(D2) + default: + error(MID.ExpectedButFound, "BasicType", token.srcText); + t = new IllegalType(); + nT(); + } + return set(t, begin); + } + + Type parseBasicType2(Type t) + { + while (1) + { + auto begin = token; + switch (token.kind) + { + case T.Mul: + t = new PointerType(t); + nT(); + break; + case T.LBracket: + t = parseArrayType(t); + continue; + case T.Function, T.Delegate: + TOK tok = token.kind; + nT(); + auto parameters = parseParameterList(); + if (tok == T.Function) + t = new FunctionType(t, parameters); + else + t = new DelegateType(t, parameters); + break; + default: + return t; + } + set(t, begin); + } + assert(0); + } + + /// Returns true if the token after the closing parenthesis + /// is of kind tok. + bool tokenAfterParenIs(TOK tok) + { + // We count nested parentheses tokens because template types + // may appear inside parameter lists. E.g.: (int x, Foo!(int) y) + assert(token.kind == T.LParen); + Token* next = token; + uint level = 1; + Loop: + while (1) + { + lexer.peek(next); + switch (next.kind) + { + case T.RParen: + if (--level == 0) + { // Last, closing parentheses found. + do + lexer.peek(next); + while (next.isWhitespace) + break Loop; + } + break; + case T.LParen: + ++level; + break; + case T.EOF: + break Loop; + default: + } + } + return next.kind == tok; + } + + /// Parse the array types after the declarator (C-style.) E.g.: int a[] + Type parseDeclaratorSuffix(Type lhsType) + { + // The Type chain should be as follows: + // int[3]* Identifier [][32] + // <- <- -> -. + // ^-----------------´ + // Resulting chain: [][32]*[3]int + Type parseNext() // Nested function required to accomplish this. + { + if (token.kind != T.LBracket) + return lhsType; // Break recursion; return Type on the left hand side of the Identifier. + + auto begin = token; + Type t; + skip(T.LBracket); + if (consumed(T.RBracket)) + t = new ArrayType(parseNext()); // [ ] + else + { + bool success; + Type parseAAType() + { + auto type = parseType(); + require(T.RBracket); + return type; + } + auto assocType = try_(&parseAAType, success); + if (success) + t = new ArrayType(parseNext(), assocType); // [ Type ] + else + { + Expression e = parseExpression(), e2; + if (consumed(T.Slice)) + e2 = parseExpression(); + require(T.RBracket); + t = new ArrayType(parseNext(), e, e2); // [ Expression .. Expression ] + } + } + set(t, begin); + return t; + } + return parseNext(); + } + + Type parseArrayType(Type t) + { + auto begin = token; + skip(T.LBracket); + if (consumed(T.RBracket)) + t = new ArrayType(t); + else + { + bool success; + Type parseAAType() + { + auto type = parseType(); + require(T.RBracket); + return type; + } + auto assocType = try_(&parseAAType, success); + if (success) + t = new ArrayType(t, assocType); + else + { + Expression e = parseExpression(), e2; + if (consumed(T.Slice)) + e2 = parseExpression(); + require(T.RBracket); + t = new ArrayType(t, e, e2); + } + } + set(t, begin); + return t; + } + + Type parseCFunctionPointerType(Type type, ref Identifier* ident, bool optionalParamList) + { + assert(type !is null); + auto begin = token; + skip(T.LParen); + + type = parseBasicType2(type); + if (token.kind == T.LParen) + { // Can be nested. + type = parseCFunctionPointerType(type, ident, true); + } + else if (token.kind == T.Identifier) + { // The identifier of the function pointer and the declaration. + ident = token.ident; + nT(); + type = parseDeclaratorSuffix(type); + } + require(T.RParen); + + Parameters params; + if (optionalParamList) + params = token.kind == T.LParen ? parseParameterList() : null; + else + params = parseParameterList(); + + type = new CFuncPointerType(type, params); + return set(type, begin); + } + + Type parseDeclarator(ref Identifier* ident, bool identOptional = false) + { + auto t = parseType(); + + if (token.kind == T.LParen) + t = parseCFunctionPointerType(t, ident, true); + else if (token.kind == T.Identifier) + { + ident = token.ident; + nT(); + t = parseDeclaratorSuffix(t); + } + + if (ident is null && !identOptional) + error(token, MSG.ExpectedDeclaratorIdentifier, token.srcText); + + return t; + } + + /// Parses a list of AssignExpressions. + /// $(PRE + /// ExpressionList := + /// AssignExpression + /// AssignExpression , ExpressionList + /// ) + Expression[] parseExpressionList() + { + Expression[] expressions; + do + expressions ~= parseAssignExpression(); + while(consumed(T.Comma)) + return expressions; + } + + /// Parses a list of Arguments. + /// $(PRE + /// Arguments := + /// ( ) + /// ( ExpressionList ) + /// ) + Expression[] parseArguments() + { + skip(T.LParen); + Expression[] args; + if (token.kind != T.RParen) + args = parseExpressionList(); + require(T.RParen); + return args; + } + + /// Parses a ParameterList. + Parameters parseParameterList() + out(params) + { + if (params.length > 1) + foreach (param; params.items[0..$-1]) + { + if (param.isVariadic()) + assert(0, "variadic arguments can only appear at the end of the parameter list."); + } + } + body + { + auto begin = token; + require(T.LParen); + + auto params = new Parameters(); + + if (consumed(T.RParen)) + return set(params, begin); + + do + { + auto paramBegin = token; + StorageClass stc, stc_; + Type type; + Identifier* ident; + Expression defValue; + + void pushParameter() + { + params ~= set(new Parameter(stc, type, ident, defValue), paramBegin); + } + + if (consumed(T.Ellipses)) + { + stc = StorageClass.Variadic; + pushParameter(); // type, ident and defValue will be null. + break; + } + + while (1) + { // Parse storage classes. + switch (token.kind) + { + version(D2) + { + case T.Invariant: // D2.0 + if (peekNext() == T.LParen) + break; + stc_ = StorageClass.Invariant; + goto Lcommon; + case T.Const: // D2.0 + if (peekNext() == T.LParen) + break; + stc_ = StorageClass.Const; + goto Lcommon; + case T.Final: // D2.0 + stc_ = StorageClass.Final; + goto Lcommon; + case T.Scope: // D2.0 + stc_ = StorageClass.Scope; + goto Lcommon; + case T.Static: // D2.0 + stc_ = StorageClass.Static; + goto Lcommon; + } + case T.In: + stc_ = StorageClass.In; + goto Lcommon; + case T.Out: + stc_ = StorageClass.Out; + goto Lcommon; + case T.Inout, T.Ref: + stc_ = StorageClass.Ref; + goto Lcommon; + case T.Lazy: + stc_ = StorageClass.Lazy; + goto Lcommon; + Lcommon: + // Check for redundancy. + if (stc & stc_) + error(MID.RedundantStorageClass, token.srcText); + else + stc |= stc_; + nT(); + version(D2) + continue; + else + break; // In D1.0 the grammar only allows one storage class. + default: + } + break; // Break out of inner loop. + } + type = parseDeclarator(ident, true); + + if (consumed(T.Assign)) + defValue = parseAssignExpression(); + + if (consumed(T.Ellipses)) + { + stc |= StorageClass.Variadic; + pushParameter(); + break; + } + pushParameter(); + + } while (consumed(T.Comma)) + require(T.RParen); + return set(params, begin); + } + + TemplateArguments parseTemplateArguments() + { + TemplateArguments targs; + require(T.LParen); + if (token.kind != T.RParen) + targs = parseTemplateArguments_(); + require(T.RParen); + return targs; + } + +version(D2) +{ + TemplateArguments parseTemplateArguments2() + { + skip(T.Comma); + TemplateArguments targs; + if (token.kind != T.RParen) + targs = parseTemplateArguments_(); + else + error(token, MSG.ExpectedTypeOrExpression); + require(T.RParen); + return targs; + } +} // version(D2) + + TemplateArguments parseTemplateArguments_() + { + auto begin = token; + auto targs = new TemplateArguments; + do + { + Type parseType_() + { + auto type = parseType(); + if (token.kind == T.Comma || token.kind == T.RParen) + return type; + errorCount++; // Cause try_() to fail. + return null; + } + bool success; + auto typeArgument = try_(&parseType_, success); + if (success) + // TemplateArgument: + // Type + // Symbol + targs ~= typeArgument; + else + // TemplateArgument: + // AssignExpression + targs ~= parseAssignExpression(); + } while (consumed(T.Comma)) + set(targs, begin); + return targs; + } + + TemplateParameters parseTemplateParameterList() + { + auto begin = token; + auto tparams = new TemplateParameters; + require(T.LParen); + if (token.kind != T.RParen) + parseTemplateParameterList_(tparams); + require(T.RParen); + return set(tparams, begin); + } + +version(D2) +{ + TemplateParameters parseTemplateParameterList2() + { + skip(T.Comma); + auto begin = token; + auto tparams = new TemplateParameters; + if (token.kind != T.RParen) + parseTemplateParameterList_(tparams); + else + error(token, MSG.ExpectedTemplateParameters); + return set(tparams, begin); + } +} // version(D2) + + /// Parses template parameters. + void parseTemplateParameterList_(TemplateParameters tparams) + { + do + { + auto paramBegin = token; + TemplateParameter tp; + Identifier* ident; + Type specType, defType; + + void parseSpecAndOrDefaultType() + { + // : SpecializationType + if (consumed(T.Colon)) + specType = parseType(); + // = DefaultType + if (consumed(T.Assign)) + defType = parseType(); + } + + switch (token.kind) + { + case T.Alias: + // TemplateAliasParameter: + // alias Identifier + skip(T.Alias); + ident = requireIdentifier(MSG.ExpectedAliasTemplateParam); + parseSpecAndOrDefaultType(); + tp = new TemplateAliasParameter(ident, specType, defType); + break; + case T.Identifier: + ident = token.ident; + switch (peekNext()) + { + case T.Ellipses: + // TemplateTupleParameter: + // Identifier ... + skip(T.Identifier); skip(T.Ellipses); + if (token.kind == T.Comma) + error(MID.TemplateTupleParameter); + tp = new TemplateTupleParameter(ident); + break; + case T.Comma, T.RParen, T.Colon, T.Assign: + // TemplateTypeParameter: + // Identifier + skip(T.Identifier); + parseSpecAndOrDefaultType(); + tp = new TemplateTypeParameter(ident, specType, defType); + break; + default: + // TemplateValueParameter: + // Declarator + ident = null; + goto LTemplateValueParameter; + } + break; + version(D2) + { + case T.This: + // TemplateThisParameter + // this TemplateTypeParameter + skip(T.This); + ident = requireIdentifier(MSG.ExpectedNameForThisTempParam); + parseSpecAndOrDefaultType(); + tp = new TemplateThisParameter(ident, specType, defType); + break; + } + default: + LTemplateValueParameter: + // TemplateValueParameter: + // Declarator + Expression specValue, defValue; + auto valueType = parseDeclarator(ident); + // : SpecializationValue + if (consumed(T.Colon)) + specValue = parseCondExpression(); + // = DefaultValue + if (consumed(T.Assign)) + defValue = parseCondExpression(); + tp = new TemplateValueParameter(valueType, ident, specValue, defValue); + } + + // Push template parameter. + tparams ~= set(tp, paramBegin); + + } while (consumed(T.Comma)) + } + + alias require expected; + + /// Requires a token of kind tok. + void require(TOK tok) + { + if (token.kind == tok) + nT(); + else + error(MID.ExpectedButFound, Token.toString(tok), token.srcText); + } + + /// Requires the next token to be of kind tok. + void requireNext(TOK tok) + { + nT(); + require(tok); + } + + /// Optionally parses an identifier. + /// Returns: null or the identifier. + Identifier* optionalIdentifier() + { + Identifier* id; + if (token.kind == T.Identifier) + (id = token.ident), skip(T.Identifier); + return id; + } + + Identifier* requireIdentifier() + { + Identifier* id; + if (token.kind == T.Identifier) + (id = token.ident), skip(T.Identifier); + else + error(MID.ExpectedButFound, "Identifier", token.srcText); + return id; + } + + /// Reports an error if the current token is not an identifier. + /// Params: + /// errorMsg = the error message to be used. + /// Returns: null or the identifier. + Identifier* requireIdentifier(char[] errorMsg) + { + Identifier* id; + if (token.kind == T.Identifier) + (id = token.ident), skip(T.Identifier); + else + error(token, errorMsg, token.srcText); + return id; + } + + /// Reports an error if the current token is not an identifier. + /// Params: + /// mid = the error message ID to be used. + /// Returns: null or the identifier. + Identifier* requireIdentifier(MID mid) + { + Identifier* id; + if (token.kind == T.Identifier) + (id = token.ident), skip(T.Identifier); + else + error(mid, token.srcText); + return id; + } + + /// Reports an error if the current token is not an identifier. + /// Returns: null or the token. + Token* requireId() + { + Token* idtok; + if (token.kind == T.Identifier) + (idtok = token), skip(T.Identifier); + else + error(MID.ExpectedButFound, "Identifier", token.srcText); + return idtok; + } + + Token* requireIdToken(char[] errorMsg) + { + Token* idtok; + if (token.kind == T.Identifier) + (idtok = token), skip(T.Identifier); + else + { + error(token, errorMsg, token.srcText); + idtok = lexer.insertEmptyTokenBefore(token); + this.prevToken = idtok; + } + return idtok; + } + + /// Returns true if the string str has an invalid UTF-8 sequence. + bool checkString(Token* begin, string str) + { + auto utf8Seq = Lexer.findInvalidUTF8Sequence(str); + if (utf8Seq.length) + error(begin, MSG.InvalidUTF8SequenceInString, utf8Seq); + return utf8Seq.length != 0; + } + + /// Forwards error parameters. + void error(Token* token, char[] formatMsg, ...) + { + error_(token, formatMsg, _arguments, _argptr); + } + + /// ditto + void error(MID mid, ...) + { + error_(this.token, GetMsg(mid), _arguments, _argptr); + } + + /// Creates an error report and appends it to a list. + /// Params: + /// token = used to get the location of where the error is. + /// formatMsg = the compiler error message. + void error_(Token* token, char[] formatMsg, TypeInfo[] _arguments, Arg _argptr) + { + if (trying) + { + ++errorCount; + return; + } + auto location = token.getErrorLocation(); + auto msg = Format(_arguments, _argptr, formatMsg); + auto error = new ParserError(location, msg); + errors ~= error; + if (infoMan !is null) + infoMan ~= error; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Analysis.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Analysis.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,106 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Analysis; + +import dil.ast.Node; +import dil.ast.Expressions; +import dil.semantic.Scope; +import dil.lexer.IdTable; +import dil.Compilation; +import common; + +/// Common semantics for pragma declarations and statements. +void pragmaSemantic(Scope scop, Token* pragmaLoc, + Identifier* ident, + Expression[] args) +{ + if (ident is Ident.msg) + pragma_msg(scop, pragmaLoc, args); + else if (ident is Ident.lib) + pragma_lib(scop, pragmaLoc, args); + // else + // scop.error(begin, "unrecognized pragma"); +} + +/// Evaluates a msg pragma. +void pragma_msg(Scope scop, Token* pragmaLoc, Expression[] args) +{ + if (args.length == 0) + return /*scop.error(pragmaLoc, "expected expression arguments to pragma")*/; + + foreach (arg; args) + { + auto e = arg/+.evaluate()+/; + if (e is null) + { + // scop.error(e.begin, "expression is not evaluatable at compile time"); + } + else if (auto stringExpr = e.Is!(StringExpression)) + // Print string to standard output. + Stdout(stringExpr.getString()); + else + { + // scop.error(e.begin, "expression must evaluate to a string"); + } + } + // Print a newline at the end. + Stdout('\n'); +} + +/// Evaluates a lib pragma. +void pragma_lib(Scope scop, Token* pragmaLoc, Expression[] args) +{ + if (args.length != 1) + return /*scop.error(pragmaLoc, "expected one expression argument to pragma")*/; + + auto e = args[0]/+.evaluate()+/; + if (e is null) + { + // scop.error(e.begin, "expression is not evaluatable at compile time"); + } + else if (auto stringExpr = e.Is!(StringExpression)) + { + // TODO: collect library paths in Module? + // scop.modul.addLibrary(stringExpr.getString()); + } + else + { + // scop.error(e.begin, "expression must evaluate to a string"); + } +} + +/// Returns true if the first branch (of a debug declaration/statement) or +/// false if the else-branch should be compiled in. +bool debugBranchChoice(Token* cond, CompilationContext context) +{ + if (cond) + { + if (cond.kind == TOK.Identifier) + { + if (context.findDebugId(cond.ident.str)) + return true; + } + else if (cond.uint_ <= context.debugLevel) + return true; + } + else if (1 <= context.debugLevel) + return true; + return false; +} + +/// Returns true if the first branch (of a version declaration/statement) or +/// false if the else-branch should be compiled in. +bool versionBranchChoice(Token* cond, CompilationContext context) +{ + assert(cond); + if (cond.kind == TOK.Identifier) + { + if (context.findVersionId(cond.ident.str)) + return true; + } + else if (cond.uint_ >= context.versionLevel) + return true; + return false; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Interpreter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Interpreter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,74 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Interpreter; + +import dil.ast.Visitor; +import dil.ast.Node, + dil.ast.Declarations, + dil.ast.Expressions, + dil.ast.Statements, + dil.ast.Types, + dil.ast.Parameters; + +import dil.semantic.Symbol, + dil.semantic.Symbols, + dil.semantic.Scope, + dil.semantic.Types; +import dil.Information; + +/// Used for compile-time evaluation of expressions. +class Interpreter : Visitor +{ + // Scope scop; + InfoManager infoMan; + + static class Result : Expression + { + override Result copy(){return null;} + } + + static const Result NAR; /// Not a Result. Similar to NAN in floating point arithmetics. + + static this() + { + NAR = new Result; + NAR.type = Types.Error; + } + + /// Evaluates the expression e. + /// Returns: NAR or a value. + static Expression interpret(Expression e, InfoManager infoMan/+, Scope scop+/) + { + return (new Interpreter(/+scop,+/ infoMan)).eval(e); + } + + /// Constructs an Interpreter object. + this(/+Scope scop, +/InfoManager infoMan) + { + // this.scop = scop; + this.infoMan = infoMan; + } + + /// Start evaluation. + Expression eval(Expression e) + { + return e; + } + + /// Returns true if e is immutable. + bool isImmutable(Expression e) + { + switch (e.kind) + { + alias NodeKind NK; + case NK.IntExpression, NK.RealExpression, + NK.ComplexExpression, NK.CharExpression, + NK.BoolExpression, NK.StringExpression: + return true; + default: + } + return false; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Module.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Module.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,158 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Module; + +import dil.ast.Node; +import dil.ast.Declarations; +import dil.parser.Parser; +import dil.lexer.Lexer; +import dil.semantic.Symbol; +import dil.semantic.Symbols; +import dil.Information; +import dil.SourceText; +import common; + +import tango.io.FilePath; +import tango.io.FileConst; + +alias FileConst.PathSeparatorChar dirSep; + +/// Represents a semantic D module and a source file. +class Module : ScopeSymbol +{ + SourceText sourceText; /// The source file of this module. + string moduleFQN; /// Fully qualified name of the module. E.g.: dil.ast.Node + string packageName; /// E.g.: dil.ast + string moduleName; /// E.g.: Node + + CompoundDeclaration root; /// The root of the parse tree. + ImportDeclaration[] imports; /// ImportDeclarations found in this file. + ModuleDeclaration moduleDecl; /// The optional ModuleDeclaration in this file. + Parser parser; /// The parser used to parse this file. + + // Module[] modules; + + InfoManager infoMan; /// Collects error messages. + + this() + { + super(SYM.Module, null, null); + } + + /// Constructs a Module object. + /// Params: + /// filePath = file path to the source text; loaded in the constructor. + /// infoMan = used for collecting error messages. + this(string filePath, InfoManager infoMan = null) + { + this(); + this.sourceText = new SourceText(filePath); + this.infoMan = infoMan; + this.sourceText.load(infoMan); + } + + /// Returns the file path of the source text. + string filePath() + { + return sourceText.filePath; + } + + /// Returns the file extension: "d" or "di". + string fileExtension() + { + foreach_reverse(i, c; filePath) + if (c == '.') + return filePath[i+1..$]; + return ""; + } + + /// Sets the parser to be used for parsing the source text. + void setParser(Parser parser) + { + this.parser = parser; + } + + /// Parses the module. + /// Throws: + /// An Exception if the there's no ModuleDeclaration and + /// the file name is an invalid or reserved D identifier. + void parse() + { + if (this.parser is null) + this.parser = new Parser(sourceText, infoMan); + + this.root = parser.start(); + this.imports = parser.imports; + + if (root.children.length) + { // moduleDecl will be null if first node isn't a ModuleDeclaration. + this.moduleDecl = root.children[0].Is!(ModuleDeclaration); + if (this.moduleDecl) + this.setFQN(moduleDecl.getFQN()); + } + + if (!this.moduleFQN.length) + { // Take base name of file path as module name. + auto str = (new FilePath(filePath)).name(); + if (Lexer.isReservedIdentifier(str)) + throw new Exception("'"~str~"' is not a valid module name; it's a reserved or invalid D identifier."); + this.moduleFQN = this.moduleName = str; + } + } + + /// Returns the first token of the module's source text. + Token* firstToken() + { + return parser.lexer.firstToken(); + } + + /// Returns true if there are errors in the source file. + bool hasErrors() + { + return parser.errors.length || parser.lexer.errors.length; + } + + /// Returns a list of import paths. + /// E.g.: ["dil/ast/Node", "dil/semantic/Module"] + string[] getImportPaths() + { + string[] result; + foreach (import_; imports) + result ~= import_.getModuleFQNs(dirSep); + return result; + } + + /// Returns the fully qualified name of this module. + /// E.g.: dil.ast.Node + string getFQN() + { + return moduleFQN; + } + + /// Set's the module's FQN. + void setFQN(string moduleFQN) + { + uint i = moduleFQN.length; + if (i != 0) // Don't decrement if string has zero length. + i--; + // Find last dot. + for (; i != 0 && moduleFQN[i] != '.'; i--) + {} + this.moduleFQN = moduleFQN; + this.packageName = moduleFQN[0..i]; + this.moduleName = moduleFQN[(i == 0 ? 0 : i+1) .. $]; + } + + /// Returns the module's FQN with slashes instead of dots. + /// E.g.: dil/ast/Node + string getFQNPath() + { + string FQNPath = moduleFQN.dup; + foreach (i, c; FQNPath) + if (c == '.') + FQNPath[i] = dirSep; + return FQNPath; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Package.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Package.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,12 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Package; + +import dil.semantic.Symbol; + +class Package : Symbol +{ + +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Pass1.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Pass1.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,504 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Pass1; + +import dil.ast.Visitor; +import dil.ast.Node, + dil.ast.Declarations, + dil.ast.Expressions, + dil.ast.Statements, + dil.ast.Types, + dil.ast.Parameters; +import dil.lexer.IdTable; +import dil.semantic.Symbol, + dil.semantic.Symbols, + dil.semantic.Types, + dil.semantic.Scope, + dil.semantic.Module, + dil.semantic.Analysis; +import dil.Compilation; +import dil.Location; +import dil.Information; +import dil.Messages; +import dil.Enums; +import dil.CompilerInfo; +import common; + +/// The first pass is the declaration pass. +/// +/// The basic task of this class is to traverse the parse tree, +/// find all kinds of declarations and add them +/// to the symbol tables of their respective scopes. +class SemanticPass1 : Visitor +{ + Scope scop; /// The current scope. + Module modul; /// The module to be semantically checked. + CompilationContext context; /// The compilation context. + + // Attributes: + LinkageType linkageType; /// Current linkage type. + Protection protection; /// Current protection attribute. + StorageClass storageClass; /// Current storage classes. + uint alignSize; /// Current align size. + + /// Constructs a SemanticPass1 object. + /// Params: + /// modul = the module to be processed. + /// context = the compilation context. + this(Module modul, CompilationContext context) + { + this.modul = modul; + this.context = new CompilationContext(context); + this.alignSize = context.structAlign; + } + + /// Starts processing the module. + void start() + { + assert(modul.root !is null); + // Create module scope. + scop = new Scope(null, modul); + visit(modul.root); + } + + /// Enters a new scope. + void enterScope(ScopeSymbol s) + { + scop = scop.enter(s); + } + + /// Exits the current scope. + void exitScope() + { + scop = scop.exit(); + } + + /// Returns true if this is the module scope. + bool isModuleScope() + { + return scop.symbol.isModule(); + } + + /// Inserts a symbol into the current scope. + void insert(Symbol symbol, Identifier* name) + { + auto symX = scop.symbol.lookup(name); + if (symX) + reportSymbolConflict(symbol, symX, name); + else + scop.symbol.insert(symbol, name); + // Set the current scope symbol as the parent. + symbol.parent = scop.symbol; + } + + /// Inserts a symbol into scopeSym. + void insert(Symbol symbol, ScopeSymbol scopeSym) + { + auto symX = scopeSym.lookup(symbol.name); + if (symX) + reportSymbolConflict(symbol, symX, symbol.name); + else + scopeSym.insert(symbol, symbol.name); + // Set the current scope symbol as the parent. + symbol.parent = scopeSym; + } + + /// Inserts a symbol, overloading on the name, into the current scope. + void insertOverload(Symbol sym, Identifier* name) + { + auto sym2 = scop.symbol.lookup(name); + if (sym2) + { + if (sym2.isOverloadSet) + (cast(OverloadSet)cast(void*)sym2).add(sym); + else + reportSymbolConflict(sym, sym2, name); + } + else + // Create a new overload set. + scop.symbol.insert(new OverloadSet(name, sym.node), name); + // Set the current scope symbol as the parent. + sym.parent = scop.symbol; + } + + /// Reports an error: new symbol s1 conflicts with existing symbol s2. + void reportSymbolConflict(Symbol s1, Symbol s2, Identifier* name) + { + auto loc = s2.node.begin.getErrorLocation(); + auto locString = Format("{}({},{})", loc.filePath, loc.lineNum, loc.colNum); + error(s1.node.begin, MSG.DeclConflictsWithDecl, name.str, locString); + } + + /// Creates an error report. + void error(Token* token, char[] formatMsg, ...) + { + if (!modul.infoMan) + return; + auto location = token.getErrorLocation(); + auto msg = Format(_arguments, _argptr, formatMsg); + modul.infoMan ~= new SemanticError(location, msg); + } + + + /// Collects info about nodes which have to be evaluated later. + static class Deferred + { + Node node; + ScopeSymbol symbol; + // Saved attributes. + LinkageType linkageType; + Protection protection; + StorageClass storageClass; + uint alignSize; + } + + /// List of mixin, static if, static assert and pragma(msg,...) declarations. + /// + /// Their analysis must be deferred because they entail + /// evaluation of expressions. + Deferred[] deferred; + + /// Adds a deferred node to the list. + void addDeferred(Node node) + { + auto d = new Deferred; + d.node = node; + d.symbol = scop.symbol; + d.linkageType = linkageType; + d.protection = protection; + d.storageClass = storageClass; + d.alignSize = alignSize; + deferred ~= d; + } + + private alias Declaration D; /// A handy alias. Saves typing. + +override +{ + D visit(CompoundDeclaration d) + { + foreach (decl; d.decls) + visitD(decl); + return d; + } + + D visit(IllegalDeclaration) + { assert(0, "semantic pass on invalid AST"); return null; } + + D visit(EmptyDeclaration ed) + { return ed; } + + D visit(ModuleDeclaration) + { return null; } + + D visit(ImportDeclaration d) + { + return d; + } + + D visit(AliasDeclaration ad) + { + return ad; + } + + D visit(TypedefDeclaration td) + { + return td; + } + + D visit(EnumDeclaration d) + { + // Create the symbol. + d.symbol = new Enum(d.name, d); + auto isAnonymous = d.name is null; + if (isAnonymous) + d.symbol.name = IdTable.genAnonEnumID(); + insert(d.symbol, d.symbol.name); + auto parentScopeSymbol = scop.symbol; + enterScope(d.symbol); + // Declare members. + foreach (member; d.members) + { + visitD(member); + if (isAnonymous) // Also insert into parent scope if enum is anonymous. + insert(member.symbol, parentScopeSymbol); + member.symbol.parent = d.symbol; + } + exitScope(); + return d; + } + + D visit(EnumMemberDeclaration d) + { + d.symbol = new EnumMember(d.name, protection, storageClass, linkageType, d); + insert(d.symbol, d.symbol.name); + return d; + } + + D visit(ClassDeclaration d) + { + if (d.symbol) + return d; + d.symbol = new Class(d.name, d); + // Insert into current scope. + insert(d.symbol, d.name); + enterScope(d.symbol); + // Continue semantic analysis. + d.decls && visitD(d.decls); + exitScope(); + return d; + } + + D visit(InterfaceDeclaration d) + { + if (d.symbol) + return d; + d.symbol = new dil.semantic.Symbols.Interface(d.name, d); + // Insert into current scope. + insert(d.symbol, d.name); + enterScope(d.symbol); + // Continue semantic analysis. + d.decls && visitD(d.decls); + exitScope(); + return d; + } + + D visit(StructDeclaration d) + { + if (d.symbol) + return d; + d.symbol = new Struct(d.name, d); + // Insert into current scope. + if (d.name) + insert(d.symbol, d.name); + enterScope(d.symbol); + // Continue semantic analysis. + d.decls && visitD(d.decls); + exitScope(); + return d; + } + + D visit(UnionDeclaration d) + { + if (d.symbol) + return d; + d.symbol = new Union(d.name, d); + // Insert into current scope. + if (d.name) + insert(d.symbol, d.name); + enterScope(d.symbol); + // Continue semantic analysis. + d.decls && visitD(d.decls); + exitScope(); + return d; + } + + D visit(ConstructorDeclaration d) + { + auto func = new Function(Ident.__ctor, d); + insertOverload(func, func.name); + return d; + } + + D visit(StaticConstructorDeclaration d) + { + auto func = new Function(Ident.__ctor, d); + insertOverload(func, func.name); + return d; + } + + D visit(DestructorDeclaration d) + { + auto func = new Function(Ident.__dtor, d); + insertOverload(func, func.name); + return d; + } + + D visit(StaticDestructorDeclaration d) + { + auto func = new Function(Ident.__dtor, d); + insertOverload(func, func.name); + return d; + } + + D visit(FunctionDeclaration d) + { + auto func = new Function(d.name, d); + insertOverload(func, func.name); + return d; + } + + D visit(VariablesDeclaration vd) + { + // Error if we are in an interface. + if (scop.symbol.isInterface && !vd.isStatic) + return error(vd.begin, MSG.InterfaceCantHaveVariables), vd; + + // Insert variable symbols in this declaration into the symbol table. + foreach (i, name; vd.names) + { + auto variable = new Variable(name, protection, storageClass, linkageType, vd); + variable.value = vd.inits[i]; + vd.variables ~= variable; + insert(variable, name); + } + return vd; + } + + D visit(InvariantDeclaration d) + { + auto func = new Function(Ident.__invariant, d); + insert(func, func.name); + return d; + } + + D visit(UnittestDeclaration d) + { + auto func = new Function(Ident.__unittest, d); + insertOverload(func, func.name); + return d; + } + + D visit(DebugDeclaration d) + { + if (d.isSpecification) + { + if (!isModuleScope()) + error(d.begin, MSG.DebugSpecModuleLevel, d.spec.srcText); + else if (d.spec.kind == TOK.Identifier) + context.addDebugId(d.spec.ident.str); + else + context.debugLevel = d.spec.uint_; + } + else + { + if (debugBranchChoice(d.cond, context)) + d.compiledDecls = d.decls; + else + d.compiledDecls = d.elseDecls; + d.compiledDecls && visitD(d.compiledDecls); + } + return d; + } + + D visit(VersionDeclaration d) + { + if (d.isSpecification) + { + if (!isModuleScope()) + error(d.begin, MSG.VersionSpecModuleLevel, d.spec.srcText); + else if (d.spec.kind == TOK.Identifier) + context.addVersionId(d.spec.ident.str); + else + context.versionLevel = d.spec.uint_; + } + else + { + if (versionBranchChoice(d.cond, context)) + d.compiledDecls = d.decls; + else + d.compiledDecls = d.elseDecls; + d.compiledDecls && visitD(d.compiledDecls); + } + return d; + } + + D visit(TemplateDeclaration d) + { + if (d.symbol) + return d; + d.symbol = new Template(d.name, d); + // Insert into current scope. + insertOverload(d.symbol, d.name); + enterScope(d.symbol); + // Continue semantic analysis. + visitD(d.decls); + exitScope(); + return d; + } + + D visit(NewDeclaration d) + { + auto func = new Function(Ident.__new, d); + insert(func, func.name); + return d; + } + + D visit(DeleteDeclaration d) + { + auto func = new Function(Ident.__delete, d); + insert(func, func.name); + return d; + } + + D visit(ProtectionDeclaration d) + { + auto saved = protection; // Save. + protection = d.prot; // Set. + visitD(d.decls); + protection = saved; // Restore. + return d; + } + + D visit(StorageClassDeclaration d) + { + auto saved = storageClass; // Save. + storageClass = d.storageClass; // Set. + visitD(d.decls); + storageClass = saved; // Restore. + return d; + } + + D visit(LinkageDeclaration d) + { + auto saved = linkageType; // Save. + linkageType = d.linkageType; // Set. + visitD(d.decls); + linkageType = saved; // Restore. + return d; + } + + D visit(AlignDeclaration d) + { + auto saved = alignSize; // Save. + alignSize = d.size; // Set. + visitD(d.decls); + alignSize = saved; // Restore. + return d; + } + + // Deferred declarations: + + D visit(StaticAssertDeclaration d) + { + addDeferred(d); + return d; + } + + D visit(StaticIfDeclaration d) + { + addDeferred(d); + return d; + } + + D visit(MixinDeclaration d) + { + addDeferred(d); + return d; + } + + D visit(PragmaDeclaration d) + { + if (d.ident is Ident.msg) + addDeferred(d); + else + { + pragmaSemantic(scop, d.begin, d.ident, d.args); + visitD(d.decls); + } + return d; + } +} // override +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Pass2.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Pass2.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,436 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Pass2; + +import dil.ast.DefaultVisitor; +import dil.ast.Node, + dil.ast.Declarations, + dil.ast.Expressions, + dil.ast.Statements, + dil.ast.Types, + dil.ast.Parameters; +import dil.lexer.Identifier; +import dil.semantic.Symbol, + dil.semantic.Symbols, + dil.semantic.Types, + dil.semantic.Scope, + dil.semantic.Module, + dil.semantic.Analysis, + dil.semantic.Interpreter; +import dil.parser.Parser; +import dil.SourceText; +import dil.Location; +import dil.Information; +import dil.Messages; +import dil.Enums; +import dil.CompilerInfo; +import common; + +/// The second pass determines the types of symbols and the types +/// of expressions and also evaluates them. +class SemanticPass2 : DefaultVisitor +{ + Scope scop; /// The current scope. + Module modul; /// The module to be semantically checked. + + /// Constructs a SemanticPass2 object. + /// Params: + /// modul = the module to be checked. + this(Module modul) + { + this.modul = modul; + } + + /// Start semantic analysis. + void start() + { + assert(modul.root !is null); + // Create module scope. + scop = new Scope(null, modul); + visit(modul.root); + } + + /// Enters a new scope. + void enterScope(ScopeSymbol s) + { + scop = scop.enter(s); + } + + /// Exits the current scope. + void exitScope() + { + scop = scop.exit(); + } + + /// Evaluates e and returns the result. + Expression interpret(Expression e) + { + return Interpreter.interpret(e, modul.infoMan/+, scop+/); + } + + /// Creates an error report. + void error(Token* token, char[] formatMsg, ...) + { + auto location = token.getErrorLocation(); + auto msg = Format(_arguments, _argptr, formatMsg); + modul.infoMan ~= new SemanticError(location, msg); + } + + /// Some handy aliases. + private alias Declaration D; + private alias Expression E; /// ditto + private alias Statement S; /// ditto + private alias TypeNode T; /// ditto + + /// The current scope symbol to use for looking up identifiers. + /// E.g.: + /// --- + /// object.method(); // *) object is looked up in the current scope. + /// // *) idScope is set if object is a ScopeSymbol. + /// // *) method will be looked up in idScope. + /// dil.ast.Node.Node node; // A fully qualified type. + /// --- + ScopeSymbol idScope; + + /// Searches for a symbol. + Symbol search(Token* idTok) + { + assert(idTok.kind == TOK.Identifier); + auto id = idTok.ident; + Symbol symbol; + + if (idScope is null) + for (auto sc = scop; sc; sc = sc.parent) + { + symbol = sc.lookup(id); + if (symbol) + return symbol; + } + else + symbol = idScope.lookup(id); + + if (symbol is null) + error(idTok, MSG.UndefinedIdentifier, id.str); + else if (auto scopSymbol = cast(ScopeSymbol)symbol) + idScope = scopSymbol; + + return symbol; + } + +override +{ + D visit(CompoundDeclaration d) + { + return super.visit(d); + } + + D visit(EnumDeclaration d) + { + d.symbol.setCompleting(); + + Type type = Types.Int; // Default to int. + if (d.baseType) + type = visitT(d.baseType).type; + d.symbol.type = new TypeEnum(d.symbol, type); + + enterScope(d.symbol); + + foreach (member; d.members) + { + Expression finalValue; + member.symbol.setCompleting(); + if (member.value) + { + member.value = visitE(member.value); + finalValue = interpret(member.value); + if (finalValue is Interpreter.NAR) + finalValue = new IntExpression(0, d.symbol.type); + } + //else + // TODO: increment a number variable and assign that to value. + member.symbol.type = d.symbol.type; // Assign TypeEnum. + member.symbol.value = finalValue; + member.symbol.setComplete(); + } + + exitScope(); + d.symbol.setComplete(); + return d; + } + + D visit(MixinDeclaration md) + { + if (md.decls) + return md.decls; + if (md.isMixinExpression) + { + md.argument = visitE(md.argument); + auto expr = interpret(md.argument); + if (expr is Interpreter.NAR) + return md; + auto stringExpr = expr.Is!(StringExpression); + if (stringExpr is null) + { + error(md.begin, MSG.MixinArgumentMustBeString); + return md; + } + else + { // Parse the declarations in the string. + auto loc = md.begin.getErrorLocation(); + auto filePath = loc.filePath; + auto sourceText = new SourceText(filePath, stringExpr.getString()); + auto parser = new Parser(sourceText, modul.infoMan); + md.decls = parser.start(); + } + } + else + { + // TODO: implement template mixin. + } + return md.decls; + } + + // Type nodes: + + T visit(TypeofType t) + { + t.e = visitE(t.e); + t.type = t.e.type; + return t; + } + + T visit(ArrayType t) + { + auto baseType = visitT(t.next).type; + if (t.isAssociative) + t.type = baseType.arrayOf(visitT(t.assocType).type); + else if (t.isDynamic) + t.type = baseType.arrayOf(); + else if (t.isStatic) + {} + else + assert(t.isSlice); + return t; + } + + T visit(PointerType t) + { + t.type = visitT(t.next).type.ptrTo(); + return t; + } + + T visit(QualifiedType t) + { + if (t.lhs.Is!(QualifiedType) is null) + idScope = null; // Reset at left-most type. + visitT(t.lhs); + visitT(t.rhs); + t.type = t.rhs.type; + return t; + } + + T visit(IdentifierType t) + { + auto idToken = t.begin; + auto symbol = search(idToken); + // TODO: save symbol or its type in t. + return t; + } + + T visit(TemplateInstanceType t) + { + auto idToken = t.begin; + auto symbol = search(idToken); + // TODO: save symbol or its type in t. + return t; + } + + T visit(ModuleScopeType t) + { + idScope = modul; + return t; + } + + T visit(IntegralType t) + { + // A table mapping the kind of a token to its corresponding semantic Type. + TypeBasic[TOK] tok2Type = [ + TOK.Char : Types.Char, TOK.Wchar : Types.Wchar, TOK.Dchar : Types.Dchar, TOK.Bool : Types.Bool, + TOK.Byte : Types.Byte, TOK.Ubyte : Types.Ubyte, TOK.Short : Types.Short, TOK.Ushort : Types.Ushort, + TOK.Int : Types.Int, TOK.Uint : Types.Uint, TOK.Long : Types.Long, TOK.Ulong : Types.Ulong, + TOK.Cent : Types.Cent, TOK.Ucent : Types.Ucent, + TOK.Float : Types.Float, TOK.Double : Types.Double, TOK.Real : Types.Real, + TOK.Ifloat : Types.Ifloat, TOK.Idouble : Types.Idouble, TOK.Ireal : Types.Ireal, + TOK.Cfloat : Types.Cfloat, TOK.Cdouble : Types.Cdouble, TOK.Creal : Types.Creal, TOK.Void : Types.Void + ]; + assert(t.tok in tok2Type); + t.type = tok2Type[t.tok]; + return t; + } + + // Expression nodes: + + E visit(ParenExpression e) + { + if (!e.type) + { + e.next = visitE(e.next); + e.type = e.next.type; + } + return e; + } + + E visit(CommaExpression e) + { + if (!e.type) + { + e.lhs = visitE(e.lhs); + e.rhs = visitE(e.rhs); + e.type = e.rhs.type; + } + return e; + } + + E visit(OrOrExpression) + { return null; } + + E visit(AndAndExpression) + { return null; } + + E visit(SpecialTokenExpression e) + { + if (e.type) + return e.value; + switch (e.specialToken.kind) + { + case TOK.LINE, TOK.VERSION: + e.value = new IntExpression(e.specialToken.uint_, Types.Uint); + break; + case TOK.FILE, TOK.DATE, TOK.TIME, TOK.TIMESTAMP, TOK.VENDOR: + e.value = new StringExpression(e.specialToken.str); + break; + default: + assert(0); + } + e.type = e.value.type; + return e.value; + } + + E visit(DollarExpression e) + { + if (e.type) + return e; + e.type = Types.Size_t; + // if (!inArraySubscript) + // error("$ can only be in an array subscript."); + return e; + } + + E visit(NullExpression e) + { + if (!e.type) + e.type = Types.Void_ptr; + return e; + } + + E visit(BoolExpression e) + { + if (e.type) + return e; + e.value = new IntExpression(e.toBool(), Types.Bool); + e.type = Types.Bool; + return e; + } + + E visit(IntExpression e) + { + if (e.type) + return e; + + if (e.number & 0x8000_0000_0000_0000) + e.type = Types.Ulong; // 0xFFFF_FFFF_FFFF_FFFF + else if (e.number & 0xFFFF_FFFF_0000_0000) + e.type = Types.Long; // 0x7FFF_FFFF_FFFF_FFFF + else if (e.number & 0x8000_0000) + e.type = Types.Uint; // 0xFFFF_FFFF + else + e.type = Types.Int; // 0x7FFF_FFFF + return e; + } + + E visit(RealExpression e) + { + if (!e.type) + e.type = Types.Double; + return e; + } + + E visit(ComplexExpression e) + { + if (!e.type) + e.type = Types.Cdouble; + return e; + } + + E visit(CharExpression e) + { + if (e.type) + return e; + if (e.character <= 0xFF) + e.type = Types.Char; + else if (e.character <= 0xFFFF) + e.type = Types.Wchar; + else + e.type = Types.Dchar; + return e; + } + + E visit(StringExpression e) + { + return e; + } + + E visit(MixinExpression me) + { + if (me.type) + return me.expr; + me.expr = visitE(me.expr); + auto expr = interpret(me.expr); + if (expr is Interpreter.NAR) + return me; + auto stringExpr = expr.Is!(StringExpression); + if (stringExpr is null) + error(me.begin, MSG.MixinArgumentMustBeString); + else + { + auto loc = me.begin.getErrorLocation(); + auto filePath = loc.filePath; + auto sourceText = new SourceText(filePath, stringExpr.getString()); + auto parser = new Parser(sourceText, modul.infoMan); + expr = parser.start2(); + expr = visitE(expr); // Check expression. + } + me.expr = expr; + me.type = expr.type; + return me.expr; + } + + E visit(ImportExpression ie) + { + if (ie.type) + return ie.expr; + ie.expr = visitE(ie.expr); + auto expr = interpret(ie.expr); + if (expr is Interpreter.NAR) + return ie; + auto stringExpr = expr.Is!(StringExpression); + //if (stringExpr is null) + // error(me.begin, MSG.ImportArgumentMustBeString); + // TODO: load file + //ie.expr = new StringExpression(loadImportFile(stringExpr.getString())); + return ie.expr; + } +} +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Scope.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Scope.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,72 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Scope; + +import dil.semantic.Symbol; +import dil.semantic.Symbols; +import dil.lexer.Identifier; +import common; + +/// Builds a hierarchy of environments. +class Scope +{ + Scope parent; /// The surrounding scope, or null if this is the root scope. + + ScopeSymbol symbol; /// The current symbol with the symbol table. + + this(Scope parent, ScopeSymbol symbol) + { + this.parent = parent; + this.symbol = symbol; + } + + /// Find a symbol in this scope. + /// Params: + /// name = the name of the symbol. + Symbol lookup(Identifier* name) + { + return symbol.lookup(name); + } + + /// Create a new inner scope and return that. + Scope enter(ScopeSymbol symbol) + { + return new Scope(this, symbol); + } + + /// Destroy this scope and return the outer scope. + Scope exit() + { + auto sc = parent; + // delete this; + return sc; + } + + /// Search for the enclosing Class scope. + Scope classScope() + { + auto scop = this; + do + { + if (scop.symbol.isClass) + return scop; + scop = scop.parent; + } while (scop) + return null; + } + + /// Search for the enclosing Module scope. + Scope moduleScope() + { + auto scop = this; + do + { + if (scop.symbol.isModule) + return scop; + scop = scop.parent; + } while (scop) + return null; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Symbol.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Symbol.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,111 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Symbol; + +import dil.ast.Node; +import dil.lexer.Identifier; +import common; + +/// Enumeration of Symbol IDs. +enum SYM +{ + Module, + Class, + Interface, + Struct, + Union, + Enum, + EnumMember, + Template, + Variable, + Function, + Alias, + OverloadSet, +// Type, +} + +/// A symbol represents an object with semantic code information. +class Symbol +{ /// Enumeration of symbol statuses. + enum Status : ushort + { + Declared, /// The symbol has been declared. + Completing, /// The symbol is being processed. + Complete /// The symbol is complete. + } + + SYM sid; /// The ID of this symbol. + Status status; /// The semantic status of this symbol. + Symbol parent; /// The parent this symbol belongs to. + Identifier* name; /// The name of this symbol. + /// The syntax tree node that produced this symbol. + /// Useful for source code location info and retrieval of doc comments. + Node node; + + /// Constructs a Symbol object. + /// Params: + /// sid = the symbol's ID. + /// name = the symbol's name. + /// node = the symbol's node. + this(SYM sid, Identifier* name, Node node) + { + this.sid = sid; + this.name = name; + this.node = node; + } + + /// Change the status to Status.Completing. + void setCompleting() + { status = Status.Completing; } + + /// Change the status to Status.Complete. + void setComplete() + { status = Status.Complete; } + + /// Returns true if the symbol is being completed. + bool isCompleting() + { return status == Status.Completing; } + + /// Returns true if the symbols is complete. + bool isComplete() + { return status == Status.Complete; } + + /// A template macro for building isXYZ() methods. + private template isX(char[] kind) + { + const char[] isX = `bool is`~kind~`(){ return sid == SYM.`~kind~`; }`; + } + mixin(isX!("Module")); + mixin(isX!("Class")); + mixin(isX!("Interface")); + mixin(isX!("Struct")); + mixin(isX!("Union")); + mixin(isX!("Enum")); + mixin(isX!("EnumMember")); + mixin(isX!("Template")); + mixin(isX!("Variable")); + mixin(isX!("Function")); + mixin(isX!("Alias")); + mixin(isX!("OverloadSet")); +// mixin(isX!("Type")); + + /// Casts the symbol to Class. + Class to(Class)() + { + assert(mixin(`this.sid == mixin("SYM." ~ typeof(Class).stringof)`)); + return cast(Class)cast(void*)this; + } + + /// Returns: the fully qualified name of this symbol. + /// E.g.: dil.semantic.Symbol.Symbol.getFQN + char[] getFQN() + { + if (!name) + return parent ? parent.getFQN() : ""; + if (parent) + return parent.getFQN() ~ '.' ~ name.str; + return name.str; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/SymbolTable.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/SymbolTable.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,30 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.SymbolTable; + +import dil.semantic.Symbol; +import dil.lexer.Identifier; +import common; + +/// Maps an identifier string to a Symbol. +struct SymbolTable +{ + Symbol[char[]] table; /// The table data structure. + + /// Looks up ident in the table. + /// Returns: the symbol if there, otherwise null. + Symbol lookup(Identifier* ident) + { + assert(ident !is null); + auto psym = ident.str in table; + return psym ? *psym : null; + } + + /// Inserts a symbol into the table. + void insert(Symbol symbol, Identifier* ident) + { + table[ident.str] = symbol; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Symbols.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Symbols.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,206 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Symbols; + +import dil.ast.Expression; +import dil.semantic.Symbol; +import dil.semantic.SymbolTable; +import dil.semantic.Types; +import dil.ast.Node; +import dil.lexer.IdTable; +import dil.Enums; +import common; + +/// A symbol that has its own scope with a symbol table. +class ScopeSymbol : Symbol +{ + SymbolTable symbolTable; /// The symbol table. + Symbol[] members; /// The member symbols (in lexical order.) + + this(SYM sid, Identifier* name, Node node) + { + super(sid, name, node); + } + + /// Look up name in the table. + Symbol lookup(Identifier* name) + { + return symbolTable.lookup(name); + } + + /// Look up name in the table. + Symbol lookup(string name) + { + auto id = IdTable.lookup(name); + return id ? symbolTable.lookup(id) : null; + } + + /// Insert a symbol into the table. + void insert(Symbol s, Identifier* name) + { + symbolTable.insert(s, name); + members ~= s; + } +} + +/// Aggregates have function and field members. +abstract class Aggregate : ScopeSymbol +{ + Function[] funcs; + Variable[] fields; + + this(SYM sid, Identifier* name, Node node) + { + super(sid, name, node); + } + + override void insert(Symbol s, Identifier* ident) + { + if (s.isVariable) + // Append variable to fields. + fields ~= cast(Variable)cast(void*)s; + else if (s.isFunction) + // Append function to funcs. + funcs ~= cast(Function)cast(void*)s; + super.insert(s, ident); + } +} + +/// A class symbol. +class Class : Aggregate +{ + this(Identifier* name, Node classNode) + { + super(SYM.Class, name, classNode); + } +} + +/// An interface symbol. +class Interface : Aggregate +{ + this(Identifier* name, Node interfaceNode) + { + super(SYM.Interface, name, interfaceNode); + } +} + +/// A union symbol. +class Union : Aggregate +{ + this(Identifier* name, Node unionNode) + { + super(SYM.Union, name, unionNode); + } +} + +/// A struct symbol. +class Struct : Aggregate +{ + this(Identifier* name, Node structNode) + { + super(SYM.Struct, name, structNode); + } +} + +/// An enum symbol. +class Enum : ScopeSymbol +{ + TypeEnum type; + this(Identifier* name, Node enumNode) + { + super(SYM.Enum, name, enumNode); + } + + void setType(TypeEnum type) + { + this.type = type; + } +} + +/// A template symbol. +class Template : ScopeSymbol +{ + this(Identifier* name, Node templateNode) + { + super(SYM.Template, name, templateNode); + } +} + +/// A function symbol. +class Function : ScopeSymbol +{ + Protection prot; /// The protection. + StorageClass stc; /// The storage classes. + LinkageType linkType; /// The linkage type. + + Type returnType; + Variable[] params; + + this(Identifier* name, Node functionNode) + { + super(SYM.Function, name, functionNode); + } +} + +/// A variable symbol. +class Variable : Symbol +{ + Protection prot; /// The protection. + StorageClass stc; /// The storage classes. + LinkageType linkType; /// The linkage type. + + Type type; /// The type of this variable. + Expression value; /// The value of this variable. + + this(Identifier* name, + Protection prot, StorageClass stc, LinkageType linkType, + Node variableNode) + { + super(SYM.Variable, name, variableNode); + + this.prot = prot; + this.stc = stc; + this.linkType = linkType; + } +} + +/// An enum member symbol. +class EnumMember : Variable +{ + this(Identifier* name, + Protection prot, StorageClass stc, LinkageType linkType, + Node enumMemberNode) + { + super(name, prot, stc, linkType, enumMemberNode); + this.sid = SYM.EnumMember; + } +} + +/// An alias symbol. +class Alias : Symbol +{ + this(Identifier* name, Node aliasNode) + { + super(SYM.Alias, name, aliasNode); + } +} + +/// A list of symbols that share the same identifier. +/// +/// These can be functions, templates and aggregates with template parameter lists. +class OverloadSet : Symbol +{ + Symbol[] symbols; + + this(Identifier* name, Node node) + { + super(SYM.OverloadSet, name, node); + } + + void add(Symbol s) + { + symbols ~= s; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/Types.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/Types.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,403 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.Types; + +import dil.semantic.Symbol; +import dil.semantic.TypesEnum; +import dil.lexer.Identifier; +import dil.CompilerInfo; + +/// The base type for all type structures. +abstract class Type/* : Symbol*/ +{ + Type next; /// The next type in the type structure. + TYP tid; /// The ID of the type. + Symbol symbol; /// Not null if this type has a symbol. + + this(){} + + /// Constructs a Type object. + /// Params: + /// next = the type's next type. + /// tid = the type's ID. + this(Type next, TYP tid) + { +// this.sid = SYM.Type; + + this.next = next; + this.tid = tid; + } + + /// Returns a pointer type to this type. + TypePointer ptrTo() + { + return new TypePointer(this); + } + + /// Returns a dynamic array type using this type as its base. + TypeDArray arrayOf() + { + return new TypeDArray(this); + } + + /// Returns an associative array type using this type as its base. + /// Params: + /// key = the key type. + TypeAArray arrayOf(Type key) + { + return new TypeAArray(this, key); + } + + /// Returns the byte size of this type. + final size_t sizeOf() + { + return MITable.getSize(this); + } + + /// Size is not in MITable. Find out via virtual method. + size_t sizeOf_() + { + return sizeOf(); + } + + /// Returns true if this type has a symbol. + bool hasSymbol() + { + return symbol !is null; + } +} + +/// All basic types. E.g.: int, char, real etc. +class TypeBasic : Type +{ + this(TYP typ) + { + super(null, typ); + } +} + +/// Dynamic array type. +class TypeDArray : Type +{ + this(Type next) + { + super(next, TYP.DArray); + } +} + +/// Associative array type. +class TypeAArray : Type +{ + Type keyType; + this(Type next, Type keyType) + { + super(next, TYP.AArray); + this.keyType = keyType; + } +} + +/// Static array type. +class TypeSArray : Type +{ + size_t dimension; + this(Type next, size_t dimension) + { + super(next, TYP.SArray); + this.dimension = dimension; + } +} + +/// Pointer type. +class TypePointer : Type +{ + this(Type next) + { + super(next, TYP.Pointer); + } +} + +/// Reference type. +class TypeReference : Type +{ + this(Type next) + { + super(next, TYP.Reference); + } +} + +/// Enum type. +class TypeEnum : Type +{ + this(Symbol symbol, Type baseType) + { + super(baseType, TYP.Enum); + this.symbol = symbol; + } + + Type baseType() + { + return next; + } +} + +/// Struct type. +class TypeStruct : Type +{ + this(Symbol symbol) + { + super(null, TYP.Struct); + this.symbol = symbol; + } +} + +/// Class type. +class TypeClass : Type +{ + this(Symbol symbol) + { + super(null, TYP.Class); + this.symbol = symbol; + } +} + +/// Typedef type. +class TypeTypedef : Type +{ + this(Type next) + { + super(next, TYP.Typedef); + } +} + +/// Function type. +class TypeFunction : Type +{ + this(Type next) + { + super(next, TYP.Function); + } +} + +/// Delegate type. +class TypeDelegate : Type +{ + this(Type next) + { + super(next, TYP.Delegate); + } +} + +/// Identifier type. +class TypeIdentifier : Type +{ + Identifier* ident; + this(Identifier* ident) + { + super(null, TYP.Identifier); + } +} + +/// Template instantiation type. +class TypeTemplInstance : Type +{ + this() + { + super(null, TYP.TInstance); + } +} + +/// Template tuple type. +class TypeTuple : Type +{ + this(Type next) + { + super(next, TYP.Tuple); + } +} + +/// Constant type. D2.0 +class TypeConst : Type +{ + this(Type next) + { + super(next, TYP.Const); + } +} + +/// Invariant type. D2.0 +class TypeInvariant : Type +{ + this(Type next) + { + super(next, TYP.Const); + } +} + +/// Represents a value related to a Type. +union Value +{ + void* pvoid; + bool bool_; + dchar dchar_; + long long_; + ulong ulong_; + int int_; + uint uint_; + float float_; + double double_; + real real_; + creal creal_; +} + +/// Information related to a Type. +struct TypeMetaInfo +{ + char mangle; /// Mangle character of the type. + ushort size; /// Byte size of the type. + Value* defaultInit; /// Default initialization value. +} + +/// Namespace for the meta info table. +struct MITable +{ +static: + const ushort SIZE_NOT_AVAILABLE = 0; /// Size not available. + const Value VZERO = {int_:0}; /// Value 0. + const Value VNULL = {pvoid:null}; /// Value null. + const Value V0xFF = {dchar_:0xFF}; /// Value 0xFF. + const Value V0xFFFF = {dchar_:0xFFFF}; /// Value 0xFFFF. + const Value VFALSE = {bool_:false}; /// Value false. + const Value VNAN = {float_:float.nan}; /// Value NAN. + const Value VCNAN = {creal_:creal.nan}; /// Value complex NAN. + private alias SIZE_NOT_AVAILABLE SNA; + private alias PTR_SIZE PS; + /// The meta info table. + private const TypeMetaInfo metaInfoTable[] = [ + {'?', SNA}, // Error + + {'a', 1, &V0xFF}, // Char + {'u', 2, &V0xFFFF}, // Wchar + {'w', 4, &V0xFFFF}, // Dchar + {'b', 1, &VFALSE}, // Bool + {'g', 1, &VZERO}, // Byte + {'h', 1, &VZERO}, // Ubyte + {'s', 2, &VZERO}, // Short + {'t', 2, &VZERO}, // Ushort + {'i', 4, &VZERO}, // Int + {'k', 4, &VZERO}, // Uint + {'l', 8, &VZERO}, // Long + {'m', 8, &VZERO}, // Ulong + {'?', 16, &VZERO}, // Cent + {'?', 16, &VZERO}, // Ucent + {'f', 4, &VNAN}, // Float + {'d', 8, &VNAN}, // Double + {'e', 12, &VNAN}, // Real + {'o', 4, &VNAN}, // Ifloat + {'p', 8, &VNAN}, // Idouble + {'j', 12, &VNAN}, // Ireal + {'q', 8, &VCNAN}, // Cfloat + {'r', 16, &VCNAN}, // Cdouble + {'c', 24, &VCNAN}, // Creal + {'v', 1}, // void + + {'n', SNA}, // None + + {'A', PS*2, &VNULL}, // Dynamic array + {'G', PS*2, &VNULL}, // Static array + {'H', PS*2, &VNULL}, // Associative array + + {'E', SNA}, // Enum + {'S', SNA}, // Struct + {'C', PS, &VNULL}, // Class + {'T', SNA}, // Typedef + {'F', PS}, // Function + {'D', PS*2, &VNULL}, // Delegate + {'P', PS, &VNULL}, // Pointer + {'R', PS, &VNULL}, // Reference + {'I', SNA}, // Identifier + {'?', SNA}, // Template instance + {'B', SNA}, // Tuple + {'x', SNA}, // Const, D2 + {'y', SNA}, // Invariant, D2 + ]; + static assert(metaInfoTable.length == TYP.max+1); + + /// Returns the size of a type. + size_t getSize(Type type) + { + auto size = metaInfoTable[type.tid].size; + if (size == SIZE_NOT_AVAILABLE) + return type.sizeOf_(); + return size; + } +} + +/// Namespace for a set of predefined types. +struct Types +{ +static: + /// Predefined basic types. + TypeBasic Char, Wchar, Dchar, Bool, + Byte, Ubyte, Short, Ushort, + Int, Uint, Long, Ulong, + Cent, Ucent, + Float, Double, Real, + Ifloat, Idouble, Ireal, + Cfloat, Cdouble, Creal, Void; + + TypeBasic Size_t; /// The size type. + TypeBasic Ptrdiff_t; /// The pointer difference type. + TypePointer Void_ptr; /// The void pointer type. + TypeBasic Error; /// The error type. + TypeBasic Undefined; /// The undefined type. + + /// Allocates an instance of TypeBasic and assigns it to typeName. + template newTB(char[] typeName) + { + const newTB = mixin(typeName~" = new TypeBasic(TYP."~typeName~")"); + } + + /// Initializes predefined types. + static this() + { + newTB!("Char"); + newTB!("Wchar"); + newTB!("Dchar"); + newTB!("Bool"); + newTB!("Byte"); + newTB!("Ubyte"); + newTB!("Short"); + newTB!("Ushort"); + newTB!("Int"); + newTB!("Uint"); + newTB!("Long"); + newTB!("Ulong"); + newTB!("Cent"); + newTB!("Ucent"); + newTB!("Float"); + newTB!("Double"); + newTB!("Real"); + newTB!("Ifloat"); + newTB!("Idouble"); + newTB!("Ireal"); + newTB!("Cfloat"); + newTB!("Cdouble"); + newTB!("Creal"); + newTB!("Void"); + version(X86_64) + { + Size_t = Ulong; + Ptrdiff_t = Long; + } + else + { + Size_t = Uint; + Ptrdiff_t = Int; + } + Void_ptr = Void.ptrTo; + Error = new TypeBasic(TYP.Error); + Undefined = new TypeBasic(TYP.Error); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/semantic/TypesEnum.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/semantic/TypesEnum.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,56 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.semantic.TypesEnum; + +/// Enumeration of Type IDs. +enum TYP +{ + Error, + // Basic types. + Char, /// char + Wchar, /// wchar + Dchar, /// dchar + Bool, /// bool + Byte, /// int8 + Ubyte, /// uint8 + Short, /// int16 + Ushort, /// uint16 + Int, /// int32 + Uint, /// uint32 + Long, /// int64 + Ulong, /// uint64 + Cent, /// int128 + Ucent, /// uint128 + Float, /// float32 + Double, /// float64 + Real, /// float80 + Ifloat, /// imaginary float32 + Idouble, /// imaginary float64 + Ireal, /// imaginary float80 + Cfloat, /// complex float32 + Cdouble, /// complex float64 + Creal, /// complex float80 + Void, /// void + + None, /// TypeNone in the specs. Why? + + DArray, /// Dynamic array. + SArray, /// Static array. + AArray, /// Associative array. + + Enum, /// An enum. + Struct, /// A struct. + Class, /// A class. + Typedef, /// A typedef. + Function, /// A function. + Delegate, /// A delegate. + Pointer, /// A pointer. + Reference, /// A reference. + Identifier, /// An identifier. + TInstance, /// Template instance. + Tuple, /// A template tuple. + Const, /// A constant type. D2.0 + Invariant, /// An invariant type. D2.0 +} diff -r a3fab8b74a7d -r bcb74c9b895c src/dil/translator/German.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/dil/translator/German.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,340 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module dil.translator.German; + +import dil.ast.DefaultVisitor; +import dil.ast.Node; +import dil.ast.Declarations, + dil.ast.Statements, + dil.ast.Types, + dil.ast.Parameters; +import tango.io.Print; + +private alias Declaration D; + +/// Translates a syntax tree into German. +class GermanTranslator : DefaultVisitor +{ + Print!(char) put; /// Output buffer. + + char[] indent; /// Current indendation string. + char[] indentStep; /// Appended to indent at each indendation level. + + Declaration inAggregate; /// Current aggregate. + Declaration inFunc; /// Current function. + + bool pluralize; /// Whether to use the plural when printing the next types. + bool pointer; /// Whether next types should consider the previous pointer. + + /// Construct a GermanTranslator. + /// Params: + /// put = buffer to print to. + /// indentStep = added at every indendation step. + this(Print!(char) put, char[] indentStep) + { + this.put = put; + this.indentStep = indentStep; + } + + /// Start translation. + void translate(Node root) + { + visitN(root); + } + + /// Increases the indentation when instantiated. + /// The indentation is restored when the instance goes out of scope. + scope class Indent + { + char[] old_indent; + this() + { + old_indent = this.outer.indent; + this.outer.indent ~= this.outer.indentStep; + } + + ~this() + { this.outer.indent = old_indent; } + + char[] toString() + { return this.outer.indent; } + } + + /// Saves an outer member when instantiated. + /// It is restored when the instance goes out of scope. + scope class Enter(T) + { + T t_save; + this(T t) + { + auto t_save = t; + static if (is(T == ClassDeclaration) || + is(T == InterfaceDeclaration) || + is(T == StructDeclaration) || + is(T == UnionDeclaration)) + this.outer.inAggregate = t; + static if (is(T == FunctionDeclaration) || + is(T == ConstructorDeclaration)) + this.outer.inFunc = t; + } + + ~this() + { + static if (is(T == ClassDeclaration) || + is(T == InterfaceDeclaration) || + is(T == StructDeclaration) || + is(T == UnionDeclaration)) + this.outer.inAggregate = t_save; + static if (is(T == FunctionDeclaration) || + is(T == ConstructorDeclaration)) + this.outer.inFunc = t_save; + } + } + + alias Enter!(ClassDeclaration) EnteredClass; + alias Enter!(InterfaceDeclaration) EnteredInterface; + alias Enter!(StructDeclaration) EnteredStruct; + alias Enter!(UnionDeclaration) EnteredUnion; + alias Enter!(FunctionDeclaration) EnteredFunction; + alias Enter!(ConstructorDeclaration) EnteredConstructor; + + /// Prints the location of a node: @(lin,col) + void printLoc(Node node) + { + auto loc = node.begin.getRealLocation(); + put(indent).formatln("@({},{})",/+ loc.filePath,+/ loc.lineNum, loc.colNum); + } + +override: + D visit(ModuleDeclaration n) + { + printLoc(n); + put.format("Dies ist das Modul '{}'", n.moduleName.str); + if (n.packages.length) + put.format(" im Paket '{}'", n.getPackageName('.')); + put(".").newline; + return n; + } + + D visit(ImportDeclaration n) + { + printLoc(n); + put("Importiert Symbole aus einem anderen Modul bzw. Module.").newline; + return n; + } + + D visit(ClassDeclaration n) + { + printLoc(n); + scope E = new EnteredClass(n); + put(indent).formatln("'{}' is eine Klasse mit den Eigenschaften:", n.name.str); + scope I = new Indent(); + n.decls && visitD(n.decls); + return n; + } + + D visit(InterfaceDeclaration n) + { + printLoc(n); + scope E = new EnteredInterface(n); + put(indent).formatln("'{}' is ein Interface mit den Eigenschaften:", n.name.str); + scope I = new Indent(); + n.decls && visitD(n.decls); + return n; + } + + D visit(StructDeclaration n) + { + printLoc(n); + scope E = new EnteredStruct(n); + put(indent).formatln("'{}' is eine Datenstruktur mit den Eigenschaften:", n.name.str); + scope I = new Indent(); + n.decls && visitD(n.decls); + return n; + } + + D visit(UnionDeclaration n) + { + printLoc(n); + scope E = new EnteredUnion(n); + put(indent).formatln("'{}' is eine Datenunion mit den Eigenschaften:", n.name.str); + scope I = new Indent(); + n.decls && visitD(n.decls); + return n; + } + + D visit(VariablesDeclaration n) + { + printLoc(n); + char[] was; + if (inAggregate) + was = "Membervariable"; + else if (inFunc) + was = "lokale Variable"; + else + was = "globale Variable"; + foreach (name; n.names) + { + put(indent).format("'{}' ist eine {} des Typs: ", name.str, was); + if (n.typeNode) + visitT(n.typeNode); + else + put("auto"); + put.newline; + } + return n; + } + + D visit(FunctionDeclaration n) + { + printLoc(n); + char[] was; + if (inAggregate) + was = "Methode"; + else if(inFunc) + was = "geschachtelte Funktion"; + else + was = "Funktion"; + scope E = new EnteredFunction(n); + put(indent).format("'{}' ist eine {} ", n.name.str, was); + if (n.params.length == 1) + put("mit dem Argument "), visitN(n.params); + else if (n.params.length > 1) + put("mit den Argumenten "), visitN(n.params); + else + put("ohne Argumente"); + put(".").newline; + scope I = new Indent(); + return n; + } + + D visit(ConstructorDeclaration n) + { + printLoc(n); + scope E = new EnteredConstructor(n); + put(indent)("Ein Konstruktor "); + if (n.params.length == 1) + put("mit dem Argument "), visitN(n.params); + else if (n.params.length > 1) + put("mit den Argumenten "), visitN(n.params); + else + put("ohne Argumente"); + put(".").newline; + return n; + } + + D visit(StaticConstructorDeclaration n) + { + printLoc(n); + put(indent)("Ein statischer Konstruktor.").newline; + return n; + } + + D visit(DestructorDeclaration n) + { + printLoc(n); + put(indent)("Ein Destruktor.").newline; + return n; + } + + D visit(StaticDestructorDeclaration n) + { + printLoc(n); + put(indent)("Ein statischer Destruktor.").newline; + return n; + } + + D visit(InvariantDeclaration n) + { + printLoc(n); + put(indent)("Eine Unveränderliche.").newline; + return n; + } + + D visit(UnittestDeclaration n) + { + printLoc(n); + put(indent)("Ein Komponententest.").newline; + return n; + } + + Node visit(Parameter n) + { + put.format("'{}' des Typs \"", n.name ? n.name.str : "unbenannt"); + n.type && visitN(n.type); + put(\"); + return n; + } + + Node visit(Parameters n) + { + if (n.length > 1) + { + visitN(n.children[0]); + foreach (node; n.children[1..$]) + put(", "), visitN(node); + } + else + super.visit(n); + return n; + } + + TypeNode visit(ArrayType n) + { + char[] c1 = "s", c2 = ""; + if (pluralize) + (c1 = pointer ? ""[] : "n"), (c2 = "s"); + pointer = false; + if (n.assocType) + put.format("assoziative{} Array{} von ", c1, c2); +// visitT(n.assocType); + else if (n.e1) + { + if (n.e2) + put.format("gescheibte{} Array{} von ", c1, c2); + else + put.format("statische{} Array{} von ", c1, c2); +// visitE(n.e), n.e2 && visitE(n.e2); + } + else + put.format("dynamische{} Array{} von ", c1, c2); + // Types following arrays should be in plural. + pluralize = true; + visitT(n.next); + pluralize = false; + return n; + } + + TypeNode visit(PointerType n) + { + char[] c = pluralize ? (pointer ? ""[] : "n") : ""; + pointer = true; + put.format("Zeiger{} auf ", c), visitT(n.next); + return n; + } + + TypeNode visit(QualifiedType n) + { + visitT(n.lhs); + put("."); + visitT(n.rhs); + return n; + } + + TypeNode visit(IdentifierType n) + { + put(n.ident.str); + return n; + } + + TypeNode visit(IntegralType n) + { + char[] c = pluralize ? "s"[] : ""; + if (n.tok == TOK.Void) // Avoid pluralizing "void" + c = ""; + put.format("{}{}", n.begin.srcText, c); + return n; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/archdoc.xmi --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/archdoc.xmi Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1066 @@ + + + + + umbrello uml modeller http://uml.sf.net + 1.5.8 + UnicodeUTF8 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/config/configurator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/config/configurator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,81 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.config.configurator; + +import docgen.config.reader; +import docgen.config.reflection; +import docgen.misc.options; + +import Integer = tango.text.convert.Integer; +import tango.io.stream.FileStream; +import tango.io.Stdout; + +/** + * Class for handling and merging doc generator options. + */ +interface Configurator { + /** + * Merges configuration options from the given file. + */ + void mergeConfiguration(char[] cfgFile); + + /** + * Returns a hierarchical structure of configuration options. + */ + DocGeneratorOptions *getConfiguration(); +} + +private DocGeneratorOptions options; +private Struct!(options) opt; +private const cases = makeTypeStringForStruct!(opt); + +class DefaultConfigurator : Configurator { + private: + + DocGeneratorOptions options; + + public: + + const defaultProfileLocation = "docgen/config/default.cfg"; + + this() { + mergeConfiguration(defaultProfileLocation); + } + + this(char[] cfgFile) { + this(); + mergeConfiguration(cfgFile); + } + + void mergeConfiguration(char[] cfgFile) { + + auto inputStream = new FileInput(cfgFile); + auto content = new char[inputStream.length]; + auto bytesRead = inputStream.read (content); + + assert(bytesRead == inputStream.length, "Error reading configuration file"); + + auto tokens = lex(content); + auto configuration = parse(tokens); + + foreach(key, val; configuration) { + bool err() { + throw new Exception( + "Configurator: Invalid key-val pair " ~ key ~ + "=" ~ (val.length ? val[0] : "null")); + } + + // cuteness, lul + mixin(_switch( + mixin(cases) ~ + `default: throw new Exception("Illegal configuration key " ~ key);` + )); + } + } + + DocGeneratorOptions *getConfiguration() { + return &options; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/config/default.cfg --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/config/default.cfg Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,49 @@ +# +# This file contains the default configuration. You don't need to +# worry about this, the system will load the settings automatically. +# +# If you need to customize settings, just take a copy of this or +# write one from scratch and pass the new file location as a +# command line parameter. +# + + +(options + (graph + (imageFormat PNG) + (depth -1) + (nodeColor white) + (cyclicNodeColor tomato) + (unlocatableNodeColor gray) + (depColor black) + (cyclicDepColor red) + (publicDepColor blue) + (clusterColor blue) + (includeUnlocatableModules true) + (highlightCyclicEdges true) + (highlightCyclicVertices true) + (groupByPackageNames true) + (groupByFullPackageName false) + ) + (listing + (literateStyle true) + (enableListings true) + ) + (templates + (title "Test project") + (versionString 1.0) + (copyright "Crashtest dummy") + (paperSize a4paper) + (shortFileNames false) + (templateStyle default) + ) + (parser + (importPaths) + (rootPaths) + (strRegexps) + (commentFormat Doxygen) + (depth -1) + ) + (outputFormats LaTeX HTML PlainText) + (outputDir tmp/) +) diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/config/reader.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/config/reader.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,144 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.config.reader; + +debug import tango.io.Stdout; + +/** + * Lexes a s-exp like input + */ +char[][] lex(char[] input) { + char[][] tokens; + + uint state = 0, level = 0; + size_t sidx = 0; + + void err(size_t i, int type = 0) { + auto b = input[i<20 ? 0 : i-20..i]; + auto e = input[i+1..i+21>$ ? $ : i+21]; + + throw new Exception("Lex: " ~ + (type == 0 ? "Illegal character" : + type == 1 ? "Wrong number of parenthesis" : + "Unexpected end of input") ~ + ": " ~ b ~ " >>>" ~ input[i] ~ "<<< " ~ e ~ "." + ); + } + void begin(size_t i) { sidx = i; } + void end(size_t i) { tokens ~= input[sidx..i]; } + + foreach(size_t i, c; input) { + if (sidx > i) continue; + switch(c) { // states space, token, textEnd + case '"': // starts a ""-string + switch(state) { + case 0: + char[] buf; + bool escape; + char d; + sidx = i; + while(!((d = input[++sidx]) == '"' && !escape) && sidx 0) end(input.length); + + debug { + foreach(i, tok; tokens) + Stdout.format("{}{}", tok, (i != tokens.length-1 ? " " : "")); + Stdout.newline; + } + + return tokens; +} + +/** + * Parser a s-exp like input used by the config files. + */ +char[][][char[]] parse(char[][] tokens) { + char[][][char[]] values; + size_t i = 1; + + void err(size_t j) { + auto b = tokens[j < 5 ? 0 : j-5..j]; + auto e = tokens[j+1..j+6>$ ? $ : j+6]; + char[] tmp; + foreach(t; b) tmp ~= t ~ " "; + tmp ~= ">>>" ~ tokens[j] ~ "<<< "; + foreach(t; e) tmp ~= t ~ " "; + + throw new Exception( + "Parse: Illegal token: " ~ tmp ~ "." + ); + } + + if (tokens[0] != "(") err(0); + + void doParse(char[] prefix = null) { + if (tokens[i] == "(" || + tokens[i] == ")") err(i); + if (prefix) prefix ~= "."; + auto v = prefix ~ tokens[i++]; + //values[v] = null; + while (tokens[i] != ")") + if (tokens[i++] == "(") + doParse(v); + else + values[v] ~= tokens[i-1]; + i++; + } + + doParse(); + + return values; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/config/reflection.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/config/reflection.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,275 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.config.reflection; + +import docgen.misc.meta; +import docgen.misc.options; + +//// +// +// Macros for reading input +// +//// + +char[] _wrong(char[] key) { + return `if (val.length != 1) throw new Exception( + "Wrong number of arguments for `~key~`");`; +} + +char[] _switch(char[] stuff) { + return "switch(key) {" ~ stuff ~ "}"; +} + +char[] _parseI(char[] key) { + return `case "` ~ key ~ `":` ~ _wrong(key) ~ key ~ + `= Integer.parse(val[0]); continue;`; +} + +char[] _parseS(char[] key) { + return `case "` ~ key ~ `":` ~ _wrong(key) ~ key ~ + `= val[0]; continue;`; +} + +char[] _parseB(char[] key) { + return `case "` ~ key ~ `":` ~ _wrong(key) ~ key ~ + `= val[0] == "true" ? true : val[0] == "false" ? false : err(); continue;`; +} + +char[] _parseList(char[] key) { + return `case "` ~ key ~ `":foreach(v; val) ` ~ + key ~ `~= v; continue;`; +} + +template _parseEnum_(bool list, char[] key, V...) { + static if (V.length>1) + const char[] _parseEnum_ = + `case "` ~ V[0] ~ `":` ~ key ~ (list ? "~" : "") ~ `=` ~ V[1] ~ `; continue;` \n ~ + _parseEnum_!(list, key, V[2..$]); + else + const char[] _parseEnum_ = ""; +} + +template _parseEnum(char[] key, V...) { + const char[] _parseEnum = `case "` ~ key ~ + `":` ~ _wrong(key) ~ `switch(val[0]) {` ~ + _parseEnum_!(false, key, V) ~ + `default: err(); } continue;`; +} + +template _parseEnumList(char[] key, V...) { + const char[] _parseEnumList = `case "` ~ key ~ + `":` ~ `foreach (item; val) switch(item) {` ~ + _parseEnum_!(true, key, V) ~ + `default: err(); } continue;`; +} + +//// +// +// Reflection for properties. This code will hopefully get better when the dmdfe bugs get fixed. +// +//// + +// dmdfe bug -- Seriously WTF? +char[] fixType(char[] type) { + return type[$-1] == ' ' ? type[0..$-1] : type; +} + +// take the last part of field name +char[] getLastName(char[] field) { + if (field.length == 0) return ""; + int t = 0; + // dmdfe bug: a while loop causes index out of bounds error + for (int i=field.length-1; i >= 0; i--) + if (field[i] == '.') { t = i+1; break; } + return field[t..$]; +} + +// dmdfe bug: cannot return evalType alias +template _evalType(char[] type) { + mixin("alias "~type~" value;"); +} + +// Note: stuple wrappers needed for tuples, otherwise: +// dmdfe bug: cc1d: ../.././gcc/d/dmd/expression.c:4865: virtual Expression* DotIdExp::semantic(Scope*): Assertion `0' failed. +template evalType(char[] type) { + alias _evalType!(type).value evalType; +} + +// wraps the reflected struct and enum data inside struct because otherwise the handling becomes impossibly hard +template getType(T) { + static if(is(T == struct)) + alias Struct!(T) getType; + else static if(is(T == enum)) + alias Enum!(T) getType; + else static if(isEnumList!(T)) + alias Enum!(enumListType!(T), true) getType; + else + alias T getType; +} + +template getTypes(alias S, int idx = S.tupleof.length) { + static if(idx) + alias Tuple!(getTypes!(S, idx-1), getType!(typeof(S.tupleof[idx-1]))) getTypes; + else + alias Tuple!() getTypes; +} + +/** + * Extracts the comma separated struct field names using .tupleof.stringof. + * This is needed since the struct.tupleof[n].stringof is broken for enums and tuples in dmdfe. + * + * Bugs: handling of tuples + */ +char[] __getNames(char[] type) { + char[] tmp; + bool end = false; + + foreach(c; type[5..$]) { + if (c != ' ' && c != '(' && c != ')' && end) tmp ~= c; + if (c == ',') end = false; + if (c == '.') end = true; + } + + return tmp; +} + +template _getNames(char[] str, T...) { + static if (str.length) { + static if (str[0] == ',') + alias _getNames!(str[1..$], T, "") _getNames; + else + alias _getNames!(str[1..$], T[0..$-1], T[$-1] ~ str[0]) _getNames; + } else + alias T _getNames; +} + +template getNames(char[] str) { + alias _getNames!(__getNames(str), "") getNames; +} + +struct Struct(alias T) { + const type = "struct"; // used for pattern matching... apparently there's no other way + const name = fixType(T.stringof); // dmdfe bug: trailing space + alias STuple!(getNames!(T.tupleof.stringof)) names; + alias STuple!(getTypes!(T)) types; +} + +struct Enum(alias T, bool list = false) { + const type = list ? "enumlist" : "enum"; // used for pattern matching... apparently there's no other way + const name = T.stringof[1..$]; // dmdfe bug: returns enum base type instead enum type + alias evalType!("___"~name).tuple elements; +} + +// determines the enumtype[] type +template isEnumList(T : T[]) { + const isEnumList = T.stringof[0] == '_'; +} + +template isEnumList(T) { + const isEnumList = false; +} + +template enumListType(T : T[]) { + alias T enumListType; +} + +template enumListType(T) { + static assert(false, "Not enum list type!"); +} + +char[] createIParser(char[] field) { + return `_parseI("` ~ field ~ `") ~` \n; +} + +char[] createBParser(char[] field) { + return `_parseB("` ~ field ~ `") ~` \n; +} + +char[] createSParser(char[] field) { + return `_parseS("` ~ field ~ `") ~` \n; +} + +char[] createLParser(char[] field) { + return `_parseList("` ~ field ~ `") ~` \n; +} + +char[] createEParser(char[] field, char[] contents) { + return `_parseEnum!("` ~ field ~ `",` ~ contents ~ `) ~` \n; +} + +char[] createELParser(char[] field, char[] contents) { + return `_parseEnumList!("` ~ field ~ `",` ~ contents ~ `) ~` \n; +} + +template _makeEnumString(char[] t, E...) { + static if (E.length) + const _makeEnumString = `"` ~ E[0] ~ `", "` ~ t ~ "." ~ E[0] ~ `",` ~ + _makeEnumString!(t, E[1..$]); + else + const _makeEnumString = ""; +} + +// avoids the following dmdfe bugs: +// - Error: elements is not a type (typeof(T).elements) +// - Error: tuple is not a valid template value argument (T.elements where T is the complex type def) +template makeEnumString(char[] t, T) { + const makeEnumString = _makeEnumString!(t, T.elements); +} + +/** + * Generates code for parsing data from the configuration data structure. + */ +template makeTypeString(int i, N, T, char[] prefix) { + static assert(N.tuple.length == T.tuple.length); + static if (i < N.tuple.length) { + static if (is(T.tuple[i] == bool)) + const makeTypeString = createBParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); + else static if (is(T.tuple[i] : int)) + const makeTypeString = createIParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); + else static if (is(T.tuple[i] == char[])) + const makeTypeString = createSParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); + else static if (is(T.tuple[i] == char[][])) + const makeTypeString = createLParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); + else static if (is(T.tuple[i] == struct)) { + static if (T.tuple[i].type == "struct") + const makeTypeString = makeTypeString!(0, typeof(T.tuple[i].names), + typeof(T.tuple[i].types), prefix~N.tuple[i]~".") ~ + makeTypeString!(i+1, N, T, prefix); + else static if (T.tuple[i].type == "enum") + const makeTypeString = createEParser(prefix ~ N.tuple[i], + makeEnumString!(T.tuple[i].name, T.tuple[i])[0..$-1]) ~ + makeTypeString!(i+1, N, T, prefix); + else static if (T.tuple[i].type == "enumlist") + const makeTypeString = createELParser(prefix ~ N.tuple[i], + makeEnumString!(T.tuple[i].name, T.tuple[i])[0..$-1]) ~ + makeTypeString!(i+1, N, T, prefix); + else { + const makeTypeString = "?" ~ makeTypeString!(i+1, N, T, prefix); + static assert(false, "Unknown type"); + } + } else { + const makeTypeString = "?" ~ makeTypeString!(i+1, N, T, prefix); + static assert(false, "Unknown type"); + } + } else + const makeTypeString = ""; +} + +template makeTypeStringForStruct(alias opt) { + const makeTypeStringForStruct = makeTypeString!(0, opt.names, opt.types, opt.name~".")[0..$-2]; +} + +/* some leftovers +template handleType(T, char[] prefix="") { + static if(is(typeof(T) == struct)) { + static if(T.type == "enum") + // another dmdfe weirdness: T.stringof == "Enum!(Type)" here, but if do + // alias T handleType;, the result.stringof is "struct Enum". + alias T handleType; + } else + alias T handleType; +} +*/ + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/docgen.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/docgen.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,131 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.docgen; + +import docgen.graphutils.writers; +import docgen.config.configurator; +import docgen.document.latexgenerator; +import docgen.document.htmlgenerator; +import docgen.document.xmlgenerator; +import docgen.document.plaintextgenerator; + +//import dil.Settings; +import dil.SettingsLoader; + +import tango.core.Array; +import tango.text.Text; +import tango.io.Stdout; + +void usage() { + Stdout( + "Usage: docgen rootpath importpath_1 ... importpath_n outputdir" + ).newline; +} + +void main(char[][] args) { + dil.SettingsLoader.loadSettings(); + + Stdout(docgen_version).newline.newline; + + if (args.length<3) { + usage(); + return; + } + + Configurator config = new DefaultConfigurator(); + + auto options = config.getConfiguration(); + options.parser.rootPaths = [ args[1] ]; + options.parser.importPaths = args[2..$-1]; + options.outputDir = args[$-1]; + + alias DepGraph.Vertex Vertex; + alias DepGraph.Edge Edge; + + Module[] cachedModules; + DepGraph cachedGraph; + + void parser(ref Module[] modules, ref DepGraph depGraph) { + Edge[] edges; + Vertex[char[]] vertices; + + if (cachedGraph !is null) { + modules = cachedModules; + depGraph = cachedGraph; + return; + } + + int id = 1; + + Parser.loadModules( + options.parser.rootPaths, + options.parser.importPaths, + options.parser.strRegexps, + options.graph.includeUnlocatableModules, + options.parser.depth, + (char[] fqn, char[] path, Module m) { + if (m is null) { + if (fqn in vertices) { + debug Stdout.format("{} already set.\n", fqn); + return; + } + auto vertex = new Vertex(fqn, path, id++); + vertices[fqn] = vertex; + debug Stdout.format("Setting {} = {}.\n", fqn, path); + } else { + vertices[m.moduleFQN] = new Vertex(m.moduleFQN, m.filePath, id++); + debug Stdout.format("Setting {} = {}.\n", m.moduleFQN, m.filePath); + } + }, + (Module imported, Module importer, bool isPublic, bool isStatic) { + debug Stdout.format("Connecting {} - {}.\n", imported.moduleFQN, importer.moduleFQN); + auto edge = vertices[imported.moduleFQN].addChild(vertices[importer.moduleFQN]); + edge.isPublic = isPublic; + edge.isStatic = isStatic; + edges ~= edge; + }, + modules + ); + + modules.sort( + (Module a, Module b) { return ((new Text!(char)(a.moduleFQN)).compare(b.moduleFQN)) < 0; } + ); + + depGraph.edges = edges; + depGraph.vertices = vertices.values; + + cachedGraph = depGraph; + cachedModules = modules; + } + + GraphCache graphcache = new DefaultGraphCache(); + + foreach(format; options.outputFormats) { + DocGenerator generator; + + switch(format) { + case DocFormat.LaTeX: + Stdout("Generating LaTeX docs.."); + generator = new LaTeXDocGenerator(*options, &parser, graphcache); + break; + case DocFormat.HTML: + Stdout("Generating HTML docs.."); + generator = new HTMLDocGenerator(*options, &parser, graphcache); + break; + case DocFormat.XML: + Stdout("Generating XML docs.."); + generator = new XMLDocGenerator(*options, &parser); + break; + case DocFormat.PlainText: + Stdout("Generating plain text docs.."); + generator = new PlainTextDocGenerator(*options, &parser, graphcache); + break; + default: throw new Exception("Format not supported"); + } + + generator.generate(); + Stdout("done.").newline; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/document/generator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/document/generator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,147 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.document.generator; + +import docgen.misc.misc; +import docgen.misc.parser; +public import docgen.misc.options; +import docgen.page.writers; +import docgen.moduledoc.writers; +import docgen.graphutils.writers; +import docgen.sourcelisting.writers; +import docgen.config.configurator; +import tango.io.stream.FileStream; +import tango.io.FilePath; +import tango.io.FileScan; +debug import tango.io.Stdout; + + +alias void delegate(ref Module[], ref DepGraph) ParserDg; + +abstract class DefaultDocGenerator : DocGenerator { + protected: + + DocFormat docFormat; + auto makeFile = "make.sh"; + char[] genDir; + + DocGeneratorOptions m_options; + ParserDg m_parser; + PageWriter docWriter; + + GraphWriterFactory graphFactory; + PageWriterFactory pageFactory; + ListingWriterFactory listingFactory; + ModuleDocWriterFactory moduleDocFactory; + + Module[] modules; + DepGraph depGraph; + + public: + + this(DocGeneratorOptions options, ParserDg parser) { + m_options = options; + m_parser = parser; + + createGraphWriterFactory(); + createPageWriterFactory(); + createListingWriterFactory(); + createModuleDocWriterFactory(); + + // create output dir + (new FilePath(options.outputDir ~ "/" ~ genDir)).create(); + + // copy static files + copyStaticContent(); + } + + DocGeneratorOptions *options() { + return &m_options; + } + + protected: + + void createGraphWriterFactory() { + graphFactory = new DefaultGraphWriterFactory(this); + } + + void createPageWriterFactory() { + pageFactory = new DefaultPageWriterFactory(this); + } + + void createListingWriterFactory() { + listingFactory = new DefaultListingWriterFactory(this); + } + + void createModuleDocWriterFactory() { + moduleDocFactory = new DefaultModuleDocWriterFactory(this); + } + + char[] outPath(char[] file) { + return options.outputDir ~ "/" ~ genDir ~ "/" ~ file; + } + + void copyStaticContent() { + auto scan = new FileScan(); + scan(templateDir~options.templates.templateStyle~"/"~formatDirs[docFormat]~"/static/"); + + foreach(filePath; scan.files) { + (new FilePath(outPath(filePath.file))).copy(filePath.toString()); + } + + debug Stdout(scan.files.length)(" static files copied.\n"); + } + + FileOutput outputFile(char[] fname) { + return new FileOutput(outPath(fname)); + } + + void parseSources() { + depGraph = new DepGraph(); + m_parser(modules, depGraph); + } + + //--- + + void writeSimpleFile(char[] fname, void delegate() dg) { + auto docFile = outputFile(fname); + + docWriter.setOutput([docFile]); + dg(); + + docFile.close(); + } + + /** + * Generates "makefile" for processing e.g. .dot files. + */ + void generateMakeFile(char[][] args ...) { + writeSimpleFile(makeFile, { docWriter.generateCustomPage("makefile", args); } ); + } + +} + +abstract class DefaultCachingDocGenerator : DefaultDocGenerator, CachingDocGenerator { + private: + + GraphCache m_graphCache; + + public: + + this(DocGeneratorOptions options, ParserDg parser, GraphCache graphCache) { + super(options, parser); + m_graphCache = graphCache; + } + + GraphCache graphCache() { + return m_graphCache; + } + + protected: + + void createGraphWriterFactory() { + graphFactory = new DefaultCachingGraphWriterFactory(this); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/document/htmlgenerator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/document/htmlgenerator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,156 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.document.htmlgenerator; + +import docgen.document.generator; +import docgen.misc.misc; +import tango.io.stream.FileStream; +import tango.text.Util : replace; + +class HTMLDocGenerator : DefaultCachingDocGenerator { + private: + + auto docFileNames = [ + "index.html"[], "toc.html"[], "classes.html"[], + "modules.html"[], "files.html"[] + ]; + + auto depGraphFile = "depgraph.dot"; + auto depGraphDocFile = "depgraph.html"; + auto styleSheetFile = "default.css"; + + public: + + this(DocGeneratorOptions options, ParserDg parser, GraphCache graphcache) { + genDir = "html"; + docFormat = DocFormat.HTML; + super(options, parser, graphcache); + } + + /** + * Generates the documentation. + */ + void generate() { + parseSources(); + + docWriter = pageFactory.createPageWriter( null, docFormat ); + + // stylesheet needs to be created first to propagate the css file name + generateStyleSheet(); + + generateDoc(); + + if (options.listing.enableListings) + generateListings(); + + generateClasses(); + generateModules(); + generateDependencies(); + generateMakeFile(imageFormatExts[options.graph.imageFormat]); + } + + protected: + + /** + * Generates document skeleton. + */ + void generateDoc() { + writeSimpleFile(docFileNames[0], { + docWriter.generateFirstPage(); + }); + /* + writeSimpleFile(docFileNames[1], { + docWriter.generateTOC(modules); + docWriter.generateCustomPage("pagetemplate2", docgen_version); + });*/ + } + + /** + * Generates a global style sheet. + */ + void generateStyleSheet() { + writeSimpleFile(styleSheetFile, { + docWriter.generateCustomPage("stylesheet"); + }); + } + + /** + * Generates documentation for classes. + */ + void generateClasses() { + writeSimpleFile(docFileNames[2], { + docWriter.generateClassSection(); + docWriter.generateCustomPage("pagetemplate2", docgen_version); + }); + } + + /** + * Generates documentation for modules. + */ + void generateModules() { + writeSimpleFile(docFileNames[3], { + docWriter.generateModuleSection(modules); + docWriter.generateCustomPage("pagetemplate2", docgen_version); + }); + +// auto mdw = moduleDocFactory.createModuleDocWriter(docWriter, docFormat); + + } + + /** + * Generates source file listings. + */ + void generateListings() { + writeSimpleFile(docFileNames[4], { + docWriter.generateListingSection(modules); + + char[][] contents; + + contents ~= "("; + + foreach(mod; modules) { + auto FQN = mod.moduleFQN; + auto dstFname = replace(mod.moduleFQN.dup, '.', '_') ~ ".html"; + contents ~= `` ~ FQN ~ ""; + } + + contents ~= ")"; + + docWriter.addList(contents, false); + + docWriter.generateCustomPage("pagetemplate2", docgen_version); + }); + + auto writer = listingFactory.createListingWriter(docWriter, docFormat); + + foreach(mod; modules) { + auto dstFname = replace(mod.moduleFQN.dup, '.', '_') ~ ".html"; + + writeSimpleFile(dstFname, { + auto srcFile = new FileInput(mod.filePath); + writer.generateListing(srcFile, null, mod.moduleFQN); + srcFile.close(); + }); + } + } + + /** + * Generates dependency graphs. + */ + void generateDependencies() { + writeSimpleFile(depGraphDocFile, { + docWriter.generateDepGraphSection(); + + auto imgFile = outputFile(depGraphFile); + + auto writer = graphFactory.createGraphWriter( docWriter, GraphFormat.Dot ); + writer.generateDepGraph(depGraph, imgFile); + + imgFile.close(); + + docWriter.generateCustomPage("pagetemplate2", docgen_version); + }); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/document/latexgenerator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/document/latexgenerator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,121 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.document.latexgenerator; + +import docgen.document.generator; +import docgen.misc.misc; +import tango.io.stream.FileStream; +import tango.text.Util : replace; + +/** + * Main routine for LaTeX doc generation. + */ +class LaTeXDocGenerator : DefaultCachingDocGenerator { + private: + + auto docFileName = "document.tex"; + auto depGraphDocFile = "dependencies.tex"; + auto depGraphFile = "depgraph.dot"; + auto listingFile = "files.tex"; + auto modulesFile = "modules.tex"; + + public: + + this(DocGeneratorOptions options, ParserDg parser, GraphCache graphcache) { + genDir = "latex"; + docFormat = DocFormat.LaTeX; + + super(options, parser, graphcache); + } + + /** + * Generates the documentation. + */ + void generate() { + parseSources(); + + generateDoc(); + + if (options.listing.enableListings) + generateListings(); + + generateClasses(); + generateModules(); + generateDependencies(); + generateMakeFile(docFileName, "pdf"); + } + + protected: + + /** + * Generates document skeleton. + */ + void generateDoc() { + auto docFile = outputFile(docFileName); + docWriter = pageFactory.createPageWriter( [ docFile ], docFormat ); + + docWriter.generateFirstPage(); + docWriter.generateTOC(modules); + docWriter.generateClassSection(); + docWriter.generateModuleSection(modules); + docWriter.generateListingSection(modules); + docWriter.generateDepGraphSection(); + docWriter.generateIndexSection(); + docWriter.generateLastPage(); + + docFile.close(); + } + + /** + * Generates documentation for classes. + */ + void generateClasses() { + auto docFile = outputFile(modulesFile); + docFile.close(); + } + + /** + * Generates documentation for modules. + */ + void generateModules() { + auto docFile = outputFile(modulesFile); + docFile.close(); + } + + /** + * Generates source file listings. + */ + void generateListings() { + writeSimpleFile(listingFile, { + auto writer = listingFactory.createListingWriter(docWriter, docFormat); + + foreach(mod; modules) { + auto dstFname = replace(mod.moduleFQN.dup, '.', '_') ~ ".d"; + + auto srcFile = new FileInput(mod.filePath); + auto dstFile = outputFile(dstFname); + + writer.generateListing(srcFile, dstFile, mod.moduleFQN); + + srcFile.close(); + dstFile.close(); + } + }); + } + + /** + * Generates dependency graphs. + */ + void generateDependencies() { + writeSimpleFile(depGraphDocFile, { + auto imgFile = outputFile(depGraphFile); + + auto writer = graphFactory.createGraphWriter( docWriter, GraphFormat.Dot ); + writer.generateDepGraph(depGraph, imgFile); + + imgFile.close(); + }); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/document/plaintextgenerator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/document/plaintextgenerator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,121 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.document.plaintextgenerator; + +import docgen.document.generator; +import docgen.misc.misc; +import tango.io.stream.FileStream; +import tango.io.FilePath; +import tango.text.Util : replace; + +class PlainTextDocGenerator : DefaultCachingDocGenerator { + private: + + auto docFileNames = [ + "index.txt"[], "toc.txt"[], "classes.txt"[], + "modules.txt"[], "files.txt"[] + ]; + + auto depGraphFile = "depgraph.dot"; + auto depGraphDocFile = "depgraph.txt"; + + public: + + this(DocGeneratorOptions options, ParserDg parser, GraphCache graphcache) { + genDir = "txt"; + docFormat = DocFormat.PlainText; + + super(options, parser, graphcache); + } + + /** + * Generates the documentation. + */ + void generate() { + parseSources(); + + docWriter = pageFactory.createPageWriter( null, docFormat ); + + generateDoc(); + + if (options.listing.enableListings) + generateListings(); + + generateClasses(); + generateModules(); + generateDependencies(); + generateMakeFile(imageFormatExts[options.graph.imageFormat]); + } + + protected: + + /** + * Generates document skeleton. + */ + void generateDoc() { + writeSimpleFile(docFileNames[0], { + docWriter.generateFirstPage(); + }); + + writeSimpleFile(docFileNames[1], { + docWriter.generateTOC(modules); + }); + } + + /** + * Generates documentation for classes. + */ + void generateClasses() { + writeSimpleFile(docFileNames[2], { + docWriter.generateClassSection(); + }); + } + + /** + * Generates documentation for modules. + */ + void generateModules() { + writeSimpleFile(docFileNames[3], { + docWriter.generateModuleSection(modules); + }); + } + + /** + * Generates source file listings. + */ + void generateListings() { + writeSimpleFile(docFileNames[4], { + docWriter.generateListingSection(modules); + + char[][] contents; + + foreach(mod; modules) { + auto FQN = mod.moduleFQN; + contents ~= FQN ~ " (see " ~ replace(FQN.dup, '.', '_') ~ ".d)"; + } + + docWriter.addList(contents, false); + }); + + foreach(mod; modules) + (new FilePath(outPath(replace(mod.moduleFQN.dup, '.', '_') ~ ".d"))).copy(mod.filePath); + } + + /** + * Generates dependency graphs. + */ + void generateDependencies() { + writeSimpleFile(depGraphDocFile, { + docWriter.generateDepGraphSection(); + + auto imgFile = outputFile(depGraphFile); + + auto writer = graphFactory.createGraphWriter( docWriter, GraphFormat.Dot ); + writer.generateDepGraph(depGraph, imgFile); + + imgFile.close(); + }); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/document/xmlgenerator.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/document/xmlgenerator.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,23 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.document.xmlgenerator; + +import docgen.document.generator; +import docgen.misc.misc; +import tango.io.stream.FileStream; +import tango.text.Util : replace; + +class XMLDocGenerator : DefaultDocGenerator { + public: + + this(DocGeneratorOptions options, ParserDg parser) { + genDir = "xml"; + docFormat = DocFormat.XML; + + super(options, parser); + } + + void generate() { /* TODO */ } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/graphutils/dotwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/graphutils/dotwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,170 @@ +/** + * Author: Aziz Köksal & Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.graphutils.dotwriter; +import docgen.graphutils.writer; + +import tango.io.Print: Print; +import tango.text.convert.Layout : Layout; +import tango.io.FilePath; +import tango.text.Util; +import tango.text.convert.Sprint; +debug import tango.io.Stdout; + +/** + * Creates a graph rule file for the dot utility. + */ +class DotWriter : AbstractGraphWriter { + public: + + this(GraphWriterFactory factory, PageWriter writer) { + super(factory, writer); + } + + void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { + generateImageTag(imageFile); + + auto image = generateDepImageFile(depGraph); + auto printer = new Print!(char)(new Layout!(char), imageFile); + printer(image); + } + + protected: + + char[] generateDepImageFile(DepGraph depGraph) { + char[] image; + auto sprint = new Sprint!(char); + + auto edges = depGraph.edges; + auto vertices = depGraph.vertices; + + DepGraph.Vertex[][char[]] verticesByPckgName; + if (factory.options.graph.groupByFullPackageName || + factory.options.graph.groupByPackageNames) { + foreach (mod; vertices) { + auto parts = mod.name.delimit("."); + + if (parts.length>1) { + auto pkg = parts[0..$-1].join("."); + verticesByPckgName[pkg] ~= mod; + } + } + } + + if (factory.options.graph.highlightCyclicVertices || + factory.options.graph.highlightCyclicEdges) + depGraph.markCycles(); + + image ~= "Digraph ModuleDependencies {\n"; + + foreach (module_; vertices) { + auto nodeName = + factory.options.graph.groupByPackageNames ? + module_.name.split(".")[$-1] : + module_.name; + + image ~= sprint.format( + ` n{} [label="{}",style=filled,fillcolor={}];`\n, + module_.id, + nodeName, + module_.cyclic && factory.options.graph.highlightCyclicVertices ? + factory.options.graph.cyclicNodeColor : + module_.incoming.length == 0 && module_.outgoing.length == 0 ? + factory.options.graph.unlocatableNodeColor : + factory.options.graph.nodeColor + ); + } + + foreach (edge; edges) + image ~= sprint.format( + ` n{} -> n{}[color={}];`\n, + edge.outgoing.id, + edge.incoming.id, + edge.cyclic ? + factory.options.graph.cyclicDepColor : + edge.isPublic ? + factory.options.graph.publicDepColor ~ ",style=bold" : + factory.options.graph.depColor + ); + + if (factory.options.graph.groupByPackageNames) + + if (!factory.options.graph.groupByFullPackageName) { + foreach (packageName, vertices; verticesByPckgName) { + auto name = packageName.split("."); + + if (name.length > 1) { + char[] pkg; + foreach(part; name) { + pkg ~= part ~ "."; + image ~= sprint.format( + `subgraph "cluster_{0}" {{`\n` label="{0}"`\n, + pkg[0..$-1], + pkg[0..$-1] + ); + } + for (int i=0; i< name.length; i++) { + image ~= "}\n"; + } + } + } + } + foreach (packageName, vertices; verticesByPckgName) { + image ~= sprint.format( + ` subgraph "cluster_{0}" {{`\n` label="{0}";color=` + ~ factory.options.graph.clusterColor ~ `;`\n` `, + packageName, + packageName + ); + + foreach (module_; vertices) + image ~= sprint.format(`n{0};`, module_.id); + image ~= "\n }\n"; + } + + image ~= "}"; + + return image; + } + + void generateImageTag(OutputStream imageFile) { + // name of the .dot file + char[] fn = (cast(Object)imageFile.conduit).toString(); + fn = FilePath(fn).file; + + fn = fn[0..$-3] ~ imageFormatExts[factory.options.graph.imageFormat]; + + writer.addGraphics(fn); + } +} + +class CachingDotWriter : DotWriter { + private: + + CachingGraphWriterFactory factory; + + public: + + this(CachingGraphWriterFactory factory, PageWriter writer) { + super(factory, writer); + this.factory = factory; + } + + override void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { + generateImageTag(imageFile); + + auto cached = factory.graphCache.getCachedGraph(depGraph, GraphFormat.Dot); + + auto printer = new Print!(char)(new Layout!(char), imageFile); + + if (cached) { + printer(cached); + } else { + auto image = generateDepImageFile(depGraph); + factory.graphCache.setCachedGraph(depGraph, GraphFormat.Dot, image); + printer(image); + } + } +} + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/graphutils/modulenamewriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/graphutils/modulenamewriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,42 @@ +/** + * Author: Aziz Köksal & Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.graphutils.modulenamewriter; +import docgen.graphutils.writer; + +import tango.io.Print: Print; +import tango.text.convert.Layout : Layout; + +class ModuleNameWriter : AbstractGraphWriter { + public: + + this(GraphWriterFactory factory, PageWriter writer) { + super(factory, writer); + } + + void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { + char[][] contents; + + auto edges = depGraph.edges; + auto vertices = depGraph.vertices; + + void doList(DepGraph.Vertex[] v, uint level) { + if (!level) return; + + contents ~= "("; + + foreach (vertex; v) { + contents ~= vertex.name; + if (vertex.outgoing.length) + doList(vertex.outgoing, level-1); + } + + contents ~= ")"; + } + + doList(vertices, factory.options.graph.depth); + + writer.addList(contents, false); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/graphutils/modulepathwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/graphutils/modulepathwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,42 @@ +/** + * Author: Aziz Köksal & Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.graphutils.modulepathwriter; +import docgen.graphutils.writer; + +import tango.io.Print: Print; +import tango.text.convert.Layout : Layout; + +class ModulePathWriter : AbstractGraphWriter { + public: + + this(GraphWriterFactory factory, PageWriter writer) { + super(factory, writer); + } + + void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { + char[][] contents; + + auto edges = depGraph.edges; + auto vertices = depGraph.vertices; + + void doList(DepGraph.Vertex[] v, uint level) { + if (!level) return; + + contents ~= "("; + + foreach (vertex; v) { + contents ~= vertex.location; + if (vertex.outgoing.length) + doList(vertex.outgoing, level-1); + } + + contents ~= ")"; + } + + doList(vertices, factory.options.graph.depth); + + writer.addList(contents, false); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/graphutils/primitives.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/graphutils/primitives.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,142 @@ +/** + * Author: Aziz Köksal & Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.graphutils.primitives; + +/** + * Extendible graph class. Should support separation of concerns now better with mixins. + * Provides a method for cycle marking. + */ +class Graph(alias V, alias E, int capacity = 1) { + static class Edge { + Vertex outgoing, incoming; + //bool cyclic = true; + + this(Vertex o, Vertex i) { + outgoing = o; + incoming = i; + o.outgoingEdges ~= this; + i.incomingEdges ~= this; + } + + bool cyclic() { + return outgoing.cyclic && incoming.cyclic; + } + + mixin E; + } + + static class Vertex { + Edge[] incomingEdges; + Edge[] outgoingEdges; + bool cyclic = true; + + Edge addChild(Vertex v) { + return new Edge(v, this); + } + + Edge addParent(Vertex v) { + return v.addChild(this); + } + + Vertex[] incoming() { + Vertex[] tmp; + + foreach(edge; incomingEdges) + tmp ~= edge.outgoing; + + return tmp; + } + + Vertex[] outgoing() { + Vertex[] tmp; + + foreach(edge; outgoingEdges) + tmp ~= edge.incoming; + + return tmp; + } + + mixin V; + } + + Vertex[] vertices; + Edge[] edges; + + this() { + vertices.length = capacity; + vertices.length = 0; + edges.length = capacity; + edges.length = 0; + } + + void add(Vertex vertex) { vertices ~= vertex; } + + void add(Edge edge) { edges ~= edge; } + + void connect(Vertex from, Vertex to) { edges ~= from.addParent(to); } + + void connect(int from, int to) { connect(vertices[from], vertices[to]); } + + /** + * Starts from non-cyclic nodes and propagates two both directions. + * Bugs: marks non-cyclic imports between two cycles as cyclic. Could be fixed later if it's really needed (slow) + */ + void markCycles() { + void mark(Vertex v) { + v.cyclic = false; + foreach(o; v.outgoing) { + if (!o.cyclic) continue; + + // propagate + bool cyclic = false; + foreach(p; o.incoming) if (p.cyclic) { cyclic = true; break; } + if (!cyclic) mark(o); + } + } + + void mark2(Vertex v) { + v.cyclic = false; + foreach(o; v.incoming) { + if (!o.cyclic) continue; + + // propagate + bool cyclic = false; + foreach(p; o.outgoing) if (p.cyclic) { cyclic = true; break; } + if (!cyclic) mark2(o); + } + } + + foreach(e; vertices) + if (e.cyclic) { + if (!e.incoming.length) mark(e); + if (!e.outgoing.length) mark2(e); + } + } +} + +template Empty() {} + + +// graph elements used in dep graphs + + +template DepEdge() { + bool isPublic; /// Public import. + bool isStatic; /// Static import. +} + +template DepVertex() { + char[] name; + char[] location; + uint id; + + this(char[] name, char[] location, uint id = 0) { + this.name = name; + this.location = location; + this.id = id; + } +} + +alias Graph!(DepVertex, DepEdge, 100) DepGraph; diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/graphutils/writer.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/graphutils/writer.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,113 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.graphutils.writer; + +public import docgen.misc.misc; +public import docgen.graphutils.primitives; +public import docgen.page.writer; +debug import tango.io.Stdout; + +interface GraphWriter { + void generateDepGraph(DepGraph depGraph, OutputStream imageFile); +} + +interface GraphWriterFactory : WriterFactory { + GraphWriter createGraphWriter(PageWriter writer, GraphFormat outputFormat); +} + +interface CachingGraphWriterFactory : GraphWriterFactory { + GraphCache graphCache(); +} +/+ +/** + * Marks all cycles in the graph. + * + * May have bugs, but is a bit simpler than the previous version. + */ +void findCycles(Vertex[] vertices, Edge[] edges) { + debug void p() { + foreach(e; edges) Stderr(e.type)(" "c); + Stderr.newline; + } + + bool visit(Edge edge) { + if (edge.cycleType == CycleType.Reserved) { + edge.cycleType = CycleType.Cyclic; + version(VerboseDebug) p(); + return true; + } + + bool wasCyclic = edge.isCyclic(); + edge.cycleType = CycleType.Reserved; + version(VerboseDebug) p(); + + foreach(edge2; edge.incoming.outgoingEdges) + if (visit(edge2)) { + if (edge.isCyclic()) { + edge.cycleType = CycleType.Reserved; + wasCyclic = true; + version(VerboseDebug) p(); + continue; + } + edge.cycleType = CycleType.Cyclic; + return true; + } + + edge.cycleType = wasCyclic ? CycleType.Cyclic : CycleType.Cyclefree; + version(VerboseDebug) p(); + return false; + } + + foreach(vertex; vertices) + foreach(edge; vertex.outgoingEdges) + if (edge.cycleType == CycleType.Unspecified) { + visit(edge); + debug Stderr("*\n"); + } +} ++/ + +abstract class AbstractGraphWriter : AbstractWriter!(GraphWriterFactory), GraphWriter { + protected: + + PageWriter writer; + + public: + + this(GraphWriterFactory factory, PageWriter writer) { + super(factory); + this.writer = writer; + } +} + +class DefaultGraphCache : GraphCache { + private: + + char[][Object][GraphFormat] m_graphCache; + + public: + + char[] getCachedGraph(Object graph, GraphFormat format) { + debug Stdout("Starting graph lookup\n"); + debug Stdout(&graph, format).newline; + debug Stdout(&m_graphCache).newline; + + auto lookup1 = format in m_graphCache; + if (lookup1) { + auto lookup2 = graph in *lookup1; + if (lookup2) { + return *lookup2; + } + } + debug Stdout("Graph cache miss!\n"); + return null; + } + + void setCachedGraph(Object graph, GraphFormat format, char[] + contents) { + m_graphCache[format][graph] = contents; + debug Stdout("Graph cache updated!\n"); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/graphutils/writers.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/graphutils/writers.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,59 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.graphutils.writers; + +public import docgen.graphutils.writer; +import docgen.graphutils.dotwriter; +import docgen.graphutils.modulepathwriter; +import docgen.graphutils.modulenamewriter; + +class DefaultGraphWriterFactory : AbstractWriterFactory, GraphWriterFactory { + public: + + this(DocGenerator generator) { + super(generator); + } + + GraphWriter createGraphWriter(PageWriter writer, GraphFormat outputFormat) { + switch (outputFormat) { + case GraphFormat.Dot: + return new DotWriter(this, writer); + case GraphFormat.ModuleNames: + return new ModuleNameWriter(this, writer); + case GraphFormat.ModulePaths: + return new ModulePathWriter(this, writer); + default: + throw new Exception("Graph writer type does not exist!"); + } + } +} + +class DefaultCachingGraphWriterFactory : AbstractWriterFactory, CachingGraphWriterFactory { + public: + + CachingDocGenerator generator; + + this(CachingDocGenerator generator) { + super(generator); + this.generator = generator; + } + + GraphCache graphCache() { + return generator.graphCache; + } + + override GraphWriter createGraphWriter(PageWriter writer, GraphFormat outputFormat) { + switch (outputFormat) { + case GraphFormat.Dot: + return new CachingDotWriter(this, writer); + case GraphFormat.ModuleNames: + return new ModuleNameWriter(this, writer); + case GraphFormat.ModulePaths: + return new ModulePathWriter(this, writer); + default: + throw new Exception("Graph writer type does not exist!"); + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/lstlang0.sty --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/lstlang0.sty Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,21 @@ +%% +%% D definition (c) 2007 Jari-Matti Mäkelä +%% +\lst@definelanguage{D}% + {morekeywords={abstract,alias,align,asm,assert,auto,body,bool,break,% + byte,case,cast,catch,cdouble,cent,cfloat,char,class,const,continue,% + creal,dchar,debug,default,delegate,delete,deprecated,do,double,% + else,enum,export,extern,false,final,finally,float,for,foreach,% + foreach_reverse,function,goto,idouble,if,ifloat,import,in,inout,% + int,interface,invariant,ireal,is,lazy,long,macro,mixin,module,new,% + null,out,override,package,pragma,private,protected,public,real,ref,% + return,scope,short,static,struct,super,switch,synchronized,template,% + this,throw,true,try,typedef,typeid,typeof,ubyte,ucent,uint,ulong,% + union,unittest,ushort,version,void,volatile,wchar,while,with},% + sensitive,% + morecomment=[s]{/*}{*/},% + morecomment=[n]{/+}{+/},% + morecomment=[l]//,% + morestring=[b]",% + morestring=[b]`% + }[keywords,comments,strings]% \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/misc/meta.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/misc/meta.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,98 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.misc.meta; + +/// tuple literal workaround +template Tuple(T...) { alias T Tuple; } + +/// another tuple literal workaround (can be nested & avoids at least one dmdfe bug) +struct STuple(T...) { alias T tuple; } + + +// (a -> b), [a] -> [b] +template map(alias S, T...) { + static if (T.length) + alias Tuple!(S!(T[0]), map!(S, T[1..$])) map; + else + alias T map; +} + +/// (a -> Bool), [a] -> [a] +template filter(alias S, T...) { + static if (!T.length) + alias Tuple!() filter; + else static if (S!(T[0])) + alias Tuple!(T[0], filter!(S, T[1..$])) filter; + else + alias filter!(S, T[1..$]) filter; +} + +/// Int -> Bool +template odd(int T) { + const odd = T%2 == 1; +} + +/// Int -> Bool +template even(int T) { + const even = !odd!(T); +} + +/// a [a] -> a -- max x y = max2 x (max y) +T max(T, U...)(T a, U b) { + static if (b.length) + return a > max(b) ? a : max(b); + else + return a; +} + +/// a [a] -> a -- min x y = min2 x (min y) +T min(T, U...)(T a, U b) { + static if (b.length) + return a < min(b) ? a : min(b); + else + return a; +} + +/// Upcasts derivatives of B to B +template UpCast(B, T) { alias T UpCast; } +template UpCast(B, T : B) { alias B UpCast; } + +/// converts integer to ascii, base 10 +char[] itoa(int i) { + char[] ret; + auto numbers = "0123456789ABCDEF"; + + do { + ret = numbers[i%10] ~ ret; + i /= 10; + } while (i) + + return ret; +} + +/// Enum stuff + +template _genList(char[] pre, char[] post, T...) { + static if (T.length) + const _genList = pre ~ T[0] ~ post ~ (T.length>1 ? "," : "") ~ + _genList!(pre, post, T[1..$]); + else + const _genList = ``; +} + +/** + * Creates + * - a typedef for enum (workaround for .tupleof.stringof) + * - the enum structure + * - string array of enum items (for runtime programming) + * - string tuple of enum items (for metaprogramming - char[][] doesn't work) + */ +template createEnum(char[] tName, char[] eName, char[] arName, char[] alName, T...) { + const createEnum = + "typedef int " ~ tName ~ ";" ~ + "enum " ~ eName ~ ":" ~ tName ~ "{" ~ _genList!("", "", T) ~ "};" ~ + "char[][] " ~ arName ~ "=[" ~ _genList!(`"`, `"[]`, T) ~ "];" ~ + "alias STuple!(" ~ _genList!(`"`, `"`, T) ~ ") " ~ alName ~ ";"; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/misc/misc.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/misc/misc.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,61 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.misc.misc; +import docgen.misc.options; +import tango.io.model.IConduit : OutputStream; + +char[] docgen_version = "Dil document generator 0.1"; + +interface DocGenerator { + DocGeneratorOptions *options(); + void generate(); +} + +interface GraphCache { + char[] getCachedGraph(Object graph, GraphFormat format); + void setCachedGraph(Object graph, GraphFormat format, char[] contents); +} + +interface CachingDocGenerator : DocGenerator { + GraphCache graphCache(); +} + +interface WriterFactory { + DocGeneratorOptions *options(); +} + +abstract class AbstractWriterFactory : WriterFactory { + protected DocGenerator generator; + + public: + + DocGeneratorOptions *options() { + return generator.options; + } + + this(DocGenerator generator) { + this.generator = generator; + } +} + + +template AbstractWriter(T, int n = 0) { + abstract class AbstractWriter { + protected T factory; + protected OutputStream[] outputs; + + static if (n > 0) { + this(T factory, OutputStream[] outputs) { + this.factory = factory; + this.outputs = outputs; + assert(outputs.length == n, "Incorrect number of outputs"); + } + } + + this(T factory) { + this.factory = factory; + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/misc/options.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/misc/options.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,112 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.misc.options; + +import docgen.misc.meta; + +/** creates reflective enums, syntax: enum name + list of elements */ +template optionEnum(char[] name, T...) { + const optionEnum = createEnum!("_" ~ name, name, "__" ~ name, "___" ~ name, T); +} + +/** Supported document output formats. */ +mixin(optionEnum!("DocFormat", "LaTeX", "XML", "HTML", "PlainText")); + +/** + * Supported comment formats. + * + * http://www.stack.nl/~dimitri/doxygen/docblocks.html + * http://www.digitalmars.com/d/ddoc.html + */ +mixin(optionEnum!("CommentFormat", "Ddoc", "Doxygen")); + +/** Supported image formats. */ +mixin(optionEnum!("ImageFormat", "PNG", "SVG", "GIF", "PDF")); + +/** Image format extensions. */ +const imageFormatExts = [ "png", "svg", "gif", "pdf" ]; + +/** Supported graph writers. */ +mixin(optionEnum!("GraphFormat", "Dot", "ModuleNames", "ModulePaths")); + +struct GraphOptions { + /// image format to use for graphs + ImageFormat imageFormat; + /// maximum depth of dependencies in graphs + uint depth; + /// color of normal modules + char[] nodeColor; + /// color of the modules in cyclic dep relation + char[] cyclicNodeColor; + /// unlocatable module color + char[] unlocatableNodeColor; + /// color of the dependencies + char[] depColor; + /// color of the dependencies in cyclic dep relation + char[] cyclicDepColor; + /// color of the public dependencies + char[] publicDepColor; + /// package color + char[] clusterColor; + /// include unlocatable modules to the dep graph + bool includeUnlocatableModules; + /// highlight imports in cyclic dep relation + bool highlightCyclicEdges; + /// highlight modules in cyclic dep relation + bool highlightCyclicVertices; + /// group modules by package names in dep graph + bool groupByPackageNames; + /// group modules hierarchically or by full package name + bool groupByFullPackageName; +} + +struct ListingOptions { + /// use literate programming symbols [LaTeX] + bool literateStyle; + /// enable source code listings + bool enableListings; +} + +struct TemplateOptions { + /// project title + char[] title; + /// project version + char[] versionString; + /// copyright notice + char[] copyright; + /// paper size [LaTeX] + char[] paperSize; + /// use short file names [HTML] + bool shortFileNames; + /// page template style to use, customizable via docgen/templates + char[] templateStyle; +} + +struct ParserOptions { + /// paths to search for imports + char[][] importPaths; + /// paths to "root files" + char[][] rootPaths; + /// regexps for excluding modules + char[][] strRegexps; + /// comment format [comment parser] + CommentFormat commentFormat; + /// maximum depth of dependencies + uint depth; +} + +struct DocGeneratorOptions { + /// location for the generated output + char[] outputDir; + + /// list of document formats to be generated + DocFormat[] outputFormats; + + GraphOptions graph; + ListingOptions listing; + TemplateOptions templates; + ParserOptions parser; +} + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/misc/parser.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/misc/parser.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,210 @@ +/** + * Author: Aziz Köksal & Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.misc.parser; + +import dil.parser.Parser; +import dil.parser.ImportParser; +import dil.File; +import dil.Settings; +public import dil.semantic.Module; +import tango.text.Regex : RegExp = Regex; +import tango.io.FilePath; +import tango.io.FileSystem; +import tango.core.Array : remove; +import tango.text.Util; +import docgen.misc.meta; +debug import tango.io.Stdout; + +alias void delegate (char[] fqn, char[] path, Module module_) modDg; +alias void delegate (Module imported, Module importer, bool isPublic, bool isStatic) importDg; + +class Parser { + protected: + +// ParserOptions m_options; + + + static char[] findModuleFilePath(char[] moduleFQNPath, char[][] importPaths) { + auto filePath = new FilePath(); + foreach (importPath; importPaths) { + filePath.set(importPath); + filePath.append(moduleFQNPath); + + foreach (moduleSuffix; [".d", ".di"/*interface file*/]) + { + filePath.suffix(moduleSuffix); + debug Stdout("Trying ")(filePath.toString()).newline; + if (filePath.exists()) + return filePath.toString(); + } + } + + debug Stdout(" * ")(moduleFQNPath)(" does not exist in imports\n")(); + return null; + } + + public: + + /** + * Imports the transitive closure of imports starting from "filePath", + * limited by recursionDepth. + * + * The search can be filtered by providing a list of regexps that match the + * FQNs of modules to be ignored. + * + * Params: + * filePath = Path of the file to parse + * importPaths = Directories to look for imports + * strRegexps = Filter regexps + * IncludeUnlocatableModules = Call the delegate also for unlocatable files + * recursionDepth = How many levels of imports to follow (-1 = no limit) + * mdg = Delegate that gets called for every module found + * idg = Delegate that gets called for every import found + * modules = List of parsed modules + */ + static void loadModules(char[] filePath, char[][] importPaths, char[][] strRegexps, + bool IncludeUnlocatableModules, int recursionDepth, + modDg mdg, importDg idg, out Module[] modules) { + + loadModules([filePath], importPaths, strRegexps, IncludeUnlocatableModules, + recursionDepth, mdg, idg, modules); + } + + /** + * Imports the transitive closure of imports starting from "filePath", + * limited by recursionDepth. + * + * The search can be filtered by providing a list of regexps that match the + * FQNs of modules to be ignored. + * + * Params: + * filePaths = Paths of the files to parse + * importPaths = Directories to look for imports + * strRegexps = Filter regexps + * IncludeUnlocatableModules = Call the delegate also for unlocatable files + * recursionDepth = How many levels of imports to follow (-1 = no limit) + * mdg = Delegate that gets called for every module found + * idg = Delegate that gets called for every import found + * modules = List of parsed modules + */ + static void loadModules(char[][] filePaths, char[][] importPaths, char[][] strRegexps, + bool IncludeUnlocatableModules, int recursionDepth, + modDg mdg, importDg idg, out Module[] modules) { + + // Initialize regular expressions. + RegExp[] regexps; + foreach (strRegexp; strRegexps) + regexps ~= new RegExp(strRegexp); + + importPaths ~= "."; + + // Add directory of file and global directories to import paths. + foreach(filePath; filePaths) { + auto fileDir = (new FilePath(filePath)).folder(); + if (fileDir.length) + importPaths ~= fileDir; + } + +// importPaths ~= GlobalSettings.importPaths; + + debug foreach(path; importPaths) { + Stdout("Import path: ")(path).newline; + } + + Module[char[]] loadedModules; + + Module loadModule(char[] moduleFQNPath, int depth) { + if (depth == 0) return null; + + debug Stdout("Loading ")(moduleFQNPath).newline; + + // Return already loaded module. + auto mod_ = moduleFQNPath in loadedModules; + if (mod_ !is null) { + debug Stdout(" Already loaded.")(moduleFQNPath).newline; + return *mod_; + } + + auto FQN = replace(moduleFQNPath.dup, dirSep, '.'); + + // Ignore module names matching regular expressions. + foreach (rx; regexps) + if (rx.test(FQN)) return null; + + auto moduleFilePath = findModuleFilePath(moduleFQNPath, importPaths); + + debug Stdout(" FQN ")(FQN).newline; + debug Stdout(" Module path ")(moduleFilePath).newline; + + Module mod = null; + + if (moduleFilePath is null) { + if (IncludeUnlocatableModules) + mdg(FQN, moduleFQNPath, null); + } else { + mod = new Module(moduleFilePath); + + // Use lightweight ImportParser. +// mod.parser = new ImportParser(loadFile(moduleFilePath), moduleFilePath); + mod.parse(); + + debug Stdout(" Parsed FQN ")(mod.getFQN()).newline; + + // autoinclude dirs (similar to Java) + // running docgen in foo/bar/abc/ also finds foo/xyz/zzz.d if module names are right + { + // foo.bar.mod -> [ "foo", "bar" ] + auto modPackage = split(mod.getFQN, ".")[0..$-1]; + auto modDir = split(FileSystem.toAbsolute(new FilePath(moduleFilePath)).standard().folder(), "/"); + auto modLocation = modDir[0..modDir.remove(".")]; + + bool matches = false; + int i; + + for(i = 1; i <= min(modPackage.length, modLocation.length); i++) { + matches = true; + debug Stdout(" Decl: ")(modPackage[$-i]).newline; + debug Stdout(" Path: ")(modLocation[$-i]).newline; + if (modPackage[$-i] != modLocation[$-i]) { + matches = false; + break; + } + } + if (matches) { + auto loc = modLocation[0..$-i+1].join("/"); + debug Stdout(" Autoadding import: ")(loc).newline; + importPaths ~= loc; + } + } + + mdg(FQN, moduleFQNPath, mod); + loadedModules[moduleFQNPath] = mod; + + foreach (importDecl; mod.imports) + foreach(moduleFQN_; importDecl.getModuleFQNs(dirSep)) { + auto loaded_mod = loadModule(moduleFQN_, depth == -1 ? depth : depth-1); + + if (loaded_mod !is null) { + idg(loaded_mod, mod, importDecl.isPublic(), importDecl.isStatic()); + } else if (IncludeUnlocatableModules) { + auto tmp = new Module(null); + tmp.setFQN(replace(moduleFQN_.dup, dirSep, '.')); + idg(tmp, mod, importDecl.isPublic(), importDecl.isStatic()); + } + } + } + + return mod; + } // loadModule + + foreach(filePath; filePaths) + loadModule(filePath, recursionDepth); + + // Finished loading modules. + + // Ordered list of loaded modules. + modules = loadedModules.values; + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/misc/textutils.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/misc/textutils.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,38 @@ +/** + * Author: Aziz Köksal & Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.misc.textutils; + +// copied from Generate.d +char[] xml_escape(char[] text) +{ + char[] result; + result.length = text.length; + result.length = 0; + foreach(c; text) + switch(c) + { + case '<': result ~= "<"; break; + case '>': result ~= ">"; break; + case '&': result ~= "&"; break; + default: result ~= c; + } + return result; +} + +char[] plainTextHeading(char[] s) { + char[] line; + line.length = 80; + line[] = '='; + + return s ~ \n ~ line[0..s.length].dup ~ \n ~ \n; +} + +char[] plainTextHorizLine(int l = 80) { + char[] line; + line.length = 80; + line[] = '-'; + + return line[0..l].dup ~ \n; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/moduledoc/htmlwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/moduledoc/htmlwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,38 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.moduledoc.htmlwriter; + +import docgen.moduledoc.writer; +import docgen.misc.textutils; + + +/** + * TODO + */ +class HTMLWriter : AbstractWriter!(ModuleDocWriterFactory), ModuleDocWriter { + PageWriter writer; + + this(ModuleDocWriterFactory factory, PageWriter writer) { + super(factory); + this.writer = writer; + } + + void generateModuleDoc(Module mod, OutputStream output) { + + /* + auto inputStream = cast(FileInput)input; + auto content = new char[inputStream.length]; + auto bytesRead = inputStream.read (content); + + assert(bytesRead == inputStream.length, "Error reading source file"); + assert(output == null); + + writer.addListing( + moduleName, + xml_escape(content) + );*/ + } +} + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/moduledoc/writer.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/moduledoc/writer.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,17 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.moduledoc.writer; + +import docgen.misc.misc; +public import docgen.page.writer; +import tango.io.model.IConduit : OutputStream, InputStream; + +interface ModuleDocWriter { + void generateModuleDoc(Module mod, OutputStream output); +} + +interface ModuleDocWriterFactory : WriterFactory { + ModuleDocWriter createModuleDocWriter(PageWriter writer, DocFormat outputFormat); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/moduledoc/writers.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/moduledoc/writers.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,30 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.moduledoc.writers; + +public import docgen.moduledoc.writer; +//import docgen.moduledoc.latexwriter; +import docgen.moduledoc.htmlwriter; +//import docgen.moduledoc.xmlwriter; + +class DefaultModuleDocWriterFactory : AbstractWriterFactory, ModuleDocWriterFactory { + this(DocGenerator generator) { + super(generator); + } + + ModuleDocWriter createModuleDocWriter(PageWriter writer, DocFormat outputFormat) { + switch (outputFormat) {/* + case DocFormat.LaTeX: + return new LaTeXWriter(this, writer); + case DocFormat.XML: + return new XMLWriter(this, writer);*/ + case DocFormat.HTML: + return new HTMLWriter(this, writer); + default: + throw new Exception("Moduledoc writer type does not exist!"); + } + } +} + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/page/htmlwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/page/htmlwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,121 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.page.htmlwriter; + +import docgen.page.writer; +import docgen.misc.textutils; +import tango.io.FileConduit : FileConduit; +import tango.text.convert.Sprint; +import tango.io.FilePath; + +/** + * Writes a HTML document skeleton. + */ +class HTMLWriter : AbstractPageWriter!("html") { + private: + + char[] styleSheetFile; + + public: + + this(PageWriterFactory factory, OutputStream[] outputs) { + super(factory); + } + + override void generateClassSection() { + print.format(getTemplate("classes"), factory.options.templates.title); + } + + override void generateModuleSection(Module[] modules) { + print.format(getTemplate("modules"), factory.options.templates.title); + } + + override void generateListingSection(Module[] modules) { + print.format(getTemplate("listings"), factory.options.templates.title); + } + + override void generateDepGraphSection() { + print.format(getTemplate("dependencies"), factory.options.templates.title); + } + + void generateFirstPage() { + print.format( + getTemplate("firstpage"), + factory.options.templates.title, + factory.options.templates.versionString, + factory.options.templates.copyright + ); + + footer(); + } + + /** + * A hack for figuring out the stylesheet file name. + */ + override void generateCustomPage(char[] name, char[][] args ...) { + super.generateCustomPage(name, args); + + if (name == "stylesheet") { + styleSheetFile = (new FilePath( + (cast(Object)outputs[0].conduit).toString())).file(); + } + } + + /** + * Overrides the default template fetcher in order to + * provide a consistent layout for all pages. + */ + override char[] getTemplate(char[] name) { + auto content = super.getTemplate(name); + + foreach(pageName; [ + "firstpage"[], "toc"[], "classes"[], "modules"[], "listing"[], + "listings"[], "dependencies"[], "lastpage"[] ]) { + if (name == pageName) { + auto sprint = new Sprint!(char)(5120); + char[] title = factory.options.templates.title ~ " "; + switch(name) { + case "firstpage": title ~= "Documentation"; break; + case "toc": title ~= "TOC"; break; + case "classes": title ~= "Class index"; break; + case "modules": title ~= "Module index"; break; + case "listing": title ~= "File contents"; break; + case "listings": title ~= "File index"; break; + case "dependencies": title ~="Dependencies"; break; + } + return + sprint.format(super.getTemplate("pagetemplate"), styleSheetFile, title) ~ + content; + } + } + + return content; + } + + void addList(char[][] contents, bool ordered) { + foreach(item; contents) { + switch(item) { + case "(": print(ordered ? "
    " : "
      "); continue; + case ")": print(ordered ? "
" : ""); continue; + default: print("
  • ")(item)("
    • "); + } + } + } + + override void addListing(char[] moduleName, char[] contents, bool inline) { + print.format(getTemplate("listing"), moduleName, contents); + + footer(); + } + + protected: + + /** + * Writes the page footer. + */ + void footer() { + print.format(getTemplate("pagetemplate2"), docgen_version); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/page/latexwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/page/latexwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,39 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.page.latexwriter; + +import docgen.page.writer; +import tango.io.FileConduit : FileConduit; + +/** + * Writes a LaTeX document skeleton. + */ +class LaTeXWriter : AbstractPageWriter!("latex", 1) { + this(PageWriterFactory factory, OutputStream[] outputs) { + super(factory, outputs); + } + + void generateFirstPage() { + print.format( + getTemplate("firstpage"), + factory.options.templates.paperSize, + factory.options.templates.title, + factory.options.templates.versionString, + docgen_version, + timeNow(), + factory.options.listing.literateStyle ? "" : "%" + ); + } + + void addList(char[][] contents, bool ordered) { + foreach(item; contents) { + switch(item) { + case "(": print(ordered ? "\\begin{enumerate}" : "\\begin{itemize}"); continue; + case ")": print(ordered ? "\\end{enumerate}" : "\\end{itemize}"); continue; + default: print("\\item")(item)(\n); + } + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/page/plaintextwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/page/plaintextwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,61 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.page.plaintextwriter; + +import docgen.page.writer; +import docgen.misc.textutils; +import tango.io.FileConduit : FileConduit; + +/** + * Writes a plain text document skeleton. + */ +class PlainTextWriter : AbstractPageWriter!("plaintext") { + this(PageWriterFactory factory, OutputStream[] outputs) { + super(factory); + } + + override void generateTOC(Module[] modules) { + print.format(getTemplate("toc")); + } + + override void generateModuleSection(Module[] modules) { + print.format(getTemplate("modules")); + } + + override void generateListingSection(Module[] modules) { + print.format(getTemplate("listings")); + } + + void generateDepGraphSection() { + print.format(getTemplate("dependencies")); + } + + void generateFirstPage() { + print.format(getTemplate("firstpage"), + plainTextHeading(factory.options.templates.title ~ " Reference Manual"), + factory.options.templates.versionString, + docgen_version, + timeNow() + ); + } + + void addList(char[][] contents, bool ordered) { + uint[] counters; + foreach(item; contents) { + switch(item) { + case "(": counters ~= 1; continue; + case ")": counters.length = counters.length - 1; continue; + default: + if (counters.length>0) + for (int i=0; i <= counters.length; i++) + print(" "); + if (ordered) + print(++counters[$-1])(". ")(item)(\n); + else + print("* ")(item)(\n); + } + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/page/writer.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/page/writer.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,231 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.page.writer; + +public import docgen.misc.misc; +public import docgen.misc.options; +public import docgen.misc.parser; +import tango.io.model.IConduit : OutputStream; +import tango.time.chrono.Gregorian; +import tango.text.locale.Core; +import tango.time.WallClock; +import tango.text.convert.Sprint; +import tango.io.stream.FileStream; +import tango.io.Stdout; +import tango.io.Print: Print; +import tango.text.convert.Layout : Layout; +import tango.io.FilePath; +import tango.io.FileScan; + +const templateDir = "docgen/templates/"; + +const formatDirs = [ "latex"[], "xml"[], "html"[], "plaintext"[] ]; + +/** + * Writes the logical subcomponents of a document, + * e.g. sections, embedded graphics, lists + */ +interface PageWriter { + /** + * Updates the outputstreams. + */ + void setOutput(OutputStream[] outputs); + + /** + * Generates the first page(s). + */ + void generateFirstPage(); + + /** + * Generates table of contents. + */ + void generateTOC(Module[] modules); + + /** + * Generates class documentation section. + */ + void generateClassSection(); + + /** + * Generates module documentation section. + */ + void generateModuleSection(Module[] modules); + + /** + * Generates source code listing section. + */ + void generateListingSection(Module[] modules); + + /** + * Generates dependency graph section. + */ + void generateDepGraphSection(); + + /** + * Generates an index section. + */ + void generateIndexSection(); + + /** + * Generates the last page(s). + */ + void generateLastPage(); + + /** + * Generates a page using a custom template file. + * + * Some examples: style sheet, DTD files, makefiles. + */ + void generateCustomPage(char[] name, char[][] args ...); + + // --- page components + // + /* + * Adds an external graphics file. + */ + void addGraphics(char[] imageFile); + + /** + * Adds a source code listing. + */ + void addListing(char[] moduleName, char[] contents, bool inline = true); + + /** + * Adds a list of items. + */ + void addList(char[][] contents, bool ordered); +} + +interface PageWriterFactory : WriterFactory { + PageWriter createPageWriter(OutputStream[] outputs, DocFormat outputFormat); +} + +template AbstractPageWriter(char[] format, int n = 0) { + abstract class AbstractPageWriter : AbstractWriter!(PageWriterFactory, n), PageWriter { + protected: + + char[][char[]] m_templates; + Print!(char) print; + + public: + + this(PageWriterFactory factory, OutputStream[] outputs) { + this(factory); + setOutput(outputs); + } + + void setOutput(OutputStream[] outputs) { + this.outputs = outputs; + static if (n > 0) + assert(outputs.length == n, "Incorrect number of outputs"); + + print = new Print!(char)(new Layout!(char), outputs[0]); + } + + void generateTOC(Module[] modules) { + print.format(getTemplate("toc")); + } + + void generateClassSection() { + print.format(getTemplate("classes")); + } + + void generateModuleSection(Module[] modules) { + print.format(getTemplate("modules")); + } + + void generateListingSection(Module[] modules) { + print.format(getTemplate("listings")); + } + + void generateDepGraphSection() { + print.format(getTemplate("dependencies")); + } + + void generateIndexSection() { + print.format(getTemplate("index")); + } + + void generateLastPage() { + print.format(getTemplate("lastpage")); + } + + void generateCustomPage(char[] name, char[][] args ...) { + switch(args.length) { + case 0: print.format(getTemplate(name)); break; + case 1: print.format(getTemplate(name), args[0]); break; + case 2: print.format(getTemplate(name), args[0], args[1]); break; + case 3: print.format(getTemplate(name), args[0], args[1], args[2]); break; + default: throw new Exception("Too many arguments"); + } + } + + //--- + + void addGraphics(char[] imageFile) { + print.format(getTemplate("graphics"), imageFile); + } + + void addListing(char[] moduleName, char[] contents, bool inline) { + print.format(getTemplate("listing"), moduleName, contents); + } + + protected: + + this(PageWriterFactory factory) { + super(factory); + + auto scan = new FileScan(); + scan(templateDir~factory.options.templates.templateStyle~"/"~format~"/", ".tpl"); + + debug Stdout(scan.files.length)(" template files loaded.\n"); + + foreach(tpl; scan.files) { + m_templates[tpl.name] = loadTemplate(tpl.toString()); + } + } + + char[] getTemplate(char[] name) { + auto tpl = name in m_templates; + assert(tpl, "Error: template ["~format~"/"~name~"] not found!"); + return *tpl; + } + + char[] loadTemplate(char[] fn) { + scope(failure) { + Stderr("Warning: error opening template "~fn~"."); + return null; + } + + auto file = new FileInput(fn); + auto content = new char[file.length]; + auto bytesRead = file.read(content); + + assert(bytesRead == file.length, "Error reading template"); + + file.close(); + + return content; + } + + char[] timeNow() { + auto n = WallClock.now; + auto c = Gregorian.generic; + auto d = c.toDate(n); + auto sprint = new Sprint!(char); + + auto culture = new Culture("en-GB"); + auto dateTimeFormat = culture.dateTimeFormat(); + + return sprint.format("{} {} {} {}", + dateTimeFormat.getAbbreviatedDayName(c.getDayOfWeek(n)), + 1,//d.day(), + //dateTimeFormat.getAbbreviatedMonthName(d.month()), + 2,//d.month(), + 3//d.year()) //FIXME: something is broken here (Error: function expected before (), not *(&d + 8u) of type uint) + ).dup; + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/page/writers.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/page/writers.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,32 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.page.writers; + +public import docgen.page.writer; +import docgen.page.htmlwriter; +import docgen.page.xmlwriter; +import docgen.page.plaintextwriter; +import docgen.page.latexwriter; + +class DefaultPageWriterFactory : AbstractWriterFactory, PageWriterFactory { + this(DocGenerator generator) { + super(generator); + } + + PageWriter createPageWriter(OutputStream[] outputs, DocFormat outputFormat) { + switch (outputFormat) { + case DocFormat.LaTeX: + return new LaTeXWriter(this, outputs); + case DocFormat.XML: + return new XMLWriter(this, outputs); + case DocFormat.HTML: + return new HTMLWriter(this, outputs); + case DocFormat.PlainText: + return new PlainTextWriter(this, outputs); + default: + throw new Exception("Document writer type does not exist!"); + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/page/xmlwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/page/xmlwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,56 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.page.xmlwriter; + +import docgen.page.writer; +import docgen.misc.textutils; +import tango.io.FileConduit : FileConduit; + +//TODO: this is mostly broken now + +/** + * TODO + */ +class XMLWriter : AbstractPageWriter!("xml", 1) { + this(PageWriterFactory factory, OutputStream[] outputs) { + super(factory, outputs); + } + + void generateTOC(Module[] modules) { + // TODO + print.format(getTemplate("toc")); + } + + void generateModuleSection() { + // TODO + print.format(getTemplate("modules")); + } + + void generateListingSection() { + // TODO + print.format(getTemplate("listings")); + } + + void generateDepGraphSection() { + // TODO + print.format(getTemplate("dependencies")); + } + + void generateIndexSection() { } + + void generateLastPage() { } + + void generateFirstPage() { } + + void addList(char[][] contents, bool ordered) { + foreach(item; contents) { + switch(item) { + case "(": print(ordered ? "
      " : "
        "); continue; + case ")": print(ordered ? "
    " : ""); continue; + default: print("
  • ")(xml_escape(item))("
    • "); + } + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/sourcelisting/htmlwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/sourcelisting/htmlwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,39 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.sourcelisting.htmlwriter; + +import docgen.sourcelisting.writer; +import docgen.misc.textutils; +//import dil.Parser; +import tango.io.stream.FileStream; + + +/** + * TODO + */ +class HTMLWriter : AbstractWriter!(ListingWriterFactory), ListingWriter { + PageWriter writer; + + this(ListingWriterFactory factory, PageWriter writer) { + super(factory); + this.writer = writer; + } + + //void generateListing(Parser parser) { /* TODO */ } + + void generateListing(InputStream input, OutputStream output, char[] moduleName) { + auto inputStream = cast(FileInput)input; + auto content = new char[inputStream.length]; + auto bytesRead = inputStream.read (content); + + assert(bytesRead == inputStream.length, "Error reading source file"); + assert(output == null); + + writer.addListing( + moduleName, + xml_escape(content) + ); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/sourcelisting/latexwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/sourcelisting/latexwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,32 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.sourcelisting.latexwriter; + +import docgen.sourcelisting.writer; +//import dil.Parser; +import tango.io.FilePath; + +/** + * Adds a code listing section for the given file. + */ +class LaTeXWriter : AbstractWriter!(ListingWriterFactory), ListingWriter { + PageWriter writer; + + this(ListingWriterFactory factory, PageWriter writer) { + super(factory); + this.writer = writer; + } + + //void generateListing(Parser parser) { /* TODO */ } + + void generateListing(InputStream input, OutputStream output, char[] moduleName) { + output.copy(input); + + writer.addListing( + moduleName, + FilePath((cast(Object)output.conduit).toString()).file + ); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/sourcelisting/writer.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/sourcelisting/writer.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,19 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.sourcelisting.writer; + +import docgen.misc.misc; +public import docgen.page.writer; +//import dil.Parser; +import tango.io.model.IConduit : OutputStream, InputStream; + +interface ListingWriter { + //void generateListing(Parser parser); + void generateListing(InputStream input, OutputStream output, char[] moduleName); +} + +interface ListingWriterFactory : WriterFactory { + ListingWriter createListingWriter(PageWriter writer, DocFormat outputFormat); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/sourcelisting/writers.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/sourcelisting/writers.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,29 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.sourcelisting.writers; + +public import docgen.sourcelisting.writer; +import docgen.sourcelisting.latexwriter; +import docgen.sourcelisting.htmlwriter; +import docgen.sourcelisting.xmlwriter; + +class DefaultListingWriterFactory : AbstractWriterFactory, ListingWriterFactory { + this(DocGenerator generator) { + super(generator); + } + + ListingWriter createListingWriter(PageWriter writer, DocFormat outputFormat) { + switch (outputFormat) { + case DocFormat.LaTeX: + return new LaTeXWriter(this, writer); + case DocFormat.XML: + return new XMLWriter(this, writer); + case DocFormat.HTML: + return new HTMLWriter(this, writer); + default: + throw new Exception("Listing writer type does not exist!"); + } + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/sourcelisting/xmlwriter.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/sourcelisting/xmlwriter.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,23 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.sourcelisting.xmlwriter; + +import docgen.sourcelisting.writer; +//import dil.Parser; + +/** + * TODO + */ +class XMLWriter : AbstractWriter!(ListingWriterFactory), ListingWriter { + PageWriter writer; + + this(ListingWriterFactory factory, PageWriter writer) { + super(factory); + this.writer = writer; + } + + //void generateListing(Parser parser) { /* TODO */ } + void generateListing(InputStream input, OutputStream output, char[] moduleName) { /* TODO */ } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/README --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/README Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,14 @@ +The docgen uses the following template file names by default + +firstpage +toc +classes +modules +listings +dependencies +index +lastpage +langdef +makefile +graphics +listing diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/classes.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/classes.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +

    {0} Class List

    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/dependencies.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/dependencies.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +

    {} Dependencies

    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/firstpage.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/firstpage.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +

    {0} Documentation

    +

    {1}

    +

    © {2}

    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/graphics.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/graphics.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +

    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/listing.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/listing.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,2 @@ +

    {0} Contents

    +
    {1}
    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/listings.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/listings.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +

    {0} File List

    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/makefile.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/makefile.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,7 @@ +#!/bin/sh + +for i in *.dot; do + F=`echo $i|sed 's/dot/{0}/'` + dot $i -T{0} -o$F +done + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/modules.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/modules.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +

    {0} Module List

    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/pagetemplate.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/pagetemplate.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,18 @@ + + + + {1} + + + + + +
    + +
    diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/pagetemplate2.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/pagetemplate2.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,4 @@ +
    +

    Generated by {0}

    + + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/static/tab_b.gif Binary file src/docgen/templates/default/html/static/tab_b.gif has changed diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/static/tab_l.gif Binary file src/docgen/templates/default/html/static/tab_l.gif has changed diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/static/tab_r.gif Binary file src/docgen/templates/default/html/static/tab_r.gif has changed diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/static/tabs.css --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/static/tabs.css Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,102 @@ +/* tabs styles, based on http://www.alistapart.com/articles/slidingdoors */ + +DIV.tabs +{ + float : left; + width : 100%; + background : url("tab_b.gif") repeat-x bottom; + margin-bottom : 4px; +} + +DIV.tabs UL +{ + margin : 0px; + padding-left : 10px; + list-style : none; +} + +DIV.tabs LI, DIV.tabs FORM +{ + display : inline; + margin : 0px; + padding : 0px; +} + +DIV.tabs FORM +{ + float : right; +} + +DIV.tabs A +{ + float : left; + background : url("tab_r.gif") no-repeat right top; + border-bottom : 1px solid #84B0C7; + font-size : x-small; + font-weight : bold; + text-decoration : none; +} + +DIV.tabs A:hover +{ + background-position: 100% -150px; +} + +DIV.tabs A:link, DIV.tabs A:visited, +DIV.tabs A:active, DIV.tabs A:hover +{ + color: #1A419D; +} + +DIV.tabs SPAN +{ + float : left; + display : block; + background : url("tab_l.gif") no-repeat left top; + padding : 5px 9px; + white-space : nowrap; +} + +DIV.tabs INPUT +{ + float : right; + display : inline; + font-size : 1em; +} + +DIV.tabs TD +{ + font-size : x-small; + font-weight : bold; + text-decoration : none; +} + + + +/* Commented Backslash Hack hides rule from IE5-Mac \*/ +DIV.tabs SPAN {float : none;} +/* End IE5-Mac hack */ + +DIV.tabs A:hover SPAN +{ + background-position: 0% -150px; +} + +DIV.tabs LI.current A +{ + background-position: 100% -150px; + border-width : 0px; +} + +DIV.tabs LI.current SPAN +{ + background-position: 0% -150px; + padding-bottom : 6px; +} + +DIV.nav +{ + background : none; + border : none; + border-bottom : 1px solid #84B0C7; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/stylesheet.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/stylesheet.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,33 @@ +@import "tabs.css"; + +body {{ +background: #fff; +} + +h1, h2 {{ +text-align: center; +} + +h1 {{ +font-size: 1.5em; +} + +h2#version {{ +font-size: 0.8em; +} + +h2 {{ +font-size: 1.1em; +} + +hr {{ +height: 0; +border: 0; +border-bottom: 1px solid black; +} + +#generator {{ +text-align: right; +font-size: small; +font-style: italic; +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/html/toc.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/html/toc.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,15 @@ + + + + {0} Reference Manual + + + + + + +

    Table of Contents

    +
    +{0} + + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/classes.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/classes.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,2 @@ +\chapter{{Class documentation} +\input{{classes} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/dependencies.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/dependencies.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,2 @@ +\chapter{{Dependency diagram} +\input{{dependencies} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/firstpage.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/firstpage.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,50 @@ +\documentclass[{0}]{{book} +\usepackage{{a4wide} +\usepackage{{makeidx} +\usepackage{{fancyhdr} +\usepackage{{graphicx} +\usepackage{{hyperref} +\usepackage{{multicol} +\usepackage{{float} +\usepackage{{textcomp} +\usepackage{{alltt} +\usepackage[utf8]{{inputenc} +\usepackage{{listings} +\lstnewenvironment{{dcode} +{{ \lstset{{language=d} } +{{} +\lstset{{ + {5} literate= + {5} {{<=}{{{{$\leq$}}1 + {5} {{>=}{{{{$\geq$}}1 + {5} {{!=}{{{{$\neq$}}1 + {5} {{...}{{{{$\dots$}}1 + {5} {{~}{{{{$\sim$}}1, + stringstyle=\ttfamily, + inputencoding=utf8, + extendedchars=false, + columns=fixed, + basicstyle=\small +} +\hypersetup{{backref,colorlinks=true} +\makeindex +\setcounter{{tocdepth}{{1} +\newcommand{{\clearemptydoublepage}{{\newpage{{\pagestyle{{empty}\cleardoublepage}} +\def\thechapter{{\Roman{{chapter}} +\def\thesection{{\arabic{{chapter}.\arabic{{section}} +% \renewcommand{{\footrulewidth}{{0.4pt} + +\begin{{document} + +\begin{{titlepage} +\vspace*{{7cm} +\begin{{center} +{{\Large {1} Reference Manual\\[1ex]\large {2} }\\ +\vspace*{{1cm} +{{\large Generated by {3} }\\ +\vspace*{{0.5cm} +{{\small {4} }\\ +\end{{center} +\end{{titlepage} + +\clearemptydoublepage diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/graphics.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/graphics.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +\includegraphics[width=1\textwidth,height=1\textheight,keepaspectratio]{{{0}} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/index.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/index.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +\printindex diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/lastpage.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/lastpage.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +\end{{document} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/listing.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/listing.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +\section{{Module {0}} +\lstinputlisting[language=d]{{{1}} +\clearpage \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/listings.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/listings.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,2 @@ +\chapter{{File listings} +\input{{files} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/makefile.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/makefile.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,10 @@ +#!/bin/sh + +for i in *.dot; do + F=`echo $i|sed 's/dot/{1}/'` + dot $i -T{1} -o$F +done + +pdflatex {0} +makeindex document +pdflatex {0} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/modules.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/modules.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,2 @@ +\chapter{{Module documentation} +\input{{modules} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/static/lstlang0.sty --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/static/lstlang0.sty Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,22 @@ +%% +%% D definition (c) 2007 Jari-Matti Mäkelä +%% +\lst@definelanguage{D}% + {morekeywords={abstract,alias,align,asm,assert,auto,body,bool,break,% + byte,case,cast,catch,cdouble,cent,cfloat,char,class,const,continue,% + creal,dchar,debug,default,delegate,delete,deprecated,do,double,% + else,enum,export,extern,false,final,finally,float,for,foreach,% + foreach_reverse,function,goto,idouble,if,ifloat,import,in,inout,% + int,interface,invariant,ireal,is,lazy,long,macro,mixin,module,new,% + null,out,override,package,pragma,private,protected,public,real,ref,% + return,scope,short,static,struct,super,switch,synchronized,template,% + this,throw,true,try,typedef,typeid,typeof,ubyte,ucent,uint,ulong,% + union,unittest,ushort,version,void,volatile,wchar,while,with},% + sensitive,% + + morecomment=[s]{/*}{*/},% + morecomment=[n]{/+}{+/},% + morecomment=[l]//,% + morestring=[b]",% + morestring=[b]`% + }[keywords,comments,strings]% diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/latex/toc.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/latex/toc.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,6 @@ +\tableofcontents +\thispagestyle{{empty} + +\clearemptydoublepage + +\setcounter{{page}{{1} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/classes.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/classes.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +Class List +---------- + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/dependencies.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/dependencies.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +Module Dependencies +------------------- + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/firstpage.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/firstpage.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,7 @@ +{0} + +Version {1} + +Generated by {2} + +{3} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/graphics.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/graphics.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,1 @@ +See {}. \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/listing.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/listing.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +Module {0} +See {1}. + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/listings.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/listings.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +File listings +------------- + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/makefile.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/makefile.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,7 @@ +#!/bin/sh + +for i in *.dot; do + F=`echo $i|sed 's/dot/{0}/'` + dot $i -T{0} -o$F +done + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/modules.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/modules.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,3 @@ +Module List +----------- + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/templates/default/plaintext/toc.tpl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/templates/default/plaintext/toc.tpl Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,10 @@ +Table of Contents +----------------- + +1. Class documentation + +2. Module documentation + +3. File listings + +4. Dependency diagram diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/tests/common.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/tests/common.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,25 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.tests.common; + +import docgen.misc.misc; +import docgen.misc.options; +import docgen.config.configurator; + +class TestDocGenerator : DocGenerator { + Configurator config; + + this() { + config = new DefaultConfigurator(); + } + + public void generate() { + + } + + public DocGeneratorOptions *options() { + return config.getConfiguration(); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/tests/doctemplate.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/tests/doctemplate.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,30 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.tests.doctemplate; + +import docgen.tests.common; +import docgen.page.writers; +import tango.io.FileConduit; + +// doc template +//@unittest +void doctemplate1() { + auto gen = new TestDocGenerator; + auto fname = "doctemplate.tex"; + + auto gwf = new DefaultPageWriterFactory(gen); + auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); + auto writer = gwf.createPageWriter( [ file ], DocFormat.LaTeX ); + + writer.generateFirstPage(); + writer.generateTOC(null); + writer.generateModuleSection(null); + writer.generateListingSection(null); + writer.generateDepGraphSection(); + writer.generateIndexSection(); + writer.generateLastPage(); + + file.close(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/tests/graphs.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/tests/graphs.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,160 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.tests.graphs; + +import docgen.tests.common; +import docgen.misc.parser; +import docgen.graphutils.writers; +import docgen.page.writers; +import tango.io.FileConduit; +import dil.semantic.Module; + +alias DepGraph.Edge Edge; +alias DepGraph.Vertex Vertex; + +void saveDefaultGraph(DepGraph depGraph, char[] fname) { + auto gen = new TestDocGenerator; + gen.options.graph.highlightCyclicVertices = true; + gen.options.graph.imageFormat = ImageFormat.SVG; + //gen.options.graph.graphFormat = GraphFormat.ModuleNames; + //gen.options.graph.graphFormat = GraphFormat.ModulePaths; + gen.options.graph.depth = 5; + auto ddf = new DefaultPageWriterFactory(gen); + auto gwf = new DefaultGraphWriterFactory(gen); + auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); + auto file2 = new FileConduit("docgen/teststuff/" ~ fname ~ "-2", FileConduit.WriteCreate); + + auto writer = gwf.createGraphWriter( + ddf.createPageWriter( [ file2 ], DocFormat.LaTeX), + GraphFormat.Dot + ); + + writer.generateDepGraph(depGraph, file); + + file.close(); + file2.close(); +} + +// no edges +//@unittest +void graph1() { + auto g = new DepGraph; + g.add(new Vertex("mod_a", "path.to.mod_a", 1)); + g.add(new Vertex("mod_b", "path.to.mod_b", 2)); + g.add(new Vertex("mod_c", "path.to.mod_c", 3)); + + saveDefaultGraph(g, "graph1.dot"); +} + + +// simple tree structure +//@unittest +void graph2() { + auto g = new DepGraph; + g.add(new Vertex("mod_a", "path.to.mod_a", 1)); + g.add(new Vertex("mod_b", "path.to.mod_b", 2)); + g.add(new Vertex("mod_c", "path.to.mod_c", 3)); + g.add(new Vertex("mod_d", "path.to.mod_d", 4)); + + g.connect(1, 0); + g.connect(2, 0); + g.connect(3, 2); + + saveDefaultGraph(g, "graph2.dot"); +} + +// circular imports +//@unittest +void graph3() { + auto g = new DepGraph; + g.add(new Vertex("mod_a", "path.to.mod_a", 1)); + g.add(new Vertex("mod_b", "path.to.mod_b", 2)); + g.add(new Vertex("mod_c", "path.to.mod_c", 3)); + g.add(new Vertex("mod_d", "path.to.mod_d", 4)); + + g.connect(1, 0); + g.connect(2, 1); + g.connect(0, 2); + + saveDefaultGraph(g, "graph3.dot"); +} + +// more complex graph +//@unittest +void graph4() { + auto g = new DepGraph; + g.add(new Vertex("mod_a", "path.to.mod_a", 1)); + g.add(new Vertex("mod_b", "path.to.mod_b", 2)); + g.add(new Vertex("mod_c", "path.to.mod_c", 3)); + g.add(new Vertex("mod_d", "path.to.mod_d", 4)); + g.add(new Vertex("mod_e", "path.to.mod_e", 5)); + g.add(new Vertex("mod_f", "path.to.mod_f", 6)); + g.add(new Vertex("mod_g", "path.to.mod_g", 7)); + + g.connect(1, 0); + g.connect(2, 1); + g.connect(0, 2); + g.connect(0, 3); + g.connect(0, 4); + g.connect(3, 1); + g.connect(4, 1); + g.connect(0, 6); + g.connect(5, 1); + g.connect(5, 6); + g.connect(6, 0); + + saveDefaultGraph(g, "graph4.dot"); +} + + +// parses the test modules and creates a dep graph +//@unittest +void graph5() { + auto gen = new TestDocGenerator; + gen.options.graph.highlightCyclicVertices = true; + gen.options.graph.imageFormat = ImageFormat.PDF; + gen.options.outputFormats = [ DocFormat.LaTeX ]; + auto fname = "dependencies.tex"; + auto imgFname = "depgraph.dot"; + + auto ddf = new DefaultPageWriterFactory(gen); + auto gwf = new DefaultGraphWriterFactory(gen); + auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); + auto imgFile = new FileConduit("docgen/teststuff/" ~ imgFname, FileConduit.WriteCreate); + + Module[] modules; + Edge[] edges; + Vertex[char[]] vertices; + int id = 1; + + Parser.loadModules( + [ "c" ], [ "docgen/teststuff/" ], + null, true, -1, + (char[] fqn, char[] path, Module m) { + vertices[m.moduleFQN] = new DepGraph.Vertex(m.moduleFQN, m.filePath, id++); + }, + (Module imported, Module importer, bool isPublic, bool isStatic) { + auto edge = vertices[imported.moduleFQN].addChild(vertices[importer.moduleFQN]); + edge.isPublic = isPublic; + edge.isStatic = isStatic; + edges ~= edge; + }, + modules + ); + + auto writer = gwf.createGraphWriter( + ddf.createPageWriter( [ file ], DocFormat.LaTeX ), + GraphFormat.Dot + ); + + auto graph = new DepGraph; + graph.edges = edges; + graph.vertices = vertices.values; + + writer.generateDepGraph(graph, imgFile); + + file.close(); + imgFile.close(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/tests/listing.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/tests/listing.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,51 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.tests.listing; + +import docgen.misc.parser; +import docgen.tests.common; +import docgen.sourcelisting.writers; +import docgen.page.writers; +import tango.io.FileConduit; +import tango.text.Util; + +// doc template +//@unittest +void listing1() { + auto gen = new TestDocGenerator; + gen.options.outputFormats = [ DocFormat.LaTeX ]; + auto fname = "files.tex"; + + auto ddf = new DefaultPageWriterFactory(gen); + auto dlwf = new DefaultListingWriterFactory(gen); + auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); + + + Module[] modules; + + Parser.loadModules( + [ "c" ], [ "docgen/teststuff/" ], + null, true, -1, + (char[] fqn, char[] path, Module m) {}, + (Module imported, Module importer, bool isPublic, bool isStatic) {}, + modules + ); + + foreach(mod; modules) { + auto dstFname = replace(mod.moduleFQN.dup, '.', '_'); + + auto srcFile = new FileConduit(mod.filePath); + auto dstFile = new FileConduit("docgen/teststuff/_" ~ dstFname ~ ".d", FileConduit.WriteCreate); + auto writer = dlwf.createListingWriter( ddf.createPageWriter( [ file ], + DocFormat.LaTeX ), DocFormat.LaTeX ); + + writer.generateListing(srcFile, dstFile, mod.moduleFQN); + + srcFile.close(); + dstFile.close(); + } + + file.close(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/tests/parse.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/tests/parse.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,65 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.tests.parse; + +import docgen.misc.parser; +import tango.io.FileConduit; +import tango.io.Print: Print; +import tango.text.convert.Layout : Layout; + +void saveToFile(char[] fname, void delegate(Print!(char) file) foo) { + auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); + auto output = new Print!(char)(new Layout!(char), file); + + foo(output); + + file.close(); +} + +// load some test files +//@unittest +void parse1() { + saveToFile("parse1.txt", (Print!(char) file){ + Module[] modules; + + Parser.loadModules( + [ "c" ], [ "docgen/teststuff/" ], + null, true, -1, + (char[] fqn, char[] path, Module m) { + file.format("{0} = {1}\n", fqn, path); + }, + (Module imported, Module importer, bool isPublic, bool isStatic) { + file.format("{0} <- {1}\n", + imported ? imported.moduleFQN : "null"[], + importer ? importer.moduleFQN : "null"[] + ); + }, + modules + ); + }); +} + +// load the imports of dil +//@unittest +void parse2() { + saveToFile("parse2.txt", (Print!(char) file){ + Module[] modules; + + Parser.loadModules( + [ "docgen/testsuite" ], [".", "/home/jm/d/tango/"], + null, true, -1, + (char[] fqn, char[] path, Module m) { + file.format("{0} = {1}\n", fqn, path); + }, + (Module imported, Module importer, bool isPublic, bool isStatic) { + file.format("{0} <- {1}\n", + imported ? imported.moduleFQN : "null"[], + importer ? importer.moduleFQN : "null"[] + ); + }, + modules + ); + }); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/teststuff/a.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/teststuff/a.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,4 @@ +module a; + +void foo() {} +void bar() {} diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/teststuff/b.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/teststuff/b.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,4 @@ +module b; + +import a; + diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/teststuff/c.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/teststuff/c.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,4 @@ +module c; + +import a; +import b; diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/teststuff/clean.sh --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/teststuff/clean.sh Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,7 @@ +rm _* +rm dep* +rm doc* +rm *.tex +rm graph* +rm parse* +touch modules.tex diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/teststuff/lstlang0.sty --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/teststuff/lstlang0.sty Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,21 @@ +%% +%% D definition (c) 2007 Jari-Matti Mäkelä +%% +\lst@definelanguage{D}% + {morekeywords={abstract,alias,align,asm,assert,auto,body,bool,break,% + byte,case,cast,catch,cdouble,cent,cfloat,char,class,const,continue,% + creal,dchar,debug,default,delegate,delete,deprecated,do,double,% + else,enum,export,extern,false,final,finally,float,for,foreach,% + foreach_reverse,function,goto,idouble,if,ifloat,import,in,inout,% + int,interface,invariant,ireal,is,lazy,long,macro,mixin,module,new,% + null,out,override,package,pragma,private,protected,public,real,ref,% + return,scope,short,static,struct,super,switch,synchronized,template,% + this,throw,true,try,typedef,typeid,typeof,ubyte,ucent,uint,ulong,% + union,unittest,ushort,version,void,volatile,wchar,while,with},% + sensitive,% + morecomment=[s]{/*}{*/},% + morecomment=[n]{/+}{+/},% + morecomment=[l]//,% + morestring=[b]",% + morestring=[b]`% + }[keywords,comments,strings]% \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/teststuff/modules.tex diff -r a3fab8b74a7d -r bcb74c9b895c src/docgen/testsuite.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/docgen/testsuite.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,42 @@ +/** + * Author: Jari-Matti Mäkelä + * License: GPL3 + */ +module docgen.testsuite; + +import docgen.tests.graphs; +import docgen.tests.parse; +import docgen.tests.doctemplate; +import docgen.tests.listing; +//import docgen.tests.sexp; +import tango.io.Stdout; + +/** + * A temporary test program for the docgen package. + * I'll replace this with proper unittests in the future. + * + */ +void main() { + Stdout("Running..\n")(); + + Stdout(" Test1\n")(); + graph1(); + Stdout(" Test2\n")(); + graph2(); + Stdout(" Test3\n")(); + graph3(); + Stdout(" Test4\n")(); + graph4(); + Stdout(" Test5\n")(); + graph5(); + Stdout(" Test6\n")(); + parse1(); + Stdout(" Test7\n")(); + parse2(); + Stdout(" Test8\n")(); + doctemplate1(); + Stdout(" Test9\n")(); + listing1(); +// loadConfig(); + Stdout("done.\n")(); +} diff -r a3fab8b74a7d -r bcb74c9b895c src/html.css --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/html.css Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,70 @@ +@charset "utf-8"; +.compilerinfo, .sourcecode, .linescolumn { + white-space: pre; + font-family: Monospace; + font-size: 0.8em; + margin:0; + padding:0; +} +.compilerinfo { + white-space: normal; + border: 1px solid #A22; + padding: 0.5em; + margin: 1em; +} +.compilerinfo .error { display: block; } +.linescolumn { + text-align: right; + padding-right: 0.3em; + border-right: 1px dotted gray; +} +.linescolumn a { + display: block; + color: #44B; + text-decoration: none; +} +/* Number */ +.n { color: teal; } +/* Keyword */ +.k { color: darkblue; font-weight: bold; } +/* Line, block and nested comments */ +.lc, .bc, .nc { color: green; } +/* Identifier */ +.i { color: black; } +/* String literal */ +.sl { color: red; } +/* Character literal */ +.cl { color: purple; } +/* All bracket types */ +.br { color: orange; } +/* Special tokens */ +.st { color: green; font-weight: bold; } +/* #line, hash line */ +.hl { color: green; } +/* filespec (e.g. #line number [filespec]) */ +.fs { color: purple;} +/* When the first line starts with #! it's a "shebang" */ +.shebang { color: gray; } +/* Operator */ +/*.op { color: royalblue; }*/ +/* Particular operators */ +/*.opaa { content: "and"; }*/ /*&& ∧*/ +/*.opoo { content: "or"; }*/ /*|| ∨*/ +/*.opn { content: "¬"; }*/ /*!*/ +/*.opne { content: "≠"; }*/ /*!=*/ +/*.ople { content: "≤"; }*/ /*<=*/ +/*.opge { content: "≥"; }*/ /*>=*/ +/*.oplg { content: "≶"; }*/ /*<>*/ +/* +d = Declaration +s = Statement +e = Expression +t = Type +o = Other +*/ +/* .d { background-color: #FFDDDD; } */ +/* .e { background-color: #DDDDFF; } */ +.d.Module .i, .d.Import .i { color: blue; } + /*.t .i,*/ .t.Identifier .i, .TemplateTypeParameter .i { color: #911; } +.t .br, .t .op { color: #911; } +.t .k { color: #911; font-weight: normal; } diff -r a3fab8b74a7d -r bcb74c9b895c src/html_map.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/html_map.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,117 @@ +/// A map of document elements and D tokens to format strings. +string[string] map = [ + "DocHead" : ``\n + ``\n + ``\n + ` `\n + ` `\n + ``\n + ``\n + ``\n, + "CompBegin" : `", + "LexerError" : `

    {0}({1},{2})L: {3}

    `\n, + "ParserError" : `

    {0}({1},{2})P: {3}

    `\n, + + "LineNumberBegin" : `\n", + + "DocEnd" : "\n
    `\n, + "CompEnd" : "
    \n
    `, + "LineNumberEnd" : "", + "LineNumber" : `{0}`, + + "SourceBegin" : `
    `\n,
+  "SourceEnd"   : "\n
    " + "\n" + "\n", + + // Node categories: + "Declaration" : "d", + "Statement" : "s", + "Expression" : "e", + "Type" : "t", + "Other" : "o", + + // {0} = node category. + // {1} = node class name: "Call", "If", "Class" etc. + // E.g.: ... + "NodeBegin" : ``, + "NodeEnd" : ``, + + "Identifier" : `{0}`, + "String" : `{0}`, + "Char" : `{0}`, + "Number" : `{0}`, + "Keyword" : `{0}`, + + "LineC" : `{0}`, + "BlockC" : `{0}`, + "NestedC" : `{0}`, + + "Shebang" : `{0}`, + "HLine" : `{0}`, // #line + "Filespec" : `{0}`, // #line N "filespec" + "Newline" : "{0}", // \n | \r | \r\n | LS | PS + "Illegal" : `{0}`, // A character not recognized by the lexer. + + "SpecialToken" : `{0}`, // __FILE__, __LINE__ etc. + + "(" : "(", + ")" : ")", + "[" : "[", + "]" : "]", + "{" : "{", + "}" : "}", + "." : ".", + ".." : "..", + "..." : "...", + "!<>=" : "!<>=", // Unordered + "!<>" : "!<>", // UorE + "!<=" : "!<=", // UorG + "!<" : "!<", // UorGorE + "!>=" : "!>=", // UorL + "!>" : "!>", // UorLorE + "<>=" : "<>=", // LorEorG + "<>" : "<>", // LorG + "=" : "=", + "==" : "==", + "!" : "!", + "!=" : "!=", + "<=" : "<=", + "<" : "<", + ">=" : ">=", + ">" : ">", + "<<=" : "<<=", + "<<" : "<<", + ">>=" : ">>=", + ">>" : ">>", + ">>>=" : ">>>=", + ">>>" : ">>>", + "|" : "|", + "||" : "||", + "|=" : "|=", + "&" : "&", + "&&" : "&&", + "&=" : "&=", + "+" : "+", + "++" : "++", + "+=" : "+=", + "-" : "-", + "--" : "--", + "-=" : "-=", + "/" : "/", + "/=" : "/=", + "*" : "*", + "*=" : "*=", + "%" : "%", + "%=" : "%=", + "^" : "^", + "^=" : "^=", + "~" : "~", + "~=" : "~=", + ":" : ":", + ";" : ";", + "?" : "?", + "," : ",", + "$" : "$", + "EOF" : "" +]; diff -r a3fab8b74a7d -r bcb74c9b895c src/lang_de.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lang_de.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,91 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ + +string lang_code = "de"; + +string[] messages = [ + // Lexer messages: + "illegales Zeichen gefunden: '{0}'", +// "ungültiges Unicodezeichen.", + "ungültige UTF-8-Sequenz: '{0}'", + // '' + "unterminiertes Zeichenliteral.", + "leeres Zeichenliteral.", + // #line + "erwartete 'line' nach '#'.", + "Ganzzahl nach #line erwartet.", +// `erwartete Dateispezifikation (z.B. "pfad\zur\datei".)`, + "unterminierte Dateispezifikation (filespec.)", + "ein Special Token muss mit einem Zeilenumbruch abgeschlossen werden.", + // "" + "unterminiertes Zeichenkettenliteral.", + // x"" + "Nicht-Hexzeichen '{0}' in Hexzeichenkette gefunden.", + "ungerade Anzahl von Hexziffern in Hexzeichenkette.", + "unterminierte Hexzeichenkette.", + // /* */ /+ +/ + "unterminierter Blockkommentar (/* */).", + "unterminierter verschachtelter Kommentar (/+ +/).", + // `` r"" + "unterminierte rohe Zeichenkette.", + "unterminierte Backquote-Zeichenkette.", + // \x \u \U + "undefinierte Escapesequenz '{0}' gefunden.", + "ungültige Unicode-Escapesequenz '{0}' gefunden.", + "unzureichende Anzahl von Hexziffern in Escapesequenz: '{0}'", + // \&[a-zA-Z][a-zA-Z0-9]+; + "undefinierte HTML-Entität '{0}'", + "unterminierte HTML-Entität '{0}'.", + "HTML-Entitäten müssen mit einem Buchstaben beginnen.", + // integer overflows + "Dezimalzahl überläuft im Vorzeichenbit.", + "Überlauf in Dezimalzahl.", + "Überlauf in Hexadezimalzahl.", + "Überlauf in Binärzahl.", + "Überlauf in Oktalzahl.", + "Überlauf in Fließkommazahl.", + "die Ziffern 8 und 9 sind in Oktalzahlen unzulässig.", + "ungültige Hexzahl; mindestens eine Hexziffer erforderlich.", + "ungültige Binärzahl; mindestens eine Binärziffer erforderlich.", + "der Exponent einer hexadezimalen Fließkommazahl ist erforderlich.", + "Hexadezimal-Exponenten müssen mit einer Dezimalziffer anfangen.", + "Exponenten müssen mit einer Dezimalziffer anfangen.", + + // Parser messages: + "erwartete '{0}', fand aber '{1}'.", + "'{0}' ist redundant.", + "Template-Tupel-Parameter dürfen nur am Ende auftreten.", + "der 'in'-Vertrag der Funktion wurde bereits geparsed.", + "der 'out'-Vertrag der Funktion wurde bereits geparsed.", + "es wurde kein Verbindungstyp angegeben.", + "unbekannter Verbindungstyp '{0}'; gültig sind C, C++, D, Windows, Pascal und System.", + "erwartete eine oder mehrere Basisklassen, nicht '{0}'.", + "Basisklassen sind in Vorwärtsdeklarationen nicht erlaubt.", + + // Help messages: + `dil v{0} +Copyright (c) 2007-2008, Aziz Köksal. Lizensiert unter der GPL3. + +Befehle: +{1} +Geben Sie 'dil help ' ein, um mehr Hilfe zu einem bestimmten Befehl zu +erhalten. + +Kompiliert mit {2} v{3} am {4}.`, + + `Generiere ein XML- oder HTML-Dokument aus einer D-Quelltextdatei. +Verwendung: + dil gen datei.d [Optionen] + +Optionen: + --syntax : generiere Elemente für den Syntaxbaum + --xml : verwende XML-Format (voreingestellt) + --html : verwende HTML-Format + +Beispiel: + dil gen Parser.d --html --syntax > Parser.html`, + + ``, +]; diff -r a3fab8b74a7d -r bcb74c9b895c src/lang_en.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lang_en.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,118 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ + +string lang_code = "en"; + +string[] messages = [ + // Lexer messages: + "illegal character found: '{0}'", +// "invalid Unicode character.", + "invalid UTF-8 sequence: '{0}'", + // '' + "unterminated character literal.", + "empty character literal.", + // #line + "expected 'line' after '#'.", + "integer expected after #line", +// `expected filespec string (e.g. "path\to\file".)`, + "unterminated filespec string.", + "expected a terminating newline after special token.", + // "" + "unterminated string literal.", + // x"" + "non-hex character '{0}' found in hex string.", + "odd number of hex digits in hex string.", + "unterminated hex string.", + // /* */ /+ +/ + "unterminated block comment (/* */).", + "unterminated nested comment (/+ +/).", + // `` r"" + "unterminated raw string.", + "unterminated back quote string.", + // \x \u \U + "found undefined escape sequence '{0}'.", + "found invalid Unicode escape sequence '{0}'.", + "insufficient number of hex digits in escape sequence: '{0}'", + // \&[a-zA-Z][a-zA-Z0-9]+; + "undefined HTML entity '{0}'", + "unterminated HTML entity '{0}'.", + "HTML entities must begin with a letter.", + // integer overflows + "decimal number overflows sign bit.", + "overflow in decimal number.", + "overflow in hexadecimal number.", + "overflow in binary number.", + "overflow in octal number.", + "overflow in float number.", + "digits 8 and 9 are not allowed in octal numbers.", + "invalid hex number; at least one hex digit expected.", + "invalid binary number; at least one binary digit expected.", + "the exponent of a hexadecimal float number is required.", + "hexadecimal float exponents must start with a digit.", + "exponents must start with a digit.", + + // Parser messages + "expected '{0}', but found '{1}'.", + "'{0}' is redundant.", + "template tuple parameters can only be last.", + "the functions 'in' contract was already parsed.", + "the functions 'out' contract was already parsed.", + "no linkage type was specified.", + "unrecognized linkage type '{0}'; valid types are C, C++, D, Windows, Pascal und System.", + "expected one or more base classes, not '{0}'.", + "base classes are not allowed in forward declarations.", + + // Help messages: + `dil v{0} +Copyright (c) 2007-2008 by Aziz Köksal. Licensed under the GPL3. + +Subcommands: +{1} +Type 'dil help ' for more help on a particular subcommand. + +Compiled with {2} v{3} on {4}.`, + + `Generate an XML or HTML document from a D source file. +Usage: + dil gen file.d [Options] + +Options: + --syntax : generate tags for the syntax tree + --xml : use XML format (default) + --html : use HTML format + +Example: + dil gen Parser.d --html --syntax > Parser.html`, + + `Parse a module and extract information from the resulting module dependency graph. +Usage: + dil igraph file.d Format [Options] + + The directory of file.d is implicitly added to the list of import paths. + +Format: + --dot : generate a dot document + Further options for --dot: + -gbp : Group modules by package names + -gbf : Group modules by full package name + -hle : highlight cyclic edges in the graph + -hlv : highlight modules in cyclic relationship + + --paths : print a list of paths to the modules imported by file.d + --list : print a list of the module names imported by file.d + Options common to --paths and --list: + -lN : print N levels. + -m : mark modules in cyclic relationships with a star. + +Options: + -Ipath : add 'path' to the list of import paths where modules are + looked for + -rREGEXP : exclude modules whose names match the regular expression + REGEXP + -i : include unlocatable modules + +Example: + dil igraph src/main.d`, +]; diff -r a3fab8b74a7d -r bcb74c9b895c src/lang_fi.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lang_fi.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,91 @@ +/++ + Author: Jari-Matti Mäkelä + License: GPL3 ++/ + +string lang_code = "fi"; + +string[] messages = [ + // Lexer messages: + "virheellinen merkki: '{0}'", +// "virheellinen Unicode-merkki.", + "virheellinen UTF-8-merkkijono: '{0}'", + // '' + "päättämätön merkkiliteraali.", + "tyhjä merkkiliteraali.", + // #line + "odotettiin rivinumeroa '#':n jälkeen.", + "odotettiin kokonaislukua #line:n jälkeen", +// `odotettiin tiedostomäärittelyn merkkijonoa (esim. "polku\tiedostoon")`, + "päättämätön tiedostomäärittely.", + "odotettiin päättävää rivinvaihtoa erikoismerkin jälkeen.", + // "" + "päättämätön merkkijonoliteraali.", + // x"" + "ei-heksamerkki '{0}' heksajonossa.", + "pariton määrä heksanumeroita heksajonossa.", + "päättämätön heksajono.", + // /* */ /+ +/ + "päättämätön lohkokommentti (/* */).", + "päättämätön sisäkkäinen kommentti (/+ +/).", + // `` r"" + "päättämätön raakamerkkijono.", + "päättämätön gravisaksenttimerkkijono.", + // \x \u \U + "määrittelemätön escape-sekvenssi {0}.", + "virheellinen Unicode escape-merkki '{0}'.", + "riittämätön määrä heksanumeroita escape-sekvenssissä: '{0}'", + // \&[a-zA-Z][a-zA-Z0-9]+; + "määrittelemätön HTML-entiteetti '{0}'", + "päättämätön HTML-entiteetti {0}.", + "HTML-entiteettien tulee alkaa kirjaimella.", + // integer overflows + "desimaaliluku ylivuotaa etumerkin.", + "desimaaliluvun ylivuoto.", + "heksadesimaaliluvun ylivuoto.", + "binääriluvun ylivuoto.", + "oktaaliluvun ylivuoto.", + "liukuluvun ylivuoto.", + "numerot 8 ja 9 eivät ole sallittuja oktaaliluvuissa.", + "virheellinen heksaluku; odotettiin vähintään yhtä heksanumeroa.", + "virheellinen binääriluku; odotettiin vähintään yhtä binäärinumeroa.", + "heksadesimaalisen liukuluvun eksponentti vaaditaan.", + "heksadesimaalisen liukuluvun eksponentin tulee alkaa numerolla.", + "eksponenttien tulee alkaa numerolla.", + + // Parser messages + "odotettiin '{0}':a, mutta luettiin '{1}'.", + "'{0}' on redundantti.", + "tupla voi esiintyä ainoastaan mallin viimeisenä parametrina.", + "funktion alkuehto jäsennettiin jo.", + "funktion loppuehto jäsennettiin jo.", + "linkitystyyppiä ei määritelty.", + "tunnistamaton linkitystyyppi '{0}'; sallittuja tyyppejä ovat C, C++, D, Windows, Pascal ja System.", + "odotettiin yhtä tai useampaa luokkaa, ei '{0}':ta.", + "kantaluokat eivät ole sallittuja etukäteismäärittelyissä.", + + // Help messages: + `dil v{0} +Copyright (c) 2007-2008, Aziz Köksal. GPL3-lisensöity. + +Alikomennot: +{1} +Lisäohjeita tietystä alitoiminnosta saa kirjoittamalla 'dil help '. + +Käännetty {2}:n versiolla {3} {4}.`, + + `Luo XML- tai HTML-dokumentti D-lähdekoodista. + +Käyttö: + dil gen tiedosto.d [Valinnat] + +Valinnat: + --syntax : luo elementtejä syntaksipuun mukaisesti + --xml : käytä XML-muotoa (oletus) + --html : käytä HTML-muotoa + +Esimerkki: + dil gen Parser.d --html --syntax > Parser.html`, + + ``, +]; diff -r a3fab8b74a7d -r bcb74c9b895c src/lang_tr.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/lang_tr.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,90 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ + +string lang_code = "tr"; + +string[] messages = [ + // Lexer messages: + "illegal karakter bulundu: '{0}'", +// "geçersiz Unikod karakteri.", + "geçersiz UTF-8 serisi: '{0}'", + // '' + "kapanmamış karakter sabiti.", + "boş karakter sabiti.", + // #line + "'#' karakter'den sonra 'line' beklendi.", + "'#line''den sonra rakam beklendi.", +// `filespec dizgisi beklendi (e.g. "yol\dosya".)`, + "kapanmamış filespec dizgisi.", + "özel belirtici'den (special token) sonra yeni bir satır beklendi.", + // "" + "kapanmamış çift tırnak dizgisi.", + // x"" + "heks sayı olmayan karakter '{0}' heks dizgisi içinde bulundu.", + "heks dizginin içindeki sayılar çifter çifter olmalıdır.", + "kapanmamış heks dizgisi.", + // /* */ /+ +/ + "kapanmamış blok açıklaması (/* */).", + "kapanmamış iç içe koyulabilen açıklaması (/+ +/).", + // `` r"" + "kapanmamış çiğ dizgisi.", + "kapanmamış ters tırnak dizgisi.", + // \x \u \U + "tanımlanmamış çıkış serisi '{0}' bulundu.", + "geçersiz Unikod çıkış serisi '{0}' bulundu.", + "heksadesimal çıkış serisi sayıları yeterli değil: '{0}'", + // \&[a-zA-Z][a-zA-Z0-9]+; + "tanımlanmamış HTML varlık '{0}'", + "kapanmamış HTML varlık '{0}'.", + "HTML varlık bir harf ile başlamalı.", + // integer overflows + "desimal rakamın bit işareti taşdı.", + "desimal rakam taşması.", + "heksadesimal rakam taşması.", + "binari rakam taşması.", + "oktal rakam taşması.", + "float rakam taşması.", + "8 ve 9 sayılar oktal rakamlar'da geçersizdir.", + "geçersiz heks rakam; minimum bir heks sayı gereklidir.", + "geçersiz binari rakam; minimum bir binari sayı gereklidir.", + "bir heksadesimal float rakamın üsü gereklidir.", + "heksadesimal float üsler desimal sayı ile başlamalı.", + "üsler desimal sayı ile başlamalı.", + + // Parser messages + "'{0}' beklendi, ama '{1}' bulundu.", + "'{0}' lüzumsuz.", + "şablon tuple parametre son sırada olmalı.", + "fonksiyonun 'in' kontratı daha önceden ayrıştırılmış.", + "fonksiyonun 'out' kontratı daha önceden ayrıştırılmış.", + "bağlantı tüp (linkage type) belirtilmedi.", + "bilinmeyen bağlantı tüpü (linkage type) '{0}'; geçerli olanlar C, C++, D, Windows, Pascal ve System.", + "expected one or more base classes, not '{0}'.", // TODO: translate + "base classes are not allowed in forward declarations.", // TODO: translate + + // Help messages: + `dil v{0} +Copyright (c) 2007-2008, Aziz Köksal. Lisans GPL3. + +Komutlar: +{1} +Belirli komut'a yardım edinmek için 'dil help ' yazınız. + +Bu yazılım {2} v{3} ile {4} tarihinde derletilmiş.`, + + `Bir D kaynak kodundan XML veya HTML dosyası oluştur. +Kullanım: + dil gen dosya.d [Seçenekler] + +Seçenekler: + --syntax : söz dizimi için etiketler yazdır + --xml : XML biçimi kullan (varsayılır) + --html : HTML biçimi kullan + +Örnek: + dil gen Parser.d --html --syntax > Parser.html`, + + ``, +]; diff -r a3fab8b74a7d -r bcb74c9b895c src/macros_dil.ddoc --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/macros_dil.ddoc Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,22 @@ +DDOC = + + + $(TITLE) + + +

    $(TITLE)

    +$(BODY) +
    +

    $(COPYRIGHT) Page generated by $(LINK2 http://code.google.com/p/dil, dil) on $(DATETIME).

    + + + +MODFQN = $(TITLE) +SRCFILE = ./htmlsrc/$(MODFQN).html +COPYRIGHT = Copyright © 2007-$(YEAR), Aziz Köksal. All rights reserved. +SYMBOL_ = $1 +SYMBOL = $1 + +PRE = 
    $0
    +DDD = --- +DMDBUG = $(LINK2 http://d.puremagic.com/issues/show_bug.cgi?id=$1, $2) diff -r a3fab8b74a7d -r bcb74c9b895c src/main.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/main.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,576 @@ +/++ + Author: Aziz Köksal + License: GPL3 ++/ +module main; + +import dil.parser.Parser; +import dil.lexer.Lexer, + dil.lexer.Token; +import dil.ast.Declarations, + dil.ast.Expressions, + dil.ast.Node, + dil.ast.Visitor; +import dil.semantic.Module; +import dil.semantic.Symbols; +import dil.semantic.Pass1, + dil.semantic.Pass2, + dil.semantic.Interpreter; +import dil.translator.German; +import dil.doc.Doc; +import dil.Messages; +import dil.CompilerInfo; +import dil.Information; +import dil.SourceText; +import dil.Compilation; + +import cmd.Generate; +import cmd.Statistics; +import cmd.ImportGraph; +import cmd.DDoc; + +import Settings; +import SettingsLoader; +// import TypeRules; +import common; + +import Integer = tango.text.convert.Integer; +import tango.stdc.stdio; +import tango.io.File; +import tango.text.Util; +import tango.time.StopWatch; +import tango.text.Ascii : icompare; + +/// Entry function of dil. +void main(char[][] args) +{ + auto infoMan = new InfoManager(); + SettingsLoader.SettingsLoader(infoMan).load(); + if (infoMan.hasInfo) + return printErrors(infoMan); + + if (args.length <= 1) + return Stdout(helpMain()).newline; + + string command = args[1]; + switch (command) + { + case "c", "compile": + if (args.length < 2) + return printHelp("compile"); + + string[] filePaths; + auto context = newCompilationContext(); + foreach (arg; args[2..$]) + { + if (parseDebugOrVersion(arg, context)) + {} + else + filePaths ~= arg; + } + infoMan = new InfoManager(); + foreach (filePath; filePaths) + { + auto mod = new Module(filePath, infoMan); + // Parse the file. + mod.parse(); + if (mod.hasErrors) + continue; + + // Start semantic analysis. + auto pass1 = new SemanticPass1(mod, context); + pass1.start(); + + void printSymbolTable(ScopeSymbol scopeSym, char[] indent) + { + foreach (member; scopeSym.members) + { + auto tokens = getDocTokens(member.node); + char[] docText; + foreach (token; tokens) + docText ~= token.srcText; + Stdout(indent).formatln("Id:{}, Symbol:{}, DocText:{}", member.name.str, member.classinfo.name, docText); + if (auto s = cast(ScopeSymbol)member) + printSymbolTable(s, indent ~ "→ "); + } + } + + printSymbolTable(mod, ""); + } + + infoMan.hasInfo && printErrors(infoMan); + break; + case "ddoc", "d": + if (args.length < 4) + return printHelp("ddoc"); + + auto destination = args[2]; + auto macroPaths = GlobalSettings.ddocFilePaths; + char[][] filePaths; + bool incUndoc; + bool writeXML; + bool verbose; + // Parse arguments. + auto context = newCompilationContext(); + foreach (arg; args[3..$]) + { + if (parseDebugOrVersion(arg, context)) + {} + else if (arg == "--xml") + writeXML = true; + else if (arg == "-i") + incUndoc = true; + else if (arg == "-v") + verbose = true; + else if (arg.length > 5 && icompare(arg[$-4..$], "ddoc") == 0) + macroPaths ~= arg; + else + filePaths ~= arg; + } + + infoMan = new InfoManager(); + // Execute command. + cmd.DDoc.execute(filePaths, destination, macroPaths, writeXML, + incUndoc, verbose, context, infoMan); + infoMan.hasInfo && printErrors(infoMan); + break; + case "gen", "generate": + char[] fileName; + GenOption options = GenOption.Tokens; + foreach (arg; args[2..$]) + { + switch (arg) + { + case "--syntax": + options |= GenOption.Syntax; break; + case "--xml": + options |= GenOption.XML; break; + case "--html": + options |= GenOption.HTML; break; + case "--lines": + options |= GenOption.PrintLines; break; + default: + fileName = arg; + } + } + if (!(options & (GenOption.XML | GenOption.HTML))) + options |= GenOption.XML; // Default to XML. + cmd.Generate.execute(fileName, options, infoMan); + infoMan.hasInfo && printErrors(infoMan); + break; + case "importgraph", "igraph": + string filePath; + string[] regexps; + string siStyle = "dashed"; // static import style + string piStyle = "bold"; // public import style + uint levels; + IGraphOption options; + auto context = newCompilationContext(); + foreach (arg; args[2..$]) + { + if (parseDebugOrVersion(arg, context)) + {} + else if (strbeg(arg, "-I")) + context.importPaths ~= arg[2..$]; + else if(strbeg(arg, "-r")) + regexps ~= arg[2..$]; + else if(strbeg(arg, "-l")) + levels = Integer.toInt(arg[2..$]); + else if(strbeg(arg, "-si")) + siStyle = arg[3..$]; + else if(strbeg(arg, "-pi")) + piStyle = arg[3..$]; + else + switch (arg) + { + case "--dot": + options |= IGraphOption.PrintDot; break; + case "--paths": + options |= IGraphOption.PrintPaths; break; + case "--list": + options |= IGraphOption.PrintList; break; + case "-i": + options |= IGraphOption.IncludeUnlocatableModules; break; + case "-hle": + options |= IGraphOption.HighlightCyclicEdges; break; + case "-hlv": + options |= IGraphOption.HighlightCyclicVertices; break; + case "-gbp": + options |= IGraphOption.GroupByPackageNames; break; + case "-gbf": + options |= IGraphOption.GroupByFullPackageName; break; + case "-m": + options |= IGraphOption.MarkCyclicModules; break; + default: + filePath = arg; + } + } + cmd.ImportGraph.execute(filePath, context, regexps, levels, siStyle, piStyle, options); + break; + case "stats", "statistics": + char[][] filePaths; + bool printTokensTable; + bool printNodesTable; + foreach (arg; args[2..$]) + if (arg == "--toktable") + printTokensTable = true; + else if (arg == "--asttable") + printNodesTable = true; + else + filePaths ~= arg; + cmd.Statistics.execute(filePaths, printTokensTable, printNodesTable); + break; + case "tok", "tokenize": + SourceText sourceText; + char[] filePath; + char[] separator; + bool ignoreWSToks; + bool printWS; + + foreach (arg; args[2..$]) + { + if (strbeg(arg, "-s")) + separator = arg[2..$]; + else if (arg == "-") + sourceText = new SourceText("stdin", readStdin()); + else if (arg == "-i") + ignoreWSToks = true; + else if (arg == "-ws") + printWS = true; + else + filePath = arg; + } + + separator || (separator = "\n"); + if (!sourceText) + sourceText = new SourceText(filePath, true); + + infoMan = new InfoManager(); + auto lx = new Lexer(sourceText, infoMan); + lx.scanAll(); + auto token = lx.firstToken(); + + for (; token.kind != TOK.EOF; token = token.next) + { + if (token.kind == TOK.Newline || ignoreWSToks && token.isWhitespace) + continue; + if (printWS && token.ws) + Stdout(token.wsChars); + Stdout(token.srcText)(separator); + } + + infoMan.hasInfo && printErrors(infoMan); + break; + case "trans", "translate": + if (args.length < 3) + return printHelp("trans"); + + if (args[2] != "German") + return Stdout.formatln("Error: unrecognized target language \"{}\"", args[2]); + + infoMan = new InfoManager(); + auto filePath = args[3]; + auto mod = new Module(filePath, infoMan); + // Parse the file. + mod.parse(); + if (!mod.hasErrors) + { // Translate + auto german = new GermanTranslator(Stdout, " "); + german.translate(mod.root); + } + printErrors(infoMan); + break; + case "profile": + if (args.length < 3) + break; + char[][] filePaths; + if (args[2] == "dstress") + { + auto text = cast(char[])(new File("dstress_files")).read(); + filePaths = split(text, "\0"); + } + else + filePaths = args[2..$]; + + StopWatch swatch; + swatch.start; + + foreach (filePath; filePaths) + (new Lexer(new SourceText(filePath, true))).scanAll(); + + Stdout.formatln("Scanned in {:f10}s.", swatch.stop); + break; + // case "parse": + // if (args.length == 3) + // parse(args[2]); + // break; + case "?", "help": + printHelp(args.length >= 3 ? args[2] : ""); + break; + // case "typerules": + // genHTMLTypeRulesTables(); + // break; + default: + } +} + +char[] readStdin() +{ + char[] text; + while (1) + { + auto c = getc(stdin); + if (c == EOF) + break; + text ~= c; + } + return text; +} + +/// Available commands. +const char[] COMMANDS = + " compile (c)\n" + " ddoc (d)\n" + " generate (gen)\n" + " help (?)\n" + " importgraph (igraph)\n" + " statistics (stats)\n" + " tokenize (tok)\n" + " translate (trans)\n"; + +bool strbeg(char[] str, char[] begin) +{ + if (str.length >= begin.length) + { + if (str[0 .. begin.length] == begin) + return true; + } + return false; +} + +/// Creates the global compilation context. +CompilationContext newCompilationContext() +{ + auto cc = new CompilationContext; + cc.importPaths = GlobalSettings.importPaths; + cc.addVersionId("dil"); + cc.addVersionId("all"); +version(D2) + cc.addVersionId("D_Version2"); + foreach (versionId; GlobalSettings.versionIds) + if (!Lexer.isReservedIdentifier(versionId)) + cc.versionIds[versionId] = true; + return cc; +} + +bool parseDebugOrVersion(string arg, CompilationContext context) +{ + if (strbeg(arg, "-debug")) + { + if (arg.length > 7) + { + auto val = arg[7..$]; + if (isdigit(val[0])) + context.debugLevel = Integer.toInt(val); + else if (!Lexer.isReservedIdentifier(val)) + context.addDebugId(val); + } + else + context.debugLevel = 1; + } + else if (arg.length > 9 && strbeg(arg, "-version=")) + { + auto val = arg[9..$]; + if (isdigit(val[0])) + context.versionLevel = Integer.toInt(val); + else if (!Lexer.isReservedIdentifier(val)) + context.addVersionId(val); + } + else + return false; + return true; +} + +/// Prints the errors collected in infoMan. +void printErrors(InfoManager infoMan) +{ + foreach (info; infoMan.info) + { + char[] errorFormat; + if (info.classinfo is LexerError.classinfo) + errorFormat = GlobalSettings.lexerErrorFormat; + else if (info.classinfo is ParserError.classinfo) + errorFormat = GlobalSettings.parserErrorFormat; + else if (info.classinfo is SemanticError.classinfo) + errorFormat = GlobalSettings.semanticErrorFormat; + else if (info.classinfo is Warning.classinfo) + errorFormat = "{0}: Warning: {3}"; + else + continue; + auto err = cast(Problem)info; + Stderr.formatln(errorFormat, err.filePath, err.loc, err.col, err.getMsg); + } +} + +/// Prints the compiler's main help message. +char[] helpMain() +{ + auto COMPILED_WITH = __VENDOR__; + auto COMPILED_VERSION = Format("{}.{,:d3}", __VERSION__/1000, __VERSION__%1000); + auto COMPILED_DATE = __TIMESTAMP__; + return FormatMsg(MID.HelpMain, VERSION, COMMANDS, COMPILED_WITH, + COMPILED_VERSION, COMPILED_DATE); +} + +/// Prints a help message for command. +void printHelp(char[] command) +{ + char[] msg; + switch (command) + { + case "c", "compile": + msg = `Compile D source files. +Usage: + dil compile file.d [file2.d, ...] [Options] + + This command only parses the source files and does little semantic analysis. + Errors are printed to standard error output. + +Options: + -debug : include debug code + -debug=level : include debug(l) code where l <= level + -debug=ident : include debug(ident) code + -version=level : include version(l) code where l >= level + -version=ident : include version(ident) code + +Example: + dil c src/main.d`; + break; + case "ddoc", "d": + msg = `Generate documentation from DDoc comments in D source files. +Usage: + dil ddoc Destination file.d [file2.d, ...] [Options] + + Destination is the folder where the documentation files are written to. + Files with the extension .ddoc are recognized as macro definition files. + +Options: + --xml : write XML instead of HTML documents + -i : include undocumented symbols + -v : verbose output + +Example: + dil d doc/ src/main.d src/macros_dil.ddoc -i`; + break; + case "gen", "generate": +// msg = GetMsg(MID.HelpGenerate); + msg = `Generate an XML or HTML document from a D source file. +Usage: + dil gen file.d [Options] + +Options: + --syntax : generate tags for the syntax tree + --xml : use XML format (default) + --html : use HTML format + --lines : print line numbers + +Example: + dil gen Parser.d --html --syntax > Parser.html`; + break; + case "importgraph", "igraph": +// msg = GetMsg(MID.HelpImportGraph); + msg = `Parse a module and build a module dependency graph based on its imports. +Usage: + dil igraph file.d Format [Options] + + The directory of file.d is implicitly added to the list of import paths. + +Format: + --dot : generate a dot document (default) + Options related to --dot: + -gbp : Group modules by package names + -gbf : Group modules by full package name + -hle : highlight cyclic edges in the graph + -hlv : highlight modules in cyclic relationships + -siSTYLE : the edge style to use for static imports + -piSTYLE : the edge style to use for public imports + STYLE can be: "dashed", "dotted", "solid", "invis" or "bold" + + --paths : print the file paths of the modules in the graph + + --list : print the names of the module in the graph + Options common to --paths and --list: + -lN : print N levels. + -m : use '*' to mark modules in cyclic relationships + +Options: + -Ipath : add 'path' to the list of import paths where modules are + looked for + -rREGEXP : exclude modules whose names match the regular expression + REGEXP + -i : include unlocatable modules + +Example: + dil igraph src/main.d --list + dil igraph src/main.d | dot -Tpng > main.png`; + break; + case "tok", "tokenize": + msg = `Print the tokens of a D source file. +Usage: + dil tok file.d [Options] + +Options: + - : reads text from the standard input. + -sSEPARATOR : print SEPARATOR instead of newline between tokens. + -i : ignore whitespace tokens (e.g. comments, shebang etc.) + -ws : print a token's preceding whitespace characters. + +Example: + echo "module foo; void func(){}" | dil tok - + dil tok main.d | grep ^[0-9]`; + break; + case "stats", "statistics": + msg = "Gather statistics about D source files. +Usage: + dil stat file.d [file2.d, ...] [Options] + +Options: + --toktable : print the count of all kinds of tokens in a table. + --asttable : print the count of all kinds of nodes in a table. + +Example: + dil stat src/dil/Parser.d src/dil/Lexer.d"; + break; + case "trans", "translate": + msg = `Translate a D source file to another language. +Usage: + dil translate Language file.d + + Languages that are supported: + *) German + +Example: + dil trans German src/main.d`; + break; + default: + msg = helpMain(); + } + Stdout(msg).newline; +} + +/+void parse(string fileName) +{ + auto mod = new Module(fileName); + mod.parse(); + + void print(Node[] decls, char[] indent) + { + foreach(decl; decls) + { + assert(decl !is null); + Stdout.formatln("{}{}: begin={} end={}", indent, decl.classinfo.name, decl.begin ? decl.begin.srcText : "\33[31mnull\33[0m", decl.end ? decl.end.srcText : "\33[31mnull\33[0m"); + print(decl.children, indent ~ " "); + } + } + print(mod.root.children, ""); +}+/ diff -r a3fab8b74a7d -r bcb74c9b895c src/predefined.ddoc --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/predefined.ddoc Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,109 @@ +DDOC = + + + $(TITLE) + + +

    $(TITLE)

    +$(BODY) +
    +

    $(COPYRIGHT) | Page generated by $(LINK2 http://code.google.com/p/dil, dil) on $(DATETIME).

    + + + +B = $0 +I = $0 +U = $0 +P =

    $0

    +DL =
    $0
    +DT =
    $0
    +DD =
    $0
    +TABLE = $0
    +TR = $0 +TH = $0 +TD = $0 +OL =
      $0
    +UL =
      $0
    +LI =
  • $0
    • +BIG = $0 +SMALL = $0 +BR =
    +LINK = $0 +LINK2 = $+ + +RED = $0 +BLUE = $0 +GREEN = $0 +YELLOW = $0 +BLACK = $0 +WHITE = $0 + +D_CODE = 
    $0
    +D_COMMENT = $(GREEN $0) +D_STRING = $(RED $0) +D_KEYWORD = $(BLUE $0) +D_PSYMBOL = $(U $0) +D_PARAM = $(I $0) + +DDOC_COMMENT = +DDOC_DECL = $(DT $(BIG $0)) +DDOC_DECL_DD = $(DD $0) +DDOC_DITTO = $(BR)$0 + +DDOC_SECTIONS = $0 +DDOC_SUMMARY = $0$(BR)$(BR) +DDOC_DESCRIPTION = $0$(BR)$(BR) +DDOC_AUTHORS = $(B Authors:)$(BR) +$0$(BR)$(BR) +DDOC_BUGS = $(RED BUGS:)$(BR) +$0$(BR)$(BR) +DDOC_COPYRIGHT = $(B Copyright:)$(BR) +$0$(BR)$(BR) +DDOC_DATE = $(B Date:)$(BR) +$0$(BR)$(BR) +DDOC_DEPRECATED = $(RED Deprecated:)$(BR) +$0$(BR)$(BR) +DDOC_EXAMPLES = $(B Examples:)$(BR) +$0$(BR)$(BR) +DDOC_HISTORY = $(B History:)$(BR) +$0$(BR)$(BR) +DDOC_LICENSE = $(B License:)$(BR) +$0$(BR)$(BR) +DDOC_RETURNS = $(B Returns:)$(BR) +$0$(BR)$(BR) +DDOC_SEE_ALSO = $(B See Also:)$(BR) +$0$(BR)$(BR) +DDOC_STANDARDS = $(B Standards:)$(BR) +$0$(BR)$(BR) +DDOC_THROWS = $(B Throws:)$(BR) +$0$(BR)$(BR) +DDOC_VERSION = $(B Version:)$(BR) +$0$(BR)$(BR) +DDOC_SECTION_H = $(B $0)$(BR) +DDOC_SECTION = $0$(BR)$(BR) + +DDOC_MEMBERS = $(DL $0) +DDOC_MODULE_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_CLASS_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_INTERFACE_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_STRUCT_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_UNION_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_TEMPLATE_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_ENUM_MEMBERS = $(DDOC_MEMBERS $0) + +DDOC_PARAMS = $(B Params:)$(BR) +$(TABLE $0)$(BR) +DDOC_PARAM_ROW = $(TR $0) +DDOC_PARAM_ID = $(TD $0) +DDOC_PARAM_DESC = $(TD $0) +DDOC_BLANKLINE = $(BR)$(BR) + +DDOC_PSYMBOL = $(U $0) +DDOC_KEYWORD = $(B $0) +DDOC_PARAM = $(I $0) + +ATTRIBUTES = [$0] +PROT = $0 +STC = $0 +LINKAGE = $0 +SYMBOL = $1 diff -r a3fab8b74a7d -r bcb74c9b895c src/predefined_xml.ddoc --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/predefined_xml.ddoc Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,98 @@ +DDOC = +$(TITLE) +$(BODY) +$(COPYRIGHT) +dil $(DDOC_VERSION) $(LINK http://code.google.com/p/dil) + $(DATETIME) + + + +B = $0 +I = $0 +U = $0 +P =

    $0

    +DL =
    $0
    +DT =
    $0
    +DD =
    $0
    +TABLE = $0
    +TR = $0 +TH = $0 +TD = $0 +OL =
      $0
    +UL =
      $0
    +LI =
  • $0
    • +BIG = $0 +SMALL = $0 +BR =
    +LINK = $0 +LINK2 = $+ + +RED = $0 +BLUE = $0 +GREEN = $0 +YELLOW = $0 +BLACK = $0 +WHITE = $0 + +D_CODE = $0 +D_COMMENT = $0 +D_STRING = $0 +D_COMMENT = $0 +D_PSYMBOL = $0 +D_PARAM = $0 + +DDOC_COMMENT = +DDOC_DECL = $2 +DDOC_DECL_DD = $0 +DDOC_DITTO = $0 + +DDOC_SECTIONS = $0 +DDOC_SECTION_T =
    $2
    +DDOC_SUMMARY = $(DDOC_SECTION_T summary, $0) +DDOC_DESCRIPTION = $(DDOC_SECTION_T description, $0) +DDOC_AUTHORS = $(DDOC_SECTION_T authors, $0) +DDOC_BUGS = $(DDOC_SECTION_T bugs, $0) +DDOC_COPYRIGHT = $(DDOC_SECTION_T copyright, $0) +DDOC_DATE = $(DDOC_SECTION_T date, $0) +DDOC_DEPRECATED = $(DDOC_SECTION_T deprecated, $0) +DDOC_EXAMPLES = $(DDOC_SECTION_T examples, $0) +DDOC_HISTORY = $(DDOC_SECTION_T history, $0) +DDOC_LICENSE = $(DDOC_SECTION_T license, $0) +DDOC_RETURNS = $(DDOC_SECTION_T returns, $0) +DDOC_SEE_ALSO = $(DDOC_SECTION_T seealso, $0) +DDOC_STANDARDS = $(DDOC_SECTION_T standards, $0) +DDOC_THROWS = $(DDOC_SECTION_T throws, $0) +DDOC_VERSION = $(DDOC_SECTION_T version, $0) +DDOC_SECTION_H = $(B $0)$(BR) +DDOC_SECTION = $0 + +DDOC_MEMBERS = $0 +DDOC_MODULE_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_CLASS_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_INTERFACE_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_STRUCT_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_UNION_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_TEMPLATE_MEMBERS = $(DDOC_MEMBERS $0) +DDOC_ENUM_MEMBERS = $(DDOC_MEMBERS $0) + +DDOC_PARAMS = $0 +DDOC_PARAM_ROW = $0 +DDOC_PARAM_ID = $0 +DDOC_PARAM_DESC = $0 +DDOC_BLANKLINE = + +DDOC_PSYMBOL = $(U $0) +DDOC_KEYWORD = $(B $0) +DDOC_PARAM = $0 + +ATTRIBUTES = $0 +ATTRIBUTE = $2 +PROT = $(ATTRIBUTE protection, $0) +STC = $(ATTRIBUTE storage, $0) +LINKAGE = $(ATTRIBUTE linkage, $0) +SYMBOL = $1 +PARENTS = $0 +TYPE = $0 +PARAMS = $0 +RETURNS = $0 +TEMPLATE_PARAMS = $0 diff -r a3fab8b74a7d -r bcb74c9b895c src/tests/forward01.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/tests/forward01.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,7 @@ +/++ + Author: Jari-Matti Mäkelä ++/ + +// Impossible circular composition. +struct A { B b; } +struct B { A a; } diff -r a3fab8b74a7d -r bcb74c9b895c src/tests/forward02.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/tests/forward02.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,13 @@ +/++ + Author: Jari-Matti Mäkelä ++/ + +// Valid circular composition because of pointer. +struct A { B* b; } +struct B { A a; } +// Equivalent to: +struct A { A* a; } + +// Valid circular composition because classes are reference types. +class C { D d; } +class D { C c; } diff -r a3fab8b74a7d -r bcb74c9b895c src/tests/forward03.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/tests/forward03.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,18 @@ +/++ + Author: Aziz Köksal ++/ + +// Impossible static circular reference. +const x = y; +const y = x; + +// Impossible static circular reference. +struct A +{ const int a = B.b; } +struct B +{ const int b = A.a; } + +struct C +{ + const x = C.x; +} \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c src/tests/forward04.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/tests/forward04.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,6 @@ +/++ + Author: Aziz Köksal ++/ + +class A { auto x = pi; } +mixin("const pi = 3.14;"); diff -r a3fab8b74a7d -r bcb74c9b895c src/tests/forward05.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/tests/forward05.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,9 @@ +/++ + Author: Aziz Köksal ++/ + +struct A +{ int a = B.x; } + +struct B +{ const int x = 1; } diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/about.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/about.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,70 @@ + + AboutDialog + + + Qt::WindowModal + + + + 0 + 0 + 414 + 157 + + + + About + + + + + + <html><head><meta name="qrichtext" content="1" /><style type="text/css"> +p, li { white-space: pre-wrap; } +</style></head><body style=" font-family:'Sans Serif'; font-size:9pt; font-weight:400; font-style:normal;"> +<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">A tool for managing message catalogues in different languages.</p> +<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">These catalogues can be used to make an application multilingual.</p> +<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"></p> +<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Copyright © 2007 by Aziz Köksal &lt;aziz.koeksal@gmail.com&gt;</p> +<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Licensed under the GPL2.</p></body></html> + + + Qt::RichText + + + Qt::AlignLeading|Qt::AlignLeft|Qt::AlignTop + + + true + + + + + + + QDialogButtonBox::Ok + + + + + + + + + buttonBox + accepted() + AboutDialog + close() + + + 346 + 274 + + + 317 + 256 + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/closing_project.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/closing_project.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,125 @@ + + ClosingProjectDialog + + + + 0 + 0 + 400 + 300 + + + + Closing Project + + + + + + The following documents have been modified: + + + + + + + + Title + + + + + Full Path + + + + + + + + Qt::Vertical + + + + 382 + 33 + + + + + + + + + + Qt::Horizontal + + + + 40 + 20 + + + + + + + + &Save Selected + + + + + + + &Discard All Changes + + + + + + + &Cancel + + + + + + + + + + + button_Save_Selected + clicked() + ClosingProjectDialog + accept() + + + 144 + 269 + + + 179 + 238 + + + + + button_Cancel + clicked() + ClosingProjectDialog + reject() + + + 337 + 268 + + + 328 + 248 + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/errors.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/errors.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,11 @@ +# -*- coding: utf-8 -*- +# Author: Aziz Köksal +# License: GPL2 +import exceptions + +class LoadingError(exceptions.Exception): + def __init__(self, msg): + self.msg = msg + return + def __str__(self): + return self.msg diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/langfile.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/langfile.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,112 @@ +# -*- coding: utf-8 -*- +# Author: Aziz Köksal +# License: GPL2 +import yaml +from errors import LoadingError + +# Avoid that all unicode strings are tagged with "!!python/unicode". +def unicode_representer(dumper, data): + return dumper.represent_scalar(u'tag:yaml.org,2002:str', data) +yaml.add_representer(unicode, unicode_representer) + +def newLangFile(langCode, authors, license): + return { + "LangCode":langCode, + "Authors":authors, + "License":license, + "Messages":[] + } + +class LangFile: + def __init__(self, filePath): + from os import path + self.filePath = path.abspath(filePath) + self.isSource = False + self.source = None + # Load language file and check data integrity. + try: + self.doc = yaml.load(open(filePath, "r")) + except yaml.YAMLError, e: + raise LoadingError(str(e)) + self.verify() + + def verify(self): + doc = self.doc + self.checkType(doc, dict) + try: + self.langCode = str(doc["LangCode"]) + self.authors = list(doc["Authors"]) + self.license = unicode(doc["License"]) + self.messages = list(doc["Messages"]) + except KeyError, e: + raise LoadingError("Missing member '%s' in '%s'" % (e.message, filePath)) + + authors = [] + for author in self.authors: + self.checkType(author, dict, "LangFile: author must be of type dict.") + try: + author["Name"] = unicode(author["Name"]) + author["EMail"] = str(author["EMail"]) + authors += [author] + except KeyError, e: + raise LoadingError("Author is missing '%s' in '%s'" % (e.message, filePath)) + self.authors = authors + + self.msgDict = {} # {ID : msg, ...} + for msg in self.messages: + self.checkType(msg, dict, "LangFile: messages must be of type dict.") + try: + msg["ID"] = int(msg["ID"]) + msg["Text"] = unicode(msg["Text"]) + msg["Annot"] = unicode(msg["Annot"]) + msg["LastEd"] = int(msg["LastEd"]) + except KeyError, e: + raise LoadingError("LangFile: a message is missing the '%s' key." % str(e)) + self.msgDict[msg["ID"]] = msg + + def checkType(self, var, type_, msg=""): + if not isinstance(var, type_): + raise LoadingError(msg) + + def setSource(self, sourceLangFile): + self.source = sourceLangFile + + def getMsg(self, ID): + for msg in self.messages: + if msg["ID"] == ID: + return msg + return None + + def createMissingMessages(self, IDs): + for ID in IDs: + if not self.msgDict.has_key(ID): + msg = self.createEmptyMsg(ID) + self.msgDict[ID] = msg + self.messages += [msg] + + def createEmptyMsg(self, ID): + return {"ID":ID,"Text":"","Annot":"","LastEd":""} + + def save(self): + self.doc["LangCode"] = self.langCode + self.doc["License"] = self.license + self.doc["Authors"] = self.authors + langFile = open(self.filePath, "w") + yaml.dump(self.doc, stream=langFile, allow_unicode=True) + langFile.close() + + def setLangCode(self, langCode): + self.langCode = langCode + + def setLicense(self, license): + self.license = license + + def setAuthors(self, authors): + self.authors = authors + + def getFileName(self): + from os import path + return path.basename(self.filePath) + + def getFilePath(self): + return self.filePath diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/langfile_properties.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/langfile_properties.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,128 @@ + + LangFilePropertiesDialog + + + + 0 + 0 + 400 + 228 + + + + Language File Properties + + + + + + + + Language code: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + langCodeField + + + + + + + + + + Creation date: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + creationDateField + + + + + + + + + + + + + Author(s): + + + Qt::AlignRight|Qt::AlignTop|Qt::AlignTrailing + + + + + + + + + Qt::Vertical + + + QSizePolicy::Fixed + + + + 20 + 10 + + + + + + + + Qt::Horizontal + + + QDialogButtonBox::Cancel|QDialogButtonBox::NoButton|QDialogButtonBox::Ok + + + + + + + + + buttonBox + accepted() + LangFilePropertiesDialog + accept() + + + 227 + 278 + + + 157 + 274 + + + + + buttonBox + rejected() + LangFilePropertiesDialog + reject() + + + 295 + 284 + + + 286 + 274 + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/make_ui.sh --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/make_ui.sh Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,5 @@ +#!/bin/bash + +ui_file=$1 +echo $ui_file \> ui_`basename $ui_file .ui`.py +pyuic4 $ui_file > ui_`basename $ui_file .ui`.py diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/make_uis.sh --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/make_uis.sh Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,5 @@ +#!/bin/bash + +for ui_file in *.ui; do + ./make_ui.sh $ui_file +done \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/msg_form.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/msg_form.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,107 @@ + + MsgForm + + + + 0 + 0 + 456 + 512 + + + + Form + + + + + + Qt::Vertical + + + + + + + Messages: + + + + + + + + 1 + + + + + + + + + + + + + + Source string: + + + + + + + + + + + + + + Source annotation: + + + + + + + + + + + + + + Translation: + + + + + + + + + + + + + + Translator's annotation: + + + + + + + + + + + + + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/new_project.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/new_project.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,127 @@ + + NewProjectDialog + + + + 0 + 0 + 401 + 131 + + + + Create New Project + + + + + + + + + + Project name: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + projectName + + + + + + + + + + Project file: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + projectFilePath + + + + + + + + + + + + ... + + + + + + + + + + + Qt::Vertical + + + + 20 + 40 + + + + + + + + Qt::Horizontal + + + QDialogButtonBox::Cancel|QDialogButtonBox::NoButton|QDialogButtonBox::Ok + + + + + + + + + + + buttonBox + accepted() + NewProjectDialog + accept() + + + 228 + 108 + + + 157 + 130 + + + + + buttonBox + rejected() + NewProjectDialog + reject() + + + 296 + 114 + + + 286 + 130 + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/project.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/project.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,109 @@ +# -*- coding: utf-8 -*- +# Author: Aziz Köksal +# License: GPL2 +import os +from errors import LoadingError +import langfile +import datetime +import yaml + +def newProjectData(projectName): + return { + "Name":projectName, + "LangFiles":[], + "SourceLangFile":'', + "MsgIDs":[], + "CreationDate":str(datetime.datetime.utcnow()), + "BuildScript":'' + } + +class Project: + # Members: + # name + # source + # langFilePaths + # langFile + # msgids + # creationDate + def __init__(self, projectPath): + self.projectPath = projectPath + # Load project file and check data integrity. + try: + self.doc = yaml.load(open(projectPath, "r")) + except yaml.YAMLError, e: + raise LoadingError(str(e)) + self.verify() + + def verify(self): + doc = self.doc + self.checkType(doc, dict) + try: + self.name = unicode(doc["Name"]) + self.srcLangFilePath = str(doc["SourceLangFile"]) + self.langFilePaths = list(doc["LangFiles"]) + self.msgIDs = list(doc["MsgIDs"]) + self.creationDate = str(doc["CreationDate"]) + self.buildScript = str(doc["BuildScript"]) + except KeyError, e: + raise LoadingError("Missing member '%s' in '%s'" % (e.message, projectPath)) + + for path in self.langFilePaths: + self.checkType(path, str) + + msgIDs = [] + for msg in self.msgIDs: + self.checkType(msg, dict) + try: + msg["ID"] = int(msg["ID"]) + msg["Name"] = unicode(msg["Name"]) + msg["Order"] = int(msg["Order"]) + except KeyError, e: + raise LoadingError("Project: a message is missing the '%s' key." % str(e)) + msgIDs += [msg] + self.msgIDs = msgIDs + + # Load language files. + self.langFiles = [] + IDList = [msg["ID"] for msgID in self.msgIDs] + for filePath in self.langFilePaths: + langFile = self.loadLangFile(filePath) + langFile.createMissingMessages(IDList) + self.langFiles += [langFile] + self.srcLangFile = self.loadLangFile(self.srcLangFilePath) + self.srcLangFile.createMissingMessages(IDList) + self.srcLangFile.isSource = True + self.langFiles += [self.srcLangFile] + + for langFile in self.langFiles: + langFile.setSource(self.srcLangFile) + + def checkType(self, var, type_): + if not isinstance(var, type_): + raise LoadingException("%s is not of type %s" % (str(var), str(type_))) + + def loadLangFile(self, filePath): + if not os.path.exists(filePath): + # Look in project directory. + projectDir = os.path.dirname(self.projectPath) + filePath2 = os.path.join(projectDir, filePath) + if not os.path.exists(filePath2): + raise LoadingError("Project: Language file '%s' doesn't exist"%filePath) + filePath = filePath2 + return langfile.LangFile(filePath) + + def setName(self, name): + self.name = name + + def setBuildScript(self, filePath): + self.buildScript = filePath + + def setCreationDate(self, date): + self.creationDate = date + + def save(self): + self.doc["Name"] = self.name + self.doc["BuildScript"] = self.buildScript + self.doc["CreationDate"] = self.creationDate + file_ = open(self.projectPath, "w") + yaml.dump(self.doc, stream=file_, allow_unicode=True) + file_.close() diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/project_properties.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/project_properties.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,136 @@ + + ProjectPropertiesDialog + + + + 0 + 0 + 400 + 152 + + + + Project Properties + + + + + + + + Project name: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + projectNameField + + + + + + + + + + Build script: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + buildScriptField + + + + + + + + + + + + ... + + + + + + + + + + + + Creation date: + + + Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter + + + + + + + + + Qt::Vertical + + + + 20 + 40 + + + + + + + + Qt::Horizontal + + + QDialogButtonBox::Cancel|QDialogButtonBox::NoButton|QDialogButtonBox::Ok + + + + + + + + + buttonBox + accepted() + ProjectPropertiesDialog + accept() + + + 227 + 278 + + + 157 + 274 + + + + + buttonBox + rejected() + ProjectPropertiesDialog + reject() + + + 295 + 284 + + + 286 + 274 + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/translator.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/translator.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,685 @@ +#! /usr/bin/python +# -*- coding: utf-8 -*- +# Author: Aziz Köksal +# License: GPL2 +import sys, os +import yaml + +from PyQt4 import QtCore, QtGui +# User interface modules +from ui_translator import Ui_MainWindow +from ui_about import Ui_AboutDialog +from ui_new_project import Ui_NewProjectDialog +from ui_project_properties import Ui_ProjectPropertiesDialog +from ui_msg_form import Ui_MsgForm +from ui_closing_project import Ui_ClosingProjectDialog + +from project import Project, newProjectData + +g_scriptDir = sys.path[0] +g_CWD = os.getcwd() +g_projectExt = ".tproj" +g_catExt = ".cat" +g_settingsFile = os.path.join(g_scriptDir, "settings.yaml") +g_settings = {} + +Qt = QtCore.Qt +Qt.connect = QtCore.QObject.connect +Qt.disconnect = QtCore.QObject.disconnect +Qt.SIGNAL = QtCore.SIGNAL +Qt.SLOT = QtCore.SLOT + +def QTabWidgetCloseAll(self): + for i in range(self.count()-1,-1,-1): + widget = self.widget(i) + self.removeTab(i) + widget.close() +QtGui.QTabWidget.closeAll = QTabWidgetCloseAll + +class MainWindow(QtGui.QMainWindow, Ui_MainWindow): + def __init__(self): + QtGui.QMainWindow.__init__(self) + self.setupUi(self) + + self.project = None + # Modifications + self.pages = QtGui.QTabWidget() + self.setCentralWidget(self.pages) + self.disableMenuItems() + self.projectDock = QtGui.QDockWidget("Project", self) + self.projectTree = ProjectTree(self.projectDock, self) + self.projectDock.setWidget(self.projectTree) + self.addDockWidget(QtCore.Qt.LeftDockWidgetArea, self.projectDock) + # Custom connections + triggered = Qt.SIGNAL("triggered()") + Qt.connect(self.action_About, triggered, self.showAboutDialog) + Qt.connect(self.action_New_Project, triggered, self.createNewProject) + Qt.connect(self.action_Open_Project, triggered, self.openProjectAction) + Qt.connect(self.action_Close_Project, triggered, self.closeProjectAction) + Qt.connect(self.action_Save, triggered, self.saveForm) + Qt.connect(self.action_Save_All, triggered, self.saveAllForms) + Qt.connect(self.action_Close, triggered, self.closeForm) + Qt.connect(self.action_Close_All, triggered, self.closeAllForms) + Qt.connect(self.action_Properties, triggered, self.showProjectProperties) + Qt.connect(self.action_Add_Catalogue, triggered, self.addCatalogue) + Qt.connect(self.action_Add_New_Catalogue, triggered, self.addNewCatalogue) + Qt.connect(self.projectTree, Qt.SIGNAL("itemDoubleClicked(QTreeWidgetItem*,int)"), self.projectTreeItemDblClicked) + Qt.connect(self.projectTree, Qt.SIGNAL("onKeyEnter"), self.projectTreeItemActivated) + Qt.connect(self.projectTree, Qt.SIGNAL("onKeyDelete"), self.projectTreeItemDeleted) + + shortcut = QtGui.QShortcut(QtGui.QKeySequence(Qt.CTRL+Qt.Key_Tab), self) + Qt.connect(shortcut, Qt.SIGNAL("activated()"), self.nextDocument) + shortcut = QtGui.QShortcut(QtGui.QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_Tab), self) + Qt.connect(shortcut, Qt.SIGNAL("activated()"), self.prevDocument) + + self.readSettings() + + def nextDocument(self): + count = self.pages.count() + if count < 1: return + index = self.pages.currentIndex()+1 + if index == count: index = 0 + self.pages.setCurrentIndex(index) + + def prevDocument(self): + count = self.pages.count() + if count < 1: return + index = self.pages.currentIndex()-1 + if index == -1: index = count-1 + self.pages.setCurrentIndex(index) + + def showAboutDialog(self): + about = QtGui.QDialog() + Ui_AboutDialog().setupUi(about) + about.exec_() + + def showProjectProperties(self): + dialog = ProjectPropertiesDialog(self.project) + dialog.exec_() + self.projectTree.updateProjectName() + + def createNewProject(self): + if self.rejectClosingProject(): + return + dialog = NewProjectDialog() + code = dialog.exec_() + if code == QtGui.QDialog.Accepted: + self.closeProject() + self.openProject(str(dialog.projectFilePath.text())) + + def openProjectAction(self): + if self.rejectClosingProject(): + return + filePath = QtGui.QFileDialog.getOpenFileName(self, "Select Project File", g_CWD, "Translator Project (*%s)" % g_projectExt); + filePath = str(filePath) + if filePath: + self.closeProject() + self.openProject(filePath) + + def openProject(self, filePath): + from errors import LoadingError + try: + self.project = Project(filePath) + except LoadingError, e: + QtGui.QMessageBox.critical(self, "Error", u"Couldn't load project file:\n\n"+str(e)) + return + self.enableMenuItems() + self.projectTree.setProject(self.project) + + def closeProjectAction(self): + if not self.rejectClosingProject(): + self.closeProject() + + def addCatalogue(self): + filePath = QtGui.QFileDialog.getOpenFileName(self, "Select Project File", g_CWD, "Catalogue (*%s)" % g_catExt); + filePath = str(filePath) + # TODO: + #self.project.addLangFile(filePath) + + def addNewCatalogue(self): + pass + + def rejectClosingProject(self): + if self.project == None: + return False + + modifiedDocs = [] + # Check if any open document is modified. + for i in range(0, self.pages.count()): + if self.pages.widget(i).isModified: + modifiedDocs += [self.pages.widget(i)] + # Display dialog if so. + if len(modifiedDocs): + dialog = ClosingProjectDialog(modifiedDocs) + code = dialog.exec_() + if code == dialog.Accepted: + for doc in dialog.getSelectedDocs(): + self.saveDocument(doc) + elif code == dialog.Rejected: + return True + elif code == dialog.DiscardAll: + pass + + return False + + def closeProject(self): + if self.project == None: + return + self.project.save() + del self.project + self.project = None + self.disableMenuItems() + self.projectTree.clear() + self.pages.closeAll() + + def enableMenuItems(self): + #self.action_Close_Project.setEnabled(True) + for action in [ self.action_Save, + self.action_Save_All, + self.action_Close, + self.action_Close_All ]: + action.setEnabled(True) + self.menubar.insertMenu(self.menu_Help.menuAction(), self.menu_Project) + + def disableMenuItems(self): + #self.action_Close_Project.setEnabled(False) + for action in [ self.action_Save, + self.action_Save_All, + self.action_Close, + self.action_Close_All ]: + action.setEnabled(False) + self.menubar.removeAction(self.menu_Project.menuAction()) + + def projectTreeItemDblClicked(self, item, int): + self.projectTreeItemActivated(item) + + def projectTreeItemActivated(self, item): + if item == None: + return + + if isinstance(item, LangFileItem): + msgForm = None + if not item.isDocOpen(): + msgForm = item.openDoc() + msgForm.setModifiedCallback(self.formModified) + else: + msgForm = item.openDoc() + index = self.pages.indexOf(msgForm) + if index == -1: + index = self.pages.addTab(msgForm, msgForm.getDocumentTitle()) + self.pages.setCurrentIndex(index) + msgForm.updateData() + + def projectTreeItemDeleted(self, item): + pass + + def formModified(self, form): + # Append an asterisk to the tab label + index = self.pages.indexOf(form) + text = form.getDocumentTitle() + "*" + self.pages.setTabText(index, text) + + def saveForm(self): + self.saveDocument(self.pages.currentWidget()) + + def saveAllForms(self): + for i in range(0, self.pages.count()): + self.saveDocument(self.pages.widget(i)) + + def saveDocument(self, form): + if form.isModified: + # Reset tab text. + index = self.pages.indexOf(form) + text = form.getDocumentTitle() + self.pages.setTabText(index, text) + + form.save() + + def closeForm(self): + if self.pages.currentWidget(): + self.closeDocument(self.pages.currentWidget()) + + def closeAllForms(self): + for i in range(self.pages.count()-1, -1, -1): + self.closeDocument(self.pages.widget(i)) + + def closeDocument(self, form): + if form.isModified: + MB = QtGui.QMessageBox + button = MB.question(self, "Closing Document", "The document '%s' has been modified.\nDo you want to save the changes?" % form.getDocumentFullPath(), MB.Save | MB.Discard | MB.Cancel, MB.Cancel) + if button == MB.Cancel: + return False + if button == MB.Save: + self.saveDocument(form) + index = self.pages.indexOf(form) + self.pages.removeTab(index) + form.close() + return True + + def closeEvent(self, event): + if self.rejectClosingProject(): + event.ignore() + return + self.closeProject() + self.writeSettings() + # Closing application + + def moveToCenterOfDesktop(self): + rect = QtGui.QApplication.desktop().geometry() + self.move(rect.center() - self.rect().center()) + + def readSettings(self): + # Set default size + self.resize(QtCore.QSize(500, 400)) + doc = {} + try: + doc = yaml.load(open(g_settingsFile, "r")) + except: + self.moveToCenterOfDesktop() + return + + g_settings = doc + if not isinstance(doc, dict): + g_settings = {} + + try: + coord = doc["Window"] + size = QtCore.QSize(coord["Size"][0], coord["Size"][1]) + point = QtCore.QPoint(coord["Pos"][0], coord["Pos"][1]) + self.resize(size) + self.move(point) + except: + self.moveToCenterOfDesktop() + + def writeSettings(self): + # Save window coordinates + g_settings["Window"] = { + "Pos" : [self.pos().x(), self.pos().y()], + "Size" : [self.size().width(), self.size().height()] + } + yaml.dump(g_settings, open(g_settingsFile, "w")) #default_flow_style=False + + +class Document(QtGui.QWidget): + def __init__(self): + QtGui.QWidget.__init__(self) + self.isModified = False + self.modifiedCallback = None + self.documentTitle = "" + self.documentFullPath = "" + + def modified(self): + if not self.isModified: + self.isModified = True + self.modifiedCallback(self) + + def setModifiedCallback(self, func): + self.modifiedCallback = func + + def save(self): + self.isModified = False + + def close(self): + self.emit(Qt.SIGNAL("closed()")) + QtGui.QWidget.close(self) + + def getDocumentTitle(self): + return self.documentTitle + + def getDocumentFullPath(self): + return self.documentFullPath + + +class MessageItem(QtGui.QTreeWidgetItem): + def __init__(self, msg): + QtGui.QTreeWidgetItem.__init__(self, [str(msg["ID"]), msg["Text"], "Done"]) + self.msg = msg + + def getID(self): + return self.msg["ID"] + + def setMsgText(self, text): + self.msg["Text"] = text + + def setMsgAnnot(self, text): + self.msg["Annot"] = text + + +class MsgForm(Document, Ui_MsgForm): + def __init__(self, langFile): + Document.__init__(self) + self.documentTitle = langFile.getFileName() + self.documentFullPath = langFile.getFilePath() + self.setupUi(self) + self.vboxlayout.setMargin(0) + + self.langFile = langFile + self.currentItem = None + self.colID = 0 + self.colText = 1 + #self.colStat = 2 + #self.treeWidget.setColumnCount(3) + self.treeWidget.setColumnCount(2) + self.treeWidget.setHeaderLabels(["ID", "Text"]) #, "Status" + self.msgItemDict = {} # Maps msg IDs to msg items. + for msg in self.langFile.messages: + item = MessageItem(msg) + self.msgItemDict[msg["ID"]] = item + self.treeWidget.addTopLevelItem(item) + + Qt.connect(self.treeWidget, Qt.SIGNAL("currentItemChanged (QTreeWidgetItem *,QTreeWidgetItem *)"), self.treeItemChanged) + Qt.connect(self.translEdit, Qt.SIGNAL("textChanged()"), self.translEditTextChanged) + Qt.connect(self.translAnnotEdit, Qt.SIGNAL("textChanged()"), self.translAnnotEditTextChanged) + + #self.translEdit.focusOutEvent = self.translEditFocusOut + + def sourceMsgChanged(self, msg): + # TODO: + pass + + def treeItemChanged(self, current, previous): + if current == None: + self.setTranslMsg("") + self.setSourceMsg("") + return + ID = current.getID() + # Set the text controls. + # The slots receiving text changed signals do nothing if self.currentItem is None. + self.currentItem = None + self.setTranslMsg(self.langFile.getMsg(ID)) + self.setSourceMsg(self.langFile.source.getMsg(ID)) + self.currentItem = current + + def setTranslMsg(self, msg): + self.translEdit.setText(msg["Text"]) + self.translAnnotEdit.setText(msg["Annot"]) + + def setSourceMsg(self, msg): + self.sourceEdit.setText(msg["Text"]) + self.sourceAnnotEdit.setText(msg["Annot"]) + + #def translEditFocusOut(self, event): + #if self.currentItem: + #print self.currentItem.text(self.colText) + #if self.translEdit.document().isModified(): + #self.translEdit.document().setModified(False) + #print "translEdit was modified" + #QtGui.QTextEdit.focusOutEvent(self.translEdit, event) + + def translEditTextChanged(self): + if self.currentItem: + text = unicode(self.translEdit.toPlainText()) + self.currentItem.setText(self.colText, text) + self.currentItem.setMsgText(text) + self.modified() + + def translAnnotEditTextChanged(self): + if self.currentItem: + text = unicode(self.translAnnotEdit.toPlainText()) + self.currentItem.setMsgAnnot(text) + self.modified() + + def updateData(self): + if self.currentItem == None: + return + ID = self.currentItem.getID() + msg = self.langFile.source.getMsg(ID) + text = self.sourceEdit.toPlainText() + if text != msg["Text"]: + self.sourceEdit.setText(msg["Text"]) + text = self.sourceAnnotEdit.toPlainText() + if text != msg["Annot"]: + self.sourceAnnotEdit.setText(msg["Annot"]) + + def save(self): + Document.save(self) + self.langFile.save() + + +class MsgFormSource(MsgForm): + def __init__(self, langFile): + MsgForm.__init__(self, langFile) + + for x in [self.translEdit, + self.translAnnotEdit, + self.label_4, + self.label_5]: + x.close() + + Qt.connect(self.sourceEdit, Qt.SIGNAL("textChanged()"), self.sourceEditTextChanged) + Qt.connect(self.sourceAnnotEdit, Qt.SIGNAL("textChanged()"), self.sourceAnnotEditTextChanged) + + def treeItemChanged(self, current, previous): + if current == None: + self.setSourceMsg("") + return + ID = current.getID() + self.currentItem = None + self.setSourceMsg(self.langFile.getMsg(ID)) + self.currentItem = current + + def sourceEditTextChanged(self): + if self.currentItem: + text = unicode(self.sourceEdit.toPlainText()) + self.currentItem.setText(self.colText, text) + self.currentItem.setMsgText(text) + self.modified() + + def sourceAnnotEditTextChanged(self): + if self.currentItem: + text = unicode(self.sourceAnnotEdit.toPlainText()) + #self.currentItem.setText(self.colAnnot, text) + self.currentItem.setMsgAnnot(text) + self.modified() + + +class MsgIDItem(QtGui.QTreeWidgetItem): + def __init__(self, parent, text): + QtGui.QTreeWidgetItem.__init__(self, parent, [text]) + self.setFlags(self.flags()|Qt.ItemIsEditable); + +class LangFileItem(QtGui.QTreeWidgetItem): + def __init__(self, parent, langFile): + QtGui.QTreeWidgetItem.__init__(self, parent, [langFile.langCode]) + self.langFile = langFile + self.msgForm = None + + def isDocOpen(self): + return self.msgForm != None + + def openDoc(self): + if self.msgForm == None: + if self.langFile.isSource: + self.msgForm = MsgFormSource(self.langFile) + else: + self.msgForm = MsgForm(self.langFile) + Qt.connect(self.msgForm, Qt.SIGNAL("closed()"), self.docClosed) + return self.msgForm + + def docClosed(self): + self.msgForm = None + +class ProjectItem(QtGui.QTreeWidgetItem): + def __init__(self, text): + QtGui.QTreeWidgetItem.__init__(self, [text]) + self.setFlags(self.flags()|Qt.ItemIsEditable); + +class ProjectTree(QtGui.QTreeWidget): + def __init__(self, parent, mainWindow): + QtGui.QTreeWidget.__init__(self, parent) + self.mainWindow = mainWindow + self.project = None + self.topItem = None + self.msgIDsItem = None + self.headerItem().setHidden(True) + self.ignoreItemChanged = False + + def itemChanged(self, item, column): + if self.ignoreItemChanged: + return + text = unicode(item.text(0)) + #if hasattr(item, "textChanged"): + #item.textChanged(text) + if isinstance(item, ProjectItem): + self.project.setName(text) + print text + + def keyReleaseEvent(self, event): + Qt = QtCore.Qt + key = event.key() + if key in [Qt.Key_Enter, Qt.Key_Return]: + self.emit(Qt.SIGNAL("onKeyEnter"), self.currentItem()) + elif key == Qt.Key_Delete: + self.emit(Qt.SIGNAL("onKeyDelete"), self.currentItem()) + + def setProject(self, project): + self.project = project + + self.topItem = ProjectItem(self.project.name) + self.addTopLevelItem(self.topItem) + + for langFile in self.project.langFiles: + langFileItem = LangFileItem(self.topItem, langFile) + + self.msgIDsItem = QtGui.QTreeWidgetItem(self.topItem, ["Message IDs"]) + for msgID in self.project.msgIDs: + MsgIDItem(self.msgIDsItem, msgID["Name"]) + + for x in [self.topItem, self.msgIDsItem]: + x.setExpanded(True) + + Qt.connect(self, Qt.SIGNAL("itemChanged(QTreeWidgetItem*,int)"), self.itemChanged) + + def contextMenuEvent(self, event): + item = self.itemAt(event.pos()) + func_map = { + None : lambda item: None, + QtGui.QTreeWidgetItem : lambda item: None, + ProjectItem : self.showMenuProjectItem, + LangFileItem : self.showMenuLangFileItem, + MsgIDItem : self.showMenuMsgIDItem + } + func_map[type(item)](item) + + def showMenuProjectItem(self, item): + mousePos = QtGui.QCursor.pos() + menu = QtGui.QMenu() + actions = {} + actions[menu.addAction("Build")] = lambda: None + actions[menu.addAction("Properties")] = lambda: self.mainWindow.showProjectProperties() + actions[menu.exec_(mousePos)]() + + def showMenuLangFileItem(self, item): + print "LangFileItem" + + def showMenuMsgIDItem(self, item): + print "MsgIDItem" + + def updateProjectName(self): + self.ignoreItemChanged = True + self.topItem.setText(0, self.project.name) + self.ignoreItemChanged = False + + def clear(self): + self.project = None + self.topItem = None + self.msgIDsItem = None + Qt.disconnect(self, Qt.SIGNAL("itemChanged(QTreeWidgetItem*,int)"), self.itemChanged) + QtGui.QTreeWidget.clear(self) + + +class ClosingProjectDialog(QtGui.QDialog, Ui_ClosingProjectDialog): + DiscardAll = 2 + def __init__(self, docs): + QtGui.QDialog.__init__(self) + self.setupUi(self) + Qt.connect(self.button_Discard_All, Qt.SIGNAL("clicked()"), self.discardAll) + + self.items = [] + for doc in docs: + title = doc.getDocumentTitle() + path = doc.getDocumentFullPath() + item = QtGui.QTreeWidgetItem([title, path]) + item.doc = doc + item.setFlags(item.flags()|Qt.ItemIsUserCheckable); + item.setCheckState(0, Qt.Checked) + self.items += [item] + self.treeWidget.addTopLevelItems(self.items) + + self.button_Cancel.setFocus() + + def getSelectedDocs(self): + return [item.doc for item in self.items if item.checkState(0)] + + def discardAll(self): + self.done(self.DiscardAll) + + +class ProjectPropertiesDialog(QtGui.QDialog, Ui_ProjectPropertiesDialog): + def __init__(self, project): + QtGui.QDialog.__init__(self) + self.setupUi(self) + Qt.connect(self.pickFileButton, Qt.SIGNAL("clicked()"), self.pickFilePath) + + self.project = project + self.projectNameField.setText(self.project.name) + self.buildScriptField.setText(self.project.buildScript) + self.creationDateField.setText(self.project.creationDate) + + def pickFilePath(self): + filePath = QtGui.QFileDialog.getOpenFileName(self, "Select Build Script File", g_CWD, "Python Script (*.py)"); + if filePath: + self.buildScriptField.setText(str(filePath)) + + def accept(self): + self.project.setName(unicode(self.projectNameField.text())) + self.project.setBuildScript(unicode(self.buildScriptField.text())) + self.project.setCreationDate(str(self.creationDateField.text())) + QtGui.QDialog.accept(self) + + +class NewProjectDialog(QtGui.QDialog, Ui_NewProjectDialog): + def __init__(self): + QtGui.QDialog.__init__(self) + self.setupUi(self) + Qt.connect(self.pickFileButton, Qt.SIGNAL("clicked()"), self.pickFilePath) + + def pickFilePath(self): + filePath = QtGui.QFileDialog.getSaveFileName(self, "New Project File", g_CWD, "Translator Project (*%s)" % g_projectExt); + if filePath: + filePath = str(filePath) # Convert QString + if os.path.splitext(filePath)[1] != g_projectExt: + filePath += g_projectExt + self.projectFilePath.setText(filePath) + + def accept(self): + projectName = str(self.projectName.text()) + filePath = str(self.projectFilePath.text()) + + MB = QtGui.QMessageBox + if projectName == "": + MB.warning(self, "Warning", "Please, enter a name for the project.") + return + if filePath == "": + MB.warning(self, "Warning", "Please, choose or enter a path for the project file.") + return + + projectData = newProjectData(projectName) + + if os.path.splitext(filePath)[1] != g_projectExt: + filePath += g_projectExt + + try: + yaml.dump(projectData, open(filePath, "w"), default_flow_style=False) + except Exception, e: + MB.critical(self, "Error", str(e)) + return + + # Accept and close dialog. + QtGui.QDialog.accept(self) + +if __name__ == "__main__": + app = QtGui.QApplication(sys.argv) + main = MainWindow() + main.show() + sys.exit(app.exec_()) diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/translator.ui --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/translator.ui Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,174 @@ + + MainWindow + + + + 0 + 0 + 608 + 464 + + + + Translator + + + + + + + + 0 + 0 + 608 + 29 + + + + + &File + + + + + + + + + + + + + + + &Help + + + + + + &Project + + + + + + + + + + + + + + + + + &Quit + + + Ctrl+Q + + + + + &About + + + + + &New Project... + + + Ctrl+N + + + + + &Add catalogue... + + + + + &Properties + + + + + &Build Project + + + + + &Open Project... + + + Ctrl+O + + + + + &Close + + + Ctrl+W + + + + + Add new catalogue... + + + + + &Close Project + + + Ctrl+F4 + + + + + &Save + + + Ctrl+S + + + + + Save &All + + + Ctrl+Shift+S + + + + + Clos&e All + + + Ctrl+Shift+W + + + + + + + action_Quit + triggered() + MainWindow + close() + + + -1 + -1 + + + 303 + 231 + + + + + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_about.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_about.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,48 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'about.ui' +# +# Created: Sun Oct 21 17:33:29 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_AboutDialog(object): + def setupUi(self, AboutDialog): + AboutDialog.setObjectName("AboutDialog") + AboutDialog.setWindowModality(QtCore.Qt.WindowModal) + AboutDialog.resize(QtCore.QSize(QtCore.QRect(0,0,414,157).size()).expandedTo(AboutDialog.minimumSizeHint())) + + self.vboxlayout = QtGui.QVBoxLayout(AboutDialog) + self.vboxlayout.setObjectName("vboxlayout") + + self.label1 = QtGui.QLabel(AboutDialog) + self.label1.setTextFormat(QtCore.Qt.RichText) + self.label1.setAlignment(QtCore.Qt.AlignLeading|QtCore.Qt.AlignLeft|QtCore.Qt.AlignTop) + self.label1.setWordWrap(True) + self.label1.setObjectName("label1") + self.vboxlayout.addWidget(self.label1) + + self.buttonBox = QtGui.QDialogButtonBox(AboutDialog) + self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Ok) + self.buttonBox.setObjectName("buttonBox") + self.vboxlayout.addWidget(self.buttonBox) + + self.retranslateUi(AboutDialog) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),AboutDialog.close) + QtCore.QMetaObject.connectSlotsByName(AboutDialog) + + def retranslateUi(self, AboutDialog): + AboutDialog.setWindowTitle(QtGui.QApplication.translate("AboutDialog", "About", None, QtGui.QApplication.UnicodeUTF8)) + self.label1.setText(QtGui.QApplication.translate("AboutDialog", "\n" + "

    A tool for managing message catalogues in different languages.

    \n" + "

    These catalogues can be used to make an application multilingual.

    \n" + "

    \n" + "

    Copyright © 2007 by Aziz Köksal <aziz.koeksal@gmail.com>

    \n" + "

    Licensed under the GPL2.

    ", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_closing_project.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_closing_project.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,65 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'closing_project.ui' +# +# Created: Sun Nov 11 23:01:28 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_ClosingProjectDialog(object): + def setupUi(self, ClosingProjectDialog): + ClosingProjectDialog.setObjectName("ClosingProjectDialog") + ClosingProjectDialog.resize(QtCore.QSize(QtCore.QRect(0,0,400,300).size()).expandedTo(ClosingProjectDialog.minimumSizeHint())) + + self.vboxlayout = QtGui.QVBoxLayout(ClosingProjectDialog) + self.vboxlayout.setObjectName("vboxlayout") + + self.label = QtGui.QLabel(ClosingProjectDialog) + self.label.setObjectName("label") + self.vboxlayout.addWidget(self.label) + + self.treeWidget = QtGui.QTreeWidget(ClosingProjectDialog) + self.treeWidget.setObjectName("treeWidget") + self.vboxlayout.addWidget(self.treeWidget) + + spacerItem = QtGui.QSpacerItem(382,33,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Expanding) + self.vboxlayout.addItem(spacerItem) + + self.hboxlayout = QtGui.QHBoxLayout() + self.hboxlayout.setObjectName("hboxlayout") + + spacerItem1 = QtGui.QSpacerItem(40,20,QtGui.QSizePolicy.Expanding,QtGui.QSizePolicy.Minimum) + self.hboxlayout.addItem(spacerItem1) + + self.button_Save_Selected = QtGui.QPushButton(ClosingProjectDialog) + self.button_Save_Selected.setObjectName("button_Save_Selected") + self.hboxlayout.addWidget(self.button_Save_Selected) + + self.button_Discard_All = QtGui.QPushButton(ClosingProjectDialog) + self.button_Discard_All.setObjectName("button_Discard_All") + self.hboxlayout.addWidget(self.button_Discard_All) + + self.button_Cancel = QtGui.QPushButton(ClosingProjectDialog) + self.button_Cancel.setDefault(True) + self.button_Cancel.setObjectName("button_Cancel") + self.hboxlayout.addWidget(self.button_Cancel) + self.vboxlayout.addLayout(self.hboxlayout) + + self.retranslateUi(ClosingProjectDialog) + QtCore.QObject.connect(self.button_Save_Selected,QtCore.SIGNAL("clicked()"),ClosingProjectDialog.accept) + QtCore.QObject.connect(self.button_Cancel,QtCore.SIGNAL("clicked()"),ClosingProjectDialog.reject) + QtCore.QMetaObject.connectSlotsByName(ClosingProjectDialog) + + def retranslateUi(self, ClosingProjectDialog): + ClosingProjectDialog.setWindowTitle(QtGui.QApplication.translate("ClosingProjectDialog", "Closing Project", None, QtGui.QApplication.UnicodeUTF8)) + self.label.setText(QtGui.QApplication.translate("ClosingProjectDialog", "The following documents have been modified:", None, QtGui.QApplication.UnicodeUTF8)) + self.treeWidget.headerItem().setText(0,QtGui.QApplication.translate("ClosingProjectDialog", "Title", None, QtGui.QApplication.UnicodeUTF8)) + self.treeWidget.headerItem().setText(1,QtGui.QApplication.translate("ClosingProjectDialog", "Full Path", None, QtGui.QApplication.UnicodeUTF8)) + self.button_Save_Selected.setText(QtGui.QApplication.translate("ClosingProjectDialog", "&Save Selected", None, QtGui.QApplication.UnicodeUTF8)) + self.button_Discard_All.setText(QtGui.QApplication.translate("ClosingProjectDialog", "&Discard All Changes", None, QtGui.QApplication.UnicodeUTF8)) + self.button_Cancel.setText(QtGui.QApplication.translate("ClosingProjectDialog", "&Cancel", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_langfile_properties.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_langfile_properties.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,73 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'langfile_properties.ui' +# +# Created: Sat Nov 17 23:14:57 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_LangFilePropertiesDialog(object): + def setupUi(self, LangFilePropertiesDialog): + LangFilePropertiesDialog.setObjectName("LangFilePropertiesDialog") + LangFilePropertiesDialog.resize(QtCore.QSize(QtCore.QRect(0,0,400,228).size()).expandedTo(LangFilePropertiesDialog.minimumSizeHint())) + + self.vboxlayout = QtGui.QVBoxLayout(LangFilePropertiesDialog) + self.vboxlayout.setObjectName("vboxlayout") + + self.gridlayout = QtGui.QGridLayout() + self.gridlayout.setObjectName("gridlayout") + + self.label_2 = QtGui.QLabel(LangFilePropertiesDialog) + self.label_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label_2.setObjectName("label_2") + self.gridlayout.addWidget(self.label_2,1,0,1,1) + + self.langCodeField = QtGui.QLineEdit(LangFilePropertiesDialog) + self.langCodeField.setObjectName("langCodeField") + self.gridlayout.addWidget(self.langCodeField,1,1,1,1) + + self.label = QtGui.QLabel(LangFilePropertiesDialog) + self.label.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label.setObjectName("label") + self.gridlayout.addWidget(self.label,2,0,1,1) + + self.creationDateField = QtGui.QLineEdit(LangFilePropertiesDialog) + self.creationDateField.setObjectName("creationDateField") + self.gridlayout.addWidget(self.creationDateField,2,1,1,1) + + self.authorsField = QtGui.QTextEdit(LangFilePropertiesDialog) + self.authorsField.setObjectName("authorsField") + self.gridlayout.addWidget(self.authorsField,0,1,1,1) + + self.label_3 = QtGui.QLabel(LangFilePropertiesDialog) + self.label_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTop|QtCore.Qt.AlignTrailing) + self.label_3.setObjectName("label_3") + self.gridlayout.addWidget(self.label_3,0,0,1,1) + self.vboxlayout.addLayout(self.gridlayout) + + spacerItem = QtGui.QSpacerItem(20,10,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Fixed) + self.vboxlayout.addItem(spacerItem) + + self.buttonBox = QtGui.QDialogButtonBox(LangFilePropertiesDialog) + self.buttonBox.setOrientation(QtCore.Qt.Horizontal) + self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Cancel|QtGui.QDialogButtonBox.NoButton|QtGui.QDialogButtonBox.Ok) + self.buttonBox.setObjectName("buttonBox") + self.vboxlayout.addWidget(self.buttonBox) + self.label_2.setBuddy(self.langCodeField) + self.label.setBuddy(self.creationDateField) + + self.retranslateUi(LangFilePropertiesDialog) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),LangFilePropertiesDialog.accept) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("rejected()"),LangFilePropertiesDialog.reject) + QtCore.QMetaObject.connectSlotsByName(LangFilePropertiesDialog) + + def retranslateUi(self, LangFilePropertiesDialog): + LangFilePropertiesDialog.setWindowTitle(QtGui.QApplication.translate("LangFilePropertiesDialog", "Language File Properties", None, QtGui.QApplication.UnicodeUTF8)) + self.label_2.setText(QtGui.QApplication.translate("LangFilePropertiesDialog", "Language code:", None, QtGui.QApplication.UnicodeUTF8)) + self.label.setText(QtGui.QApplication.translate("LangFilePropertiesDialog", "Creation date:", None, QtGui.QApplication.UnicodeUTF8)) + self.label_3.setText(QtGui.QApplication.translate("LangFilePropertiesDialog", "Author(s):", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_msg_form.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_msg_form.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,105 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'msg_form.ui' +# +# Created: Fri Nov 9 15:38:32 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_MsgForm(object): + def setupUi(self, MsgForm): + MsgForm.setObjectName("MsgForm") + MsgForm.resize(QtCore.QSize(QtCore.QRect(0,0,456,512).size()).expandedTo(MsgForm.minimumSizeHint())) + + self.vboxlayout = QtGui.QVBoxLayout(MsgForm) + self.vboxlayout.setObjectName("vboxlayout") + + self.splitter = QtGui.QSplitter(MsgForm) + self.splitter.setOrientation(QtCore.Qt.Vertical) + self.splitter.setObjectName("splitter") + + self.layoutWidget = QtGui.QWidget(self.splitter) + self.layoutWidget.setObjectName("layoutWidget") + + self.vboxlayout1 = QtGui.QVBoxLayout(self.layoutWidget) + self.vboxlayout1.setObjectName("vboxlayout1") + + self.label = QtGui.QLabel(self.layoutWidget) + self.label.setObjectName("label") + self.vboxlayout1.addWidget(self.label) + + self.treeWidget = QtGui.QTreeWidget(self.layoutWidget) + self.treeWidget.setObjectName("treeWidget") + self.vboxlayout1.addWidget(self.treeWidget) + + self.layoutWidget1 = QtGui.QWidget(self.splitter) + self.layoutWidget1.setObjectName("layoutWidget1") + + self.gridlayout = QtGui.QGridLayout(self.layoutWidget1) + self.gridlayout.setObjectName("gridlayout") + + self.vboxlayout2 = QtGui.QVBoxLayout() + self.vboxlayout2.setObjectName("vboxlayout2") + + self.label_2 = QtGui.QLabel(self.layoutWidget1) + self.label_2.setObjectName("label_2") + self.vboxlayout2.addWidget(self.label_2) + + self.sourceEdit = QtGui.QTextEdit(self.layoutWidget1) + self.sourceEdit.setObjectName("sourceEdit") + self.vboxlayout2.addWidget(self.sourceEdit) + self.gridlayout.addLayout(self.vboxlayout2,0,0,1,1) + + self.vboxlayout3 = QtGui.QVBoxLayout() + self.vboxlayout3.setObjectName("vboxlayout3") + + self.label_3 = QtGui.QLabel(self.layoutWidget1) + self.label_3.setObjectName("label_3") + self.vboxlayout3.addWidget(self.label_3) + + self.sourceAnnotEdit = QtGui.QTextEdit(self.layoutWidget1) + self.sourceAnnotEdit.setObjectName("sourceAnnotEdit") + self.vboxlayout3.addWidget(self.sourceAnnotEdit) + self.gridlayout.addLayout(self.vboxlayout3,0,1,1,1) + + self.vboxlayout4 = QtGui.QVBoxLayout() + self.vboxlayout4.setObjectName("vboxlayout4") + + self.label_4 = QtGui.QLabel(self.layoutWidget1) + self.label_4.setObjectName("label_4") + self.vboxlayout4.addWidget(self.label_4) + + self.translEdit = QtGui.QTextEdit(self.layoutWidget1) + self.translEdit.setObjectName("translEdit") + self.vboxlayout4.addWidget(self.translEdit) + self.gridlayout.addLayout(self.vboxlayout4,1,0,1,1) + + self.vboxlayout5 = QtGui.QVBoxLayout() + self.vboxlayout5.setObjectName("vboxlayout5") + + self.label_5 = QtGui.QLabel(self.layoutWidget1) + self.label_5.setObjectName("label_5") + self.vboxlayout5.addWidget(self.label_5) + + self.translAnnotEdit = QtGui.QTextEdit(self.layoutWidget1) + self.translAnnotEdit.setObjectName("translAnnotEdit") + self.vboxlayout5.addWidget(self.translAnnotEdit) + self.gridlayout.addLayout(self.vboxlayout5,1,1,1,1) + self.vboxlayout.addWidget(self.splitter) + + self.retranslateUi(MsgForm) + QtCore.QMetaObject.connectSlotsByName(MsgForm) + + def retranslateUi(self, MsgForm): + MsgForm.setWindowTitle(QtGui.QApplication.translate("MsgForm", "Form", None, QtGui.QApplication.UnicodeUTF8)) + self.label.setText(QtGui.QApplication.translate("MsgForm", "Messages:", None, QtGui.QApplication.UnicodeUTF8)) + self.treeWidget.headerItem().setText(0,QtGui.QApplication.translate("MsgForm", "1", None, QtGui.QApplication.UnicodeUTF8)) + self.label_2.setText(QtGui.QApplication.translate("MsgForm", "Source string:", None, QtGui.QApplication.UnicodeUTF8)) + self.label_3.setText(QtGui.QApplication.translate("MsgForm", "Source annotation:", None, QtGui.QApplication.UnicodeUTF8)) + self.label_4.setText(QtGui.QApplication.translate("MsgForm", "Translation:", None, QtGui.QApplication.UnicodeUTF8)) + self.label_5.setText(QtGui.QApplication.translate("MsgForm", "Translator\'s annotation:", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_new_project.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_new_project.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,76 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'new_project.ui' +# +# Created: Sun Oct 21 17:33:29 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_NewProjectDialog(object): + def setupUi(self, NewProjectDialog): + NewProjectDialog.setObjectName("NewProjectDialog") + NewProjectDialog.resize(QtCore.QSize(QtCore.QRect(0,0,401,131).size()).expandedTo(NewProjectDialog.minimumSizeHint())) + + self.vboxlayout = QtGui.QVBoxLayout(NewProjectDialog) + self.vboxlayout.setObjectName("vboxlayout") + + self.gridlayout = QtGui.QGridLayout() + self.gridlayout.setObjectName("gridlayout") + + self.gridlayout1 = QtGui.QGridLayout() + self.gridlayout1.setObjectName("gridlayout1") + + self.label = QtGui.QLabel(NewProjectDialog) + self.label.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label.setObjectName("label") + self.gridlayout1.addWidget(self.label,0,0,1,1) + + self.projectName = QtGui.QLineEdit(NewProjectDialog) + self.projectName.setObjectName("projectName") + self.gridlayout1.addWidget(self.projectName,0,1,1,1) + + self.label_2 = QtGui.QLabel(NewProjectDialog) + self.label_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label_2.setObjectName("label_2") + self.gridlayout1.addWidget(self.label_2,1,0,1,1) + + self.hboxlayout = QtGui.QHBoxLayout() + self.hboxlayout.setObjectName("hboxlayout") + + self.projectFilePath = QtGui.QLineEdit(NewProjectDialog) + self.projectFilePath.setObjectName("projectFilePath") + self.hboxlayout.addWidget(self.projectFilePath) + + self.pickFileButton = QtGui.QToolButton(NewProjectDialog) + self.pickFileButton.setObjectName("pickFileButton") + self.hboxlayout.addWidget(self.pickFileButton) + self.gridlayout1.addLayout(self.hboxlayout,1,1,1,1) + self.gridlayout.addLayout(self.gridlayout1,0,0,1,1) + + spacerItem = QtGui.QSpacerItem(20,40,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Expanding) + self.gridlayout.addItem(spacerItem,1,0,1,1) + + self.buttonBox = QtGui.QDialogButtonBox(NewProjectDialog) + self.buttonBox.setOrientation(QtCore.Qt.Horizontal) + self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Cancel|QtGui.QDialogButtonBox.NoButton|QtGui.QDialogButtonBox.Ok) + self.buttonBox.setObjectName("buttonBox") + self.gridlayout.addWidget(self.buttonBox,2,0,1,1) + self.vboxlayout.addLayout(self.gridlayout) + self.label.setBuddy(self.projectName) + self.label_2.setBuddy(self.projectFilePath) + + self.retranslateUi(NewProjectDialog) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),NewProjectDialog.accept) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("rejected()"),NewProjectDialog.reject) + QtCore.QMetaObject.connectSlotsByName(NewProjectDialog) + + def retranslateUi(self, NewProjectDialog): + NewProjectDialog.setWindowTitle(QtGui.QApplication.translate("NewProjectDialog", "Create New Project", None, QtGui.QApplication.UnicodeUTF8)) + self.label.setText(QtGui.QApplication.translate("NewProjectDialog", "Project name:", None, QtGui.QApplication.UnicodeUTF8)) + self.label_2.setText(QtGui.QApplication.translate("NewProjectDialog", "Project file:", None, QtGui.QApplication.UnicodeUTF8)) + self.pickFileButton.setText(QtGui.QApplication.translate("NewProjectDialog", "...", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_project_properties.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_project_properties.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,82 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'project_properties.ui' +# +# Created: Sat Nov 17 23:14:55 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_ProjectPropertiesDialog(object): + def setupUi(self, ProjectPropertiesDialog): + ProjectPropertiesDialog.setObjectName("ProjectPropertiesDialog") + ProjectPropertiesDialog.resize(QtCore.QSize(QtCore.QRect(0,0,400,152).size()).expandedTo(ProjectPropertiesDialog.minimumSizeHint())) + + self.vboxlayout = QtGui.QVBoxLayout(ProjectPropertiesDialog) + self.vboxlayout.setObjectName("vboxlayout") + + self.gridlayout = QtGui.QGridLayout() + self.gridlayout.setObjectName("gridlayout") + + self.label_2 = QtGui.QLabel(ProjectPropertiesDialog) + self.label_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label_2.setObjectName("label_2") + self.gridlayout.addWidget(self.label_2,0,0,1,1) + + self.projectNameField = QtGui.QLineEdit(ProjectPropertiesDialog) + self.projectNameField.setObjectName("projectNameField") + self.gridlayout.addWidget(self.projectNameField,0,1,1,1) + + self.label = QtGui.QLabel(ProjectPropertiesDialog) + self.label.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label.setObjectName("label") + self.gridlayout.addWidget(self.label,1,0,1,1) + + self.hboxlayout = QtGui.QHBoxLayout() + self.hboxlayout.setObjectName("hboxlayout") + + self.buildScriptField = QtGui.QLineEdit(ProjectPropertiesDialog) + self.buildScriptField.setObjectName("buildScriptField") + self.hboxlayout.addWidget(self.buildScriptField) + + self.pickFileButton = QtGui.QToolButton(ProjectPropertiesDialog) + self.pickFileButton.setObjectName("pickFileButton") + self.hboxlayout.addWidget(self.pickFileButton) + self.gridlayout.addLayout(self.hboxlayout,1,1,1,1) + + self.creationDateField = QtGui.QLineEdit(ProjectPropertiesDialog) + self.creationDateField.setObjectName("creationDateField") + self.gridlayout.addWidget(self.creationDateField,2,1,1,1) + + self.label_3 = QtGui.QLabel(ProjectPropertiesDialog) + self.label_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) + self.label_3.setObjectName("label_3") + self.gridlayout.addWidget(self.label_3,2,0,1,1) + self.vboxlayout.addLayout(self.gridlayout) + + spacerItem = QtGui.QSpacerItem(20,40,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Expanding) + self.vboxlayout.addItem(spacerItem) + + self.buttonBox = QtGui.QDialogButtonBox(ProjectPropertiesDialog) + self.buttonBox.setOrientation(QtCore.Qt.Horizontal) + self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Cancel|QtGui.QDialogButtonBox.NoButton|QtGui.QDialogButtonBox.Ok) + self.buttonBox.setObjectName("buttonBox") + self.vboxlayout.addWidget(self.buttonBox) + self.label_2.setBuddy(self.projectNameField) + self.label.setBuddy(self.buildScriptField) + + self.retranslateUi(ProjectPropertiesDialog) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),ProjectPropertiesDialog.accept) + QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("rejected()"),ProjectPropertiesDialog.reject) + QtCore.QMetaObject.connectSlotsByName(ProjectPropertiesDialog) + + def retranslateUi(self, ProjectPropertiesDialog): + ProjectPropertiesDialog.setWindowTitle(QtGui.QApplication.translate("ProjectPropertiesDialog", "Project Properties", None, QtGui.QApplication.UnicodeUTF8)) + self.label_2.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "Project name:", None, QtGui.QApplication.UnicodeUTF8)) + self.label.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "Build script:", None, QtGui.QApplication.UnicodeUTF8)) + self.pickFileButton.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "...", None, QtGui.QApplication.UnicodeUTF8)) + self.label_3.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "Creation date:", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/translator/ui_translator.py --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/translator/ui_translator.py Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,133 @@ +# -*- coding: utf-8 -*- + +# Form implementation generated from reading ui file 'translator.ui' +# +# Created: Fri Nov 9 20:02:57 2007 +# by: PyQt4 UI code generator 4.1 +# +# WARNING! All changes made in this file will be lost! + +import sys +from PyQt4 import QtCore, QtGui + +class Ui_MainWindow(object): + def setupUi(self, MainWindow): + MainWindow.setObjectName("MainWindow") + MainWindow.resize(QtCore.QSize(QtCore.QRect(0,0,608,464).size()).expandedTo(MainWindow.minimumSizeHint())) + + self.centralwidget = QtGui.QWidget(MainWindow) + self.centralwidget.setObjectName("centralwidget") + + self.vboxlayout = QtGui.QVBoxLayout(self.centralwidget) + self.vboxlayout.setObjectName("vboxlayout") + MainWindow.setCentralWidget(self.centralwidget) + + self.menubar = QtGui.QMenuBar(MainWindow) + self.menubar.setGeometry(QtCore.QRect(0,0,608,29)) + self.menubar.setObjectName("menubar") + + self.menu_File = QtGui.QMenu(self.menubar) + self.menu_File.setObjectName("menu_File") + + self.menu_Help = QtGui.QMenu(self.menubar) + self.menu_Help.setObjectName("menu_Help") + + self.menu_Project = QtGui.QMenu(self.menubar) + self.menu_Project.setObjectName("menu_Project") + MainWindow.setMenuBar(self.menubar) + + self.statusbar = QtGui.QStatusBar(MainWindow) + self.statusbar.setObjectName("statusbar") + MainWindow.setStatusBar(self.statusbar) + + self.action_Quit = QtGui.QAction(MainWindow) + self.action_Quit.setObjectName("action_Quit") + + self.action_About = QtGui.QAction(MainWindow) + self.action_About.setObjectName("action_About") + + self.action_New_Project = QtGui.QAction(MainWindow) + self.action_New_Project.setObjectName("action_New_Project") + + self.action_Add_Catalogue = QtGui.QAction(MainWindow) + self.action_Add_Catalogue.setObjectName("action_Add_Catalogue") + + self.action_Properties = QtGui.QAction(MainWindow) + self.action_Properties.setObjectName("action_Properties") + + self.action_Build_Project = QtGui.QAction(MainWindow) + self.action_Build_Project.setObjectName("action_Build_Project") + + self.action_Open_Project = QtGui.QAction(MainWindow) + self.action_Open_Project.setObjectName("action_Open_Project") + + self.action_Close = QtGui.QAction(MainWindow) + self.action_Close.setObjectName("action_Close") + + self.action_Add_New_Catalogue = QtGui.QAction(MainWindow) + self.action_Add_New_Catalogue.setObjectName("action_Add_New_Catalogue") + + self.action_Close_Project = QtGui.QAction(MainWindow) + self.action_Close_Project.setObjectName("action_Close_Project") + + self.action_Save = QtGui.QAction(MainWindow) + self.action_Save.setObjectName("action_Save") + + self.action_Save_All = QtGui.QAction(MainWindow) + self.action_Save_All.setObjectName("action_Save_All") + + self.action_Close_All = QtGui.QAction(MainWindow) + self.action_Close_All.setObjectName("action_Close_All") + self.menu_File.addAction(self.action_Open_Project) + self.menu_File.addAction(self.action_New_Project) + self.menu_File.addSeparator() + self.menu_File.addAction(self.action_Save) + self.menu_File.addAction(self.action_Save_All) + self.menu_File.addSeparator() + self.menu_File.addAction(self.action_Close) + self.menu_File.addAction(self.action_Close_All) + self.menu_File.addSeparator() + self.menu_File.addAction(self.action_Quit) + self.menu_Help.addAction(self.action_About) + self.menu_Project.addAction(self.action_Add_Catalogue) + self.menu_Project.addAction(self.action_Add_New_Catalogue) + self.menu_Project.addAction(self.action_Build_Project) + self.menu_Project.addSeparator() + self.menu_Project.addAction(self.action_Properties) + self.menu_Project.addSeparator() + self.menu_Project.addAction(self.action_Close_Project) + self.menubar.addAction(self.menu_File.menuAction()) + self.menubar.addAction(self.menu_Project.menuAction()) + self.menubar.addAction(self.menu_Help.menuAction()) + + self.retranslateUi(MainWindow) + QtCore.QObject.connect(self.action_Quit,QtCore.SIGNAL("triggered()"),MainWindow.close) + QtCore.QMetaObject.connectSlotsByName(MainWindow) + + def retranslateUi(self, MainWindow): + MainWindow.setWindowTitle(QtGui.QApplication.translate("MainWindow", "Translator", None, QtGui.QApplication.UnicodeUTF8)) + self.menu_File.setTitle(QtGui.QApplication.translate("MainWindow", "&File", None, QtGui.QApplication.UnicodeUTF8)) + self.menu_Help.setTitle(QtGui.QApplication.translate("MainWindow", "&Help", None, QtGui.QApplication.UnicodeUTF8)) + self.menu_Project.setTitle(QtGui.QApplication.translate("MainWindow", "&Project", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Quit.setText(QtGui.QApplication.translate("MainWindow", "&Quit", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Quit.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+Q", None, QtGui.QApplication.UnicodeUTF8)) + self.action_About.setText(QtGui.QApplication.translate("MainWindow", "&About", None, QtGui.QApplication.UnicodeUTF8)) + self.action_New_Project.setText(QtGui.QApplication.translate("MainWindow", "&New Project...", None, QtGui.QApplication.UnicodeUTF8)) + self.action_New_Project.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+N", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Add_Catalogue.setText(QtGui.QApplication.translate("MainWindow", "&Add catalogue...", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Properties.setText(QtGui.QApplication.translate("MainWindow", "&Properties", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Build_Project.setText(QtGui.QApplication.translate("MainWindow", "&Build Project", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Open_Project.setText(QtGui.QApplication.translate("MainWindow", "&Open Project...", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Open_Project.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+O", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Close.setText(QtGui.QApplication.translate("MainWindow", "&Close", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Close.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+W", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Add_New_Catalogue.setText(QtGui.QApplication.translate("MainWindow", "Add new catalogue...", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Close_Project.setText(QtGui.QApplication.translate("MainWindow", "&Close Project", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Close_Project.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+F4", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Save.setText(QtGui.QApplication.translate("MainWindow", "&Save", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Save.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+S", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Save_All.setText(QtGui.QApplication.translate("MainWindow", "Save &All", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Save_All.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+Shift+S", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Close_All.setText(QtGui.QApplication.translate("MainWindow", "Clos&e All", None, QtGui.QApplication.UnicodeUTF8)) + self.action_Close_All.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+Shift+W", None, QtGui.QApplication.UnicodeUTF8)) + diff -r a3fab8b74a7d -r bcb74c9b895c src/util/uni.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/util/uni.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,632 @@ + +// Written in the D programming language. + +/* + * Placed into the Public Domain. + * Digital Mars, www.digitalmars.com + * Written by Walter Bright + */ + +/** + * Simple Unicode character classification functions. + * For ASCII classification, see $(LINK2 std_ctype.html, std.ctype). + * Macros: + * WIKI=Phobos/StdUni + * References: + * $(LINK2 http://www.digitalmars.com/d/ascii-table.html, ASCII Table), + * $(LINK2 http://en.wikipedia.org/wiki/Unicode, Wikipedia), + * $(LINK2 http://www.unicode.org, The Unicode Consortium) + * Trademarks: + * Unicode(tm) is a trademark of Unicode, Inc. + * Copyright: + * Public Domain. + */ + + +module util.uni; + +/** + * Returns !=0 if c is a Unicode lower case character. + */ +int isUniLower(dchar c) +{ + if (c <= 0x7F) + return (c >= 'a' && c <= 'z'); + + return isUniAlpha(c) && c == toUniLower(c); +} + +/** + * Returns !=0 if c is a Unicode upper case character. + */ +int isUniUpper(dchar c) +{ + if (c <= 0x7F) + return (c >= 'A' && c <= 'Z'); + + return isUniAlpha(c) && c == toUniUpper(c); +} + +/** + * If c is a Unicode upper case character, return the lower case + * equivalent, otherwise return c. + */ +dchar toUniLower(dchar c) +{ + if (c >= 'A' && c <= 'Z') + { + c += 32; + } + else if (c >= 0x00C0) + { + if ((c >= 0x00C0 && c <= 0x00D6) || (c >= 0x00D8 && c<=0x00DE)) + { + c += 32; + } + else if ((c >= 0x0100 && c < 0x0138) || (c > 0x0149 && c < 0x0178)) + { + if (c == 0x0130) + c = 0x0069; + else if ((c & 1) == 0) + c += 1; + } + else if (c == 0x0178) + { + c = 0x00FF; + } + else if ((c >= 0x0139 && c < 0x0149) || (c > 0x0178 && c < 0x017F)) + { + if (c & 1) + c += 1; + } + else if (c >= 0x0200 && c <= 0x0217) + { + if ((c & 1) == 0) + c += 1; + } + else if ((c >= 0x0401 && c <= 0x040C) || (c>= 0x040E && c <= 0x040F)) + { + c += 80; + } + else if (c >= 0x0410 && c <= 0x042F) + { + c += 32; + } + else if (c >= 0x0460 && c <= 0x047F) + { + if ((c & 1) == 0) + c += 1; + } + else if (c >= 0x0531 && c <= 0x0556) + { + c += 48; + } + else if (c >= 0x10A0 && c <= 0x10C5) + { + c += 48; + } + else if (c >= 0xFF21 && c <= 0xFF3A) + { + c += 32; + } + } + return c; +} + +/** + * If c is a Unicode lower case character, return the upper case + * equivalent, otherwise return c. + */ +dchar toUniUpper(dchar c) +{ + if (c >= 'a' && c <= 'z') + { + c -= 32; + } + else if (c >= 0x00E0) + { + if ((c >= 0x00E0 && c <= 0x00F6) || (c >= 0x00F8 && c <= 0x00FE)) + { + c -= 32; + } + else if (c == 0x00FF) + { + c = 0x0178; + } + else if ((c >= 0x0100 && c < 0x0138) || (c > 0x0149 && c < 0x0178)) + { + if (c == 0x0131) + c = 0x0049; + else if (c & 1) + c -= 1; + } + else if ((c >= 0x0139 && c < 0x0149) || (c > 0x0178 && c < 0x017F)) + { + if ((c & 1) == 0) + c = c-1; + } + else if (c == 0x017F) + { + c = 0x0053; + } + else if (c >= 0x0200 && c <= 0x0217) + { + if (c & 1) + c = c-1; + } + else if (c >= 0x0430 && c<= 0x044F) + { + c -= 32; + } + else if ((c >= 0x0451 && c <= 0x045C) || (c >=0x045E && c<= 0x045F)) + { + c -= 80; + } + else if (c >= 0x0460 && c <= 0x047F) + { + if (c & 1) + c -= 1; + } + else if (c >= 0x0561 && c < 0x0587) + { + c -= 48; + } + else if (c >= 0xFF41 && c <= 0xFF5A) + { + c -= 32; + } + } + return c; +} + + +/******************************* + * Return !=0 if u is a Unicode alpha character. + * (general Unicode category: Lu, Ll, Lt, Lm and Lo) + * + * Standards: Unicode 5.0.0 + */ + +int isUniAlpha(dchar u) +{ + static dchar table[][2] = + [ + [ 'A', 'Z' ], + [ 'a', 'z' ], + [ 0x00AA, 0x00AA ], + [ 0x00B5, 0x00B5 ], + [ 0x00BA, 0x00BA ], + [ 0x00C0, 0x00D6 ], + [ 0x00D8, 0x00F6 ], + [ 0x00F8, 0x02C1 ], + [ 0x02C6, 0x02D1 ], + [ 0x02E0, 0x02E4 ], + [ 0x02EE, 0x02EE ], + [ 0x037A, 0x037D ], + [ 0x0386, 0x0386 ], + [ 0x0388, 0x038A ], + [ 0x038C, 0x038C ], + [ 0x038E, 0x03A1 ], + [ 0x03A3, 0x03CE ], + [ 0x03D0, 0x03F5 ], + [ 0x03F7, 0x0481 ], + [ 0x048A, 0x0513 ], + [ 0x0531, 0x0556 ], + [ 0x0559, 0x0559 ], + [ 0x0561, 0x0587 ], + [ 0x05D0, 0x05EA ], + [ 0x05F0, 0x05F2 ], + [ 0x0621, 0x063A ], + [ 0x0640, 0x064A ], + [ 0x066E, 0x066F ], + [ 0x0671, 0x06D3 ], + [ 0x06D5, 0x06D5 ], + [ 0x06E5, 0x06E6 ], + [ 0x06EE, 0x06EF ], + [ 0x06FA, 0x06FC ], + [ 0x06FF, 0x06FF ], + [ 0x0710, 0x0710 ], + [ 0x0712, 0x072F ], + [ 0x074D, 0x076D ], + [ 0x0780, 0x07A5 ], + [ 0x07B1, 0x07B1 ], + [ 0x07CA, 0x07EA ], + [ 0x07F4, 0x07F5 ], + [ 0x07FA, 0x07FA ], + [ 0x0904, 0x0939 ], + [ 0x093D, 0x093D ], + [ 0x0950, 0x0950 ], + [ 0x0958, 0x0961 ], + [ 0x097B, 0x097F ], + [ 0x0985, 0x098C ], + [ 0x098F, 0x0990 ], + [ 0x0993, 0x09A8 ], + [ 0x09AA, 0x09B0 ], + [ 0x09B2, 0x09B2 ], + [ 0x09B6, 0x09B9 ], + [ 0x09BD, 0x09BD ], + [ 0x09CE, 0x09CE ], + [ 0x09DC, 0x09DD ], + [ 0x09DF, 0x09E1 ], + [ 0x09F0, 0x09F1 ], + [ 0x0A05, 0x0A0A ], + [ 0x0A0F, 0x0A10 ], + [ 0x0A13, 0x0A28 ], + [ 0x0A2A, 0x0A30 ], + [ 0x0A32, 0x0A33 ], + [ 0x0A35, 0x0A36 ], + [ 0x0A38, 0x0A39 ], + [ 0x0A59, 0x0A5C ], + [ 0x0A5E, 0x0A5E ], + [ 0x0A72, 0x0A74 ], + [ 0x0A85, 0x0A8D ], + [ 0x0A8F, 0x0A91 ], + [ 0x0A93, 0x0AA8 ], + [ 0x0AAA, 0x0AB0 ], + [ 0x0AB2, 0x0AB3 ], + [ 0x0AB5, 0x0AB9 ], + [ 0x0ABD, 0x0ABD ], + [ 0x0AD0, 0x0AD0 ], + [ 0x0AE0, 0x0AE1 ], + [ 0x0B05, 0x0B0C ], + [ 0x0B0F, 0x0B10 ], + [ 0x0B13, 0x0B28 ], + [ 0x0B2A, 0x0B30 ], + [ 0x0B32, 0x0B33 ], + [ 0x0B35, 0x0B39 ], + [ 0x0B3D, 0x0B3D ], + [ 0x0B5C, 0x0B5D ], + [ 0x0B5F, 0x0B61 ], + [ 0x0B71, 0x0B71 ], + [ 0x0B83, 0x0B83 ], + [ 0x0B85, 0x0B8A ], + [ 0x0B8E, 0x0B90 ], + [ 0x0B92, 0x0B95 ], + [ 0x0B99, 0x0B9A ], + [ 0x0B9C, 0x0B9C ], + [ 0x0B9E, 0x0B9F ], + [ 0x0BA3, 0x0BA4 ], + [ 0x0BA8, 0x0BAA ], + [ 0x0BAE, 0x0BB9 ], + [ 0x0C05, 0x0C0C ], + [ 0x0C0E, 0x0C10 ], + [ 0x0C12, 0x0C28 ], + [ 0x0C2A, 0x0C33 ], + [ 0x0C35, 0x0C39 ], + [ 0x0C60, 0x0C61 ], + [ 0x0C85, 0x0C8C ], + [ 0x0C8E, 0x0C90 ], + [ 0x0C92, 0x0CA8 ], + [ 0x0CAA, 0x0CB3 ], + [ 0x0CB5, 0x0CB9 ], + [ 0x0CBD, 0x0CBD ], + [ 0x0CDE, 0x0CDE ], + [ 0x0CE0, 0x0CE1 ], + [ 0x0D05, 0x0D0C ], + [ 0x0D0E, 0x0D10 ], + [ 0x0D12, 0x0D28 ], + [ 0x0D2A, 0x0D39 ], + [ 0x0D60, 0x0D61 ], + [ 0x0D85, 0x0D96 ], + [ 0x0D9A, 0x0DB1 ], + [ 0x0DB3, 0x0DBB ], + [ 0x0DBD, 0x0DBD ], + [ 0x0DC0, 0x0DC6 ], + [ 0x0E01, 0x0E30 ], + [ 0x0E32, 0x0E33 ], + [ 0x0E40, 0x0E46 ], + [ 0x0E81, 0x0E82 ], + [ 0x0E84, 0x0E84 ], + [ 0x0E87, 0x0E88 ], + [ 0x0E8A, 0x0E8A ], + [ 0x0E8D, 0x0E8D ], + [ 0x0E94, 0x0E97 ], + [ 0x0E99, 0x0E9F ], + [ 0x0EA1, 0x0EA3 ], + [ 0x0EA5, 0x0EA5 ], + [ 0x0EA7, 0x0EA7 ], + [ 0x0EAA, 0x0EAB ], + [ 0x0EAD, 0x0EB0 ], + [ 0x0EB2, 0x0EB3 ], + [ 0x0EBD, 0x0EBD ], + [ 0x0EC0, 0x0EC4 ], + [ 0x0EC6, 0x0EC6 ], + [ 0x0EDC, 0x0EDD ], + [ 0x0F00, 0x0F00 ], + [ 0x0F40, 0x0F47 ], + [ 0x0F49, 0x0F6A ], + [ 0x0F88, 0x0F8B ], + [ 0x1000, 0x1021 ], + [ 0x1023, 0x1027 ], + [ 0x1029, 0x102A ], + [ 0x1050, 0x1055 ], + [ 0x10A0, 0x10C5 ], + [ 0x10D0, 0x10FA ], + [ 0x10FC, 0x10FC ], + [ 0x1100, 0x1159 ], + [ 0x115F, 0x11A2 ], + [ 0x11A8, 0x11F9 ], + [ 0x1200, 0x1248 ], + [ 0x124A, 0x124D ], + [ 0x1250, 0x1256 ], + [ 0x1258, 0x1258 ], + [ 0x125A, 0x125D ], + [ 0x1260, 0x1288 ], + [ 0x128A, 0x128D ], + [ 0x1290, 0x12B0 ], + [ 0x12B2, 0x12B5 ], + [ 0x12B8, 0x12BE ], + [ 0x12C0, 0x12C0 ], + [ 0x12C2, 0x12C5 ], + [ 0x12C8, 0x12D6 ], + [ 0x12D8, 0x1310 ], + [ 0x1312, 0x1315 ], + [ 0x1318, 0x135A ], + [ 0x1380, 0x138F ], + [ 0x13A0, 0x13F4 ], + [ 0x1401, 0x166C ], + [ 0x166F, 0x1676 ], + [ 0x1681, 0x169A ], + [ 0x16A0, 0x16EA ], + [ 0x1700, 0x170C ], + [ 0x170E, 0x1711 ], + [ 0x1720, 0x1731 ], + [ 0x1740, 0x1751 ], + [ 0x1760, 0x176C ], + [ 0x176E, 0x1770 ], + [ 0x1780, 0x17B3 ], + [ 0x17D7, 0x17D7 ], + [ 0x17DC, 0x17DC ], + [ 0x1820, 0x1877 ], + [ 0x1880, 0x18A8 ], + [ 0x1900, 0x191C ], + [ 0x1950, 0x196D ], + [ 0x1970, 0x1974 ], + [ 0x1980, 0x19A9 ], + [ 0x19C1, 0x19C7 ], + [ 0x1A00, 0x1A16 ], + [ 0x1B05, 0x1B33 ], + [ 0x1B45, 0x1B4B ], + [ 0x1D00, 0x1DBF ], + [ 0x1E00, 0x1E9B ], + [ 0x1EA0, 0x1EF9 ], + [ 0x1F00, 0x1F15 ], + [ 0x1F18, 0x1F1D ], + [ 0x1F20, 0x1F45 ], + [ 0x1F48, 0x1F4D ], + [ 0x1F50, 0x1F57 ], + [ 0x1F59, 0x1F59 ], + [ 0x1F5B, 0x1F5B ], + [ 0x1F5D, 0x1F5D ], + [ 0x1F5F, 0x1F7D ], + [ 0x1F80, 0x1FB4 ], + [ 0x1FB6, 0x1FBC ], + [ 0x1FBE, 0x1FBE ], + [ 0x1FC2, 0x1FC4 ], + [ 0x1FC6, 0x1FCC ], + [ 0x1FD0, 0x1FD3 ], + [ 0x1FD6, 0x1FDB ], + [ 0x1FE0, 0x1FEC ], + [ 0x1FF2, 0x1FF4 ], + [ 0x1FF6, 0x1FFC ], + [ 0x2071, 0x2071 ], + [ 0x207F, 0x207F ], + [ 0x2090, 0x2094 ], + [ 0x2102, 0x2102 ], + [ 0x2107, 0x2107 ], + [ 0x210A, 0x2113 ], + [ 0x2115, 0x2115 ], + [ 0x2119, 0x211D ], + [ 0x2124, 0x2124 ], + [ 0x2126, 0x2126 ], + [ 0x2128, 0x2128 ], + [ 0x212A, 0x212D ], + [ 0x212F, 0x2139 ], + [ 0x213C, 0x213F ], + [ 0x2145, 0x2149 ], + [ 0x214E, 0x214E ], + [ 0x2183, 0x2184 ], + [ 0x2C00, 0x2C2E ], + [ 0x2C30, 0x2C5E ], + [ 0x2C60, 0x2C6C ], + [ 0x2C74, 0x2C77 ], + [ 0x2C80, 0x2CE4 ], + [ 0x2D00, 0x2D25 ], + [ 0x2D30, 0x2D65 ], + [ 0x2D6F, 0x2D6F ], + [ 0x2D80, 0x2D96 ], + [ 0x2DA0, 0x2DA6 ], + [ 0x2DA8, 0x2DAE ], + [ 0x2DB0, 0x2DB6 ], + [ 0x2DB8, 0x2DBE ], + [ 0x2DC0, 0x2DC6 ], + [ 0x2DC8, 0x2DCE ], + [ 0x2DD0, 0x2DD6 ], + [ 0x2DD8, 0x2DDE ], + [ 0x3005, 0x3006 ], + [ 0x3031, 0x3035 ], + [ 0x303B, 0x303C ], + [ 0x3041, 0x3096 ], + [ 0x309D, 0x309F ], + [ 0x30A1, 0x30FA ], + [ 0x30FC, 0x30FF ], + [ 0x3105, 0x312C ], + [ 0x3131, 0x318E ], + [ 0x31A0, 0x31B7 ], + [ 0x31F0, 0x31FF ], + [ 0x3400, 0x4DB5 ], + [ 0x4E00, 0x9FBB ], + [ 0xA000, 0xA48C ], + [ 0xA717, 0xA71A ], + [ 0xA800, 0xA801 ], + [ 0xA803, 0xA805 ], + [ 0xA807, 0xA80A ], + [ 0xA80C, 0xA822 ], + [ 0xA840, 0xA873 ], + [ 0xAC00, 0xD7A3 ], + [ 0xF900, 0xFA2D ], + [ 0xFA30, 0xFA6A ], + [ 0xFA70, 0xFAD9 ], + [ 0xFB00, 0xFB06 ], + [ 0xFB13, 0xFB17 ], + [ 0xFB1D, 0xFB1D ], + [ 0xFB1F, 0xFB28 ], + [ 0xFB2A, 0xFB36 ], + [ 0xFB38, 0xFB3C ], + [ 0xFB3E, 0xFB3E ], + [ 0xFB40, 0xFB41 ], + [ 0xFB43, 0xFB44 ], + [ 0xFB46, 0xFBB1 ], + [ 0xFBD3, 0xFD3D ], + [ 0xFD50, 0xFD8F ], + [ 0xFD92, 0xFDC7 ], + [ 0xFDF0, 0xFDFB ], + [ 0xFE70, 0xFE74 ], + [ 0xFE76, 0xFEFC ], + [ 0xFF21, 0xFF3A ], + [ 0xFF41, 0xFF5A ], + [ 0xFF66, 0xFFBE ], + [ 0xFFC2, 0xFFC7 ], + [ 0xFFCA, 0xFFCF ], + [ 0xFFD2, 0xFFD7 ], + [ 0xFFDA, 0xFFDC ], + [ 0x10000, 0x1000B ], + [ 0x1000D, 0x10026 ], + [ 0x10028, 0x1003A ], + [ 0x1003C, 0x1003D ], + [ 0x1003F, 0x1004D ], + [ 0x10050, 0x1005D ], + [ 0x10080, 0x100FA ], + [ 0x10300, 0x1031E ], + [ 0x10330, 0x10340 ], + [ 0x10342, 0x10349 ], + [ 0x10380, 0x1039D ], + [ 0x103A0, 0x103C3 ], + [ 0x103C8, 0x103CF ], + [ 0x10400, 0x1049D ], + [ 0x10800, 0x10805 ], + [ 0x10808, 0x10808 ], + [ 0x1080A, 0x10835 ], + [ 0x10837, 0x10838 ], + [ 0x1083C, 0x1083C ], + [ 0x1083F, 0x1083F ], + [ 0x10900, 0x10915 ], + [ 0x10A00, 0x10A00 ], + [ 0x10A10, 0x10A13 ], + [ 0x10A15, 0x10A17 ], + [ 0x10A19, 0x10A33 ], + [ 0x12000, 0x1236E ], + [ 0x1D400, 0x1D454 ], + [ 0x1D456, 0x1D49C ], + [ 0x1D49E, 0x1D49F ], + [ 0x1D4A2, 0x1D4A2 ], + [ 0x1D4A5, 0x1D4A6 ], + [ 0x1D4A9, 0x1D4AC ], + [ 0x1D4AE, 0x1D4B9 ], + [ 0x1D4BB, 0x1D4BB ], + [ 0x1D4BD, 0x1D4C3 ], + [ 0x1D4C5, 0x1D505 ], + [ 0x1D507, 0x1D50A ], + [ 0x1D50D, 0x1D514 ], + [ 0x1D516, 0x1D51C ], + [ 0x1D51E, 0x1D539 ], + [ 0x1D53B, 0x1D53E ], + [ 0x1D540, 0x1D544 ], + [ 0x1D546, 0x1D546 ], + [ 0x1D54A, 0x1D550 ], + [ 0x1D552, 0x1D6A5 ], + [ 0x1D6A8, 0x1D6C0 ], + [ 0x1D6C2, 0x1D6DA ], + [ 0x1D6DC, 0x1D6FA ], + [ 0x1D6FC, 0x1D714 ], + [ 0x1D716, 0x1D734 ], + [ 0x1D736, 0x1D74E ], + [ 0x1D750, 0x1D76E ], + [ 0x1D770, 0x1D788 ], + [ 0x1D78A, 0x1D7A8 ], + [ 0x1D7AA, 0x1D7C2 ], + [ 0x1D7C4, 0x1D7CB ], + [ 0x20000, 0x2A6D6 ], + [ 0x2F800, 0x2FA1D ], + ]; + + debug + { + for (int i = 0; i < table.length; i++) + { + assert(table[i][0] <= table[i][1]); + if (i < table.length - 1) + { +// if (table[i][1] >= table[i + 1][0]) +// printf("table[%d][1] = x%x, table[%d][0] = x%x\n", i, table[i][1], i + 1, table[i + 1][0]); + assert(table[i][1] < table[i + 1][0]); + } + } + } + + if (u < 0xAA) + { + if (u < 'A') + goto Lisnot; + if (u <= 'Z') + goto Lis; + if (u < 'a') + goto Lisnot; + if (u <= 'z') + goto Lis; + goto Lisnot; + } + + // Binary search + uint mid; + uint low; + uint high; + + low = 0; + high = table.length - 1; + while (cast(int)low <= cast(int)high) + { + mid = (low + high) >> 1; + if (u < table[mid][0]) + high = mid - 1; + else if (u > table[mid][1]) + low = mid + 1; + else + goto Lis; + } + +Lisnot: + debug + { + for (int i = 0; i < table.length; i++) + { + assert(u < table[i][0] || u > table[i][1]); + } + } + return 0; + +Lis: + debug + { + for (int i = 0; i < table.length; i++) + { + if (u >= table[i][0] && u <= table[i][1]) + return 1; + } + assert(0); // should have been in table + } + return 1; +} + +unittest +{ + for (uint i = 0; i < 0x80; i++) + { + if (i >= 'A' && i <= 'Z') + assert(isUniAlpha(i)); + else if (i >= 'a' && i <= 'z') + assert(isUniAlpha(i)); + else + assert(!isUniAlpha(i)); + } +} diff -r a3fab8b74a7d -r bcb74c9b895c src/xml.css --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/xml.css Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,71 @@ +@charset "utf-8"; +compilerinfo, sourcecode, linescolumn { + white-space: pre; + font-family: Monospace; + font-size: 0.8em; +} +compilerinfo, sourcecode { + display: block; +} +compilerinfo { + white-space: normal; + border: 1px solid #A22; + padding: 0.5em; + margin: 1em; +} +compilerinfo error { display: block; } +linescolumn { + display: block; + float: left; + text-align: right; + margin-right: 0.2em; + border-right: 1px solid gray; +} +linescolumn a { display: block; color: #555; } +/* Number */ +n { color: teal; } +/* Keyword */ +k { color: darkblue; font-weight: bold; } +/* Line, block and nested comments */ +lc, bc, nc { color: green; } +/* Identifier */ +i { color: black; } +/* String literal */ +sl { color: red; } +/* Character literal */ +cl { color: purple; } +/* All bracket types */ +br { color: orange; } +/* Special tokens */ +st { color: green; font-weight: bold; } +/* #line, hash line */ +hl { color: green; } +/* filespec (e.g. #line number [filespec]) */ +fs { color: purple;} +/* When the first line starts with #! it's a "shebang" */ +shebang { color: gray; } +/* Deprecated styles. */ +/* Operator */ +/*op { color: royalblue; }*/ +/* Particular operators */ +/*op[t=aa] { content: "and"; }*/ /*&& ∧*/ +/*op[t=oo] { content: "or"; }*/ /*|| ∨*/ +/*op[t=n] { content: "¬"; }*/ /*!*/ +/*op[t=ne] { content: "≠"; }*/ /*!=*/ +/*op[t=le] { content: "≤"; }*/ /*<=*/ +/*op[t=ge] { content: "≥"; }*/ /*>=*/ +/*op[t=lg] { content: "≶"; }*/ /*<>*/ +/* +d = Declaration +s = Statement +e = Expression +t = Type +o = Other +*/ +/* d { background-color: #FFDDDD; } */ +/* e { background-color: #DDDDFF; } */ +d[t=Illegal], s[t=Illegal] { background-color: #DD4422; } +d[t=Module] i, d[t=Import] i { color: blue; } +t > i { color: #911; } +t > br, t > op { color: #911; } +t[t=Integral] k { color: #911; font-weight: normal; } diff -r a3fab8b74a7d -r bcb74c9b895c src/xml_map.d --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/xml_map.d Sun Mar 09 00:12:19 2008 +0100 @@ -0,0 +1,107 @@ +/// A map of document elements and D tokens to format strings. +string[string] map = [ + "DocHead" : ``\n + ``\n + "\n", + "DocEnd" : "\n", + "SourceBegin" : "", + "SourceEnd" : "\n", + "CompBegin" : "\n", + "CompEnd" : "\n", + "LexerError" : `{0}({1},{2})L: {3}`\n, + "ParserError" : `{0}({1},{2})P: {3}`\n, + "LineNumberBegin" : ``, + "LineNumberEnd" : ``, + "LineNumber" : `{0}`, + + // Node categories: + "Declaration" : "d", + "Statement" : "s", + "Expression" : "e", + "Type" : "t", + "Other" : "o", + + // {0} = node category. + // {1} = node class name: "Call", "If", "Class" etc. + // E.g.: ... + "NodeBegin" : `<{0} t="{1}">`, + "NodeEnd" : ``, + + "Identifier" : "{0}", + "String" : "{0}", + "Char" : "{0}", + "Number" : "{0}", + "Keyword" : "{0}", + + "LineC" : "{0}", + "BlockC" : "{0}", + "NestedC" : "{0}", + + "Shebang" : "{0}", + "HLine" : "{0}", // #line + "Filespec" : "{0}", // #line N "filespec" + "Newline" : "{0}", // \n | \r | \r\n | LS | PS + "Illegal" : "{0}", // A character not recognized by the lexer. + + "SpecialToken" : "{0}", // __FILE__, __LINE__ etc. + + "(" : "
    (
    ", + ")" : "
    )
    ", + "[" : "
    [
    ", + "]" : "
    ]
    ", + "{" : "
    {
    ", + "}" : "
    }
    ", + "." : ".", + ".." : "..", + "..." : "...", + "!<>=" : "!<>=", // Unordered + "!<>" : "!<>", // UorE + "!<=" : "!<=", // UorG + "!<" : "!<", // UorGorE + "!>=" : "!>=", // UorL + "!>" : "!>", // UorLorE + "<>=" : "<>=", // LorEorG + "<>" : "<>", // LorG + "=" : "=", + "==" : "==", + "!" : "!", + "!=" : "!=", + "<=" : "<=", + "<" : "<", + ">=" : ">=", + ">" : ">", + "<<=" : "<<=", + "<<" : "<<", + ">>=" : ">>=", + ">>" : ">>", + ">>>=" : ">>>=", + ">>>" : ">>>", + "|" : "|", + "||" : "||", + "|=" : "|=", + "&" : "&", + "&&" : "&&", + "&=" : "&=", + "+" : "+", + "++" : "++", + "+=" : "+=", + "-" : "-", + "--" : "--", + "-=" : "-=", + "/" : "/", + "/=" : "/=", + "*" : "*", + "*=" : "*=", + "%" : "%", + "%=" : "%=", + "^" : "^", + "^=" : "^=", + "~" : "~", + "~=" : "~=", + ":" : ":", + ";" : ";", + "?" : "?", + "," : ",", + "$" : "$", + "EOF" : "" +]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/AUTHORS --- a/trunk/AUTHORS Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,13 +0,0 @@ -Founder: - Aziz Köksal - "All rights to the code I've written will pass over to Jari-Matti Mäkelä, - in case I catch the bus or the bus catches me. - My death will probably not be made known anywhere on the internet, - therefore my testament will become effective if I don't - show any signs of life for 6 months on the internet. - Within this time limit I may always revoke or edit this testament, - by committing to the hg repository at http://hg.sharesource.org/dil/ - Only the latest revision of this file is to be considered valid." - - Aziz Köksal -Contributors: - Jari-Matti Mäkelä diff -r a3fab8b74a7d -r bcb74c9b895c trunk/COPYING --- a/trunk/COPYING Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,674 +0,0 @@ - GNU GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The GNU General Public License is a free, copyleft license for -software and other kinds of works. - - The licenses for most software and other practical works are designed -to take away your freedom to share and change the works. By contrast, -the GNU General Public License is intended to guarantee your freedom to -share and change all versions of a program--to make sure it remains free -software for all its users. We, the Free Software Foundation, use the -GNU General Public License for most of our software; it applies also to -any other work released this way by its authors. You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -them if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs, and that you know you can do these things. - - To protect your rights, we need to prevent others from denying you -these rights or asking you to surrender the rights. Therefore, you have -certain responsibilities if you distribute copies of the software, or if -you modify it: responsibilities to respect the freedom of others. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must pass on to the recipients the same -freedoms that you received. You must make sure that they, too, receive -or can get the source code. And you must show them these terms so they -know their rights. - - Developers that use the GNU GPL protect your rights with two steps: -(1) assert copyright on the software, and (2) offer you this License -giving you legal permission to copy, distribute and/or modify it. - - For the developers' and authors' protection, the GPL clearly explains -that there is no warranty for this free software. For both users' and -authors' sake, the GPL requires that modified versions be marked as -changed, so that their problems will not be attributed erroneously to -authors of previous versions. - - Some devices are designed to deny users access to install or run -modified versions of the software inside them, although the manufacturer -can do so. This is fundamentally incompatible with the aim of -protecting users' freedom to change the software. The systematic -pattern of such abuse occurs in the area of products for individuals to -use, which is precisely where it is most unacceptable. Therefore, we -have designed this version of the GPL to prohibit the practice for those -products. If such problems arise substantially in other domains, we -stand ready to extend this provision to those domains in future versions -of the GPL, as needed to protect the freedom of users. - - Finally, every program is threatened constantly by software patents. -States should not allow patents to restrict development and use of -software on general-purpose computers, but in those that do, we wish to -avoid the special danger that patents applied to a free program could -make it effectively proprietary. To prevent this, the GPL assures that -patents cannot be used to render the program non-free. - - The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - - 0. Definitions. - - "This License" refers to version 3 of the GNU General Public License. - - "Copyright" also means copyright-like laws that apply to other kinds of -works, such as semiconductor masks. - - "The Program" refers to any copyrightable work licensed under this -License. Each licensee is addressed as "you". "Licensees" and -"recipients" may be individuals or organizations. - - To "modify" a work means to copy from or adapt all or part of the work -in a fashion requiring copyright permission, other than the making of an -exact copy. The resulting work is called a "modified version" of the -earlier work or a work "based on" the earlier work. - - A "covered work" means either the unmodified Program or a work based -on the Program. - - To "propagate" a work means to do anything with it that, without -permission, would make you directly or secondarily liable for -infringement under applicable copyright law, except executing it on a -computer or modifying a private copy. Propagation includes copying, -distribution (with or without modification), making available to the -public, and in some countries other activities as well. - - To "convey" a work means any kind of propagation that enables other -parties to make or receive copies. Mere interaction with a user through -a computer network, with no transfer of a copy, is not conveying. - - An interactive user interface displays "Appropriate Legal Notices" -to the extent that it includes a convenient and prominently visible -feature that (1) displays an appropriate copyright notice, and (2) -tells the user that there is no warranty for the work (except to the -extent that warranties are provided), that licensees may convey the -work under this License, and how to view a copy of this License. If -the interface presents a list of user commands or options, such as a -menu, a prominent item in the list meets this criterion. - - 1. Source Code. - - The "source code" for a work means the preferred form of the work -for making modifications to it. "Object code" means any non-source -form of a work. - - A "Standard Interface" means an interface that either is an official -standard defined by a recognized standards body, or, in the case of -interfaces specified for a particular programming language, one that -is widely used among developers working in that language. - - The "System Libraries" of an executable work include anything, other -than the work as a whole, that (a) is included in the normal form of -packaging a Major Component, but which is not part of that Major -Component, and (b) serves only to enable use of the work with that -Major Component, or to implement a Standard Interface for which an -implementation is available to the public in source code form. A -"Major Component", in this context, means a major essential component -(kernel, window system, and so on) of the specific operating system -(if any) on which the executable work runs, or a compiler used to -produce the work, or an object code interpreter used to run it. - - The "Corresponding Source" for a work in object code form means all -the source code needed to generate, install, and (for an executable -work) run the object code and to modify the work, including scripts to -control those activities. However, it does not include the work's -System Libraries, or general-purpose tools or generally available free -programs which are used unmodified in performing those activities but -which are not part of the work. For example, Corresponding Source -includes interface definition files associated with source files for -the work, and the source code for shared libraries and dynamically -linked subprograms that the work is specifically designed to require, -such as by intimate data communication or control flow between those -subprograms and other parts of the work. - - The Corresponding Source need not include anything that users -can regenerate automatically from other parts of the Corresponding -Source. - - The Corresponding Source for a work in source code form is that -same work. - - 2. Basic Permissions. - - All rights granted under this License are granted for the term of -copyright on the Program, and are irrevocable provided the stated -conditions are met. This License explicitly affirms your unlimited -permission to run the unmodified Program. The output from running a -covered work is covered by this License only if the output, given its -content, constitutes a covered work. This License acknowledges your -rights of fair use or other equivalent, as provided by copyright law. - - You may make, run and propagate covered works that you do not -convey, without conditions so long as your license otherwise remains -in force. You may convey covered works to others for the sole purpose -of having them make modifications exclusively for you, or provide you -with facilities for running those works, provided that you comply with -the terms of this License in conveying all material for which you do -not control copyright. Those thus making or running the covered works -for you must do so exclusively on your behalf, under your direction -and control, on terms that prohibit them from making any copies of -your copyrighted material outside their relationship with you. - - Conveying under any other circumstances is permitted solely under -the conditions stated below. Sublicensing is not allowed; section 10 -makes it unnecessary. - - 3. Protecting Users' Legal Rights From Anti-Circumvention Law. - - No covered work shall be deemed part of an effective technological -measure under any applicable law fulfilling obligations under article -11 of the WIPO copyright treaty adopted on 20 December 1996, or -similar laws prohibiting or restricting circumvention of such -measures. - - When you convey a covered work, you waive any legal power to forbid -circumvention of technological measures to the extent such circumvention -is effected by exercising rights under this License with respect to -the covered work, and you disclaim any intention to limit operation or -modification of the work as a means of enforcing, against the work's -users, your or third parties' legal rights to forbid circumvention of -technological measures. - - 4. Conveying Verbatim Copies. - - You may convey verbatim copies of the Program's source code as you -receive it, in any medium, provided that you conspicuously and -appropriately publish on each copy an appropriate copyright notice; -keep intact all notices stating that this License and any -non-permissive terms added in accord with section 7 apply to the code; -keep intact all notices of the absence of any warranty; and give all -recipients a copy of this License along with the Program. - - You may charge any price or no price for each copy that you convey, -and you may offer support or warranty protection for a fee. - - 5. Conveying Modified Source Versions. - - You may convey a work based on the Program, or the modifications to -produce it from the Program, in the form of source code under the -terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified - it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is - released under this License and any conditions added under section - 7. This requirement modifies the requirement in section 4 to - "keep intact all notices". - - c) You must license the entire work, as a whole, under this - License to anyone who comes into possession of a copy. This - License will therefore apply, along with any applicable section 7 - additional terms, to the whole of the work, and all its parts, - regardless of how they are packaged. This License gives no - permission to license the work in any other way, but it does not - invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display - Appropriate Legal Notices; however, if the Program has interactive - interfaces that do not display Appropriate Legal Notices, your - work need not make them do so. - - A compilation of a covered work with other separate and independent -works, which are not by their nature extensions of the covered work, -and which are not combined with it such as to form a larger program, -in or on a volume of a storage or distribution medium, is called an -"aggregate" if the compilation and its resulting copyright are not -used to limit the access or legal rights of the compilation's users -beyond what the individual works permit. Inclusion of a covered work -in an aggregate does not cause this License to apply to the other -parts of the aggregate. - - 6. Conveying Non-Source Forms. - - You may convey a covered work in object code form under the terms -of sections 4 and 5, provided that you also convey the -machine-readable Corresponding Source under the terms of this License, -in one of these ways: - - a) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by the - Corresponding Source fixed on a durable physical medium - customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by a - written offer, valid for at least three years and valid for as - long as you offer spare parts or customer support for that product - model, to give anyone who possesses the object code either (1) a - copy of the Corresponding Source for all the software in the - product that is covered by this License, on a durable physical - medium customarily used for software interchange, for a price no - more than your reasonable cost of physically performing this - conveying of source, or (2) access to copy the - Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the - written offer to provide the Corresponding Source. This - alternative is allowed only occasionally and noncommercially, and - only if you received the object code with such an offer, in accord - with subsection 6b. - - d) Convey the object code by offering access from a designated - place (gratis or for a charge), and offer equivalent access to the - Corresponding Source in the same way through the same place at no - further charge. You need not require recipients to copy the - Corresponding Source along with the object code. If the place to - copy the object code is a network server, the Corresponding Source - may be on a different server (operated by you or a third party) - that supports equivalent copying facilities, provided you maintain - clear directions next to the object code saying where to find the - Corresponding Source. Regardless of what server hosts the - Corresponding Source, you remain obligated to ensure that it is - available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided - you inform other peers where the object code and Corresponding - Source of the work are being offered to the general public at no - charge under subsection 6d. - - A separable portion of the object code, whose source code is excluded -from the Corresponding Source as a System Library, need not be -included in conveying the object code work. - - A "User Product" is either (1) a "consumer product", which means any -tangible personal property which is normally used for personal, family, -or household purposes, or (2) anything designed or sold for incorporation -into a dwelling. In determining whether a product is a consumer product, -doubtful cases shall be resolved in favor of coverage. For a particular -product received by a particular user, "normally used" refers to a -typical or common use of that class of product, regardless of the status -of the particular user or of the way in which the particular user -actually uses, or expects or is expected to use, the product. A product -is a consumer product regardless of whether the product has substantial -commercial, industrial or non-consumer uses, unless such uses represent -the only significant mode of use of the product. - - "Installation Information" for a User Product means any methods, -procedures, authorization keys, or other information required to install -and execute modified versions of a covered work in that User Product from -a modified version of its Corresponding Source. The information must -suffice to ensure that the continued functioning of the modified object -code is in no case prevented or interfered with solely because -modification has been made. - - If you convey an object code work under this section in, or with, or -specifically for use in, a User Product, and the conveying occurs as -part of a transaction in which the right of possession and use of the -User Product is transferred to the recipient in perpetuity or for a -fixed term (regardless of how the transaction is characterized), the -Corresponding Source conveyed under this section must be accompanied -by the Installation Information. But this requirement does not apply -if neither you nor any third party retains the ability to install -modified object code on the User Product (for example, the work has -been installed in ROM). - - The requirement to provide Installation Information does not include a -requirement to continue to provide support service, warranty, or updates -for a work that has been modified or installed by the recipient, or for -the User Product in which it has been modified or installed. Access to a -network may be denied when the modification itself materially and -adversely affects the operation of the network or violates the rules and -protocols for communication across the network. - - Corresponding Source conveyed, and Installation Information provided, -in accord with this section must be in a format that is publicly -documented (and with an implementation available to the public in -source code form), and must require no special password or key for -unpacking, reading or copying. - - 7. Additional Terms. - - "Additional permissions" are terms that supplement the terms of this -License by making exceptions from one or more of its conditions. -Additional permissions that are applicable to the entire Program shall -be treated as though they were included in this License, to the extent -that they are valid under applicable law. If additional permissions -apply only to part of the Program, that part may be used separately -under those permissions, but the entire Program remains governed by -this License without regard to the additional permissions. - - When you convey a copy of a covered work, you may at your option -remove any additional permissions from that copy, or from any part of -it. (Additional permissions may be written to require their own -removal in certain cases when you modify the work.) You may place -additional permissions on material, added by you to a covered work, -for which you have or can give appropriate copyright permission. - - Notwithstanding any other provision of this License, for material you -add to a covered work, you may (if authorized by the copyright holders of -that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the - terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or - author attributions in that material or in the Appropriate Legal - Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or - requiring that modified versions of such material be marked in - reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or - authors of the material; or - - e) Declining to grant rights under trademark law for use of some - trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that - material by anyone who conveys the material (or modified versions of - it) with contractual assumptions of liability to the recipient, for - any liability that these contractual assumptions directly impose on - those licensors and authors. - - All other non-permissive additional terms are considered "further -restrictions" within the meaning of section 10. If the Program as you -received it, or any part of it, contains a notice stating that it is -governed by this License along with a term that is a further -restriction, you may remove that term. If a license document contains -a further restriction but permits relicensing or conveying under this -License, you may add to a covered work material governed by the terms -of that license document, provided that the further restriction does -not survive such relicensing or conveying. - - If you add terms to a covered work in accord with this section, you -must place, in the relevant source files, a statement of the -additional terms that apply to those files, or a notice indicating -where to find the applicable terms. - - Additional terms, permissive or non-permissive, may be stated in the -form of a separately written license, or stated as exceptions; -the above requirements apply either way. - - 8. Termination. - - You may not propagate or modify a covered work except as expressly -provided under this License. Any attempt otherwise to propagate or -modify it is void, and will automatically terminate your rights under -this License (including any patent licenses granted under the third -paragraph of section 11). - - However, if you cease all violation of this License, then your -license from a particular copyright holder is reinstated (a) -provisionally, unless and until the copyright holder explicitly and -finally terminates your license, and (b) permanently, if the copyright -holder fails to notify you of the violation by some reasonable means -prior to 60 days after the cessation. - - Moreover, your license from a particular copyright holder is -reinstated permanently if the copyright holder notifies you of the -violation by some reasonable means, this is the first time you have -received notice of violation of this License (for any work) from that -copyright holder, and you cure the violation prior to 30 days after -your receipt of the notice. - - Termination of your rights under this section does not terminate the -licenses of parties who have received copies or rights from you under -this License. If your rights have been terminated and not permanently -reinstated, you do not qualify to receive new licenses for the same -material under section 10. - - 9. Acceptance Not Required for Having Copies. - - You are not required to accept this License in order to receive or -run a copy of the Program. Ancillary propagation of a covered work -occurring solely as a consequence of using peer-to-peer transmission -to receive a copy likewise does not require acceptance. However, -nothing other than this License grants you permission to propagate or -modify any covered work. These actions infringe copyright if you do -not accept this License. Therefore, by modifying or propagating a -covered work, you indicate your acceptance of this License to do so. - - 10. Automatic Licensing of Downstream Recipients. - - Each time you convey a covered work, the recipient automatically -receives a license from the original licensors, to run, modify and -propagate that work, subject to this License. You are not responsible -for enforcing compliance by third parties with this License. - - An "entity transaction" is a transaction transferring control of an -organization, or substantially all assets of one, or subdividing an -organization, or merging organizations. If propagation of a covered -work results from an entity transaction, each party to that -transaction who receives a copy of the work also receives whatever -licenses to the work the party's predecessor in interest had or could -give under the previous paragraph, plus a right to possession of the -Corresponding Source of the work from the predecessor in interest, if -the predecessor has it or can get it with reasonable efforts. - - You may not impose any further restrictions on the exercise of the -rights granted or affirmed under this License. For example, you may -not impose a license fee, royalty, or other charge for exercise of -rights granted under this License, and you may not initiate litigation -(including a cross-claim or counterclaim in a lawsuit) alleging that -any patent claim is infringed by making, using, selling, offering for -sale, or importing the Program or any portion of it. - - 11. Patents. - - A "contributor" is a copyright holder who authorizes use under this -License of the Program or a work on which the Program is based. The -work thus licensed is called the contributor's "contributor version". - - A contributor's "essential patent claims" are all patent claims -owned or controlled by the contributor, whether already acquired or -hereafter acquired, that would be infringed by some manner, permitted -by this License, of making, using, or selling its contributor version, -but do not include claims that would be infringed only as a -consequence of further modification of the contributor version. For -purposes of this definition, "control" includes the right to grant -patent sublicenses in a manner consistent with the requirements of -this License. - - Each contributor grants you a non-exclusive, worldwide, royalty-free -patent license under the contributor's essential patent claims, to -make, use, sell, offer for sale, import and otherwise run, modify and -propagate the contents of its contributor version. - - In the following three paragraphs, a "patent license" is any express -agreement or commitment, however denominated, not to enforce a patent -(such as an express permission to practice a patent or covenant not to -sue for patent infringement). To "grant" such a patent license to a -party means to make such an agreement or commitment not to enforce a -patent against the party. - - If you convey a covered work, knowingly relying on a patent license, -and the Corresponding Source of the work is not available for anyone -to copy, free of charge and under the terms of this License, through a -publicly available network server or other readily accessible means, -then you must either (1) cause the Corresponding Source to be so -available, or (2) arrange to deprive yourself of the benefit of the -patent license for this particular work, or (3) arrange, in a manner -consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have -actual knowledge that, but for the patent license, your conveying the -covered work in a country, or your recipient's use of the covered work -in a country, would infringe one or more identifiable patents in that -country that you have reason to believe are valid. - - If, pursuant to or in connection with a single transaction or -arrangement, you convey, or propagate by procuring conveyance of, a -covered work, and grant a patent license to some of the parties -receiving the covered work authorizing them to use, propagate, modify -or convey a specific copy of the covered work, then the patent license -you grant is automatically extended to all recipients of the covered -work and works based on it. - - A patent license is "discriminatory" if it does not include within -the scope of its coverage, prohibits the exercise of, or is -conditioned on the non-exercise of one or more of the rights that are -specifically granted under this License. You may not convey a covered -work if you are a party to an arrangement with a third party that is -in the business of distributing software, under which you make payment -to the third party based on the extent of your activity of conveying -the work, and under which the third party grants, to any of the -parties who would receive the covered work from you, a discriminatory -patent license (a) in connection with copies of the covered work -conveyed by you (or copies made from those copies), or (b) primarily -for and in connection with specific products or compilations that -contain the covered work, unless you entered into that arrangement, -or that patent license was granted, prior to 28 March 2007. - - Nothing in this License shall be construed as excluding or limiting -any implied license or other defenses to infringement that may -otherwise be available to you under applicable patent law. - - 12. No Surrender of Others' Freedom. - - If conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot convey a -covered work so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you -to collect a royalty for further conveying from those to whom you convey -the Program, the only way you could satisfy both those terms and this -License would be to refrain entirely from conveying the Program. - - 13. Use with the GNU Affero General Public License. - - Notwithstanding any other provision of this License, you have -permission to link or combine any covered work with a work licensed -under version 3 of the GNU Affero General Public License into a single -combined work, and to convey the resulting work. The terms of this -License will continue to apply to the part which is the covered work, -but the special requirements of the GNU Affero General Public License, -section 13, concerning interaction through a network will apply to the -combination as such. - - 14. Revised Versions of this License. - - The Free Software Foundation may publish revised and/or new versions of -the GNU General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - - Each version is given a distinguishing version number. If the -Program specifies that a certain numbered version of the GNU General -Public License "or any later version" applies to it, you have the -option of following the terms and conditions either of that numbered -version or of any later version published by the Free Software -Foundation. If the Program does not specify a version number of the -GNU General Public License, you may choose any version ever published -by the Free Software Foundation. - - If the Program specifies that a proxy can decide which future -versions of the GNU General Public License can be used, that proxy's -public statement of acceptance of a version permanently authorizes you -to choose that version for the Program. - - Later license versions may give you additional or different -permissions. However, no additional obligations are imposed on any -author or copyright holder as a result of your choosing to follow a -later version. - - 15. Disclaimer of Warranty. - - THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY -APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT -HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY -OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, -THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM -IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF -ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - - 16. Limitation of Liability. - - IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS -THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY -GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE -USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF -DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD -PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), -EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF -SUCH DAMAGES. - - 17. Interpretation of Sections 15 and 16. - - If the disclaimer of warranty and limitation of liability provided -above cannot be given local legal effect according to their terms, -reviewing courts shall apply local law that most closely approximates -an absolute waiver of all civil liability in connection with the -Program, unless a warranty or assumption of liability accompanies a -copy of the Program in return for a fee. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -state the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - - If the program does terminal interaction, make it output a short -notice like this when it starts in an interactive mode: - - Copyright (C) - This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, your program's commands -might be different; for a GUI interface, you would use an "about box". - - You should also get your employer (if you work as a programmer) or school, -if any, to sign a "copyright disclaimer" for the program, if necessary. -For more information on this, and how to apply and follow the GNU GPL, see -. - - The GNU General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications with -the library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License. But first, please read -. diff -r a3fab8b74a7d -r bcb74c9b895c trunk/README --- a/trunk/README Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,32 +0,0 @@ -dil -=== -Copyright (c) 2008 by Aziz Köksal -This program is free software, licensed under the GPL3. -Please, read the license file, COPYING, for further information. - -Description -=========== -This software is a compiler written in D for the D programming language. - -How To Compile dil -================== -In order to compile dil you need to have: - *) DMD 1.028 (http://www.digitalmars.com/d/1.0/changelog.html) - *) Tango 0.99.5 (http://dsource.org/projects/tango/) - *) DSSS 0.71 (http://dsource.org/projects/dsss/) - -If you can't compile dil because you have a newer version of these programs -then please report the problem to me (see Bugs section.) - -Before you run dil, make sure that the executable can find config.d and -lang_en.d (which are located in trunk/src/.) -The language can be configured in config.d. - -Bugs And Patches -================ -"errare humanum est, ignoscere divinum" - to err is human, to forgive divine. - - Cicero - -Users can report problems with this software or submit patches by: - *) contacting me: aziz.koeksal@gmail.com - *) filing a bug report here: http://code.google.com/p/dil/issues/list diff -r a3fab8b74a7d -r bcb74c9b895c trunk/dsss.conf --- a/trunk/dsss.conf Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,10 +0,0 @@ -name = dil -version = 0.1 -[src/main.d] -type = binary -target = dil -version(GNU) { - buildflags = -Isrc/ -Ldsss_objs/G/cmd.DDoc.o -Ldsss_objs/G/cmd.DDocXML.o -} else { - buildflags = -Isrc/ -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/i18n/build.py --- a/trunk/i18n/build.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -# -*- coding: utf-8 -*- -# Author: Aziz Köksal -# License: GPL2 diff -r a3fab8b74a7d -r bcb74c9b895c trunk/i18n/de.cat --- a/trunk/i18n/de.cat Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,86 +0,0 @@ -Authors: -- {EMail: aziz.koeksal@gmail.com, Name: Aziz Köksal} -LangCode: de -License: GPL3 -Messages: -- {Annot: '', ID: 0, LastEd: 0, Text: 'illegales Zeichen gefunden: ''{0}'''} -- {Annot: '', ID: 1, LastEd: 0, Text: ungültiges Unicodezeichen.} -- {Annot: '', ID: 2, LastEd: 0, Text: ungültige UTF-8-Sequenz.} -- {Annot: '', ID: 3, LastEd: 0, Text: unterminiertes Zeichenliteral.} -- {Annot: '', ID: 4, LastEd: 0, Text: leeres Zeichenliteral.} -- {Annot: '', ID: 5, LastEd: 0, Text: erwartete 'line' nach '#'.} -- {Annot: '', ID: 6, LastEd: 0, Text: 'Ganzzahl nach #line erwartet.'} -- {Annot: '', ID: 7, LastEd: 0, Text: erwartete Dateispezifikation (z.B. "pfad\zur\datei".)} -- {Annot: '', ID: 8, LastEd: 0, Text: unterminierte Dateispezifikation (filespec.)} -- {Annot: '', ID: 9, LastEd: 0, Text: ein Special Token muss mit einem Zeilenumbruch - abgeschlossen werden.} -- {Annot: '', ID: 10, LastEd: 0, Text: unterminiertes Zeichenkettenliteral.} -- {Annot: '', ID: 11, LastEd: 0, Text: 'Nicht-Hexzeichen ''{0}'' in Hexzeichenkette - gefunden.'} -- {Annot: '', ID: 12, LastEd: 0, Text: ungerade Anzahl von Hexziffern in Hexzeichenkette.} -- {Annot: '', ID: 13, LastEd: 0, Text: unterminierte Hexzeichenkette.} -- {Annot: '', ID: 14, LastEd: 0, Text: unterminierter Blockkommentar (/* */).} -- {Annot: '', ID: 15, LastEd: 0, Text: unterminierter verschachtelter Kommentar (/+ - +/).} -- {Annot: '', ID: 16, LastEd: 0, Text: unterminierte rohe Zeichenkette.} -- {Annot: '', ID: 17, LastEd: 0, Text: unterminierte Backquote-Zeichenkette.} -- {Annot: '', ID: 18, LastEd: 0, Text: 'undefinierte Escapesequenz ''{0}'' gefunden.'} -- {Annot: '', ID: 19, LastEd: 0, Text: 'ungültige Unicode-Escapesequenz ''{0}'' gefunden.'} -- {Annot: '', ID: 20, LastEd: 0, Text: unzureichende Anzahl von Hexziffern in Escapesequenz.} -- {Annot: '', ID: 21, LastEd: 0, Text: 'undefinierte HTML-Entität ''{0}'''} -- {Annot: '', ID: 22, LastEd: 0, Text: 'unterminierte HTML-Entität ''{0}''.'} -- {Annot: '', ID: 23, LastEd: 0, Text: HTML-Entitäten müssen mit einem Buchstaben - beginnen.} -- {Annot: '', ID: 24, LastEd: 0, Text: Dezimalzahl überläuft im Vorzeichenbit.} -- {Annot: '', ID: 25, LastEd: 0, Text: Überlauf in Dezimalzahl.} -- {Annot: '', ID: 26, LastEd: 0, Text: Überlauf in Hexadezimalzahl.} -- {Annot: '', ID: 27, LastEd: 0, Text: Überlauf in Binärzahl.} -- {Annot: '', ID: 28, LastEd: 0, Text: Überlauf in Oktalzahl.} -- {Annot: '', ID: 29, LastEd: 0, Text: Überlauf in Fließkommazahl.} -- {Annot: '', ID: 30, LastEd: 0, Text: die Ziffern 8 und 9 sind in Oktalzahlen unzulässig.} -- {Annot: '', ID: 31, LastEd: 0, Text: ungültige Hexzahl; mindestens eine Hexziffer - erforderlich.} -- {Annot: '', ID: 32, LastEd: 0, Text: ungültige Binärzahl; mindestens eine Binärziffer - erforderlich.} -- {Annot: '', ID: 33, LastEd: 0, Text: der Exponent einer hexadezimalen Fließkommazahl - ist erforderlich.} -- {Annot: '', ID: 34, LastEd: 0, Text: Hexadezimal-Exponenten müssen mit einer Dezimalziffer - anfangen.} -- {Annot: '', ID: 35, LastEd: 0, Text: Exponenten müssen mit einer Dezimalziffer anfangen.} -- {Annot: '', ID: 36, LastEd: 0, Text: 'erwartete ''{0}'', fand aber ''{1}''.'} -- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' ist redundant.'} -- {Annot: '', ID: 38, LastEd: 0, Text: Template-Tupel-Parameter dürfen nur am Ende - auftreten.} -- {Annot: '', ID: 39, LastEd: 0, Text: der 'in'-Vertrag der Funktion wurde bereits - geparsed.} -- {Annot: '', ID: 40, LastEd: 0, Text: der 'out'-Vertrag der Funktion wurde bereits - geparsed.} -- {Annot: '', ID: 41, LastEd: 0, Text: es wurde kein Verbindungstyp angegeben.} -- {Annot: '', ID: 42, LastEd: 0, Text: 'unbekannter Verbindungstyp ''{0}''; gültig - sind C, C++, D, Windows, Pascal und System.'} -- {Annot: '', ID: 43, LastEd: 0, Text: 'erwartete eine oder mehrere Basisklassen, - nicht ''{0}''.'} -- {Annot: '', ID: 44, LastEd: 0, Text: Basisklassen sind in Vorwärtsdeklarationen - nicht erlaubt.} -- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} - - Copyright (c) 2007, Aziz Köksal. Lizensiert unter der GPL3. - - - Befehle: - - {1} - - Geben Sie ''dil help '' ein, um mehr Hilfe zu einem bestimmten Befehl - zu - - erhalten. - - - Kompiliert mit {2} v{3} am {4}.'} -- {Annot: '', ID: 46, LastEd: 0, Text: "Generiere ein XML- oder HTML-Dokument aus\ - \ einer D-Quelltextdatei.\nVerwendung:\n dil gen datei.d [Optionen]\n\nOptionen:\n\ - \ --syntax : generiere Elemente für den Syntaxbaum\n --xml \ - \ : verwende XML-Format (voreingestellt)\n --html : verwende HTML-Format\n\ - \nBeispiel:\n dil gen Parser.d --html --syntax > Parser.html"} -- {Annot: '', ID: 47, LastEd: 0, Text: ''} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/i18n/dil.tproj --- a/trunk/i18n/dil.tproj Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,54 +0,0 @@ -BuildScript: build.py -CreationDate: '2007-10-12 09:41:17.868084' -LangFiles: [de.cat, tr.cat, fi.cat] -MsgIDs: -- {ID: 0, Name: IllegalCharacter, Order: 0} -- {ID: 1, Name: InvalidUnicodeCharacter, Order: 1} -- {ID: 2, Name: InvalidUTF8Sequence, Order: 2} -- {ID: 3, Name: UnterminatedCharacterLiteral, Order: 3} -- {ID: 4, Name: EmptyCharacterLiteral, Order: 4} -- {ID: 5, Name: ExpectedIdentifierSTLine, Order: 5} -- {ID: 6, Name: ExpectedIntegerAfterSTLine, Order: 6} -- {ID: 7, Name: ExpectedFilespec, Order: 7} -- {ID: 8, Name: UnterminatedFilespec, Order: 8} -- {ID: 9, Name: UnterminatedSpecialToken, Order: 9} -- {ID: 10, Name: UnterminatedString, Order: 10} -- {ID: 11, Name: NonHexCharInHexString, Order: 11} -- {ID: 12, Name: OddNumberOfDigitsInHexString, Order: 12} -- {ID: 13, Name: UnterminatedHexString, Order: 13} -- {ID: 14, Name: UnterminatedBlockComment, Order: 14} -- {ID: 15, Name: UnterminatedNestedComment, Order: 15} -- {ID: 16, Name: UnterminatedRawString, Order: 16} -- {ID: 17, Name: UnterminatedBackQuoteString, Order: 17} -- {ID: 18, Name: UndefinedEscapeSequence, Order: 18} -- {ID: 19, Name: InvalidUnicodeEscapeSequence, Order: 19} -- {ID: 20, Name: InsufficientHexDigits, Order: 20} -- {ID: 21, Name: UndefinedHTMLEntity, Order: 21} -- {ID: 22, Name: UnterminatedHTMLEntity, Order: 22} -- {ID: 23, Name: InvalidBeginHTMLEntity, Order: 23} -- {ID: 24, Name: OverflowDecimalSign, Order: 24} -- {ID: 25, Name: OverflowDecimalNumber, Order: 25} -- {ID: 26, Name: OverflowHexNumber, Order: 26} -- {ID: 27, Name: OverflowBinaryNumber, Order: 27} -- {ID: 28, Name: OverflowOctalNumber, Order: 28} -- {ID: 29, Name: OverflowFloatNumber, Order: 29} -- {ID: 30, Name: OctalNumberHasDecimals, Order: 30} -- {ID: 31, Name: NoDigitsInHexNumber, Order: 31} -- {ID: 32, Name: NoDigitsInBinNumber, Order: 32} -- {ID: 33, Name: HexFloatExponentRequired, Order: 33} -- {ID: 34, Name: HexFloatExpMustStartWithDigit, Order: 34} -- {ID: 35, Name: FloatExpMustStartWithDigit, Order: 35} -- {ID: 36, Name: ExpectedButFound, Order: 36} -- {ID: 37, Name: RedundantStorageClass, Order: 37} -- {ID: 38, Name: TemplateTupleParameter, Order: 38} -- {ID: 39, Name: InContract, Order: 39} -- {ID: 40, Name: OutContract, Order: 40} -- {ID: 41, Name: MissingLinkageType, Order: 41} -- {ID: 42, Name: UnrecognizedLinkageType, Order: 42} -- {ID: 43, Name: ExpectedBaseClasses, Order: 43} -- {ID: 44, Name: BaseClassInForwardDeclaration, Order: 44} -- {ID: 45, Name: HelpMain, Order: 45} -- {ID: 46, Name: HelpGenerate, Order: 46} -- {ID: 47, Name: HelpImportGraph, Order: 47} -Name: dil -SourceLangFile: en.cat diff -r a3fab8b74a7d -r bcb74c9b895c trunk/i18n/en.cat --- a/trunk/i18n/en.cat Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,90 +0,0 @@ -Authors: -- {EMail: aziz.koeksal@gmail.com, Name: Aziz Köksal} -LangCode: en -License: GPL3 -Messages: -- {Annot: '', ID: 0, LastEd: 0, Text: 'illegal character found: ''{0}'''} -- {Annot: '', ID: 1, LastEd: 0, Text: invalid Unicode character.} -- {Annot: '', ID: 2, LastEd: 0, Text: invalid UTF-8 sequence.} -- {Annot: '', ID: 3, LastEd: 0, Text: unterminated character literal.} -- {Annot: '', ID: 4, LastEd: 0, Text: empty character literal.} -- {Annot: '', ID: 5, LastEd: 0, Text: expected 'line' after '#'.} -- {Annot: '', ID: 6, LastEd: 0, Text: 'integer expected after #line'} -- {Annot: '', ID: 7, LastEd: 0, Text: expected filespec string (e.g. "path\to\file".)} -- {Annot: '', ID: 8, LastEd: 0, Text: unterminated filespec string.} -- {Annot: '', ID: 9, LastEd: 0, Text: expected a terminating newline after special - token.} -- {Annot: '', ID: 10, LastEd: 0, Text: unterminated string literal.} -- {Annot: '', ID: 11, LastEd: 0, Text: 'non-hex character ''{0}'' found in hex string.'} -- {Annot: '', ID: 12, LastEd: 0, Text: odd number of hex digits in hex string.} -- {Annot: '', ID: 13, LastEd: 0, Text: unterminated hex string.} -- {Annot: '', ID: 14, LastEd: 0, Text: unterminated block comment (/* */).} -- {Annot: '', ID: 15, LastEd: 0, Text: unterminated nested comment (/+ +/).} -- {Annot: '', ID: 16, LastEd: 0, Text: unterminated raw string.} -- {Annot: '', ID: 17, LastEd: 0, Text: unterminated back quote string.} -- {Annot: '', ID: 18, LastEd: 0, Text: 'found undefined escape sequence ''{0}''.'} -- {Annot: '', ID: 19, LastEd: 0, Text: 'found invalid Unicode escape sequence ''{0}''.'} -- {Annot: '', ID: 20, LastEd: 0, Text: insufficient number of hex digits in escape - sequence.} -- {Annot: '', ID: 21, LastEd: 0, Text: 'undefined HTML entity ''{0}'''} -- {Annot: '', ID: 22, LastEd: 0, Text: 'unterminated HTML entity ''{0}''.'} -- {Annot: '', ID: 23, LastEd: 0, Text: HTML entities must begin with a letter.} -- {Annot: '', ID: 24, LastEd: 0, Text: decimal number overflows sign bit.} -- {Annot: '', ID: 25, LastEd: 0, Text: overflow in decimal number.} -- {Annot: '', ID: 26, LastEd: 0, Text: overflow in hexadecimal number.} -- {Annot: '', ID: 27, LastEd: 0, Text: overflow in binary number.} -- {Annot: '', ID: 28, LastEd: 0, Text: overflow in octal number.} -- {Annot: '', ID: 29, LastEd: 0, Text: overflow in float number.} -- {Annot: '', ID: 30, LastEd: 0, Text: digits 8 and 9 are not allowed in octal numbers.} -- {Annot: '', ID: 31, LastEd: 0, Text: invalid hex number; at least one hex digit - expected.} -- {Annot: '', ID: 32, LastEd: 0, Text: invalid binary number; at least one binary - digit expected.} -- {Annot: '', ID: 33, LastEd: 0, Text: the exponent of a hexadecimal float number - is required.} -- {Annot: '', ID: 34, LastEd: 0, Text: hexadecimal float exponents must start with - a digit.} -- {Annot: '', ID: 35, LastEd: 0, Text: exponents must start with a digit.} -- {Annot: '', ID: 36, LastEd: 0, Text: 'expected ''{0}'', but found ''{1}''.'} -- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' is redundant.'} -- {Annot: '', ID: 38, LastEd: 0, Text: template tuple parameters can only be last.} -- {Annot: '', ID: 39, LastEd: 0, Text: the functions 'in' contract was already parsed.} -- {Annot: '', ID: 40, LastEd: 0, Text: the functions 'out' contract was already parsed.} -- {Annot: '', ID: 41, LastEd: 0, Text: no linkage type was specified.} -- {Annot: '', ID: 42, LastEd: 0, Text: 'unrecognized linkage type ''{0}''; valid types - are C, C++, D, Windows, Pascal und System.'} -- {Annot: '', ID: 43, LastEd: 0, Text: 'expected one or more base classes, not ''{0}''.'} -- {Annot: '', ID: 44, LastEd: 0, Text: base classes are not allowed in forward declarations.} -- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} - - Copyright (c) 2007 by Aziz Köksal. Licensed under the GPL3. - - - Subcommands: - - {1} - - Type ''dil help '' for more help on a particular subcommand. - - - Compiled with {2} v{3} on {4}.'} -- {Annot: '', ID: 46, LastEd: 0, Text: "Generate an XML or HTML document from a D\ - \ source file.\nUsage:\n dil gen file.d [Options]\n\nOptions:\n --syntax \ - \ : generate tags for the syntax tree\n --xml : use XML format\ - \ (default)\n --html : use HTML format\n\nExample:\n dil gen Parser.d\ - \ --html --syntax > Parser.html"} -- {Annot: '', ID: 47, LastEd: 0, Text: "Parse a module and extract information from\ - \ the resulting module dependency graph.\nUsage:\n dil igraph file.d Format [Options]\n\ - \n The directory of file.d is implicitly added to the list of import paths.\n\ - \nFormat:\n --dot : generate a dot document\n Further options for\ - \ --dot:\n -gbp : Group modules by package names\n -gbf \ - \ : Group modules by full package name\n -hle : highlight cyclic\ - \ edges in the graph\n -hlv : highlight modules in cyclic relationship\n\ - \n --paths : print a list of paths to the modules imported by file.d\n\ - \ --list : print a list of the module names imported by file.d\n Options\ - \ common to --paths and --list:\n -lN : print N levels.\n -m \ - \ : mark modules in cyclic relationships with a star.\n\nOptions:\n\ - \ -Ipath : add 'path' to the list of import paths where modules are\n\ - \ looked for\n -rREGEXP : exclude modules whose names\ - \ match the regular expression\n REGEXP\n -i \ - \ : include unlocatable modules\n\nExample:\n dil igraph src/main.d"} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/i18n/fi.cat --- a/trunk/i18n/fi.cat Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,76 +0,0 @@ -Authors: -- {EMail: jmjm@iki.fi, Name: Jari-Matti Mäkelä} -LangCode: fi -License: GPL3 -Messages: -- {Annot: '', ID: 0, LastEd: 0, Text: ''} -- {Annot: '', ID: 1, LastEd: 0, Text: virheellinen Unicode-merkki.} -- {Annot: '', ID: 2, LastEd: 0, Text: virheellinen UTF-8-merkkijono.} -- {Annot: '', ID: 3, LastEd: 0, Text: päättämätön merkkiliteraali.} -- {Annot: '', ID: 4, LastEd: 0, Text: tyhjä merkkiliteraali.} -- {Annot: '', ID: 5, LastEd: 0, Text: 'odotettiin rivinumeroa ''#'':n jälkeen.'} -- {Annot: '', ID: 6, LastEd: 0, Text: 'odotettiin kokonaislukua #line:n jälkeen'} -- {Annot: '', ID: 7, LastEd: 0, Text: odotettiin tiedostomäärittelyn merkkijonoa (esim. - "polku\tiedostoon")} -- {Annot: '', ID: 8, LastEd: 0, Text: päättämätön tiedostomäärittely.} -- {Annot: '', ID: 9, LastEd: 0, Text: odotettiin päättävää rivinvaihtoa erikoismerkin - jälkeen.} -- {Annot: '', ID: 10, LastEd: 0, Text: päättämätön merkkijonoliteraali.} -- {Annot: '', ID: 11, LastEd: 0, Text: 'ei-heksamerkki ''{0}'' löytyi heksajonossa.'} -- {Annot: '', ID: 12, LastEd: 0, Text: pariton määrä heksanumeroita heksajonossa.} -- {Annot: '', ID: 13, LastEd: 0, Text: päättämätön heksajono.} -- {Annot: '', ID: 14, LastEd: 0, Text: päättämätön lohkokommentti (/* */).} -- {Annot: '', ID: 15, LastEd: 0, Text: päättämätön sisäkkäinen kommentti (/+ +/).} -- {Annot: '', ID: 16, LastEd: 0, Text: päättämätön raakamerkkijono.} -- {Annot: '', ID: 17, LastEd: 0, Text: päättämätön gravisaksenttimerkkijono.} -- {Annot: '', ID: 18, LastEd: 0, Text: löydettiin määrittelemätön escape-sekvenssi.} -- {Annot: '', ID: 19, LastEd: 0, Text: 'found invalid Unicode escape sequence ''{0}''.'} -- {Annot: '', ID: 20, LastEd: 0, Text: riittämätön määrä heksanumeroita escape-sekvenssissä.} -- {Annot: '', ID: 21, LastEd: 0, Text: 'määrittelemätön HTML-entiteetti ''{0}'''} -- {Annot: '', ID: 22, LastEd: 0, Text: päättämätön HTML-entiteetti.} -- {Annot: '', ID: 23, LastEd: 0, Text: HTML-entiteettien tulee alkaa kirjaimella.} -- {Annot: '', ID: 24, LastEd: 0, Text: desimaaliluku ylivuotaa etumerkin.} -- {Annot: '', ID: 25, LastEd: 0, Text: desimaaliluvun ylivuoto.} -- {Annot: '', ID: 26, LastEd: 0, Text: heksadesimaaliluvun ylivuoto.} -- {Annot: '', ID: 27, LastEd: 0, Text: binääriluvun ylivuoto.} -- {Annot: '', ID: 28, LastEd: 0, Text: oktaaliluvun ylivuoto.} -- {Annot: '', ID: 29, LastEd: 0, Text: liukuluvun ylivuoto.} -- {Annot: '', ID: 30, LastEd: 0, Text: numerot 8 ja 9 eivät ole sallittuja oktaaliluvuissa.} -- {Annot: '', ID: 31, LastEd: 0, Text: virheellinen heksaluku; odotettiin vähintään - yhtä heksanumeroa.} -- {Annot: '', ID: 32, LastEd: 0, Text: virheellinen binääriluku; odotettiin vähintään - yhtä binäärinumeroa.} -- {Annot: '', ID: 33, LastEd: 0, Text: heksadesimaalisen liukuluvun eksponentti vaaditaan.} -- {Annot: '', ID: 34, LastEd: 0, Text: heksadesimaalisen liukuluvun eksponentista - puuttui numeroita.} -- {Annot: '', ID: 35, LastEd: 0, Text: eksponenttien tulee alkaa numerolla.} -- {Annot: '', ID: 36, LastEd: 0, Text: 'odotettiin ''{0}'':a, mutta löydettiin ''{1}''.'} -- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' on redundantti.'} -- {Annot: '', ID: 38, LastEd: 0, Text: tupla voi esiintyä ainoastaan mallin viimeisenä - parametrina.} -- {Annot: '', ID: 39, LastEd: 0, Text: funktion alkuehto jäsennettiin jo.} -- {Annot: '', ID: 40, LastEd: 0, Text: funktion loppuehto jäsennettiin jo.} -- {Annot: '', ID: 41, LastEd: 0, Text: linkitystyyppiä ei määritelty.} -- {Annot: '', ID: 42, LastEd: 0, Text: 'tunnistamaton linkitystyyppi ''{0}''; sallittuja - tyyppejä ovat C, C++, D, Windows, Pascal ja System.'} -- {Annot: '', ID: 43, LastEd: 0, Text: 'expected one or more base classes, not ''{0}''.'} -- {Annot: '', ID: 44, LastEd: 0, Text: base classes are not allowed in forward declarations.} -- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} - - Copyright (c) 2007, Aziz Köksal. GPL3-lisensöity. - - - Alikomennot: - - {1} - - Lisäohjeita tietystä alitoiminnosta saa kirjoittamalla ''dil help ''. - - - Käännetty {2}:n versiolla {3} {4}.'} -- {Annot: '', ID: 46, LastEd: 0, Text: "Luo XML- tai HTML-dokumentti D-lähdekoodista.\n\ - Käyttö:\n dil gen tiedosto.d [Valinnat]\n\nValinnat:\n --syntax : luo\ - \ elementtejä syntaksipuun mukaisesti\n --xml : käytä XML-muotoa (oletus)\n\ - \ --html : käytä HTML-muotoa\n\nEsimerkki:\n dil gen Parser.d --html\ - \ --syntax > Parser.html"} -- {Annot: '', ID: 47, LastEd: 0, Text: ''} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/i18n/tr.cat --- a/trunk/i18n/tr.cat Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,73 +0,0 @@ -Authors: -- {EMail: aziz.koeksal@gmail.com, Name: Aziz Köksal} -LangCode: tr -License: GPL3 -Messages: -- {Annot: '', ID: 0, LastEd: 0, Text: 'illegal karakter bulundu: ''{0}'''} -- {Annot: '', ID: 1, LastEd: 0, Text: geçersiz Unikod karakteri.} -- {Annot: '', ID: 2, LastEd: 0, Text: geçersiz UTF-8 serisi.} -- {Annot: '', ID: 3, LastEd: 0, Text: kapanmamış karakter sabiti.} -- {Annot: '', ID: 4, LastEd: 0, Text: boş karakter sabiti.} -- {Annot: '', ID: 5, LastEd: 0, Text: '''#'' karakter''den sonra ''line'' beklendi.'} -- {Annot: '', ID: 6, LastEd: 0, Text: '''#line''''den sonra rakam beklendi.'} -- {Annot: '', ID: 7, LastEd: 0, Text: filespec dizgisi beklendi (e.g. "yol\dosya".)} -- {Annot: '', ID: 8, LastEd: 0, Text: kapanmamış filespec dizgisi.} -- {Annot: '', ID: 9, LastEd: 0, Text: özel belirtici'den (special token) sonra yeni - bir satır beklendi.} -- {Annot: '', ID: 10, LastEd: 0, Text: kapanmamış çift tırnak dizgisi.} -- {Annot: '', ID: 11, LastEd: 0, Text: 'heks sayı olmayan karakter ''{0}'' heks dizgisi - içinde bulundu.'} -- {Annot: '', ID: 12, LastEd: 0, Text: heks dizginin içindeki sayılar çifter çifter - olmalıdır.} -- {Annot: '', ID: 13, LastEd: 0, Text: kapanmamış heks dizgisi.} -- {Annot: '', ID: 14, LastEd: 0, Text: kapanmamış blok açıklaması (/* */).} -- {Annot: '', ID: 15, LastEd: 0, Text: kapanmamış iç içe koyulabilen açıklaması (/+ - +/).} -- {Annot: '', ID: 16, LastEd: 0, Text: kapanmamış çiğ dizgisi.} -- {Annot: '', ID: 17, LastEd: 0, Text: kapanmamış ters tırnak dizgisi.} -- {Annot: '', ID: 18, LastEd: 0, Text: 'tanımlanmamış çıkış serisi ''{0}'' bulundu.'} -- {Annot: '', ID: 19, LastEd: 0, Text: 'geçersiz Unikod çıkış serisi ''{0}'' bulundu.'} -- {Annot: '', ID: 20, LastEd: 0, Text: heksadesimal çıkış serisi sayıları yeterli - değil.} -- {Annot: '', ID: 21, LastEd: 0, Text: 'tanımlanmamış HTML varlık ''{0}'''} -- {Annot: '', ID: 22, LastEd: 0, Text: 'kapanmamış HTML varlık ''{0}''.'} -- {Annot: '', ID: 23, LastEd: 0, Text: HTML varlık bir harf ile başlamalı.} -- {Annot: '', ID: 24, LastEd: 0, Text: desimal rakamın bit işareti taşdı.} -- {Annot: '', ID: 25, LastEd: 0, Text: desimal rakam taşması.} -- {Annot: '', ID: 26, LastEd: 0, Text: heksadesimal rakam taşması.} -- {Annot: '', ID: 27, LastEd: 0, Text: binari rakam taşması.} -- {Annot: '', ID: 28, LastEd: 0, Text: oktal rakam taşması.} -- {Annot: '', ID: 29, LastEd: 0, Text: float rakam taşması.} -- {Annot: '', ID: 30, LastEd: 0, Text: 8 ve 9 sayılar oktal rakamlar'da geçersizdir.} -- {Annot: '', ID: 31, LastEd: 0, Text: geçersiz heks rakam; minimum bir heks sayı - gereklidir.} -- {Annot: '', ID: 32, LastEd: 0, Text: geçersiz binari rakam; minimum bir binari sayı - gereklidir.} -- {Annot: '', ID: 33, LastEd: 0, Text: bir heksadesimal float rakamın üsü gereklidir.} -- {Annot: '', ID: 34, LastEd: 0, Text: heksadesimal float üsler desimal sayı ile başlamalı.} -- {Annot: '', ID: 35, LastEd: 0, Text: üsler desimal sayı ile başlamalı.} -- {Annot: '', ID: 36, LastEd: 0, Text: '''{0}'' beklendi, ama ''{1}'' bulundu.'} -- {Annot: '', ID: 37, LastEd: 0, Text: '''{0}'' lüzumsuz.'} -- {Annot: '', ID: 38, LastEd: 0, Text: şablon tuple parametre son sırada olmalı.} -- {Annot: '', ID: 39, LastEd: 0, Text: fonksiyonun 'in' kontratı daha önceden ayrıştırılmış.} -- {Annot: '', ID: 40, LastEd: 0, Text: fonksiyonun 'out' kontratı daha önceden ayrıştırılmış.} -- {Annot: '', ID: 41, LastEd: 0, Text: bağlantı tüp (linkage type) belirtilmedi.} -- {Annot: '', ID: 42, LastEd: 0, Text: 'bilinmeyen bağlantı tüpü (linkage type) ''{0}''; - geçerli olanlar C, C++, D, Windows, Pascal ve System.'} -- {Annot: '', ID: 43, LastEd: 0, Text: ''} -- {Annot: '', ID: 44, LastEd: 0, Text: ''} -- {Annot: '', ID: 45, LastEd: 0, Text: 'dil v{0} - - Copyright (c) 2007, Aziz Köksal. Lisans GPL3. - - - Komutlar: {1} Belirli komut''a yardım edinmek için ''dil help '' yazınız. - - - Bu yazılım {2} v{3} ile {4} tarihinde derletilmiş.'} -- {Annot: '', ID: 46, LastEd: 0, Text: "Bir D kaynak kodundan XML veya HTML dosyası\ - \ oluştur.\nKullanım:\n dil gen dosya.d [Seçenekler]\n\nSeçenekler:\n --syntax\ - \ : söz dizimi için etiketler yazdır\n --xml : XML biçimi\ - \ kullan (varsayılır)\n --html : HTML biçimi kullan\n\nÖrnek:\n dil\ - \ gen Parser.d --html --syntax > Parser.html"} -- {Annot: '', ID: 47, LastEd: 0, Text: ''} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/release.py --- a/trunk/release.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,8 +0,0 @@ -# -*- coding: utf-8 -*- -# Author: Aziz Köksal - -# TODO: port release.sh to Python - -# TODO: write subcommand that creates a Makefile -def writeMakefile(): - pass diff -r a3fab8b74a7d -r bcb74c9b895c trunk/release.sh --- a/trunk/release.sh Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,96 +0,0 @@ -#!/bin/bash - -if [[ $1 != [0-9].[0-9][0-9][0-9] ]]; then - echo Wrong version format. Expected: d.ddd - exit; -fi - -BUILD="./build" -DIR="dil.$1" -DEST="$BUILD/$DIR" -FRESH_REPOS="$BUILD/fresh_repos" - -# Create build directory if it doesn't exist. -[ ! -e $BUILD ] && mkdir $BUILD - -# Convert Unix newlines to Windows newlines -# function unix2win -# { -# sed {s/$/\\r/} $* -# } - -# We need dil to get a list of all modules to be compiled. -if [ ! -s ./dil ]; then - dsss build -full &> /dev/null -fi - -if [ ! -s ./dil ]; then - echo "Couldn't build DIL. Can't get list of modules to be built." - exit; -fi - -# Used by doc generation and winbuild function. -SRC_FILES=`./dil igraph src/main.d --paths` - -# Recreate destination directory. -rm -rf $DEST -mkdir -p $DEST/{bin,doc/htmlsrc,src} - -# Create documentation. -./dil ddoc $DEST/doc/ -v src/macros_dil.ddoc -version=DDoc src/config.d $SRC_FILES -# Generate syntax highlighted HTML files. -HTMLSRC="$DEST/doc/htmlsrc" -for filepath in $SRC_FILES; -do - htmlfile=`echo $filepath | sed -e 's@^src/@@' -e 's@/@.@g' -e 's@.d$@@'`.html - echo "FILE: $filepath > $HTMLSRC/$htmlfile"; - ./dil gen --lines --syntax --html $filepath > "$HTMLSRC/$htmlfile"; -done - -# Linux Debug -echo "***** Building Linux binaries *****" -dsss build -clean -full -version=D2 -cp dil $DEST/bin/dil2_d -dsss build -clean -full -cp dil $DEST/bin/dil_d -# Linux Release -dsss build -clean -full -release -O -inline -version=D2 -cp dil $DEST/bin/dil2 -dsss build -clean -full -release -O -inline -cp dil $DEST/bin/dil - -if [ -s ~/bin/dmd.exe ]; then - echo "***** Building Windows binaries *****" - function winbuild - { # obj dir is winobj. -op = don't strip paths from obj files. - wine ~/bin/dmd.exe -odwinobj -op -ofdil $* $SRC_FILES - } - # Windows Debug - winbuild -version=D2 - cp dil.exe $DEST/bin/dil2_d.exe - winbuild - cp dil.exe $DEST/bin/dil_d.exe - # Windows Release - winbuild -release -O -inline -version=D2 - cp dil.exe $DEST/bin/dil2.exe - winbuild -release -O -inline - cp dil.exe $DEST/bin/dil.exe -fi - -# Copy source and other files. -rm -rf $FRESH_REPOS -hg archive -r tip -t files $FRESH_REPOS -cp -r $FRESH_REPOS/trunk/* $DEST - -cp $FRESH_REPOS/trunk/src/config.d $DEST/bin/ -cp $FRESH_REPOS/trunk/src/lang_*.d $DEST/bin/ -cp $FRESH_REPOS/trunk/src/*_map.d $DEST/bin/ -cp $FRESH_REPOS/trunk/src/*.css $DEST/bin/ -cp $FRESH_REPOS/trunk/src/predefined.ddoc $DEST/bin/ -cp $FRESH_REPOS/trunk/src/html.css $HTMLSRC - -# Build archives -# tar.gz doesn't compress well -tar --owner root --group root -czf $DEST.tar.gz $DEST -tar --owner root --group root --bzip2 -cf $DEST.tar.bz2 $DEST -zip -q -9 -r $DEST.zip $DEST diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/Settings.d --- a/trunk/src/Settings.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,29 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module Settings; -import common; - -/// Global application settings. -struct GlobalSettings -{ -static: - /// Predefined version identifiers. - string[] versionIds; - /// Path to the language file. - string langFile = "lang_en.d"; - /// Language code of loaded messages catalogue. - string langCode = "en"; - /// Table of localized compiler messages. - string[] messages; - /// Array of import paths to look for modules. - string[] importPaths; - /// Array of DDoc macro file paths. - string[] ddocFilePaths; - string xmlMapFile = "xml_map.d"; /// XML map file. - string htmlMapFile = "html_map.d"; /// HTML map file. - string lexerErrorFormat = "{0}({1},{2})L: {3}"; /// Lexer error. - string parserErrorFormat = "{0}({1},{2})P: {3}"; /// Parser error. - string semanticErrorFormat = "{0}({1},{2})S: {3}"; /// Semantic error. -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/SettingsLoader.d --- a/trunk/src/SettingsLoader.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,252 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module SettingsLoader; - -import Settings; -import dil.Messages; -import dil.ast.Node, dil.ast.Declarations, dil.ast.Expressions; -import dil.semantic.Module; -import dil.semantic.Pass1; -import dil.semantic.Symbol; -import dil.semantic.Symbols; -import dil.Information; -import dil.Compilation; -import common; - -import tango.io.FilePath; - -/// Loads settings from a D module file. -class SettingsLoader -{ - InfoManager infoMan; /// Collects error messages. - Module mod; /// Current module. - - this(InfoManager infoMan) - { - this.infoMan = infoMan; - } - - static SettingsLoader opCall(InfoManager infoMan) - { - return new SettingsLoader(infoMan); - } - - /// Creates an error report. - /// Params: - /// token = where the error occurred. - /// formatMsg = error message. - void error(Token* token, char[] formatMsg, ...) - { - auto location = token.getErrorLocation(); - auto msg = Format(_arguments, _argptr, formatMsg); - infoMan ~= new SemanticError(location, msg); - } - - T getValue(T)(char[] name) - { - auto var = mod.lookup(name); - if (!var) // Returning T.init instead of null, because dmd gives an error. - return error(mod.firstToken, "variable '{}' is not defined", name), T.init; - auto t = var.node.begin; - if (!var.isVariable) - return error(t, "'{}' is not a variable declaration", name), T.init; - auto value = var.to!(Variable).value; - if (!value) - return error(t, "'{}' variable has no value set", name), T.init; - T val = value.Is!(T); // Try casting to T. - if (!val) - error(value.begin, "the value of '{}' is not of type {}", name, typeof(T).stringof); - return val; - } - - T castTo(T)(Node n) - { - char[] type; - is(T == StringExpression) && (type = "char[]"); - if (!n.Is!(T)) - error(n.begin, "expression is not of type {}", type); - return n.Is!(T); - } - - void load() - { - scope execPath = new FilePath(GetExecutableFilePath()); - execPath = new FilePath(execPath.folder()); - - // Load config.d - auto filePath = resolvePath(execPath, "config.d"); - mod = new Module(filePath, infoMan); - mod.parse(); - - if (mod.hasErrors) - return; - - auto context = new CompilationContext; - auto pass1 = new SemanticPass1(mod, context); - pass1.start(); - - if (auto array = getValue!(ArrayInitExpression)("version_ids")) - foreach (value; array.values) - if (auto str = castTo!(StringExpression)(value)) - GlobalSettings.versionIds ~= str.getString(); - if (auto val = getValue!(StringExpression)("langfile")) - GlobalSettings.langFile = val.getString(); - if (auto array = getValue!(ArrayInitExpression)("import_paths")) - foreach (value; array.values) - if (auto str = castTo!(StringExpression)(value)) - GlobalSettings.importPaths ~= str.getString(); - if (auto array = getValue!(ArrayInitExpression)("ddoc_files")) - foreach (value; array.values) - if (auto str = castTo!(StringExpression)(value)) - GlobalSettings.ddocFilePaths ~= resolvePath(execPath, str.getString()); - if (auto val = getValue!(StringExpression)("xml_map")) - GlobalSettings.xmlMapFile = val.getString(); - if (auto val = getValue!(StringExpression)("html_map")) - GlobalSettings.htmlMapFile = val.getString(); - if (auto val = getValue!(StringExpression)("lexer_error")) - GlobalSettings.lexerErrorFormat = val.getString(); - if (auto val = getValue!(StringExpression)("parser_error")) - GlobalSettings.parserErrorFormat = val.getString(); - if (auto val = getValue!(StringExpression)("semantic_error")) - GlobalSettings.semanticErrorFormat = val.getString(); - - // Load language file. - filePath = resolvePath(execPath, GlobalSettings.langFile); - mod = new Module(filePath); - mod.parse(); - - if (mod.hasErrors) - return; - - pass1 = new SemanticPass1(mod, context); - pass1.start(); - - if (auto array = getValue!(ArrayInitExpression)("messages")) - { - char[][] messages; - foreach (value; array.values) - if (auto str = castTo!(StringExpression)(value)) - messages ~= str.getString(); - if (messages.length != MID.max+1) - error(mod.firstToken, - "messages table in {} must exactly have {} entries, but not {}.", - filePath, MID.max+1, messages.length); - GlobalSettings.messages = messages; - dil.Messages.SetMessages(messages); - } - if (auto val = getValue!(StringExpression)("lang_code")) - GlobalSettings.langCode = val.getString(); - } -} - -/// Loads an associative array from a D module file. -class TagMapLoader : SettingsLoader -{ - this(InfoManager infoMan) - { - super(infoMan); - } - - static TagMapLoader opCall(InfoManager infoMan) - { - return new TagMapLoader(infoMan); - } - - string[string] load(string filePath) - { - mod = new Module(filePath, infoMan); - mod.parse(); - if (mod.hasErrors) - return null; - - auto context = new CompilationContext; - auto pass1 = new SemanticPass1(mod, context); - pass1.start(); - - string[string] map; - if (auto array = getValue!(ArrayInitExpression)("map")) - foreach (i, value; array.values) - { - auto key = array.keys[i]; - if (auto valExp = castTo!(StringExpression)(value)) - if (!key) - error(value.begin, "expected key : value"); - else if (auto keyExp = castTo!(StringExpression)(key)) - map[keyExp.getString()] = valExp.getString(); - } - return map; - } -} - -/// Resolves the path to a file from the executable's dir path -/// if it is relative. -/// Returns: filePath if it is absolute or execPath + filePath. -string resolvePath(FilePath execPath, string filePath) -{ - if ((new FilePath(filePath)).isAbsolute()) - return filePath; - return execPath.dup.append(filePath).toString(); -} - -version(DDoc) -{ - /// Returns the fully qualified path to this executable. - char[] GetExecutableFilePath(); -} -else version(Windows) -{ -private extern(Windows) uint GetModuleFileNameA(void*, char*, uint); - -char[] GetExecutableFilePath() -{ - alias GetModuleFileNameA GetModuleFileName; - char[] buffer = new char[256]; - uint count; - - while (1) - { - if (buffer is null) - return null; - - count = GetModuleFileName(null, buffer.ptr, buffer.length); - if (count == 0) - return null; - if (buffer.length != count && buffer[count] == 0) - break; - // Increase size of buffer - buffer.length = buffer.length * 2; - } - assert(buffer[count] == 0); - // Reduce buffer to the actual length of the string (excluding '\0'.) - if (count < buffer.length) - buffer.length = count; - return buffer; -} -} -else version(linux) -{ -private extern(C) size_t readlink(char* path, char* buf, size_t bufsize); - -char[] GetExecutableFilePath() -{ - char[] buffer = new char[256]; - size_t count; - - while (1) - { - // This won't work on very old Linux systems. - count = readlink("/proc/self/exe".ptr, buffer.ptr, buffer.length); - if (count == -1) - return null; - if (count < buffer.length) - break; - buffer.length = buffer.length * 2; - } - buffer.length = count; - return buffer; -} -} -else - static assert(0, "GetExecutableFilePath() is not implemented on this platform."); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/TypeRules.d --- a/trunk/src/TypeRules.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,129 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module TypeRules; - -import cmd.Generate : xml_escape; - -import TypeRulesData; -import common; - -static const string[] basicTypes = [ - "char"[], "wchar", "dchar", "bool", - "byte", "ubyte", "short", "ushort", - "int", "uint", "long", "ulong", - /+"cent", "ucent",+/ - "float", "double", "real", - "ifloat", "idouble", "ireal", - "cfloat", "cdouble", "creal"/+, "void"+/ -]; - -static const string[] unaryExpressions = [ - "!x", - "&x", - "~x", - "+x", - "-x", - "++x", - "--x", - "x++", - "x--", -]; - -static const string[] binaryExpressions = [ - "x!<>=y", - "x!<>y", - "x!<=y", - "x!=y", - "x!>y", - "x<>=y", - "x<>y", - - "x=y", "x==y", "x!=y", - "x<=y", "x=y", "x>y", - "x<<=y", "x<>=y","x>>y", - "x>>>=y", "x>>>y", - "x|=y", "x||y", "x|y", - "x&=y", "x&&y", "x&y", - "x+=y", "x+y", - "x-=y", "x-y", - "x/=y", "x/y", - "x*=y", "x*y", - "x%=y", "x%y", - "x^=y", "x^y", - "x~=y", - "x~y", - "x,y" -]; - -void genHTMLTypeRulesTables() -{ - Stdout( - ``\n - ``\n - ``\n - ` `\n - ` `\n - ` `\n - ``\n - ``\n - `

    These tables show what the type results of certain expressions are.

    `\n - ); - - Stdout.format("\n\n", unaryExpressions.length); - Stdout(""); - foreach (unaryExpression; unaryExpressions) - Stdout.format("", { - if (unaryExpression[0] == 'x') - return `x` ~ xml_escape(unaryExpression[1..$]); - else - return xml_escape(unaryExpression[0..$-1]) ~ `x`; - }()); - Stdout("\n"); - foreach (i, basicType; basicTypes) - { - Stdout.format("\n"``, basicType); - foreach (expResults; unaryExpsResults) - { - auto result = expResults[i]; - Stdout.format(``, result[0] == 'E' ? `Error`[] : result); - } - Stdout("\n\n"); - } - Stdout("
    Unary Expressions
    {}
    {}{}
    \n"); - - foreach (i, expResults; binaryExpsResults) - { - auto binaryExpression = binaryExpressions[i]; - binaryExpression = `x ` ~ - xml_escape(binaryExpression[1..$-1]) ~ - ` y`; - Stdout.format("\n\n", basicTypes.length, binaryExpression); - Stdout.format(""); - foreach (basicType; basicTypes) - Stdout.format(``, basicType); - Stdout("\n\n"); - foreach (j, results; expResults) - { - Stdout.format("\n"``, basicTypes[j]); - foreach (result; results) - Stdout.format(``, result[0] == 'E' ? `Error`[] : result); - Stdout("\n\n"); - } - Stdout("
    {}
    {}
    {}{}
    \n"); - } - - Stdout( - "\n" - "\n" - ); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/TypeRulesData.d --- a/trunk/src/TypeRulesData.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -// Run TypeRulesGenerator.d diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/TypeRulesGenerator.d --- a/trunk/src/TypeRulesGenerator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,174 +0,0 @@ -#! /usr/bin/rdmd -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module TypeRulesGenerator; - -import tango.io.File; -import tango.io.FilePath; - -alias char[] string; - -void main(char[][] args) -{ - char[] text = "// Generated by TypeRulesGenerator.d\n" - "module TypeRules.d;\n\n"; - text ~= "char[][][] unaryExpsResults = [\n"; - foreach (results; unaryExpsResults) - { - text ~= " ["; - foreach (result; results) - text ~= '"' ~ result ~ '"' ~ ", "; - text[$-2] = ']'; - text[$-1] = ','; - text ~= \n; - } - text[$-2] = '\n'; - text[$-1] = ']'; - text ~= ";\n\n"; - - text ~= "char[][][][] binaryExpsResults = [\n"; - foreach (expResults; binaryExpsResults) - { - text ~= " [\n"; - foreach (results; expResults) - { - text ~= " ["; - foreach (result; results) - text ~= '"' ~ result ~ '"' ~ ", "; - text[$-2] = ']'; - text[$-1] = ','; - text ~= \n; - } - text[$-2] = '\n'; - text[$-1] = ' '; - text ~= " ],\n"; - } - text[$-2] = '\n'; - text[$-1] = ']'; - text ~= ";\n"; - - // Write the text to a D module. - auto file = new File("TypeRulesData.d"); - file.write(text); -} - -template ExpressionType(char[] T1, char[] T2, char[] expression) -{ - mixin("alias "~T1~" X;"); - mixin("alias "~T2~" Y;"); - X x; - Y y; - static if(is(typeof(mixin(expression)) ResultType)) - const char[] result = ResultType.stringof; - else - const char[] result = "Error"; -} -alias ExpressionType EType; - -// pragma(msg, EType!("char", "int", "&x").result); - -static const string[] basicTypes = [ - "char"[], "wchar", "dchar", "bool", - "byte", "ubyte", "short", "ushort", - "int", "uint", "long", "ulong", - /+"cent", "ucent",+/ - "float", "double", "real", - "ifloat", "idouble", "ireal", - "cfloat", "cdouble", "creal"/+, "void"+/ -]; - -static const string[] unaryExpressions = [ - "!x", - "&x", - "~x", - "+x", - "-x", - "++x", - "--x", - "x++", - "x--", -]; - -static const string[] binaryExpressions = [ - "x!<>=y", - "x!<>y", - "x!<=y", - "x!=y", - "x!>y", - "x<>=y", - "x<>y", - - "x=y", "x==y", "x!=y", - "x<=y", "x=y", "x>y", - "x<<=y", "x<>=y","x>>y", - "x>>>=y", "x>>>y", - "x|=y", "x||y", "x|y", - "x&=y", "x&&y", "x&y", - "x+=y", "x+y", - "x-=y", "x-y", - "x/=y", "x/y", - "x*=y", "x*y", - "x%=y", "x%y", - "x^=y", "x^y", - "x~=y", - "x~y", - "x,y" -]; - -char[] genBinaryExpArray(char[] expression) -{ - char[] result = "[\n"; - foreach (t1; basicTypes) - { - result ~= "[\n"; - foreach (t2; basicTypes) - result ~= `EType!("`~t1~`", "`~t2~`", "`~expression~`").result,`\n; - result[result.length-2] = ']'; // Overwrite last comma. - result[result.length-1] = ','; // Overwrite last \n. - } - result[result.length-1] = ']'; // Overwrite last comma. - return result; -} -// pragma(msg, mixin(genBinaryExpArray("x%y")).stringof); - -char[] genBinaryExpsArray() -{ - char[] result = "[\n"; - foreach (expression; binaryExpressions) - { - result ~= genBinaryExpArray(expression); - result ~= ",\n"; - } - result[result.length-2] = ']'; - return result; -} - -// pragma(msg, mixin(genBinaryExpsArray()).stringof); - -char[] genUnaryExpArray(char[] expression) -{ - char[] result = "[\n"; - foreach (t1; basicTypes) - result ~= `EType!("`~t1~`", "int", "`~expression~`").result,`\n; - result[result.length-2] = ']'; // Overwrite last comma. - return result; -} - -char[] genUnaryExpsArray() -{ - char[] result = "[\n"; - foreach (expression; unaryExpressions) - result ~= genUnaryExpArray(expression) ~ ",\n"; - result[result.length-2] = ']'; - return result; -} - -// pragma(msg, mixin(genUnaryExpsArray()).stringof); - -auto unaryExpsResults = mixin(genUnaryExpsArray()); -auto binaryExpsResults = mixin(genBinaryExpsArray()); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/cmd/ASTStats.d --- a/trunk/src/cmd/ASTStats.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,33 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module cmd.ASTStats; - -import dil.ast.DefaultVisitor; -import dil.ast.Node, - dil.ast.Declaration, - dil.ast.Statement, - dil.ast.Expression, - dil.ast.Types; - -/// Counts the nodes in a syntax tree. -class ASTStats : DefaultVisitor -{ - uint[] table; /// Table for counting nodes. - - /// Starts counting. - uint[] count(Node root) - { - table = new uint[g_classNames.length]; - super.visitN(root); - return table; - } - - // Override dispatch function. - override Node dispatch(Node n) - { - table[n.kind]++; - return super.dispatch(n); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/cmd/DDoc.d --- a/trunk/src/cmd/DDoc.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,849 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module cmd.DDoc; - -import cmd.DDocXML; -import cmd.Generate; -import dil.doc.Parser; -import dil.doc.Macro; -import dil.doc.Doc; -import dil.ast.Node; -import dil.ast.Declarations, - dil.ast.Statements, - dil.ast.Expression, - dil.ast.Parameters, - dil.ast.Types; -import dil.ast.DefaultVisitor; -import dil.lexer.Token; -import dil.lexer.Funcs; -import dil.semantic.Module; -import dil.semantic.Pass1; -import dil.semantic.Symbol; -import dil.semantic.Symbols; -import dil.Compilation; -import dil.Information; -import dil.Converter; -import dil.SourceText; -import dil.Enums; -import dil.Time; -import common; - -import tango.text.Ascii : toUpper; -import tango.io.File; -import tango.io.FilePath; - -/// Executes the doc generation command. -void execute(string[] filePaths, string destDir, string[] macroPaths, - bool writeXML, bool incUndoc, bool verbose, - CompilationContext context, InfoManager infoMan) -{ - // Parse macro files. - MacroTable mtable; - MacroParser mparser; - foreach (macroPath; macroPaths) - { - auto macros = mparser.parse(loadMacroFile(macroPath, infoMan)); - mtable = new MacroTable(mtable); - mtable.insert(macros); - } - -// foreach (k, v; mtable.table) -// Stdout(k)("=")(v.text); - - // For DDoc code sections. - auto tokenHL = new TokenHighlighter(infoMan, writeXML == false); - - // Process D files. - foreach (filePath; filePaths) - { - auto mod = new Module(filePath, infoMan); - // Parse the file. - mod.parse(); - if (mod.hasErrors) - continue; - - // Start semantic analysis. - auto pass1 = new SemanticPass1(mod, context); - pass1.start(); - - // Generate documentation. - auto dest = new FilePath(destDir); - dest.append(mod.getFQN() ~ (writeXML ? ".xml" : ".html")); - - InfoManager infoMan2; // Collects warnings from the macro expander. - if (verbose) - { - Stdout.formatln("{} > {}", mod.filePath, dest); - infoMan2 = new InfoManager(); - } - - writeDocFile(dest.toString(), mod, mtable, writeXML, incUndoc, tokenHL, infoMan2); - - if (infoMan2) - infoMan ~= infoMan2.info; - } -} - -void writeDocFile(string dest, Module mod, MacroTable mtable, - bool writeXML, bool incUndoc, - TokenHighlighter tokenHL, InfoManager infoMan) -{ - // Create a macro environment for this module. - mtable = new MacroTable(mtable); - // Define runtime macros. - // MODPATH is not in the specs. - mtable.insert("MODPATH", mod.getFQNPath() ~ "." ~ mod.fileExtension()); - mtable.insert("TITLE", mod.getFQN()); - mtable.insert("DOCFILENAME", mod.getFQN() ~ (writeXML ? ".xml" : ".html")); - auto timeStr = Time.toString(); - mtable.insert("DATETIME", timeStr); - mtable.insert("YEAR", Time.year(timeStr)); - - DDocEmitter docEmitter; - if (writeXML) - docEmitter = new DDocXMLEmitter(mod, mtable, incUndoc, tokenHL); - else - docEmitter = new DDocEmitter(mod, mtable, incUndoc, tokenHL); - docEmitter.emit(); - // Set BODY macro to the text produced by the DDocEmitter. - mtable.insert("BODY", docEmitter.text); - // Do the macro expansion pass. - auto fileText = MacroExpander.expand(mtable, "$(DDOC)", mod.filePath, infoMan); -// fileText ~= "\n
    \n" ~ doc.text ~ "\n
    "; - // Finally write the file out to the harddisk. - auto file = new File(dest); - file.write(fileText); -} - -/// Loads a macro file. Converts any Unicode encoding to UTF-8. -string loadMacroFile(string filePath, InfoManager infoMan) -{ - auto src = new SourceText(filePath); - src.load(infoMan); - auto text = src.data[0..$-1]; // Exclude '\0'. - return sanitizeText(text); -} - -/// Traverses the syntax tree and writes DDoc macros to a string buffer. -class DDocEmitter : DefaultVisitor -{ - char[] text; /// The buffer that is written to. - bool includeUndocumented; - MacroTable mtable; - Module modul; - TokenHighlighter tokenHL; - - /// Constructs a DDocEmitter object. - /// Params: - /// modul = the module to generate text for. - /// mtable = the macro table. - /// includeUndocumented = whether to include undocumented symbols. - /// tokenHL = used to highlight code sections. - this(Module modul, MacroTable mtable, bool includeUndocumented, - TokenHighlighter tokenHL) - { - this.mtable = mtable; - this.includeUndocumented = includeUndocumented; - this.modul = modul; - this.tokenHL = tokenHL; - } - - /// Entry method. - char[] emit() - { - if (auto d = modul.moduleDecl) - { - if (ddoc(d)) - { - if (auto copyright = cmnt.takeCopyright()) - mtable.insert(new Macro("COPYRIGHT", copyright.text)); - writeComment(); - } - } - MEMBERS("MODULE", { visitD(modul.root); }); - return text; - } - - char[] textSpan(Token* left, Token* right) - { - //assert(left && right && (left.end <= right.start || left is right)); - //char[] result; - //TODO: filter out whitespace tokens. - return Token.textSpan(left, right); - } - - TemplateParameters tparams; /// The template parameters of the current declaration. - - DDocComment cmnt; /// Current comment. - DDocComment prevCmnt; /// Previous comment in scope. - /// An empty comment. Used for undocumented symbols. - static const DDocComment emptyCmnt; - - /// Initializes the empty comment. - static this() - { - this.emptyCmnt = new DDocComment(null, null, null); - } - - /// Keeps track of previous comments in each scope. - scope class Scope - { - DDocComment saved_prevCmnt; - bool saved_cmntIsDitto; - uint saved_prevDeclOffset; - this() - { // Save the previous comment of the parent scope. - saved_prevCmnt = this.outer.prevCmnt; - saved_cmntIsDitto = this.outer.cmntIsDitto; - saved_prevDeclOffset = this.outer.prevDeclOffset; - // Entering a new scope. Clear variables. - this.outer.prevCmnt = null; - this.outer.cmntIsDitto = false; - this.outer.prevDeclOffset = 0; - } - - ~this() - { // Restore the previous comment of the parent scope. - this.outer.prevCmnt = saved_prevCmnt; - this.outer.cmntIsDitto = saved_cmntIsDitto; - this.outer.prevDeclOffset = saved_prevDeclOffset; - } - } - - bool cmntIsDitto; /// True if current comment is "ditto". - - /// Returns the DDocComment for node. - DDocComment ddoc(Node node) - { - auto c = getDDocComment(node); - this.cmnt = null; - if (c) - { - if (c.isDitto) - { - this.cmnt = this.prevCmnt; - this.cmntIsDitto = true; - } - else - { - this.cmntIsDitto = false; - this.cmnt = c; - this.prevCmnt = c; - } - } - else if (includeUndocumented) - this.cmnt = this.emptyCmnt; - return this.cmnt; - } - - /// List of predefined, special sections. - static char[][char[]] specialSections; - static this() - { - foreach (name; ["AUTHORS", "BUGS", "COPYRIGHT", "DATE", "DEPRECATED", - "EXAMPLES", "HISTORY", "LICENSE", "RETURNS", "SEE_ALSO", - "STANDARDS", "THROWS", "VERSION"]) - specialSections[name] = name; - } - - /// Writes the DDoc comment to the text buffer. - void writeComment() - { - auto c = this.cmnt; - assert(c !is null); - if (c.sections.length == 0) - return; - write("$(DDOC_SECTIONS "); - foreach (s; c.sections) - { - if (s is c.summary) - write("\n$(DDOC_SUMMARY "); - else if (s is c.description) - write("\n$(DDOC_DESCRIPTION "); - else if (auto name = toUpper(s.name.dup) in specialSections) - write("\n$(DDOC_" ~ *name ~ " "); - else if (s.Is("params")) - { // Process parameters section. - auto ps = new ParamsSection(s.name, s.text); - write("\n$(DDOC_PARAMS "); - foreach (i, paramName; ps.paramNames) - write("\n$(DDOC_PARAM_ROW ", - "$(DDOC_PARAM_ID $(DDOC_PARAM ", paramName, "))", - "$(DDOC_PARAM_DESC ", ps.paramDescs[i], ")", - ")"); - write(")"); - continue; - } - else if (s.Is("macros")) - { // Declare the macros in this section. - auto ms = new MacrosSection(s.name, s.text); - mtable.insert(ms.macroNames, ms.macroTexts); - continue; - } - else - write("\n$(DDOC_SECTION $(DDOC_SECTION_H " ~ s.name ~ ":)"); - write(scanCommentText(s.text), ")"); - } - write(")"); - } - - /// Scans the comment text and: - /// $(UL - /// $(LI skips and leaves macro invocations unchanged) - /// $(LI skips HTML tags) - /// $(LI escapes '(', ')', '<', '>' and '&') - /// $(LI inserts $(DDOC_BLANKLINE) in place of \n\n) - /// $(LI highlights code in code sections) - /// ) - char[] scanCommentText(char[] text) - { - char* p = text.ptr; - char* end = p + text.length; - char[] result = new char[text.length]; // Reserve space. - result.length = 0; - - while (p < end) - { - switch (*p) - { - case '$': - if (auto macroEnd = MacroParser.scanMacro(p, end)) - { - result ~= makeString(p, macroEnd); // Copy macro invocation as is. - p = macroEnd; - continue; - } - goto default; - case '<': - auto begin = p; - p++; - if (p+2 < end && *p == '!' && p[1] == '-' && p[2] == '-') // ". - while (++p < end) - if (p+2 < end && *p == '-' && p[1] == '-' && p[2] == '>') - { - p += 3; // Point one past '>'. - break; - } - result ~= makeString(begin, p); - } // or - else if (p < end && (isalpha(*p) || *p == '/')) - { - while (++p < end && *p != '>') // Skip to closing '>'. - {} - if (p == end) - { // No closing '>' found. - p = begin + 1; - result ~= "<"; - continue; - } - p++; // Skip '>'. - result ~= makeString(begin, p); - } - else - result ~= "<"; - continue; - case '(': result ~= "("; break; - case ')': result ~= ")"; break; - // case '\'': result ~= "'"; break; // ' - // case '"': result ~= """; break; - case '>': result ~= ">"; break; - case '&': - if (p+1 < end && (isalpha(p[1]) || p[1] == '#')) - goto default; - result ~= "&"; - break; - case '\n': - if (!(p+1 < end && p[1] == '\n')) - goto default; - ++p; - result ~= "$(DDOC_BLANKLINE)"; - break; - case '-': - if (p+2 < end && p[1] == '-' && p[2] == '-') - { - while (p < end && *p == '-') - p++; - auto codeBegin = p; - p--; - while (++p < end) - if (p+2 < end && *p == '-' && p[1] == '-' && p[2] == '-') - break; - auto codeText = makeString(codeBegin, p); - result ~= tokenHL.highlight(codeText, modul.filePath); - while (p < end && *p == '-') - p++; - continue; - } - //goto default; - default: - result ~= *p; - } - p++; - } - return result; - } - - /// Escapes '<', '>' and '&' with named HTML entities. - char[] escape(char[] text) - { - char[] result = new char[text.length]; // Reserve space. - result.length = 0; - foreach(c; text) - switch(c) - { - case '<': result ~= "<"; break; - case '>': result ~= ">"; break; - case '&': result ~= "&"; break; - default: result ~= c; - } - if (result.length != text.length) - return result; - // Nothing escaped. Return original text. - delete result; - return text; - } - - /// Writes an array of strings to the text buffer. - void write(char[][] strings...) - { - foreach (s; strings) - text ~= s; - } - - /// Writes params to the text buffer. - void writeParams(Parameters params) - { - if (!params.items.length) - return write("()"); - write("("); - auto lastParam = params.items[$-1]; - foreach (param; params.items) - { - if (param.isCVariadic) - write("..."); - else - { - assert(param.type); - // Write storage classes. - auto typeBegin = param.type.baseType.begin; - if (typeBegin !is param.begin) // Write storage classes. - write(textSpan(param.begin, typeBegin.prevNWS), " "); - write(escape(textSpan(typeBegin, param.type.end))); // Write type. - if (param.name) - write(" $(DDOC_PARAM ", param.name.str, ")"); - if (param.isDVariadic) - write("..."); - if (param.defValue) - write(" = ", escape(textSpan(param.defValue.begin, param.defValue.end))); - } - if (param !is lastParam) - write(", "); - } - write(")"); - } - - /// Writes the current template parameters to the text buffer. - void writeTemplateParams() - { - if (!tparams) - return; - write(escape(textSpan(tparams.begin, tparams.end))); - tparams = null; - } - - /// Writes bases to the text buffer. - void writeInheritanceList(BaseClassType[] bases) - { - if (bases.length == 0) - return; - auto basesBegin = bases[0].begin.prevNWS; - if (basesBegin.kind == TOK.Colon) - basesBegin = bases[0].begin; - write(" : ", escape(textSpan(basesBegin, bases[$-1].end))); - } - - /// Writes a symbol to the text buffer. E.g: $(SYMBOL Buffer, 123) - void SYMBOL(char[] name, Declaration d) - { - auto loc = d.begin.getRealLocation(); - auto str = Format("$(SYMBOL {}, {})", name, loc.lineNum); - write(str); - // write("$(DDOC_PSYMBOL ", name, ")"); - } - - /// Offset at which to insert a declaration which have a "ditto" comment. - uint prevDeclOffset; - - /// Writes a declaration to the text buffer. - void DECL(void delegate() dg, Declaration d, bool writeSemicolon = true) - { - if (cmntIsDitto) - { alias prevDeclOffset offs; - assert(offs != 0); - auto savedText = text; - text = ""; - write("\n$(DDOC_DECL "); - dg(); - writeSemicolon && write(";"); - writeAttributes(d); - write(")"); - // Insert text at offset. - auto len = text.length; - text = savedText[0..offs] ~ text ~ savedText[offs..$]; - offs += len; // Add length of the inserted text to the offset. - return; - } - write("\n$(DDOC_DECL "); - dg(); - writeSemicolon && write(";"); - writeAttributes(d); - write(")"); - prevDeclOffset = text.length; - } - - /// Wraps the DDOC_DECL_DD macro around the text written by dg(). - void DESC(void delegate() dg) - { - if (cmntIsDitto) - return; - write("\n$(DDOC_DECL_DD "); - dg(); - write(")"); - } - - /// Wraps the DDOC_kind_MEMBERS macro around the text written by dg(). - void MEMBERS(char[] kind, void delegate() dg) - { - write("\n$(DDOC_"~kind~"_MEMBERS "); - dg(); - write(")"); - } - - /// Writes a class or interface declaration. - void writeClassOrInterface(T)(T d) - { - if (!ddoc(d)) - return d; - DECL({ - write(d.begin.srcText, " "); - SYMBOL(d.name.str, d); - writeTemplateParams(); - writeInheritanceList(d.bases); - }, d); - DESC({ - writeComment(); - MEMBERS(is(T == ClassDeclaration) ? "CLASS" : "INTERFACE", { - scope s = new Scope(); - d.decls && super.visit(d.decls); - }); - }); - } - - // templated decls are not virtual so we need these: - - /// Writes a class declaration. - void writeClass(ClassDeclaration d) { - writeClassOrInterface(d); - } - - /// Writes an interface declaration. - void writeInterface(InterfaceDeclaration d) { - writeClassOrInterface(d); - } - - /// Writes a struct or union declaration. - void writeStructOrUnion(T)(T d) - { - if (!ddoc(d)) - return d; - DECL({ - write(d.begin.srcText, d.name ? " " : ""); - if (d.name) - SYMBOL(d.name.str, d); - writeTemplateParams(); - }, d); - DESC({ - writeComment(); - MEMBERS(is(T == StructDeclaration) ? "STRUCT" : "UNION", { - scope s = new Scope(); - d.decls && super.visit(d.decls); - }); - }); - } - - // templated decls are not virtual so we need these: - - /// Writes a struct declaration. - void writeStruct(StructDeclaration d) { - writeStructOrUnion(d); - } - - /// Writes an union declaration. - void writeUnion(UnionDeclaration d) { - writeStructOrUnion(d); - } - - /// Writes an alias or typedef declaration. - void writeAliasOrTypedef(T)(T d) - { - auto prefix = is(T == AliasDeclaration) ? "alias " : "typedef "; - if (auto vd = d.decl.Is!(VariablesDeclaration)) - { - auto type = textSpan(vd.typeNode.baseType.begin, vd.typeNode.end); - foreach (name; vd.names) - DECL({ write(prefix); write(escape(type), " "); SYMBOL(name.str, d); }, d); - } - else if (auto fd = d.decl.Is!(FunctionDeclaration)) - {} - // DECL({ write(textSpan(d.begin, d.end)); }, false); - DESC({ writeComment(); }); - } - - /// Writes the attributes of a declaration in brackets. - void writeAttributes(Declaration d) - { - char[][] attributes; - - if (d.prot != Protection.None) - attributes ~= "$(PROT " ~ .toString(d.prot) ~ ")"; - - auto stc = d.stc; - stc &= ~StorageClass.Auto; // Ignore auto. - foreach (stcStr; .toStrings(stc)) - attributes ~= "$(STC " ~ stcStr ~ ")"; - - LinkageType ltype; - if (auto vd = d.Is!(VariablesDeclaration)) - ltype = vd.linkageType; - else if (auto fd = d.Is!(FunctionDeclaration)) - ltype = fd.linkageType; - - if (ltype != LinkageType.None) - attributes ~= "$(LINKAGE extern(" ~ .toString(ltype) ~ "))"; - - if (!attributes.length) - return; - - write(" $(ATTRIBUTES "); - write(attributes[0]); - foreach (attribute; attributes[1..$]) - write(", ", attribute); - write(")"); - } - - alias Declaration D; - -override: - D visit(AliasDeclaration d) - { - if (!ddoc(d)) - return d; - writeAliasOrTypedef(d); - return d; - } - - D visit(TypedefDeclaration d) - { - if (!ddoc(d)) - return d; - writeAliasOrTypedef(d); - return d; - } - - D visit(EnumDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ - write("enum", d.name ? " " : ""); - d.name && SYMBOL(d.name.str, d); - }, d); - DESC({ - writeComment(); - MEMBERS("ENUM", { scope s = new Scope(); super.visit(d); }); - }); - return d; - } - - D visit(EnumMemberDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ SYMBOL(d.name.str, d); }, d, false); - DESC({ writeComment(); }); - return d; - } - - D visit(TemplateDeclaration d) - { - this.tparams = d.tparams; - if (d.begin.kind != TOK.Template) - { // This is a templatized class/interface/struct/union/function. - super.visit(d.decls); - this.tparams = null; - return d; - } - if (!ddoc(d)) - return d; - DECL({ - write("template "); - SYMBOL(d.name.str, d); - writeTemplateParams(); - }, d); - DESC({ - writeComment(); - MEMBERS("TEMPLATE", { - scope s = new Scope(); - super.visit(d.decls); - }); - }); - return d; - } - - D visit(ClassDeclaration d) - { - writeClass(d); - return d; - } - - D visit(InterfaceDeclaration d) - { - writeInterface(d); - return d; - } - - D visit(StructDeclaration d) - { - writeStruct(d); - return d; - } - - D visit(UnionDeclaration d) - { - writeUnion(d); - return d; - } - - D visit(ConstructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ SYMBOL("this", d); writeParams(d.params); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(StaticConstructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("static "); SYMBOL("this", d); write("()"); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(DestructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("~"); SYMBOL("this", d); write("()"); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(StaticDestructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("static ~"); SYMBOL("this", d); write("()"); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(FunctionDeclaration d) - { - if (!ddoc(d)) - return d; - auto type = textSpan(d.returnType.baseType.begin, d.returnType.end); - DECL({ - write(escape(type), " "); - SYMBOL(d.name.str, d); - writeTemplateParams(); - writeParams(d.params); - }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(NewDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ SYMBOL("new", d); writeParams(d.params); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(DeleteDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ SYMBOL("delete", d); writeParams(d.params); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(VariablesDeclaration d) - { - if (!ddoc(d)) - return d; - char[] type = "auto"; - if (d.typeNode) - type = textSpan(d.typeNode.baseType.begin, d.typeNode.end); - foreach (name; d.names) - DECL({ write(escape(type), " "); SYMBOL(name.str, d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(InvariantDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ SYMBOL("invariant", d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(UnittestDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ SYMBOL("unittest", d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(DebugDeclaration d) - { - d.compiledDecls && visitD(d.compiledDecls); - return d; - } - - D visit(VersionDeclaration d) - { - d.compiledDecls && visitD(d.compiledDecls); - return d; - } - - D visit(StaticIfDeclaration d) - { - d.ifDecls && visitD(d.ifDecls); - return d; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/cmd/DDocXML.d --- a/trunk/src/cmd/DDocXML.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,406 +0,0 @@ -/++ - Authors: Aziz Köksal & Jari-Matti Mäkelä - License: GPL3 -+/ -module cmd.DDocXML; - -import cmd.DDoc; -import cmd.Generate; -import dil.doc.Parser; -import dil.doc.Macro; -import dil.doc.Doc; -import dil.ast.Node; -import dil.ast.Declarations, - dil.ast.Statements, - dil.ast.Expression, - dil.ast.Parameters, - dil.ast.Types; -import dil.ast.DefaultVisitor; -import dil.lexer.Token; -import dil.lexer.Funcs; -import dil.semantic.Module; -import dil.semantic.Pass1; -import dil.semantic.Symbol; -import dil.semantic.Symbols; -import dil.Compilation; -import dil.Information; -import dil.Converter; -import dil.SourceText; -import dil.Enums; -import dil.Time; -import common; - -import tango.text.Ascii : toUpper; -import tango.io.File; -import tango.io.FilePath; - -/// Traverses the syntax tree and writes DDoc macros to a string buffer. -class DDocXMLEmitter : DDocEmitter -{ - this(Module modul, MacroTable mtable, bool includeUndocumented, - TokenHighlighter tokenHL) - { - super(modul, mtable, includeUndocumented, tokenHL); - } - - /// Writes params to the text buffer. - void writeParams(Parameters params) - { - if (!params.items.length) - return; - - write("$(PARAMS "); - auto lastParam = params.items[$-1]; - foreach (param; params.items) - { - if (param.isCVariadic) - write("..."); - else - { - assert(param.type); - // Write storage classes. - auto typeBegin = param.type.baseType.begin; - if (typeBegin !is param.begin) // Write storage classes. - write(textSpan(param.begin, typeBegin.prevNWS), " "); - write(escape(textSpan(typeBegin, param.type.end))); // Write type. - if (param.name) - write(" $(DDOC_PARAM ", param.name.str, ")"); - if (param.isDVariadic) - write("..."); - if (param.defValue) - write(" = ", escape(textSpan(param.defValue.begin, param.defValue.end))); - } - if (param !is lastParam) - write(", "); - } - write(")"); - } - - /// Writes the current template parameters to the text buffer. - void writeTemplateParams() - { - if (!tparams) - return; - write("$(TEMPLATE_PARAMS ", escape(textSpan(tparams.begin, tparams.end))[1..$-1], ")"); - tparams = null; - } - - /// Writes bases to the text buffer. - void writeInheritanceList(BaseClassType[] bases) - { - if (bases.length == 0) - return; - auto basesBegin = bases[0].begin.prevNWS; - if (basesBegin.kind == TOK.Colon) - basesBegin = bases[0].begin; - write("$(PARENTS ", escape(textSpan(basesBegin, bases[$-1].end)), ")"); - } - - /// Writes a symbol to the text buffer. E.g: $(SYMBOL Buffer, 123) - void SYMBOL(char[] name, Declaration d) - { - auto loc = d.begin.getRealLocation(); - auto str = Format("$(SYMBOL {}, {})", name, loc.lineNum); - write(str); - // write("$(DDOC_PSYMBOL ", name, ")"); - } - - /// Writes a declaration to the text buffer. - void DECL(void delegate() dg, Declaration d, bool writeSemicolon = true) - { - if (cmntIsDitto) - { alias prevDeclOffset offs; - assert(offs != 0); - auto savedText = text; - text = ""; - write("\n$(DDOC_DECL "); - dg(); - writeAttributes(d); - write(")"); - // Insert text at offset. - auto len = text.length; - text = savedText[0..offs] ~ text ~ savedText[offs..$]; - offs += len; // Add length of the inserted text to the offset. - return; - } - write("\n$(DDOC_DECL "); - dg(); - writeAttributes(d); - write(")"); - prevDeclOffset = text.length; - } - - - /// Writes a class or interface declaration. - void writeClassOrInterface(T)(T d) - { - if (!ddoc(d)) - return d; - DECL({ - write(d.begin.srcText, ", "); - SYMBOL(d.name.str, d); - writeTemplateParams(); - writeInheritanceList(d.bases); - }, d); - DESC({ - writeComment(); - MEMBERS(is(T == ClassDeclaration) ? "CLASS" : "INTERFACE", { - scope s = new Scope(); - d.decls && DefaultVisitor.visit(d.decls); - }); - }); - } - - // templated decls are not virtual so we need these: - - /// Writes a class declaration. - void writeClass(ClassDeclaration d) { - writeClassOrInterface(d); - } - - /// Writes an interface declaration. - void writeInterface(InterfaceDeclaration d) { - writeClassOrInterface(d); - } - - /// Writes a struct or union declaration. - void writeStructOrUnion(T)(T d) - { - if (!ddoc(d)) - return d; - DECL({ - write(d.begin.srcText, d.name ? ", " : ""); - if (d.name) - SYMBOL(d.name.str, d); - writeTemplateParams(); - }, d); - DESC({ - writeComment(); - MEMBERS(is(T == StructDeclaration) ? "STRUCT" : "UNION", { - scope s = new Scope(); - d.decls && DefaultVisitor.visit(d.decls); - }); - }); - } - - // templated decls are not virtual so we need these: - - /// Writes a struct declaration. - void writeStruct(StructDeclaration d) { - writeStructOrUnion(d); - } - - /// Writes an union declaration. - void writeUnion(UnionDeclaration d) { - writeStructOrUnion(d); - } - - /// Writes an alias or typedef declaration. - void writeAliasOrTypedef(T)(T d) - { - auto prefix = is(T == AliasDeclaration) ? "alias " : "typedef "; - if (auto vd = d.decl.Is!(VariablesDeclaration)) - { - auto type = textSpan(vd.typeNode.baseType.begin, vd.typeNode.end); - foreach (name; vd.names) - DECL({ write(prefix, ", "); write(escape(type), " "); SYMBOL(name.str, d); }, d); - } - else if (auto fd = d.decl.Is!(FunctionDeclaration)) - {} - // DECL({ write(textSpan(d.begin, d.end)); }, false); - DESC({ writeComment(); }); - } - - - /// Writes the attributes of a declaration in brackets. - void writeAttributes(Declaration d) - { - char[][] attributes; - - if (d.prot != Protection.None) - attributes ~= "$(PROT " ~ .toString(d.prot) ~ ")"; - - auto stc = d.stc; - stc &= ~StorageClass.Auto; // Ignore auto. - foreach (stcStr; .toStrings(stc)) - attributes ~= "$(STC " ~ stcStr ~ ")"; - - LinkageType ltype; - if (auto vd = d.Is!(VariablesDeclaration)) - ltype = vd.linkageType; - else if (auto fd = d.Is!(FunctionDeclaration)) - ltype = fd.linkageType; - - if (ltype != LinkageType.None) - attributes ~= "$(LINKAGE extern(" ~ .toString(ltype) ~ "))"; - - if (!attributes.length) - return; - - write("$(ATTRIBUTES "); - foreach (attribute; attributes) - write(attribute); - write(")"); - } - - alias Declaration D; - - alias DDocEmitter.visit visit; - - D visit(EnumDeclaration d) - { - /+ FIXME: broken, infinite recursion :/ - if (!ddoc(d)) - return d; - DECL({ - write("enum, ", d.name ? " " : ""); - d.name && SYMBOL(d.name.str, d); - }, d); - DESC({ - writeComment(); - Stdout("help\n"); -///*FIXME*/ MEMBERS("ENUM", { scope s = new Scope(); DDocEmitter.visit(d); }); - }); - +/ - return d; - } - - D visit(EnumMemberDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("member, "); SYMBOL(d.name.str, d); }, d, false); - DESC({ writeComment(); }); - return d; - } - - D visit(TemplateDeclaration d) - { - this.tparams = d.tparams; - if (d.begin.kind != TOK.Template) - { // This is a templatized class/interface/struct/union/function. - DefaultVisitor.visit(d.decls); - this.tparams = null; - return d; - } - if (!ddoc(d)) - return d; - DECL({ - write("template, "); - SYMBOL(d.name.str, d); - writeTemplateParams(); - }, d); - DESC({ - writeComment(); - MEMBERS("TEMPLATE", { - scope s = new Scope(); - DefaultVisitor.visit(d.decls); - }); - }); - return d; - } - - D visit(ConstructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("constructor, "); SYMBOL("this", d); writeParams(d.params); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(StaticConstructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("static constructor, "); SYMBOL("this", d); write("()"); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(DestructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("destructor, ~"); SYMBOL("this", d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(StaticDestructorDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("static destructor, ~"); SYMBOL("this", d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(FunctionDeclaration d) - { - if (!ddoc(d)) - return d; - auto type = textSpan(d.returnType.baseType.begin, d.returnType.end); - DECL({ - write("function, "); - write("$(TYPE "); - write("$(RETURNS ", escape(type), ")"); - writeTemplateParams(); - writeParams(d.params); - write(")"); - SYMBOL(d.name.str, d); - }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(NewDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("new, "); SYMBOL("new", d); writeParams(d.params); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(DeleteDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("delete, "); SYMBOL("delete", d); writeParams(d.params); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(VariablesDeclaration d) - { - if (!ddoc(d)) - return d; - char[] type = "auto"; - if (d.typeNode) - type = textSpan(d.typeNode.baseType.begin, d.typeNode.end); - foreach (name; d.names) - DECL({ write("variable, "); write("$(TYPE ", escape(type), ")"); SYMBOL(name.str, d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(InvariantDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("invariant, "); SYMBOL("invariant", d); }, d); - DESC({ writeComment(); }); - return d; - } - - D visit(UnittestDeclaration d) - { - if (!ddoc(d)) - return d; - DECL({ write("unittest, "); SYMBOL("unittest", d); }, d); - DESC({ writeComment(); }); - return d; - } - -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/cmd/Generate.d --- a/trunk/src/cmd/Generate.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,489 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module cmd.Generate; - -import dil.ast.DefaultVisitor; -import dil.ast.Node, - dil.ast.Declaration, - dil.ast.Statement, - dil.ast.Expression, - dil.ast.Types; -import dil.lexer.Lexer; -import dil.parser.Parser; -import dil.semantic.Module; -import dil.SourceText; -import dil.Information; -import SettingsLoader; -import Settings; -import common; - -import tango.io.GrowBuffer; -import tango.io.Print; - -/// Options for the generate command. -enum GenOption -{ - Empty, - Tokens = 1, - Syntax = 1<<1, - HTML = 1<<2, - XML = 1<<3, - PrintLines = 1<<4 -} - -/// Executes the generate command. -void execute(string filePath, GenOption options, InfoManager infoMan) -{ - assert(options != GenOption.Empty); - auto mapFilePath = options & GenOption.HTML ? GlobalSettings.htmlMapFile - : GlobalSettings.xmlMapFile; - auto map = TagMapLoader(infoMan).load(mapFilePath); - auto tags = new TagMap(map); - - if (infoMan.hasInfo) - return; - - if (options & GenOption.Syntax) - highlightSyntax(filePath, tags, Stdout, options); - else - highlightTokens(filePath, tags, Stdout, options); -} - -/// Escapes the characters '<', '>' and '&' with named character entities. -char[] xml_escape(char[] text) -{ - char[] result; - foreach(c; text) - switch(c) - { - case '<': result ~= "<"; break; - case '>': result ~= ">"; break; - case '&': result ~= "&"; break; - default: result ~= c; - } - if (result.length != text.length) - return result; - // Nothing escaped. Return original text. - delete result; - return text; -} - -/// Maps tokens to (format) strings. -class TagMap -{ - string[string] table; - string[TOK.MAX] tokenTable; - - this(string[string] table) - { - this.table = table; - Identifier = this["Identifier", "{0}"]; - String = this["String", "{0}"]; - Char = this["Char", "{0}"]; - Number = this["Number", "{0}"]; - Keyword = this["Keyword", "{0}"]; - LineC = this["LineC", "{0}"]; - BlockC = this["BlockC", "{0}"]; - NestedC = this["NestedC", "{0}"]; - Shebang = this["Shebang", "{0}"]; - HLine = this["HLine", "{0}"]; - Filespec = this["Filespec", "{0}"]; - Illegal = this["Illegal", "{0}"]; - Newline = this["Newline", "{0}"]; - SpecialToken = this["SpecialToken", "{0}"]; - Declaration = this["Declaration", "d"]; - Statement = this["Statement", "s"]; - Expression = this["Expression", "e"]; - Type = this["Type", "t"]; - Other = this["Other", "o"]; - EOF = this["EOF", ""]; - - foreach (i, tokStr; tokToString) - if (auto pStr = tokStr in this.table) - tokenTable[i] = *pStr; - } - - /// Returns the value for str, or 'fallback' if str is not in the table. - string opIndex(string str, string fallback = "") - { - auto p = str in table; - if (p) - return *p; - return fallback; - } - - /// Returns the value for tok in O(1) time. - string opIndex(TOK tok) - { - return tokenTable[tok]; - } - - /// Shortcuts for quick access. - string Identifier, String, Char, Number, Keyword, LineC, BlockC, - NestedC, Shebang, HLine, Filespec, Illegal, Newline, SpecialToken, - Declaration, Statement, Expression, Type, Other, EOF; - - /// Returns the tag for the category 'nc'. - string getTag(NodeCategory nc) - { - string tag; - switch (nc) - { alias NodeCategory NC; - case NC.Declaration: tag = Declaration; break; - case NC.Statement: tag = Statement; break; - case NC.Expression: tag = Expression; break; - case NC.Type: tag = Type; break; - case NC.Other: tag = Other; break; - default: assert(0); - } - return tag; - } -} - -/// Find the last occurrence of object in subject. -/// Returns: the index if found, or -1 if not. -int rfind(char[] subject, char object) -{ - foreach_reverse(i, c; subject) - if (c == object) - return i; - return -1; -} - -/// Returns the short class name of a class descending from Node.$(BR) -/// E.g.: dil.ast.Declarations.ClassDeclaration -> Class -char[] getShortClassName(Node node) -{ - static char[][] name_table; - if (name_table is null) - name_table = new char[][NodeKind.max+1]; // Create a new table. - // Look up in table. - char[] name = name_table[node.kind]; - if (name !is null) - return name; // Return cached name. - - name = node.classinfo.name; // Get the fully qualified name of the class. - name = name[rfind(name, '.')+1 .. $]; // Remove package and module name. - - uint suffixLength; - switch (node.category) - { - alias NodeCategory NC; - case NC.Declaration: - suffixLength = "Declaration".length; - break; - case NC.Statement: - suffixLength = "Statement".length; - break; - case NC.Expression: - suffixLength = "Expression".length; - break; - case NC.Type: - suffixLength = "Type".length; - break; - case NC.Other: - break; - default: - assert(0); - } - // Remove common suffix. - name = name[0 .. $ - suffixLength]; - // Store the name in the table. - name_table[node.kind] = name; - return name; -} - -/// Extended token structure. -struct TokenEx -{ - Token* token; /// The lexer token. - Node[] beginNodes; /// beginNodes[n].begin == token - Node[] endNodes; /// endNodes[n].end == token -} - -/// Builds an array of TokenEx items. -class TokenExBuilder : DefaultVisitor -{ - private TokenEx*[Token*] tokenTable; - - TokenEx[] build(Node root, Token* first) - { - auto token = first; - - uint count; // Count tokens. - for (; token; token = token.next) - count++; - // Creat the exact number of TokenEx instances. - auto toks = new TokenEx[count]; - token = first; - foreach (ref tokEx; toks) - { - tokEx.token = token; - if (!token.isWhitespace) - tokenTable[token] = &tokEx; - token = token.next; - } - - super.visitN(root); - tokenTable = null; - return toks; - } - - TokenEx* getTokenEx()(Token* t) - { - auto p = t in tokenTable; - assert(p, t.srcText~" is not in tokenTable"); - return *p; - } - - // Override dispatch function. - override Node dispatch(Node n) - { - auto begin = n.begin; - if (begin) - { assert(n.end); - auto txbegin = getTokenEx(begin); - auto txend = getTokenEx(n.end); - txbegin.beginNodes ~= n; - txend.endNodes ~= n; - } - return super.dispatch(n); - } -} - -void printErrors(Lexer lx, TagMap tags, Print!(char) print) -{ - foreach (e; lx.errors) - print.format(tags["LexerError"], e.filePath, e.loc, e.col, xml_escape(e.getMsg)); -} - -void printErrors(Parser parser, TagMap tags, Print!(char) print) -{ - foreach (e; parser.errors) - print.format(tags["ParserError"], e.filePath, e.loc, e.col, xml_escape(e.getMsg)); -} - -void printLines(uint lines, TagMap tags, Print!(char) print) -{ - auto lineNumberFormat = tags["LineNumber"]; - for (auto lineNum = 1; lineNum <= lines; lineNum++) - print.format(lineNumberFormat, lineNum); -} - -// void printMultiline(Token* token, TagMap tags, Print!(char) print) -// { -// } - -/// Highlights the syntax in a source file. -void highlightSyntax(string filePath, TagMap tags, Print!(char) print, GenOption options) -{ - auto parser = new Parser(new SourceText(filePath, true)); - auto root = parser.start(); - auto lx = parser.lexer; - - auto builder = new TokenExBuilder(); - auto tokenExList = builder.build(root, lx.firstToken()); - - print(tags["DocHead"]); - if (lx.errors.length || parser.errors.length) - { // Output error messages. - print(tags["CompBegin"]); - printErrors(lx, tags, print); - printErrors(parser, tags, print); - print(tags["CompEnd"]); - } - - if (options & GenOption.PrintLines) - { - print(tags["LineNumberBegin"]); - printLines(lx.lineNum, tags, print); - print(tags["LineNumberEnd"]); - } - - print(tags["SourceBegin"]); - - auto tagNodeBegin = tags["NodeBegin"]; - auto tagNodeEnd = tags["NodeEnd"]; - - // Iterate over list of tokens. - foreach (ref tokenEx; tokenExList) - { - auto token = tokenEx.token; - - token.ws && print(token.wsChars); // Print preceding whitespace. - if (token.isWhitespace) { - printToken(token, tags, print); - continue; - } - // - foreach (node; tokenEx.beginNodes) - print.format(tagNodeBegin, tags.getTag(node.category), getShortClassName(node)); - // Token text. - printToken(token, tags, print); - // - if (options & GenOption.HTML) - foreach_reverse (node; tokenEx.endNodes) - print(tagNodeEnd); - else - foreach_reverse (node; tokenEx.endNodes) - print.format(tagNodeEnd, tags.getTag(node.category)); - } - print(tags["SourceEnd"]); - print(tags["DocEnd"]); -} - -/// Highlights all tokens of a source file. -void highlightTokens(string filePath, TagMap tags, Print!(char) print, GenOption options) -{ - auto lx = new Lexer(new SourceText(filePath, true)); - lx.scanAll(); - - print(tags["DocHead"]); - if (lx.errors.length) - { - print(tags["CompBegin"]); - printErrors(lx, tags, print); - print(tags["CompEnd"]); - } - - if (options & GenOption.PrintLines) - { - print(tags["LineNumberBegin"]); - printLines(lx.lineNum, tags, print); - print(tags["LineNumberEnd"]); - } - - print(tags["SourceBegin"]); - // Traverse linked list and print tokens. - for (auto token = lx.firstToken(); token; token = token.next) { - token.ws && print(token.wsChars); // Print preceding whitespace. - printToken(token, tags, print); - } - print(tags["SourceEnd"]); - print(tags["DocEnd"]); -} - -/// A token highlighter designed for DDoc. -class TokenHighlighter -{ - TagMap tags; - this(InfoManager infoMan, bool useHTML = true) - { - string filePath = GlobalSettings.htmlMapFile; - if (!useHTML) - filePath = GlobalSettings.xmlMapFile; - auto map = TagMapLoader(infoMan).load(filePath); - tags = new TagMap(map); - } - - /// Highlights tokens in a DDoc code section. - /// Returns: a string with the highlighted tokens (in HTML tags.) - string highlight(string text, string filePath) - { - auto buffer = new GrowBuffer(text.length); - auto print = new Print!(char)(Format, buffer); - - auto lx = new Lexer(new SourceText(filePath, text)); - lx.scanAll(); - - // Traverse linked list and print tokens. - print("$(D_CODE\n"); - if (lx.errors.length) - { // Output error messages. - print(tags["CompBegin"]); - printErrors(lx, tags, print); - print(tags["CompEnd"]); - } - // Traverse linked list and print tokens. - for (auto token = lx.firstToken(); token; token = token.next) { - token.ws && print(token.wsChars); // Print preceding whitespace. - printToken(token, tags, print); - } - print("\n)"); - return cast(char[])buffer.slice(); - } -} - -/// Prints a token to the stream print. -void printToken(Token* token, TagMap tags, Print!(char) print) -{ - switch(token.kind) - { - case TOK.Identifier: - print.format(tags.Identifier, token.srcText); - break; - case TOK.Comment: - string formatStr; - switch (token.start[1]) - { - case '/': formatStr = tags.LineC; break; - case '*': formatStr = tags.BlockC; break; - case '+': formatStr = tags.NestedC; break; - default: assert(0); - } - print.format(formatStr, xml_escape(token.srcText)); - break; - case TOK.String: - print.format(tags.String, xml_escape(token.srcText)); - break; - case TOK.CharLiteral: - print.format(tags.Char, xml_escape(token.srcText)); - break; - case TOK.Int32, TOK.Int64, TOK.Uint32, TOK.Uint64, - TOK.Float32, TOK.Float64, TOK.Float80, - TOK.Imaginary32, TOK.Imaginary64, TOK.Imaginary80: - print.format(tags.Number, token.srcText); - break; - case TOK.Shebang: - print.format(tags.Shebang, xml_escape(token.srcText)); - break; - case TOK.HashLine: - auto formatStr = tags.HLine; - // The text to be inserted into formatStr. - auto buffer = new GrowBuffer; - auto print2 = new Print!(char)(Format, buffer); - - void printWS(char* start, char* end) - { - start != end && print2(start[0 .. end - start]); - } - - auto num = token.tokLineNum; - if (num is null) - { // Malformed #line - print.format(formatStr, token.srcText); - break; - } - - // Print whitespace between #line and number. - printWS(token.start, num.start); // Prints "#line" as well. - printToken(num, tags, print2); // Print the number. - - if (auto filespec = token.tokLineFilespec) - { // Print whitespace between number and filespec. - printWS(num.end, filespec.start); - print2.format(tags.Filespec, xml_escape(filespec.srcText)); - } - // Finally print the whole token. - print.format(formatStr, cast(char[])buffer.slice()); - break; - case TOK.Illegal: - print.format(tags.Illegal, token.srcText()); - break; - case TOK.Newline: - print.format(tags.Newline, token.srcText()); - break; - case TOK.EOF: - print(tags.EOF); - break; - default: - if (token.isKeyword()) - print.format(tags.Keyword, token.srcText); - else if (token.isSpecialToken) - print.format(tags.SpecialToken, token.srcText); - else - print(tags[token.kind]); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/cmd/ImportGraph.d --- a/trunk/src/cmd/ImportGraph.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,424 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module cmd.ImportGraph; - -import dil.ast.Node; -import dil.ast.Declarations; -import dil.semantic.Module; -import dil.parser.ImportParser; -import dil.SourceText; -import dil.Compilation; -import Settings; -import common; - -import tango.text.Regex : RegExp = Regex; -import tango.io.FilePath; -import tango.io.FileConst; -import tango.text.Util; - -alias FileConst.PathSeparatorChar dirSep; - -/// Options for the importgraph command. -enum IGraphOption -{ - None, - IncludeUnlocatableModules = 1, - PrintDot = 1<<1, - HighlightCyclicEdges = 1<<2, - HighlightCyclicVertices = 1<<3, - GroupByPackageNames = 1<<4, - GroupByFullPackageName = 1<<5, - PrintPaths = 1<<6, - PrintList = 1<<7, - MarkCyclicModules = 1<<8, -} - -/// Represents a module dependency graph. -class Graph -{ - Vertex[] vertices; /// The vertices or modules. - Edge[] edges; /// The edges or import statements. - - void addVertex(Vertex vertex) - { - vertex.id = vertices.length; - vertices ~= vertex; - } - - Edge addEdge(Vertex from, Vertex to) - { - auto edge = new Edge(from, to); - edges ~= edge; - from.outgoing ~= to; - to.incoming ~= from; - return edge; - } - - /// Walks the graph and marks cyclic vertices and edges. - void detectCycles() - { // Cycles could also be detected in the GraphBuilder, - // but having the code here makes things much clearer. - - // Commented out because this algorithm doesn't work. - // Returns true if the vertex is in status Visiting. - /+bool visit(Vertex vertex) - { - switch (vertex.status) - { - case Vertex.Status.Visiting: - vertex.isCyclic = true; - return true; - case Vertex.Status.None: - vertex.status = Vertex.Status.Visiting; // Flag as visiting. - foreach (outVertex; vertex.outgoing) // Visit successors. - vertex.isCyclic |= visit(outVertex); - vertex.status = Vertex.Status.Visited; // Flag as visited. - break; - case Vertex.Status.Visited: - break; - default: - assert(0, "unknown vertex status"); - } - return false; // return (vertex.status == Vertex.Status.Visiting); - } - // Start visiting vertices. - visit(vertices[0]);+/ - - //foreach (edge; edges) - // if (edge.from.isCyclic && edge.to.isCyclic) - // edge.isCyclic = true; - - // Use functioning algorithm. - analyzeGraph(vertices, edges); - } -} - -/// Represents a directed connection between two vertices. -class Edge -{ - Vertex from; /// Coming from vertex. - Vertex to; /// Going to vertex. - bool isCyclic; /// Edge connects cyclic vertices. - bool isPublic; /// Public import. - bool isStatic; /// Static import. - - this(Vertex from, Vertex to) - { - this.from = from; - this.to = to; - } -} - -/// Represents a module in the graph. -class Vertex -{ - Module modul; /// The module represented by this vertex. - uint id; /// The nth vertex in the graph. - Vertex[] incoming; /// Also called predecessors. - Vertex[] outgoing; /// Also called successors. - bool isCyclic; /// Whether this vertex is in a cyclic relationship with other vertices. - - enum Status : ubyte - { None, Visiting, Visited } - Status status; /// Used by the cycle detection algorithm. -} - -/// Searches for a module in the file system looking in importPaths. -/// Returns: the file path to the module, or null if it wasn't found. -string findModuleFilePath(string moduleFQNPath, string[] importPaths) -{ - auto filePath = new FilePath(); - foreach (importPath; importPaths) - { - filePath.set(importPath); - filePath.append(moduleFQNPath); - foreach (moduleSuffix; [".d", ".di"/*interface file*/]) - { - filePath.suffix(moduleSuffix); - if (filePath.exists()) - return filePath.toString(); - } - } - return null; -} - -/// Builds a module dependency graph. -class GraphBuilder -{ - Graph graph; - IGraphOption options; - string[] importPaths; /// Where to look for modules. - Vertex[string] loadedModulesTable; /// Maps FQN paths to modules. - bool delegate(string) filterPredicate; - - this() - { - this.graph = new Graph; - } - - /// Start building the graph and return that. - /// Params: - /// fileName = the file name of the root module. - Graph start(string fileName) - { - loadModule(fileName); - return graph; - } - - /// Loads all modules recursively and builds the graph at the same time. - /// Params: - /// moduleFQNPath = the path version of the module FQN.$(BR) - /// E.g.: FQN = dil.ast.Node -> FQNPath = dil/ast/Node - Vertex loadModule(string moduleFQNPath) - { - // Look up in table if the module is already loaded. - auto pVertex = moduleFQNPath in loadedModulesTable; - if (pVertex !is null) - return *pVertex; // Returns null for filtered or unlocatable modules. - - // Filter out modules. - if (filterPredicate && filterPredicate(moduleFQNPath)) - { // Store null for filtered modules. - loadedModulesTable[moduleFQNPath] = null; - return null; - } - - // Locate the module in the file system. - auto moduleFilePath = findModuleFilePath(moduleFQNPath, importPaths); - - Vertex vertex; - - if (moduleFilePath is null) - { // Module not found. - if (options & IGraphOption.IncludeUnlocatableModules) - { // Include module nevertheless. - vertex = new Vertex; - vertex.modul = new Module(""); - vertex.modul.setFQN(replace(moduleFQNPath, dirSep, '.')); - graph.addVertex(vertex); - } - // Store vertex in the table (vertex may be null.) - loadedModulesTable[moduleFQNPath] = vertex; - } - else - { - auto modul = new Module(moduleFilePath); - // Use lightweight ImportParser. - modul.setParser(new ImportParser(modul.sourceText)); - modul.parse(); - - vertex = new Vertex; - vertex.modul = modul; - - graph.addVertex(vertex); - loadedModulesTable[modul.getFQNPath()] = vertex; - - // Load the modules which this module depends on. - foreach (importDecl; modul.imports) - { - foreach (moduleFQNPath2; importDecl.getModuleFQNs(dirSep)) - { - auto loaded = loadModule(moduleFQNPath2); - if (loaded !is null) - { - auto edge = graph.addEdge(vertex, loaded); - edge.isPublic = importDecl.isPublic(); - edge.isStatic = importDecl.isStatic(); - } - } - } - } - return vertex; - } -} - -/// Executes the importgraph command. -void execute(string filePathString, CompilationContext context, string[] strRegexps, - uint levels, string siStyle, string piStyle, IGraphOption options) -{ - // Init regular expressions. - RegExp[] regexps; - foreach (strRegexp; strRegexps) - regexps ~= new RegExp(strRegexp); - - // Add the directory of the file to the import paths. - auto filePath = new FilePath(filePathString); - auto fileDir = filePath.folder(); - context.importPaths ~= fileDir; - - auto gbuilder = new GraphBuilder; - - gbuilder.importPaths = context.importPaths; - gbuilder.options = options; - gbuilder.filterPredicate = (string moduleFQNPath) { - foreach (rx; regexps) - // Replace slashes: dil/ast/Node -> dil.ast.Node - if (rx.test(replace(moduleFQNPath.dup, dirSep, '.'))) - return true; - return false; - }; - - auto graph = gbuilder.start(filePath.name()); - - if (options & (IGraphOption.PrintList | IGraphOption.PrintPaths)) - { - if (options & IGraphOption.MarkCyclicModules) - graph.detectCycles(); - - if (options & IGraphOption.PrintPaths) - printModulePaths(graph.vertices, levels+1, ""); - else - printModuleList(graph.vertices, levels+1, ""); - } - else - printDotDocument(graph, siStyle, piStyle, options); -} - -/// Prints the file paths to the modules. -void printModulePaths(Vertex[] vertices, uint level, char[] indent) -{ - if (level == 0) - return; - foreach (vertex; vertices) - { - Stdout(indent)((vertex.isCyclic?"*":"")~vertex.modul.filePath).newline; - if (vertex.outgoing.length) - printModulePaths(vertex.outgoing, level-1, indent~" "); - } -} - -/// Prints a list of module FQNs. -void printModuleList(Vertex[] vertices, uint level, char[] indent) -{ - if (level == 0) - return; - foreach (vertex; vertices) - { - Stdout(indent)((vertex.isCyclic?"*":"")~vertex.modul.getFQN()).newline; - if (vertex.outgoing.length) - printModuleList(vertex.outgoing, level-1, indent~" "); - } -} - -/// Prints the graph as a graphviz dot document. -void printDotDocument(Graph graph, string siStyle, string piStyle, - IGraphOption options) -{ - Vertex[][string] verticesByPckgName; - if (options & IGraphOption.GroupByFullPackageName) - foreach (vertex; graph.vertices) - verticesByPckgName[vertex.modul.packageName] ~= vertex; - - if (options & (IGraphOption.HighlightCyclicVertices | - IGraphOption.HighlightCyclicEdges)) - graph.detectCycles(); - - // Output header of the dot document. - Stdout("Digraph ImportGraph\n{\n"); - // Output nodes. - // 'i' and vertex.id should be the same. - foreach (i, vertex; graph.vertices) - Stdout.formatln(` n{} [label="{}"{}];`, i, vertex.modul.getFQN(), (vertex.isCyclic ? ",style=filled,fillcolor=tomato" : "")); - - // Output edges. - foreach (edge; graph.edges) - { - string edgeStyles = ""; - if (edge.isStatic || edge.isPublic) - { - edgeStyles = `[style="`; - edge.isStatic && (edgeStyles ~= siStyle ~ ","); - edge.isPublic && (edgeStyles ~= piStyle); - edgeStyles[$-1] == ',' && (edgeStyles = edgeStyles[0..$-1]); // Remove last comma. - edgeStyles ~= `"]`; - } - edge.isCyclic && (edgeStyles ~= "[color=red]"); - Stdout.formatln(` n{} -> n{} {};`, edge.from.id, edge.to.id, edgeStyles); - } - - if (options & IGraphOption.GroupByFullPackageName) - foreach (packageName, vertices; verticesByPckgName) - { // Output nodes in a cluster. - Stdout.format(` subgraph "cluster_{}" {`\n` label="{}";color=blue;`"\n ", packageName, packageName); - foreach (vertex; vertices) - Stdout.format(`n{};`, vertex.id); - Stdout("\n }\n"); - } - - Stdout("}\n"); -} - -// This is the old algorithm that was used to detect cycles in a directed graph. -void analyzeGraph(Vertex[] vertices_init, Edge[] edges) -{ - edges = edges.dup; - void recursive(Vertex[] vertices) - { - foreach (idx, vertex; vertices) - { - uint outgoing, incoming; - foreach (j, edge; edges) - { - if (edge.from is vertex) - outgoing++; - if (edge.to is vertex) - incoming++; - } - - if (outgoing == 0) - { - if (incoming != 0) - { - // Vertex is a sink. - alias outgoing i; // Reuse - alias incoming j; // Reuse - // Remove edges. - for (i=j=0; i < edges.length; i++) - if (edges[i].to !is vertex) - edges[j++] = edges[i]; - edges.length = j; - vertices = vertices[0..idx] ~ vertices[idx+1..$]; - recursive(vertices); - return; - } - else - { - // Edges to this vertex were removed previously. - // Only remove vertex now. - vertices = vertices[0..idx] ~ vertices[idx+1..$]; - recursive(vertices); - return; - } - } - else if (incoming == 0) - { - // Vertex is a source - alias outgoing i; // Reuse - alias incoming j; // Reuse - // Remove edges. - for (i=j=0; i < edges.length; i++) - if (edges[i].from !is vertex) - edges[j++] = edges[i]; - edges.length = j; - vertices = vertices[0..idx] ~ vertices[idx+1..$]; - recursive(vertices); - return; - } -// else -// { -// // source && sink -// // continue loop. -// } - } - - // When reaching this point it means only cylic edges and vertices are left. - foreach (vertex; vertices) - vertex.isCyclic = true; - foreach (edge; edges) - if (edge) - edge.isCyclic = true; - } - recursive(vertices_init); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/cmd/Statistics.d --- a/trunk/src/cmd/Statistics.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,212 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module cmd.Statistics; - -import cmd.ASTStats; -import dil.lexer.Lexer; -import dil.lexer.Token; -import dil.parser.Parser; -import dil.ast.NodesEnum; -import dil.SourceText; -import common; - -/// A group of statistics variables. -struct Statistics -{ - uint whitespaceCount; /// Counter for whitespace characters. - uint wsTokenCount; /// Counter for all whitespace tokens. - uint keywordCount; /// Counter for keywords. - uint identCount; /// Counter for identifiers. - uint numberCount; /// Counter for number literals. - uint commentCount; /// Counter for comments. - uint tokenCount; /// Counter for all tokens produced by the Lexer. - uint linesOfCode; /// Number of lines. - uint[] tokensTable; /// Table of counters for all token kinds. - uint[] nodesTable; /// Table of counters for all node kinds. - - static Statistics opCall(bool allocateTokensTable, bool allocateNodesTable = false) - { - Statistics s; - if (allocateTokensTable) - s.tokensTable = new uint[TOK.MAX]; - if (allocateNodesTable) - s.nodesTable = new uint[g_classNames.length]; - return s; - } - - void opAddAssign(Statistics s) - { - this.whitespaceCount += s.whitespaceCount; - this.wsTokenCount += s.wsTokenCount; - this.keywordCount += s.keywordCount; - this.identCount += s.identCount; - this.numberCount += s.numberCount; - this.commentCount += s.commentCount; - this.tokenCount += s.tokenCount; - this.linesOfCode += s.linesOfCode; - foreach (i, count; s.tokensTable) - this.tokensTable[i] += count; - foreach (i, count; s.nodesTable) - this.nodesTable[i] += count; - } -} - -/// Executes the statistics command. -void execute(string[] filePaths, bool printTokensTable, bool printNodesTable) -{ - Statistics[] stats; - foreach (filePath; filePaths) - stats ~= getStatistics(filePath, printTokensTable, printNodesTable); - - auto total = Statistics(printTokensTable, printNodesTable); - - foreach (i, ref stat; stats) - { - total += stat; - Stdout.formatln( - "----\n" - "File: {}\n" - "Whitespace character count: {}\n" - "Whitespace token count: {}\n" - "Keyword count: {}\n" - "Identifier count: {}\n" - "Number count: {}\n" - "Comment count: {}\n" - "All tokens count: {}\n" - "Lines of code: {}", - filePaths[i], - stat.whitespaceCount, - stat.wsTokenCount, - stat.keywordCount, - stat.identCount, - stat.numberCount, - stat.commentCount, - stat.tokenCount, - stat.linesOfCode - ); - } - - if (filePaths.length > 1) - { - Stdout.formatln( - "--------------------------------------------------------------------------------\n" - "Total of {} files:\n" - "Whitespace character count: {}\n" - "Whitespace token count: {}\n" - "Keyword count: {}\n" - "Identifier count: {}\n" - "Number count: {}\n" - "Comment count: {}\n" - "All tokens count: {}\n" - "Lines of code: {}", - filePaths.length, - total.whitespaceCount, - total.wsTokenCount, - total.keywordCount, - total.identCount, - total.numberCount, - total.commentCount, - total.tokenCount, - total.linesOfCode - ); - } - - if (printTokensTable) - { - Stdout("Table of tokens:").newline; - Stdout.formatln(" {,10} | {}", "Count", "Token kind"); - Stdout("-----------------------------").newline; - foreach (i, count; total.tokensTable) - Stdout.formatln(" {,10} | {}", count, Token.toString(cast(TOK)i)); - Stdout("// End of tokens table.").newline; - } - - if(printNodesTable) - { - Stdout("Table of nodes:").newline; - Stdout.formatln(" {,10} | {}", "Count", "Node kind"); - Stdout("-----------------------------").newline; - foreach (i, count; total.nodesTable) - Stdout.formatln(" {,10} | {}", count, g_classNames[i]); - Stdout("// End of nodes table.").newline; - } -} - -/// Returns the statistics for a D source file. -Statistics getStatistics(string filePath, bool printTokensTable, bool printNodesTable) -{ - // Create a new record. - auto stats = Statistics(printTokensTable); - - auto sourceText = new SourceText(filePath, true); - Parser parser; - Lexer lx; - if (printNodesTable) - { - parser = new Parser(sourceText); - auto rootNode = parser.start(); - // Count nodes. - stats.nodesTable = (new ASTStats).count(rootNode); - lx = parser.lexer; - } - else - { - lx = new Lexer(sourceText); - lx.scanAll(); - } - - auto token = lx.firstToken(); - - // Count tokens. - // Lexer creates HEAD + Newline, which are not in the source text. - // No token left behind! - stats.tokenCount = 2; - stats.linesOfCode = lx.lineNum; - if (printTokensTable) - { - stats.tokensTable[TOK.HEAD] = 1; - stats.tokensTable[TOK.Newline] = 1; - } - // Traverse linked list. - while (1) - { - stats.tokenCount += 1; - - if (printTokensTable) - stats.tokensTable[token.kind] += 1; - - // Count whitespace characters - if (token.ws !is null) - stats.whitespaceCount += token.start - token.ws; - - switch (token.kind) - { - case TOK.Identifier: - stats.identCount++; - break; - case TOK.Comment: - stats.commentCount++; - break; - case TOK.Int32, TOK.Int64, TOK.Uint32, TOK.Uint64, - TOK.Float32, TOK.Float64, TOK.Float80, - TOK.Imaginary32, TOK.Imaginary64, TOK.Imaginary80: - stats.numberCount++; - break; - case TOK.Newline: - break; - default: - if (token.isKeyword) - stats.keywordCount++; - else if (token.isWhitespace) - stats.wsTokenCount++; - } - - if (token.next is null) - break; - token = token.next; - } - assert(token.kind == TOK.EOF); - return stats; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/common.d --- a/trunk/src/common.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,20 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module common; - -public import tango.io.Stdout; -public import tango.text.convert.Layout; - -/// String aliases. -alias char[] string; -alias wchar[] wstring; /// ditto -alias dchar[] dstring; /// ditto - -/// Global formatter instance. -static Layout!(char) Format; -static this() -{ - Format = new typeof(Format); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/config.d --- a/trunk/src/config.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,34 +0,0 @@ -/// The configuration file of dil. -/// -/// Relative paths are resolved from the directory of the executable. -module config; - -/// Predefined version identifiers. -var version_ids = ["X86", "linux", "LittleEndian"]; -// "X86_64", "Windows", "Win32", "Win64", "BigEndian" - -/// Path to the language file. -var langfile = "lang_en.d"; - -/// An array of import paths to look for modules. -var import_paths = []; /// E.g.: ["src/", "import/"] - -/// DDoc macro file paths. -/// -/// Macro definitions in ddoc_files[n] override the ones in ddoc_files[n-1]. -var ddoc_files = ["predefined.ddoc"]; /// E.g.: ["src/mymacros.ddoc", "othermacros.ddoc"] - -var xml_map = "xml_map.d"; -var html_map = "html_map.d"; - -/// Customizable formats for error messages. -/// -///
      -///
    • 0: file path to the source text.
      • -///
    • 1: line number.
      • -///
    • 2: column number.
      • -///
    • 3: error message.
      • -///
    -var lexer_error = "{0}({1},{2})L: {3}"; -var parser_error = "{0}({1},{2})P: {3}"; /// ditto -var semantic_error = "{0}({1},{2})S: {3}"; /// ditto diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Compilation.d --- a/trunk/src/dil/Compilation.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,69 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Compilation; - -import common; - -/// A group of settings relevant to the compilation process. -class CompilationContext -{ - alias typeof(this) CC; - CC parent; - string[] importPaths; - uint debugLevel; - uint versionLevel; - bool[string] debugIds; - bool[string] versionIds; - bool releaseBuild; - uint structAlign = 4; - - this(CC parent = null) - { - this.parent = parent; - if (parent) - { - this.importPaths = parent.importPaths.dup; - this.debugLevel = parent.debugLevel; - this.versionLevel = parent.versionLevel; - this.releaseBuild = parent.releaseBuild; - this.structAlign = parent.structAlign; - } - } - - void addDebugId(string id) - { - debugIds[id] = true; - } - - void addVersionId(string id) - { - versionIds[id] = true; - } - - bool findDebugId(string id) - { - auto pId = id in debugIds; - if (pId) - return true; - if (!isRoot()) - return parent.findDebugId(id); - return false; - } - - bool findVersionId(string id) - { - auto pId = id in versionIds; - if (pId) - return true; - if (!isRoot()) - return parent.findVersionId(id); - return false; - } - - bool isRoot() - { - return parent is null; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/CompilerInfo.d --- a/trunk/src/dil/CompilerInfo.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,50 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.CompilerInfo; - -version(D2) - /// The major version number of this compiler. - const uint VERSION_MAJOR = 2; -else - /// The major version number of this compiler. - const uint VERSION_MAJOR = 1; - -/// The minor version number of this compiler. -const uint VERSION_MINOR = 0; - -private char[] toString(uint x) -{ - char[] str; - do - str = cast(char)('0' + (x % 10)) ~ str; - while (x /= 10) - return str; -} - -private char[] toString(uint x, uint pad) -{ - char[] str = toString(x); - if (pad < str.length) - return str; - for (uint i = pad-str.length; i; i--) - str = "0" ~ str; - return str; -} - -/// The compiler version formatted as a string. -const char[] VERSION = toString(VERSION_MAJOR)~"."~toString(VERSION_MINOR, 3); -/// The name of the compiler. -const char[] VENDOR = "dil"; - -/// The global, default alignment size for struct fields. -const uint DEFAULT_ALIGN_SIZE = 4; - -version(DDoc) - const uint PTR_SIZE = 0; /// The pointer size depending on the platform. -else -version(X86_64) - const uint PTR_SIZE = 8; // Pointer size on 64-bit platforms. -else - const uint PTR_SIZE = 4; // Pointer size on 32-bit platforms. diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Converter.d --- a/trunk/src/dil/Converter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,334 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Converter; - -import dil.Information; -import dil.Location; -import dil.Unicode; -import dil.FileBOM; -import dil.lexer.Funcs; -import dil.Messages; -import common; - -/// Converts various Unicode encoding formats to UTF-8. -struct Converter -{ - char[] filePath; /// For error messages. - InfoManager infoMan; - - static Converter opCall(char[] filePath, InfoManager infoMan) - { - Converter conv; - conv.filePath = filePath; - conv.infoMan = infoMan; - return conv; - } - - /// Byte-swaps c. - dchar swapBytes(dchar c) - { - return c = (c << 24) | - ((c >> 8) & 0xFF00) | - ((c << 8) & 0xFF0000) | - (c >> 24); - } - - /// Byte-swaps c. - wchar swapBytes(wchar c) - { - return (c << 8) | (c >> 8); - } - - /// Swaps the bytes of c on a little-endian machine. - dchar BEtoMachineDword(dchar c) - { - version(LittleEndian) - return swapBytes(c); - else - return c; - } - - /// Swaps the bytes of c on a big-endian machine. - dchar LEtoMachineDword(dchar c) - { - version(LittleEndian) - return c; - else - return swapBytes(c); - } - - /// Swaps the bytes of c on a little-endian machine. - wchar BEtoMachineWord(wchar c) - { - version(LittleEndian) - return swapBytes(c); - else - return c; - } - - /// Swaps the bytes of c on a big-endian machine. - wchar LEtoMachineWord(wchar c) - { - version(LittleEndian) - return c; - else - return swapBytes(c); - } - - /// Converts a UTF-32 text to UTF-8. - char[] UTF32toUTF8(bool isBigEndian)(ubyte[] data) - { - if (data.length == 0) - return null; - - char[] result; - uint lineNum = 1; - dchar[] text = cast(dchar[]) data[0 .. $-($%4)]; // Trim to multiple of 4. - foreach (dchar c; text) - { - static if (isBigEndian) - c = BEtoMachineDword(c); - else - c = LEtoMachineDword(c); - - if (!isValidChar(c)) - { - infoMan ~= new LexerError( - new Location(filePath, lineNum), - Format(MSG.InvalidUTF32Character, c) - ); - c = REPLACEMENT_CHAR; - } - - if (isNewline(c)) - ++lineNum; - dil.Unicode.encode(result, c); - } - - if (data.length % 4) - infoMan ~= new LexerError( - new Location(filePath, lineNum), - MSG.UTF32FileMustBeDivisibleBy4 - ); - - return result; - } - - alias UTF32toUTF8!(true) UTF32BEtoUTF8; /// Instantiation for UTF-32 BE. - alias UTF32toUTF8!(false) UTF32LEtoUTF8; /// Instantiation for UTF-32 LE. - - /// Converts a UTF-16 text to UTF-8. - char[] UTF16toUTF8(bool isBigEndian)(ubyte[] data) - { - if (data.length == 0) - return null; - - wchar[] text = cast(wchar[]) data[0 .. $-($%2)]; // Trim to multiple of two. - wchar* p = text.ptr, - end = text.ptr + text.length; - char[] result; - uint lineNum = 1; - - for (; p < end; p++) - { - dchar c = *p; - static if (isBigEndian) - c = BEtoMachineWord(c); - else - c = LEtoMachineWord(c); - - if (0xD800 > c || c > 0xDFFF) - {} - else if (c <= 0xDBFF && p+1 < end) - { // Decode surrogate pairs. - wchar c2 = p[1]; - static if (isBigEndian) - c2 = BEtoMachineWord(c2); - else - c2 = LEtoMachineWord(c2); - - if (0xDC00 <= c2 && c2 <= 0xDFFF) - { - c = (c - 0xD7C0) << 10; - c |= (c2 & 0x3FF); - ++p; - } - } - else - { - infoMan ~= new LexerError( - new Location(filePath, lineNum), - Format(MSG.InvalidUTF16Character, c) - ); - c = REPLACEMENT_CHAR; - } - - if (isNewline(c)) - ++lineNum; - dil.Unicode.encode(result, c); - } - - if (data.length % 2) - infoMan ~= new LexerError( - new Location(filePath, lineNum), - MSG.UTF16FileMustBeDivisibleBy2 - ); - return result; - } - - alias UTF16toUTF8!(true) UTF16BEtoUTF8; /// Instantiation for UTF-16 BE. - alias UTF16toUTF8!(false) UTF16LEtoUTF8; /// Instantiation for UTF-16 LE. - - /// Converts the text in data to UTF-8. - /// Leaves data unchanged if it is in UTF-8 already. - char[] data2UTF8(ubyte[] data) - { - if (data.length == 0) - return ""; - - char[] text; - BOM bom = tellBOM(data); - - switch (bom) - { - case BOM.None: - // No BOM found. According to the specs the first character - // must be an ASCII character. - if (data.length >= 4) - { - if (data[0..3] == cast(ubyte[3])x"00 00 00") - { - text = UTF32BEtoUTF8(data); // UTF-32BE: 00 00 00 XX - break; - } - else if (data[1..4] == cast(ubyte[3])x"00 00 00") - { - text = UTF32LEtoUTF8(data); // UTF-32LE: XX 00 00 00 - break; - } - } - if (data.length >= 2) - { - if (data[0] == 0) // UTF-16BE: 00 XX - { - text = UTF16BEtoUTF8(data); - break; - } - else if (data[1] == 0) // UTF-16LE: XX 00 - { - text = UTF16LEtoUTF8(data); - break; - } - } - text = cast(char[])data; // UTF-8 - break; - case BOM.UTF8: - text = cast(char[])data[3..$]; - break; - case BOM.UTF16BE: - text = UTF16BEtoUTF8(data[2..$]); - break; - case BOM.UTF16LE: - text = UTF16LEtoUTF8(data[2..$]); - break; - case BOM.UTF32BE: - text = UTF32BEtoUTF8(data[4..$]); - break; - case BOM.UTF32LE: - text = UTF32LEtoUTF8(data[4..$]); - break; - default: - assert(0); - } - return text; - } -} - -/// Replaces invalid UTF-8 sequences with U+FFFD (if there's enough space,) -/// and Newlines with '\n'. -string sanitizeText(string text) -{ - if (!text.length) - return null; - - char* p = text.ptr; - char* end = p + text.length; - char* q = p; - - for (; p < end; p++, q++) - { - assert(q <= p); - switch (*p) - { - case '\r': - if (p+1 < end && p[1] == '\n') - p++; - case '\n': - *q = '\n'; - continue; - default: - if (isascii(*p)) - break; - if (p+2 < end && isUnicodeNewline(p)) - { - p += 2; - goto case '\n'; - } - auto p2 = p; // Beginning of the UTF-8 sequence. - dchar c = decode(p, end); - if (c == ERROR_CHAR) - { // Skip to next ASCII character or valid UTF-8 sequence. - while (++p < end && isTrailByte(*p)) - {} - alias REPLACEMENT_STR R; - if (q+2 < p) // Copy replacement char if there is enough space. - (*q = R[0]), (*++q = R[1]), (*++q = R[2]); - p--; - } - else - { // Copy the valid UTF-8 sequence. - while (p2 <= p) // p points to the last trail byte. - *q++ = *p2++; // Copy code units. - q--; - } - continue; - } - assert(isascii(*p)); - *q = *p; - } - assert(p == end); - text.length = text.length - (p - q); - //text = text.ptr[0 .. q - text.ptr]; // Another way. - return text; -} - -unittest -{ - Stdout("Testing function Converter.\n"); - struct Data2Text - { - char[] text; - char[] expected = "source"; - ubyte[] data() - { return cast(ubyte[])text; } - } - const Data2Text[] map = [ - // Without BOM - {"source"}, - {"s\0o\0u\0r\0c\0e\0"}, - {"\0s\0o\0u\0r\0c\0e"}, - {"s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e\0\0\0"}, - {"\0\0\0s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e"}, - // With BOM - {"\xEF\xBB\xBFsource"}, - {"\xFE\xFF\0s\0o\0u\0r\0c\0e"}, - {"\xFF\xFEs\0o\0u\0r\0c\0e\0"}, - {"\x00\x00\xFE\xFF\0\0\0s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e"}, - {"\xFF\xFE\x00\x00s\0\0\0o\0\0\0u\0\0\0r\0\0\0c\0\0\0e\0\0\0"}, - ]; - auto converter = Converter("", new InfoManager); - foreach (i, pair; map) - assert(converter.data2UTF8(pair.data) == pair.expected, Format("failed at item {}", i)); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Enums.d --- a/trunk/src/dil/Enums.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,121 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Enums; - -import common; - -/// Enumeration of storage classes. -enum StorageClass -{ - None = 0, - Abstract = 1, - Auto = 1<<2, - Const = 1<<3, - Deprecated = 1<<4, - Extern = 1<<5, - Final = 1<<6, - Invariant = 1<<7, - Override = 1<<8, - Scope = 1<<9, - Static = 1<<10, - Synchronized = 1<<11, - In = 1<<12, - Out = 1<<13, - Ref = 1<<14, - Lazy = 1<<15, - Variadic = 1<<16, -} - -/// Enumeration of protection attributes. -enum Protection -{ - None, - Private/+ = 1+/, - Protected/+ = 1<<1+/, - Package/+ = 1<<2+/, - Public/+ = 1<<3+/, - Export/+ = 1<<4+/ -} - -/// Enumeration of linkage types. -enum LinkageType -{ - None, - C, - Cpp, - D, - Windows, - Pascal, - System -} - -/// Returns the string for prot. -string toString(Protection prot) -{ - switch (prot) - { alias Protection P; - case P.None: return ""; - case P.Private: return "private"; - case P.Protected: return "protected"; - case P.Package: return "package"; - case P.Public: return "public"; - case P.Export: return "export"; - default: - assert(0); - } -} - -/// Returns the string of a storage class. Only one bit may be set. -string toString(StorageClass stc) -{ - switch (stc) - { alias StorageClass SC; - case SC.Abstract: return "abstract"; - case SC.Auto: return "auto"; - case SC.Const: return "const"; - case SC.Deprecated: return "deprecated"; - case SC.Extern: return "extern"; - case SC.Final: return "final"; - case SC.Invariant: return "invariant"; - case SC.Override: return "override"; - case SC.Scope: return "scope"; - case SC.Static: return "static"; - case SC.Synchronized: return "synchronized"; - case SC.In: return "in"; - case SC.Out: return "out"; - case SC.Ref: return "ref"; - case SC.Lazy: return "lazy"; - case SC.Variadic: return "variadic"; - default: - assert(0); - } -} - -/// Returns the strings for stc. Any number of bits may be set. -string[] toStrings(StorageClass stc) -{ - string[] result; - for (auto i = StorageClass.max; i; i >>= 1) - if (stc & i) - result ~= toString(i); - return result; -} - -/// Returns the string for ltype. -string toString(LinkageType ltype) -{ - switch (ltype) - { alias LinkageType LT; - case LT.None: return ""; - case LT.C: return "C"; - case LT.Cpp: return "Cpp"; - case LT.D: return "D"; - case LT.Windows: return "Windows"; - case LT.Pascal: return "Pascal"; - case LT.System: return "System"; - default: - assert(0); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/FileBOM.d --- a/trunk/src/dil/FileBOM.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,86 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.FileBOM; -import common; - -/// Enumeration of byte order marks. -enum BOM -{ - None, /// No BOM - UTF8, /// UTF-8: EF BB BF - UTF16BE, /// UTF-16 Big Endian: FE FF - UTF16LE, /// UTF-16 Little Endian: FF FE - UTF32BE, /// UTF-32 Big Endian: 00 00 FE FF - UTF32LE /// UTF-32 Little Endian: FF FE 00 00 -} - -/// Looks at the first bytes of data and returns the corresponding BOM. -BOM tellBOM(ubyte[] data) -{ - BOM bom = BOM.None; - if (data.length < 2) - return bom; - - if (data[0..2] == cast(ubyte[2])x"FE FF") - { - bom = BOM.UTF16BE; // FE FF - } - else if (data[0..2] == cast(ubyte[2])x"FF FE") - { - if (data.length >= 4 && data[2..4] == cast(ubyte[2])x"00 00") - bom = BOM.UTF32LE; // FF FE 00 00 - else - bom = BOM.UTF16LE; // FF FE XX XX - } - else if (data[0..2] == cast(ubyte[2])x"00 00") - { - if (data.length >= 4 && data[2..4] == cast(ubyte[2])x"FE FF") - bom = BOM.UTF32BE; // 00 00 FE FF - } - else if (data[0..2] == cast(ubyte[2])x"EF BB") - { - if (data.length >= 3 && data[2] == '\xBF') - bom = BOM.UTF8; // EF BB BF - } - return bom; -} - -unittest -{ - Stdout("Testing function tellBOM().\n"); - - struct Data2BOM - { - ubyte[] data; - BOM bom; - } - alias ubyte[] ub; - const Data2BOM[] map = [ - {cast(ub)x"12", BOM.None}, - {cast(ub)x"12 34", BOM.None}, - {cast(ub)x"00 00 FF FE", BOM.None}, - {cast(ub)x"EF BB FF", BOM.None}, - - {cast(ub)x"EF", BOM.None}, - {cast(ub)x"EF BB", BOM.None}, - {cast(ub)x"FE", BOM.None}, - {cast(ub)x"FF", BOM.None}, - {cast(ub)x"00", BOM.None}, - {cast(ub)x"00 00", BOM.None}, - {cast(ub)x"00 00 FE", BOM.None}, - - {cast(ub)x"FE FF 00", BOM.UTF16BE}, - {cast(ub)x"FE FF 00 FF", BOM.UTF16BE}, - - {cast(ub)x"EF BB BF", BOM.UTF8}, - {cast(ub)x"FE FF", BOM.UTF16BE}, - {cast(ub)x"FF FE", BOM.UTF16LE}, - {cast(ub)x"00 00 FE FF", BOM.UTF32BE}, - {cast(ub)x"FF FE 00 00", BOM.UTF32LE} - ]; - - foreach (pair; map) - assert(tellBOM(pair.data) == pair.bom, Format("Failed at {0}", pair.data)); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/HtmlEntities.d --- a/trunk/src/dil/HtmlEntities.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,376 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.HtmlEntities; - -import common; - -/// A named HTML entity. -struct Entity -{ - char[] name; - dchar value; -} - -/// The table of named HTML entities. -static const Entity[] namedEntities = [ - {"Aacute", '\u00C1'}, - {"aacute", '\u00E1'}, - {"Acirc", '\u00C2'}, - {"acirc", '\u00E2'}, - {"acute", '\u00B4'}, - {"AElig", '\u00C6'}, - {"aelig", '\u00E6'}, - {"Agrave", '\u00C0'}, - {"agrave", '\u00E0'}, - {"alefsym", '\u2135'}, - {"Alpha", '\u0391'}, - {"alpha", '\u03B1'}, - {"amp", '\u0026'}, - {"and", '\u2227'}, - {"ang", '\u2220'}, - {"Aring", '\u00C5'}, - {"aring", '\u00E5'}, - {"asymp", '\u2248'}, - {"Atilde", '\u00C3'}, - {"atilde", '\u00E3'}, - {"Auml", '\u00C4'}, - {"auml", '\u00E4'}, - {"bdquo", '\u201E'}, - {"Beta", '\u0392'}, - {"beta", '\u03B2'}, - {"brvbar", '\u00A6'}, - {"bull", '\u2022'}, - {"cap", '\u2229'}, - {"Ccedil", '\u00C7'}, - {"ccedil", '\u00E7'}, - {"cedil", '\u00B8'}, - {"cent", '\u00A2'}, - {"Chi", '\u03A7'}, - {"chi", '\u03C7'}, - {"circ", '\u02C6'}, - {"clubs", '\u2663'}, - {"cong", '\u2245'}, - {"copy", '\u00A9'}, - {"crarr", '\u21B5'}, - {"cup", '\u222A'}, - {"curren", '\u00A4'}, - {"Dagger", '\u2021'}, - {"dagger", '\u2020'}, - {"dArr", '\u21D3'}, - {"darr", '\u2193'}, - {"deg", '\u00B0'}, - {"Delta", '\u0394'}, - {"delta", '\u03B4'}, - {"diams", '\u2666'}, - {"divide", '\u00F7'}, - {"Eacute", '\u00C9'}, - {"eacute", '\u00E9'}, - {"Ecirc", '\u00CA'}, - {"ecirc", '\u00EA'}, - {"Egrave", '\u00C8'}, - {"egrave", '\u00E8'}, - {"empty", '\u2205'}, - {"emsp", '\u2003'}, - {"ensp", '\u2002'}, - {"Epsilon", '\u0395'}, - {"epsilon", '\u03B5'}, - {"equiv", '\u2261'}, - {"Eta", '\u0397'}, - {"eta", '\u03B7'}, - {"ETH", '\u00D0'}, - {"eth", '\u00F0'}, - {"Euml", '\u00CB'}, - {"euml", '\u00EB'}, - {"euro", '\u20AC'}, - {"exist", '\u2203'}, - {"fnof", '\u0192'}, - {"forall", '\u2200'}, - {"frac12", '\u00BD'}, - {"frac14", '\u00BC'}, - {"frac34", '\u00BE'}, - {"frasl", '\u2044'}, - {"Gamma", '\u0393'}, - {"gamma", '\u03B3'}, - {"ge", '\u2265'}, - {"gt", '\u003E'}, - {"hArr", '\u21D4'}, - {"harr", '\u2194'}, - {"hearts", '\u2665'}, - {"hellip", '\u2026'}, - {"Iacute", '\u00CD'}, - {"iacute", '\u00ED'}, - {"Icirc", '\u00CE'}, - {"icirc", '\u00EE'}, - {"iexcl", '\u00A1'}, - {"Igrave", '\u00CC'}, - {"igrave", '\u00EC'}, - {"image", '\u2111'}, - {"infin", '\u221E'}, - {"int", '\u222B'}, - {"Iota", '\u0399'}, - {"iota", '\u03B9'}, - {"iquest", '\u00BF'}, - {"isin", '\u2208'}, - {"Iuml", '\u00CF'}, - {"iuml", '\u00EF'}, - {"Kappa", '\u039A'}, - {"kappa", '\u03BA'}, - {"Lambda", '\u039B'}, - {"lambda", '\u03BB'}, - {"lang", '\u2329'}, - {"laquo", '\u00AB'}, - {"lArr", '\u21D0'}, - {"larr", '\u2190'}, - {"lceil", '\u2308'}, - {"ldquo", '\u201C'}, - {"le", '\u2264'}, - {"lfloor", '\u230A'}, - {"lowast", '\u2217'}, - {"loz", '\u25CA'}, - {"lrm", '\u200E'}, - {"lsaquo", '\u2039'}, - {"lsquo", '\u2018'}, - {"lt", '\u003C'}, - {"macr", '\u00AF'}, - {"mdash", '\u2014'}, - {"micro", '\u00B5'}, - {"middot", '\u00B7'}, - {"minus", '\u2212'}, - {"Mu", '\u039C'}, - {"mu", '\u03BC'}, - {"nabla", '\u2207'}, - {"nbsp", '\u00A0'}, - {"ndash", '\u2013'}, - {"ne", '\u2260'}, - {"ni", '\u220B'}, - {"not", '\u00AC'}, - {"notin", '\u2209'}, - {"nsub", '\u2284'}, - {"Ntilde", '\u00D1'}, - {"ntilde", '\u00F1'}, - {"Nu", '\u039D'}, - {"nu", '\u03BD'}, - {"Oacute", '\u00D3'}, - {"oacute", '\u00F3'}, - {"Ocirc", '\u00D4'}, - {"ocirc", '\u00F4'}, - {"OElig", '\u0152'}, - {"oelig", '\u0153'}, - {"Ograve", '\u00D2'}, - {"ograve", '\u00F2'}, - {"oline", '\u203E'}, - {"Omega", '\u03A9'}, - {"omega", '\u03C9'}, - {"Omicron", '\u039F'}, - {"omicron", '\u03BF'}, - {"oplus", '\u2295'}, - {"or", '\u2228'}, - {"ordf", '\u00AA'}, - {"ordm", '\u00BA'}, - {"Oslash", '\u00D8'}, - {"oslash", '\u00F8'}, - {"Otilde", '\u00D5'}, - {"otilde", '\u00F5'}, - {"otimes", '\u2297'}, - {"Ouml", '\u00D6'}, - {"ouml", '\u00F6'}, - {"para", '\u00B6'}, - {"part", '\u2202'}, - {"permil", '\u2030'}, - {"perp", '\u22A5'}, - {"Phi", '\u03A6'}, - {"phi", '\u03C6'}, - {"Pi", '\u03A0'}, - {"pi", '\u03C0'}, - {"piv", '\u03D6'}, - {"plusmn", '\u00B1'}, - {"pound", '\u00A3'}, - {"Prime", '\u2033'}, - {"prime", '\u2032'}, - {"prod", '\u220F'}, - {"prop", '\u221D'}, - {"Psi", '\u03A8'}, - {"psi", '\u03C8'}, - {"quot", '\u0022'}, - {"radic", '\u221A'}, - {"rang", '\u232A'}, - {"raquo", '\u00BB'}, - {"rArr", '\u21D2'}, - {"rarr", '\u2192'}, - {"rceil", '\u2309'}, - {"rdquo", '\u201D'}, - {"real", '\u211C'}, - {"reg", '\u00AE'}, - {"rfloor", '\u230B'}, - {"Rho", '\u03A1'}, - {"rho", '\u03C1'}, - {"rlm", '\u200F'}, - {"rsaquo", '\u203A'}, - {"rsquo", '\u2019'}, - {"sbquo", '\u201A'}, - {"Scaron", '\u0160'}, - {"scaron", '\u0161'}, - {"sdot", '\u22C5'}, - {"sect", '\u00A7'}, - {"shy", '\u00AD'}, - {"Sigma", '\u03A3'}, - {"sigma", '\u03C3'}, - {"sigmaf", '\u03C2'}, - {"sim", '\u223C'}, - {"spades", '\u2660'}, - {"sub", '\u2282'}, - {"sube", '\u2286'}, - {"sum", '\u2211'}, - {"sup", '\u2283'}, - {"sup1", '\u00B9'}, - {"sup2", '\u00B2'}, - {"sup3", '\u00B3'}, - {"supe", '\u2287'}, - {"szlig", '\u00DF'}, - {"Tau", '\u03A4'}, - {"tau", '\u03C4'}, - {"there4", '\u2234'}, - {"Theta", '\u0398'}, - {"theta", '\u03B8'}, - {"thetasym", '\u03D1'}, - {"thinsp", '\u2009'}, - {"THORN", '\u00DE'}, - {"thorn", '\u00FE'}, - {"tilde", '\u02DC'}, - {"times", '\u00D7'}, - {"trade", '\u2122'}, - {"Uacute", '\u00DA'}, - {"uacute", '\u00FA'}, - {"uArr", '\u21D1'}, - {"uarr", '\u2191'}, - {"Ucirc", '\u00DB'}, - {"ucirc", '\u00FB'}, - {"Ugrave", '\u00D9'}, - {"ugrave", '\u00F9'}, - {"uml", '\u00A8'}, - {"upsih", '\u03D2'}, - {"Upsilon", '\u03A5'}, - {"upsilon", '\u03C5'}, - {"Uuml", '\u00DC'}, - {"uuml", '\u00FC'}, - {"weierp", '\u2118'}, - {"Xi", '\u039E'}, - {"xi", '\u03BE'}, - {"Yacute", '\u00DD'}, - {"yacute", '\u00FD'}, - {"yen", '\u00A5'}, - {"Yuml", '\u0178'}, - {"yuml", '\u00FF'}, - {"Zeta", '\u0396'}, - {"zeta", '\u03B6'}, - {"zwj", '\u200D'}, - {"zwnj", '\u200C'} -]; - -uint stringToHash(char[] str) -{ - uint hash; - foreach(c; str) { - hash *= 11; - hash += c; - } - return hash; -} - -char[] toString(uint x) -{ - char[] str; - do - str = cast(char)('0' + (x % 10)) ~ str; - while (x /= 10) - return str; -} - -char[] generateHashAndValueArrays() -{ - uint[] hashes; // String hashes. - dchar[] values; // Unicode codepoints. - // Build arrays: - foreach (entity; namedEntities) - { - auto hash = stringToHash(entity.name); - auto value = entity.value; - assert(hash != 0); - // Find insertion place. - uint i; - for (; i < hashes.length; ++i) - { - assert(hash != hashes[i], "bad hash function: conflicting hashes"); - if (hash < hashes[i]) - break; - } - // Insert hash and value into tables. - if (i == hashes.length) - { - hashes ~= hash; - values ~= value; - } - else - { - hashes = hashes[0..i] ~ hash ~ hashes[i..$]; // Insert before index. - values = values[0..i] ~ value ~ values[i..$]; // Insert before index. - } - assert(hashes[i] == hash && values[i] == value); - } - // Build source text: - char[] hashesText = "private static const uint[] hashes = [", - valuesText = "private static const dchar[] values = ["; - foreach (i, hash; hashes) - { - hashesText ~= toString(hash) ~ ","; - valuesText ~= toString(values[i]) ~ ","; - } - hashesText ~= "];"; - valuesText ~= "];"; - return hashesText ~"\n"~ valuesText; -} - -version(DDoc) -{ - /// Table of hash values of the entities' names. - private static const uint[] hashes; - /// Table of Unicode codepoints. - private static const dchar[] values; -} -else - mixin(generateHashAndValueArrays); -// pragma(msg, generateHashAndValueArrays()); - -/// Converts a named HTML entity into its equivalent Unicode codepoint. -/// Returns: the entity's value or 0xFFFF if it doesn't exist. -dchar entity2Unicode(char[] entity) -{ - auto hash = stringToHash(entity); - // Binary search: - size_t lower = void, index = void, upper = void; - lower = 0; - upper = hashes.length -1; - while (lower <= upper) - { - index = (lower + upper) / 2; - if (hash < hashes[index]) - upper = index - 1; - else if (hash > hashes[index]) - lower = index + 1; - else - return values[index]; // Return the Unicode codepoint. - } - return 0xFFFF; // Return error value. -} - -unittest -{ - Stdout("Testing entity2Unicode().").newline; - alias entity2Unicode f; - foreach (entity; namedEntities) - assert(f(entity.name) == entity.value, - Format("'&{};' == \\u{:X4}, not \\u{:X4}", entity.name, entity.value, cast(uint)f(entity.name)) - ); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Information.d --- a/trunk/src/dil/Information.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,123 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Information; - -import dil.Messages; -import common; - -public import dil.Location; - -/// Information that can be displayed to the user. -class Information -{ - -} - -/// Collects information. -class InfoManager -{ - Information[] info; - - bool hasInfo() - { - return info.length != 0; - } - - void opCatAssign(Information info) - { - this.info ~= info; - } - - void opCatAssign(Information[] info) - { - this.info ~= info; - } -} - -/// For reporting a problem in the compilation process. -class Problem : Information -{ - Location location; - uint column; /// Cache variable for column. - string message; - - this(Location location, string message) - { - assert(location !is null); - this.location = location; - this.message = message; - } - - /// Returns the message. - string getMsg() - { - return this.message; - } - - /// Returns the line of code. - size_t loc() - { - return location.lineNum; - } - - /// Returns the column. - size_t col() - { - if (column == 0) - column = location.calculateColumn(); - return column; - } - - /// Returns the file path. - string filePath() - { - return location.filePath; - } -} - -/// For reporting warnings. -class Warning : Problem -{ - this(Location location, string message) - { - super(location, message); - } -} - -/// For reporting a compiler error. -class Error : Problem -{ - this(Location location, string message) - { - super(location, message); - } -} - -/// An error reported by the Lexer. -class LexerError : Error -{ - this(Location location, string message) - { - super(location, message); - } -} - -/// An error reported by the Parser. -class ParserError : Error -{ - this(Location location, string message) - { - super(location, message); - } -} - -/// An error reported by a semantic analyzer. -class SemanticError : Error -{ - this(Location location, string message) - { - super(location, message); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Location.d --- a/trunk/src/dil/Location.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,85 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Location; - -import dil.lexer.Funcs; -import dil.Unicode; - -/// Represents a location in a source text. -final class Location -{ - char[] filePath; /// The file path. - size_t lineNum; /// The line number. - char* lineBegin, to; /// Used to calculate the column. - - /// Forwards the parameters to the second constructor. - this(char[] filePath, size_t lineNum) - { - set(filePath, lineNum); - } - - /// Constructs a Location object. - this(char[] filePath, size_t lineNum, char* lineBegin, char* to) - { - set(filePath, lineNum, lineBegin, to); - } - - void set(char[] filePath, size_t lineNum) - { - set(filePath, lineNum, null, null); - } - - void set(char[] filePath, size_t lineNum, char* lineBegin, char* to) - { - this.filePath = filePath; - set(lineNum, lineBegin, to); - } - - void set(size_t lineNum, char* lineBegin, char* to) - { - assert(lineBegin <= to); - this.lineNum = lineNum; - this.lineBegin = lineBegin; - this.to = to; - } - - void setFilePath(char[] filePath) - { - this.filePath = filePath; - } - - /// This is a primitive method to count the number of characters in a string. - /// Note: Unicode compound characters and other special characters are not - /// taken into account. The tabulator character is counted as one. - uint calculateColumn() - { - uint col; - auto p = lineBegin; - if (!p) - return 0; - for (; p <= to; ++p) - { - assert(delegate () - { - // Check that there is no newline between p and to. - // But 'to' may point to a newline. - if (p != to && isNewline(*p)) - return false; - if (to-p >= 2 && isUnicodeNewline(p)) - return false; - return true; - }() == true - ); - - // Skip this byte if it is a trail byte of a UTF-8 sequence. - if (isTrailByte(*p)) - continue; // *p == 0b10xx_xxxx - // Only count ASCII characters and the first byte of a UTF-8 sequence. - ++col; - } - return col; - } - alias calculateColumn colNum; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Messages.d --- a/trunk/src/dil/Messages.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,164 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Messages; - -import common; - -/// Enumeration of indices into the table of compiler messages. -enum MID -{ - // Lexer messages: - IllegalCharacter, -// InvalidUnicodeCharacter, - InvalidUTF8Sequence, - // '' - UnterminatedCharacterLiteral, - EmptyCharacterLiteral, - // #line - ExpectedIdentifierSTLine, - ExpectedIntegerAfterSTLine, -// ExpectedFilespec, - UnterminatedFilespec, - UnterminatedSpecialToken, - // "" - UnterminatedString, - // x"" - NonHexCharInHexString, - OddNumberOfDigitsInHexString, - UnterminatedHexString, - // /* */ /+ +/ - UnterminatedBlockComment, - UnterminatedNestedComment, - // `` r"" - UnterminatedRawString, - UnterminatedBackQuoteString, - // \x \u \U - UndefinedEscapeSequence, - InvalidUnicodeEscapeSequence, - InsufficientHexDigits, - // \&[a-zA-Z][a-zA-Z0-9]+; - UndefinedHTMLEntity, - UnterminatedHTMLEntity, - InvalidBeginHTMLEntity, - // integer overflows - OverflowDecimalSign, - OverflowDecimalNumber, - OverflowHexNumber, - OverflowBinaryNumber, - OverflowOctalNumber, - OverflowFloatNumber, - OctalNumberHasDecimals, - NoDigitsInHexNumber, - NoDigitsInBinNumber, - HexFloatExponentRequired, - HexFloatExpMustStartWithDigit, - FloatExpMustStartWithDigit, - - // Parser messages: - ExpectedButFound, - RedundantStorageClass, - TemplateTupleParameter, - InContract, - OutContract, - MissingLinkageType, - UnrecognizedLinkageType, - ExpectedBaseClasses, - BaseClassInForwardDeclaration, - - // Help messages: - HelpMain, - HelpGenerate, - HelpImportGraph, -} - -/// The table of compiler messages. -private string[] g_compilerMessages; - -static this() -{ - g_compilerMessages = new string[MID.max+1]; -} - -/// Sets the compiler messages. -void SetMessages(string[] msgs) -{ - assert(MID.max+1 == msgs.length); - g_compilerMessages = msgs; -} - -/// Returns the compiler message for mid. -string GetMsg(MID mid) -{ - assert(mid < g_compilerMessages.length); - return g_compilerMessages[mid]; -} - -/// Returns a formatted string. -char[] FormatMsg(MID mid, ...) -{ - return Format(_arguments, _argptr, GetMsg(mid)); -} - -/// Collection of error messages with no MID yet. -struct MSG -{ -static: - // Converter: - auto InvalidUTF16Character = "invalid UTF-16 character '\\u{:X4}'."; - auto InvalidUTF32Character = "invalid UTF-32 character '\\U{:X8}'."; - auto UTF16FileMustBeDivisibleBy2 = "the byte length of a UTF-16 source file must be divisible by 2."; - auto UTF32FileMustBeDivisibleBy4 = "the byte length of a UTF-32 source file must be divisible by 4."; - // DDoc macros: - auto UndefinedDDocMacro = "DDoc macro '{}' is undefined"; - auto UnterminatedDDocMacro = "DDoc macro '{}' has no closing ')'"; - // Lexer messages: - auto InvalidOctalEscapeSequence = "value of octal escape sequence is greater than 0xFF: '{}'"; - // Parser messages: - auto InvalidUTF8SequenceInString = "invalid UTF-8 sequence in string literal: '{0}'"; - auto ModuleDeclarationNotFirst = "a module declaration is only allowed as the first declaration in a file"; - auto StringPostfixMismatch = "string literal has mistmatching postfix character"; - auto ExpectedIdAfterTypeDot = "expected identifier after '(Type).', not '{}'"; - auto ExpectedModuleIdentifier = "expected module identifier, not '{}'"; - auto IllegalDeclaration = "illegal declaration found: {}"; - auto ExpectedFunctionName = "expected function name, not '{}'"; - auto ExpectedVariableName = "expected variable name, not '{}'"; - auto ExpectedFunctionBody = "expected function body, not '{}'"; - auto RedundantLinkageType = "redundant linkage type: {}"; - auto ExpectedPragmaIdentifier = "expected pragma identifier, not '{}'"; - auto ExpectedAliasModuleName = "expected alias module name, not '{}'"; - auto ExpectedAliasImportName = "expected alias name, not '{}'"; - auto ExpectedImportName = "expected an identifier, not '{}'"; - auto ExpectedEnumMember = "expected enum member, not '{}'"; - auto ExpectedEnumBody = "expected enum body, not '{}'"; - auto ExpectedClassName = "expected class name, not '{}'"; - auto ExpectedClassBody = "expected class body, not '{}'"; - auto ExpectedInterfaceName = "expected interface name, not '{}'"; - auto ExpectedInterfaceBody = "expected interface body, not '{}'"; - auto ExpectedStructBody = "expected struct body, not '{}'"; - auto ExpectedUnionBody = "expected union body, not '{}'"; - auto ExpectedTemplateName = "expected template name, not '{}'"; - auto ExpectedAnIdentifier = "expected an identifier, not '{}'"; - auto IllegalStatement = "illegal statement found: {}"; - auto ExpectedNonEmptyStatement = "didn't expect ';', use {{ } instead"; - auto ExpectedScopeIdentifier = "expected 'exit', 'success' or 'failure', not '{}'"; - auto InvalidScopeIdentifier = "'exit', 'success', 'failure' are valid scope identifiers, but not '{}'"; - auto ExpectedIntegerAfterAlign = "expected an integer after align, not '{}'"; - auto IllegalAsmStatement = "illegal asm statement found: {}"; - auto ExpectedDeclaratorIdentifier = "expected declarator identifier, not '{}'"; - auto ExpectedTemplateParameters = "expected one or more template parameters, not ')'"; - auto ExpectedTypeOrExpression = "expected a type or and expression, not ')'"; - auto ExpectedAliasTemplateParam = "expected name for alias template parameter, not '{}'"; - auto ExpectedNameForThisTempParam = "expected name for 'this' template parameter, not '{}'"; - auto ExpectedIdentOrInt = "expected an identifier or an integer, not '{}'"; - auto MissingCatchOrFinally = "try statement is missing a catch or finally body."; - // Semantic analysis: - auto UndefinedIdentifier = "undefined identifier '{}'"; - auto DeclConflictsWithDecl = "declaration '{}' conflicts with declaration @{}"; - auto VariableConflictsWithDecl = "variable '{}' conflicts with declaration @{}"; - auto InterfaceCantHaveVariables = "an interface can't have member variables"; - auto MixinArgumentMustBeString = "the mixin argument must evaluate to a string"; - auto DebugSpecModuleLevel = "debug={} must be a module level"; - auto VersionSpecModuleLevel = "version={} must be a module level"; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/SourceText.d --- a/trunk/src/dil/SourceText.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,62 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.SourceText; - -import dil.Converter; -import dil.Information; -import common; - -import tango.io.File; - -/// Represents D source code. -/// -/// The source text may come from a file or from a memory buffer. -final class SourceText -{ - string filePath; /// The file path to the source text. Mainly used for error messages. - char[] data; /// The UTF-8, zero-terminated source text. - - /// Constructs a SourceText object. - /// Params: - /// filePath = file path to the source file. - /// loadFile = whether to load the file in the constructor. - this(string filePath, bool loadFile = false) - { - this.filePath = filePath; - loadFile && load(); - } - - /// Constructs a SourceText object. - /// Params: - /// filePath = file path for error messages. - /// data = memory buffer. - this(string filePath, char[] data) - { - this(filePath); - this.data = data; - addSentinelCharacter(); - } - - /// Loads the source text from a file. - void load(InfoManager infoMan = null) - { - if (!infoMan) - infoMan = new InfoManager; - assert(filePath.length); - // Read the file. - auto rawdata = cast(ubyte[]) (new File(filePath)).read(); - // Convert the data. - auto converter = Converter(filePath, infoMan); - data = converter.data2UTF8(rawdata); - addSentinelCharacter(); - } - - /// Adds '\0' to the text (if not already there.) - private void addSentinelCharacter() - { - if (data.length == 0 || data[$-1] != 0) - data ~= 0; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Time.d --- a/trunk/src/dil/Time.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,41 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Time; - -import tango.stdc.time : time_t, time, ctime; -import tango.stdc.string : strlen; - -/// Some convenience functions for dealing with C's time functions. -struct Time -{ -static: - /// Returns the current date as a string. - char[] toString() - { - time_t time_val; - .time(&time_val); - char* str = ctime(&time_val); // ctime returns a pointer to a static array. - char[] timeStr = str[0 .. strlen(str)-1]; // -1 removes trailing '\n'. - return timeStr.dup; - } - - /// Returns the time of timeStr: hh:mm:ss - char[] time(char[] timeStr) - { - return timeStr[11..19]; - } - - /// Returns the month and day of timeStr: Mmm dd - char[] month_day(char[] timeStr) - { - return timeStr[4..10]; - } - - /// Returns the year of timeStr: yyyy - char[] year(char[] timeStr) - { - return timeStr[20..24]; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/Unicode.d --- a/trunk/src/dil/Unicode.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,345 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.Unicode; -public import util.uni : isUniAlpha; - -/// U+FFFD = �. Used to replace invalid Unicode characters. -const dchar REPLACEMENT_CHAR = '\uFFFD'; -const char[3] REPLACEMENT_STR = \uFFFD; /// Ditto -/// Invalid character, returned on errors. -const dchar ERROR_CHAR = 0xD800; - -/// Returns: true if this character is not a surrogate -/// code point and not higher than 0x10FFFF. -bool isValidChar(dchar d) -{ - return d < 0xD800 || d > 0xDFFF && d <= 0x10FFFF; -} - -/// There are a total of 66 noncharacters. -/// Returns: true if this is one of them. -/// See_also: Chapter 16.7 Noncharacters in Unicode 5.0 -bool isNoncharacter(dchar d) -{ - return 0xFDD0 <= d && d <= 0xFDEF || // 32 - d <= 0x10FFFF && (d & 0xFFFF) >= 0xFFFE; // 34 -} - -/// Returns: true if this is a trail byte of a UTF-8 sequence. -bool isTrailByte(ubyte b) -{ - return (b & 0xC0) == 0x80; // 10xx_xxxx -} - -/// Returns: true if this is a lead byte of a UTF-8 sequence. -bool isLeadByte(ubyte b) -{ - return (b & 0xC0) == 0xC0; // 11xx_xxxx -} - -/// Advances ref_p only if this is a valid Unicode alpha character. -bool isUnicodeAlpha(ref char* ref_p, char* end) -in { assert(ref_p && ref_p < end); } -body -{ - if (*ref_p < 0x80) - return false; - auto p = ref_p; - auto c = decode(p, end); - if (!isUniAlpha(c)) - return false; - ref_p = p; - return true; -} - -/// Decodes a character from str at index. -/// Params: -/// index = set to one past the ASCII char or one past the last trail byte -/// of the valid UTF-8 sequence. -dchar decode(char[] str, ref size_t index) -in { assert(str.length && index < str.length); } -out { assert(index <= str.length); } -body -{ - char* p = str.ptr + index; - char* end = str.ptr + str.length; - dchar c = decode(p, end); - if (c != ERROR_CHAR) - index = p - str.ptr; - return c; -} - -/// Decodes a character starting at ref_p. -/// Params: -/// ref_p = set to one past the ASCII char or one past the last trail byte -/// of the valid UTF-8 sequence. -dchar decode(ref char* ref_p, char* end) -in { assert(ref_p && ref_p < end); } -out(c) { assert(ref_p <= end && (isValidChar(c) || c == ERROR_CHAR)); } -body -{ - char* p = ref_p; - dchar c = *p; - - if (c < 0x80) - return ref_p++, c; - - p++; // Move to second byte. - if (!(p < end)) - return ERROR_CHAR; - - // Error if second byte is not a trail byte. - if (!isTrailByte(*p)) - return ERROR_CHAR; - - // Check for overlong sequences. - switch (c) - { - case 0xE0, // 11100000 100xxxxx - 0xF0, // 11110000 1000xxxx - 0xF8, // 11111000 10000xxx - 0xFC: // 11111100 100000xx - if ((*p & c) == 0x80) - return ERROR_CHAR; - default: - if ((c & 0xFE) == 0xC0) // 1100000x - return ERROR_CHAR; - } - - const char[] checkNextByte = "if (!(++p < end && isTrailByte(*p)))" - " return ERROR_CHAR;"; - const char[] appendSixBits = "c = (c << 6) | *p & 0b0011_1111;"; - - // Decode - if ((c & 0b1110_0000) == 0b1100_0000) - { - // 110xxxxx 10xxxxxx - c &= 0b0001_1111; - mixin(appendSixBits); - } - else if ((c & 0b1111_0000) == 0b1110_0000) - { - // 1110xxxx 10xxxxxx 10xxxxxx - c &= 0b0000_1111; - mixin(appendSixBits ~ - checkNextByte ~ appendSixBits); - } - else if ((c & 0b1111_1000) == 0b1111_0000) - { - // 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx - c &= 0b0000_0111; - mixin(appendSixBits ~ - checkNextByte ~ appendSixBits ~ - checkNextByte ~ appendSixBits); - } - else - // 5 and 6 byte UTF-8 sequences are not allowed yet. - // 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx - // 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx - return ERROR_CHAR; - - assert(isTrailByte(*p)); - - if (!isValidChar(c)) - return ERROR_CHAR; - ref_p = p+1; - return c; -} - -/// Encodes c and appends it to str. -void encode(ref char[] str, dchar c) -{ - assert(isValidChar(c), "check if character is valid before calling encode()."); - - char[6] b = void; - if (c < 0x80) - str ~= c; - else if (c < 0x800) - { - b[0] = 0xC0 | (c >> 6); - b[1] = 0x80 | (c & 0x3F); - str ~= b[0..2]; - } - else if (c < 0x10000) - { - b[0] = 0xE0 | (c >> 12); - b[1] = 0x80 | ((c >> 6) & 0x3F); - b[2] = 0x80 | (c & 0x3F); - str ~= b[0..3]; - } - else if (c < 0x200000) - { - b[0] = 0xF0 | (c >> 18); - b[1] = 0x80 | ((c >> 12) & 0x3F); - b[2] = 0x80 | ((c >> 6) & 0x3F); - b[3] = 0x80 | (c & 0x3F); - str ~= b[0..4]; - } - /+ // There are no 5 and 6 byte UTF-8 sequences yet. - else if (c < 0x4000000) - { - b[0] = 0xF8 | (c >> 24); - b[1] = 0x80 | ((c >> 18) & 0x3F); - b[2] = 0x80 | ((c >> 12) & 0x3F); - b[3] = 0x80 | ((c >> 6) & 0x3F); - b[4] = 0x80 | (c & 0x3F); - str ~= b[0..5]; - } - else if (c < 0x80000000) - { - b[0] = 0xFC | (c >> 30); - b[1] = 0x80 | ((c >> 24) & 0x3F); - b[2] = 0x80 | ((c >> 18) & 0x3F); - b[3] = 0x80 | ((c >> 12) & 0x3F); - b[4] = 0x80 | ((c >> 6) & 0x3F); - b[5] = 0x80 | (c & 0x3F); - str ~= b[0..6]; - } - +/ - else - assert(0); -} - -/// Encodes c and appends it to str. -void encode(ref wchar[] str, dchar c) -in { assert(isValidChar(c)); } -body -{ - if (c < 0x10000) - str ~= cast(wchar)c; - else - { // Encode with surrogate pair. - wchar[2] pair = void; - c -= 0x10000; // c' - // higher10bits(c') | 0b1101_10xx_xxxx_xxxx - pair[0] = (c >> 10) | 0xD800; - // lower10bits(c') | 0b1101_11yy_yyyy_yyyy - pair[1] = (c & 0x3FF) | 0xDC00; - str ~= pair; - } -} - -/// Decodes a character from a UTF-16 sequence. -/// Params: -/// str = the UTF-16 sequence. -/// index = where to start from. -/// Returns: ERROR_CHAR in case of an error in the sequence. -dchar decode(wchar[] str, ref size_t index) -{ - assert(str.length && index < str.length); - dchar c = str[index]; - if (0xD800 > c || c > 0xDFFF) - { - ++index; - return c; - } - if (c <= 0xDBFF && index+1 != str.length) - { - wchar c2 = str[index+1]; - if (0xDC00 <= c2 && c2 <= 0xDFFF) - { // Decode surrogate pair. - // (c - 0xD800) << 10 + 0x10000 -> - // (c - 0xD800 + 0x40) << 10 -> - c = (c - 0xD7C0) << 10; - c |= (c2 & 0x3FF); - index += 2; - return c; - } - } - return ERROR_CHAR; -} - -/// Decodes a character from a UTF-16 sequence. -/// Params: -/// p = start of the UTF-16 sequence. -/// end = one past the end of the sequence. -/// Returns: ERROR_CHAR in case of an error in the sequence. -dchar decode(ref wchar* p, wchar* end) -{ - assert(p && p < end); - dchar c = *p; - if (0xD800 > c || c > 0xDFFF) - { - ++p; - return c; - } - if (c <= 0xDBFF && p+1 != end) - { - wchar c2 = p[1]; - if (0xDC00 <= c2 && c2 <= 0xDFFF) - { - c = (c - 0xD7C0) << 10; - c |= (c2 & 0x3FF); - p += 2; - return c; - } - } - return ERROR_CHAR; -} - -/// Decodes a character from a zero-terminated UTF-16 string. -/// Params: -/// p = start of the UTF-16 sequence. -/// Returns: ERROR_CHAR in case of an error in the sequence. -dchar decode(ref wchar* p) -{ - assert(p); - dchar c = *p; - if (0xD800 > c || c > 0xDFFF) - { - ++p; - return c; - } - if (c <= 0xDBFF) - { - wchar c2 = p[1]; - if (0xDC00 <= c2 && c2 <= 0xDFFF) - { - c = (c - 0xD7C0) << 10; - c |= (c2 & 0x3FF); - p += 2; - return c; - } - } - return ERROR_CHAR; -} - -/// Converts a UTF-8 string to a UTF-16 string. -wchar[] toUTF16(char[] str) -{ - wchar[] result; - size_t idx; - while (idx < str.length) - { - auto c = decode(str, idx); - if (c == ERROR_CHAR) - { // Skip trail bytes. - while (++idx < str.length && isTrailByte(str[idx])) - {} - c = REPLACEMENT_CHAR; - } - encode(result, c); - } - return result; -} - -/// Converts a UTF-8 string to a UTF-32 string. -dchar[] toUTF32(char[] str) -{ - dchar[] result; - size_t idx; - while (idx < str.length) - { - auto c = decode(str, idx); - if (c == ERROR_CHAR) - { // Skip trail bytes. - while (++idx < str.length && isTrailByte(str[idx])) - {} - c = REPLACEMENT_CHAR; - } - result ~= c; - } - return result; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Declaration.d --- a/trunk/src/dil/ast/Declaration.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,44 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Declaration; - -import dil.ast.Node; -import dil.Enums; - -/// The root class of all declarations. -abstract class Declaration : Node -{ - bool hasBody; - this() - { - super(NodeCategory.Declaration); - } - - // Members relevant to semantic phase. - StorageClass stc; /// The storage classes of this declaration. - Protection prot; /// The protection attribute of this declaration. - - final bool isStatic() - { - return !!(stc & StorageClass.Static); - } - - final bool isPublic() - { - return !!(prot & Protection.Public); - } - - final void setStorageClass(StorageClass stc) - { - this.stc = stc; - } - - final void setProtection(Protection prot) - { - this.prot = prot; - } - - override abstract Declaration copy(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Declarations.d --- a/trunk/src/dil/ast/Declarations.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,720 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Declarations; - -public import dil.ast.Declaration; -import dil.ast.Node; -import dil.ast.Expression; -import dil.ast.Types; -import dil.ast.Statements; -import dil.ast.Parameters; -import dil.ast.NodeCopier; -import dil.lexer.IdTable; -import dil.semantic.Symbols; -import dil.Enums; -import common; - -class CompoundDeclaration : Declaration -{ - this() - { - hasBody = true; - mixin(set_kind); - } - - void opCatAssign(Declaration d) - { - addChild(d); - } - - void opCatAssign(CompoundDeclaration ds) - { - addChildren(ds.children); - } - - Declaration[] decls() - { - return cast(Declaration[])this.children; - } - - void decls(Declaration[] decls) - { - this.children = decls; - } - - mixin(copyMethod); -} - -/// Single semicolon. -class EmptyDeclaration : Declaration -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -/// Illegal declarations encompass all tokens that don't -/// start a DeclarationDefinition. -/// See_Also: dil.lexer.Token.isDeclDefStartToken() -class IllegalDeclaration : Declaration -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -/// FQN = fully qualified name -alias Identifier*[] ModuleFQN; // Identifier(.Identifier)* - -class ModuleDeclaration : Declaration -{ - Identifier* moduleName; - Identifier*[] packages; - this(ModuleFQN moduleFQN) - { - mixin(set_kind); - assert(moduleFQN.length != 0); - this.moduleName = moduleFQN[$-1]; - this.packages = moduleFQN[0..$-1]; - } - - char[] getFQN() - { - auto pname = getPackageName('.'); - if (pname.length) - return pname ~ "." ~ getName(); - else - return getName(); - } - - char[] getName() - { - if (moduleName) - return moduleName.str; - return null; - } - - char[] getPackageName(char separator) - { - char[] pname; - foreach (pckg; packages) - if (pckg) - pname ~= pckg.str ~ separator; - if (pname.length) - pname = pname[0..$-1]; // Remove last separator - return pname; - } - - mixin(copyMethod); -} - -class ImportDeclaration : Declaration -{ - private alias Identifier*[] Ids; - ModuleFQN[] moduleFQNs; - Ids moduleAliases; - Ids bindNames; - Ids bindAliases; - - this(ModuleFQN[] moduleFQNs, Ids moduleAliases, Ids bindNames, Ids bindAliases, bool isStatic) - { - mixin(set_kind); - this.moduleFQNs = moduleFQNs; - this.moduleAliases = moduleAliases; - this.bindNames = bindNames; - this.bindAliases = bindAliases; - if (isStatic) - this.stc |= StorageClass.Static; - } - - char[][] getModuleFQNs(char separator) - { - char[][] FQNs; - foreach (moduleFQN; moduleFQNs) - { - char[] FQN; - foreach (ident; moduleFQN) - if (ident) - FQN ~= ident.str ~ separator; - FQNs ~= FQN[0..$-1]; // Remove last separator - } - return FQNs; - } - - mixin(copyMethod); -} - -class AliasDeclaration : Declaration -{ - Declaration decl; - this(Declaration decl) - { - mixin(set_kind); - addChild(decl); - this.decl = decl; - } - mixin(copyMethod); -} - -class TypedefDeclaration : Declaration -{ - Declaration decl; - this(Declaration decl) - { - mixin(set_kind); - addChild(decl); - this.decl = decl; - } - mixin(copyMethod); -} - -class EnumDeclaration : Declaration -{ - Identifier* name; - TypeNode baseType; - EnumMemberDeclaration[] members; - this(Identifier* name, TypeNode baseType, EnumMemberDeclaration[] members, bool hasBody) - { - super.hasBody = hasBody; - mixin(set_kind); - addOptChild(baseType); - addOptChildren(members); - - this.name = name; - this.baseType = baseType; - this.members = members; - } - - Enum symbol; - - mixin(copyMethod); -} - -class EnumMemberDeclaration : Declaration -{ - Identifier* name; - Expression value; - this(Identifier* name, Expression value) - { - mixin(set_kind); - addOptChild(value); - - this.name = name; - this.value = value; - } - - EnumMember symbol; - - mixin(copyMethod); -} - -class TemplateDeclaration : Declaration -{ - Identifier* name; - TemplateParameters tparams; - CompoundDeclaration decls; - this(Identifier* name, TemplateParameters tparams, CompoundDeclaration decls) - { - super.hasBody = true; - mixin(set_kind); - addChild(tparams); - addChild(decls); - - this.name = name; - this.tparams = tparams; - this.decls = decls; - } - - Template symbol; /// The template symbol for this declaration. - - mixin(copyMethod); -} - -abstract class AggregateDeclaration : Declaration -{ - Identifier* name; -// TemplateParameters tparams; - CompoundDeclaration decls; - this(Identifier* name, /+TemplateParameters tparams, +/CompoundDeclaration decls) - { - super.hasBody = decls !is null; - this.name = name; -// this.tparams = tparams; - this.decls = decls; - } - mixin(copyMethod); -} - -class ClassDeclaration : AggregateDeclaration -{ - BaseClassType[] bases; - this(Identifier* name, /+TemplateParameters tparams, +/BaseClassType[] bases, CompoundDeclaration decls) - { - super(name, /+tparams, +/decls); - mixin(set_kind); -// addChild(tparams); - addOptChildren(bases); - addOptChild(decls); - - this.bases = bases; - } - - Class symbol; /// The class symbol for this declaration. - - mixin(copyMethod); -} - -class InterfaceDeclaration : AggregateDeclaration -{ - BaseClassType[] bases; - this(Identifier* name, /+TemplateParameters tparams, +/BaseClassType[] bases, CompoundDeclaration decls) - { - super(name, /+tparams, +/decls); - mixin(set_kind); -// addChild(tparams); - addOptChildren(bases); - addOptChild(decls); - - this.bases = bases; - } - - alias dil.semantic.Symbols.Interface Interface; - - Interface symbol; /// The interface symbol for this declaration. - - mixin(copyMethod); -} - -class StructDeclaration : AggregateDeclaration -{ - uint alignSize; - this(Identifier* name, /+TemplateParameters tparams, +/CompoundDeclaration decls) - { - super(name, /+tparams, +/decls); - mixin(set_kind); -// addChild(tparams); - addOptChild(decls); - } - - void setAlignSize(uint alignSize) - { - this.alignSize = alignSize; - } - - Struct symbol; /// The struct symbol for this declaration. - - mixin(copyMethod); -} - -class UnionDeclaration : AggregateDeclaration -{ - this(Identifier* name, /+TemplateParameters tparams, +/CompoundDeclaration decls) - { - super(name, /+tparams, +/decls); - mixin(set_kind); -// addChild(tparams); - addOptChild(decls); - } - - Union symbol; /// The union symbol for this declaration. - - mixin(copyMethod); -} - -class ConstructorDeclaration : Declaration -{ - Parameters params; - FuncBodyStatement funcBody; - this(Parameters params, FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(params); - addChild(funcBody); - - this.params = params; - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -class StaticConstructorDeclaration : Declaration -{ - FuncBodyStatement funcBody; - this(FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(funcBody); - - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -class DestructorDeclaration : Declaration -{ - FuncBodyStatement funcBody; - this(FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(funcBody); - - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -class StaticDestructorDeclaration : Declaration -{ - FuncBodyStatement funcBody; - this(FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(funcBody); - - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -class FunctionDeclaration : Declaration -{ - TypeNode returnType; - Identifier* name; -// TemplateParameters tparams; - Parameters params; - FuncBodyStatement funcBody; - LinkageType linkageType; - this(TypeNode returnType, Identifier* name,/+ TemplateParameters tparams,+/ - Parameters params, FuncBodyStatement funcBody) - { - super.hasBody = funcBody.funcBody !is null; - mixin(set_kind); - addChild(returnType); -// addChild(tparams); - addChild(params); - addChild(funcBody); - - this.returnType = returnType; - this.name = name; -// this.tparams = tparams; - this.params = params; - this.funcBody = funcBody; - } - - void setLinkageType(LinkageType linkageType) - { - this.linkageType = linkageType; - } - - bool isTemplatized() - { // E.g.: void func(T)(T t) - // ^ params.begin.prevNWS - return params.begin.prevNWS.kind == TOK.RParen; - } - - mixin(copyMethod); -} - -/// VariablesDeclaration := Type? Identifier ("=" Init)? ("," Identifier ("=" Init)?)* ";" -class VariablesDeclaration : Declaration -{ - TypeNode typeNode; - Identifier*[] names; - Expression[] inits; - this(TypeNode typeNode, Identifier*[] names, Expression[] inits) - { - // No empty arrays allowed. Both arrays must be of same size. - assert(names.length != 0 && names.length == inits.length); - // If no type (in case of AutoDeclaration), first value mustn't be null. - assert(typeNode ? 1 : inits[0] !is null); - mixin(set_kind); - addOptChild(typeNode); - foreach(init; inits) - addOptChild(init); - - this.typeNode = typeNode; - this.names = names; - this.inits = inits; - } - - LinkageType linkageType; - - void setLinkageType(LinkageType linkageType) - { - this.linkageType = linkageType; - } - - Variable[] variables; - - mixin(copyMethod); -} - -class InvariantDeclaration : Declaration -{ - FuncBodyStatement funcBody; - this(FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(funcBody); - - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -class UnittestDeclaration : Declaration -{ - FuncBodyStatement funcBody; - this(FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(funcBody); - - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -abstract class ConditionalCompilationDeclaration : Declaration -{ - Token* spec; - Token* cond; - Declaration decls, elseDecls; - - this(Token* spec, Token* cond, Declaration decls, Declaration elseDecls) - { - super.hasBody = decls !is null; - addOptChild(decls); - addOptChild(elseDecls); - - this.spec = spec; - this.cond = cond; - this.decls = decls; - this.elseDecls = elseDecls; - } - - bool isSpecification() - { - return decls is null; - } - - bool isCondition() - { - return decls !is null; - } - - /// The branch to be compiled in. - Declaration compiledDecls; -} - -class DebugDeclaration : ConditionalCompilationDeclaration -{ - this(Token* spec, Token* cond, Declaration decls, Declaration elseDecls) - { - super(spec, cond, decls, elseDecls); - mixin(set_kind); - } - mixin(copyMethod); -} - -class VersionDeclaration : ConditionalCompilationDeclaration -{ - this(Token* spec, Token* cond, Declaration decls, Declaration elseDecls) - { - super(spec, cond, decls, elseDecls); - mixin(set_kind); - } - mixin(copyMethod); -} - -class StaticIfDeclaration : Declaration -{ - Expression condition; - Declaration ifDecls, elseDecls; - this(Expression condition, Declaration ifDecls, Declaration elseDecls) - { - super.hasBody = true; - mixin(set_kind); - addChild(condition); - addChild(ifDecls); - addOptChild(elseDecls); - - this.condition = condition; - this.ifDecls = ifDecls; - this.elseDecls = elseDecls; - } - mixin(copyMethod); -} - -class StaticAssertDeclaration : Declaration -{ - Expression condition, message; - this(Expression condition, Expression message) - { - super.hasBody = true; - mixin(set_kind); - addChild(condition); - addOptChild(message); - - this.condition = condition; - this.message = message; - } - mixin(copyMethod); -} - -class NewDeclaration : Declaration -{ - Parameters params; - FuncBodyStatement funcBody; - this(Parameters params, FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(params); - addChild(funcBody); - - this.params = params; - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -class DeleteDeclaration : Declaration -{ - Parameters params; - FuncBodyStatement funcBody; - this(Parameters params, FuncBodyStatement funcBody) - { - super.hasBody = true; - mixin(set_kind); - addChild(params); - addChild(funcBody); - - this.params = params; - this.funcBody = funcBody; - } - mixin(copyMethod); -} - -abstract class AttributeDeclaration : Declaration -{ - Declaration decls; - this(Declaration decls) - { - super.hasBody = true; - addChild(decls); - this.decls = decls; - } -} - -class ProtectionDeclaration : AttributeDeclaration -{ - Protection prot; - this(Protection prot, Declaration decls) - { - super(decls); - mixin(set_kind); - this.prot = prot; - } - mixin(copyMethod); -} - -class StorageClassDeclaration : AttributeDeclaration -{ - StorageClass storageClass; - this(StorageClass storageClass, Declaration decl) - { - super(decl); - mixin(set_kind); - - this.storageClass = storageClass; - } - mixin(copyMethod); -} - -class LinkageDeclaration : AttributeDeclaration -{ - LinkageType linkageType; - this(LinkageType linkageType, Declaration decls) - { - super(decls); - mixin(set_kind); - - this.linkageType = linkageType; - } - mixin(copyMethod); -} - -class AlignDeclaration : AttributeDeclaration -{ - int size; - this(int size, Declaration decls) - { - super(decls); - mixin(set_kind); - this.size = size; - } - mixin(copyMethod); -} - -class PragmaDeclaration : AttributeDeclaration -{ - Identifier* ident; - Expression[] args; - this(Identifier* ident, Expression[] args, Declaration decls) - { - addOptChildren(args); // Add args before calling super(). - super(decls); - mixin(set_kind); - - this.ident = ident; - this.args = args; - } - mixin(copyMethod); -} - -class MixinDeclaration : Declaration -{ - /// IdExpression := IdentifierExpression | TemplateInstanceExpression - /// MixinTemplate := IdExpression ("." IdExpression)* - Expression templateExpr; - Identifier* mixinIdent; /// Optional mixin identifier. - Expression argument; /// "mixin" "(" AssignExpression ")" - Declaration decls; /// Initialized in the semantic phase. - - this(Expression templateExpr, Identifier* mixinIdent) - { - mixin(set_kind); - addChild(templateExpr); - - this.templateExpr = templateExpr; - this.mixinIdent = mixinIdent; - } - - this(Expression argument) - { - mixin(set_kind); - addChild(argument); - - this.argument = argument; - } - - bool isMixinExpression() - { - return argument !is null; - } - - mixin(copyMethod); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/DefaultVisitor.d --- a/trunk/src/dil/ast/DefaultVisitor.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,375 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.DefaultVisitor; - -import dil.ast.Visitor; - -import dil.ast.Node; -import dil.ast.Declarations, - dil.ast.Expressions, - dil.ast.Statements, - dil.ast.Types, - dil.ast.Parameters; -import common; - -/// This huge template function, when instantiated for a certain node class, -/// generates a body of calls to visit() on the subnodes. -returnType!(T.stringof) visitDefault(T)(T t) -{ - assert(t !is null, "node passed to visitDefault() is null"); - //Stdout(t).newline; - - alias t d, s, e, n; // Variable aliases of t. - - static if (is(T : Declaration)) - { - alias T D; - static if (is(D == CompoundDeclaration)) - foreach (decl; d.decls) - visitD(decl); - //EmptyDeclaration, - //IllegalDeclaration, - //ModuleDeclaration have no subnodes. - static if (is(D == AliasDeclaration) || - is(D == TypedefDeclaration)) - visitD(d.decl); - static if (is(D == EnumDeclaration)) - { - d.baseType && visitT(d.baseType); - foreach (member; d.members) - visitD(member); - } - static if (is(D == EnumMemberDeclaration)) - d.value && visitE(d.value); - static if (is(D == ClassDeclaration) || is( D == InterfaceDeclaration)) - { -// visitN(d.tparams); - foreach (base; d.bases) - visitT(base); - d.decls && visitD(d.decls); - } - static if (is(D == StructDeclaration) || is(D == UnionDeclaration)) -// visitN(d.tparams), - d.decls && visitD(d.decls); - static if (is(D == ConstructorDeclaration)) - visitN(d.params), visitS(d.funcBody); - static if (is(D == StaticConstructorDeclaration) || - is(D == DestructorDeclaration) || - is(D == StaticDestructorDeclaration) || - is(D == InvariantDeclaration) || - is(D == UnittestDeclaration)) - visitS(d.funcBody); - static if (is(D == FunctionDeclaration)) - visitT(d.returnType), -// visitN(d.tparams), - visitN(d.params), - visitS(d.funcBody); - static if (is(D == VariablesDeclaration)) - { - d.typeNode && visitT(d.typeNode); - foreach(init; d.inits) - init && visitE(init); - } - static if (is(D == DebugDeclaration) || is(D == VersionDeclaration)) - d.decls && visitD(d.decls), - d.elseDecls && visitD(d.elseDecls); - static if (is(D == StaticIfDeclaration)) - visitE(d.condition), - visitD(d.ifDecls), - d.elseDecls && visitD(d.elseDecls); - static if (is(D == StaticAssertDeclaration)) - visitE(d.condition), - d.message && visitE(d.message); - static if (is(D == TemplateDeclaration)) - visitN(d.tparams), - visitD(d.decls); - static if (is(D == NewDeclaration) || is(D == DeleteDeclaration)) - visitN(d.params), - visitS(d.funcBody); - static if (is(D == ProtectionDeclaration) || - is(D == StorageClassDeclaration) || - is(D == LinkageDeclaration) || - is(D == AlignDeclaration)) - visitD(d.decls); - static if (is(D == PragmaDeclaration)) - { - foreach (arg; d.args) - visitE(arg); - visitD(d.decls); - } - static if (is(D == MixinDeclaration)) - d.templateExpr ? visitE(d.templateExpr) : visitE(d.argument); - } - else - static if (is(T : Expression)) - { - alias T E; - static if (is(E == IllegalExpression)) - {} - else - static if (is(E == CondExpression)) - visitE(e.condition), visitE(e.lhs), visitE(e.rhs); - else - static if (is(E : BinaryExpression)) - visitE(e.lhs), visitE(e.rhs); - else - static if (is(E : UnaryExpression)) - { - static if (is(E == CastExpression)) - visitT(e.type); - visitE(e.e); // Visit member in base class UnaryExpression. - static if (is(E == IndexExpression)) - foreach (arg; e.args) - visitE(arg); - static if (is(E == SliceExpression)) - e.left && (visitE(e.left), visitE(e.right)); - static if (is(E == AsmPostBracketExpression)) - visitE(e.e2); - } - else - { - static if (is(E == NewExpression)) - { - foreach (arg; e.newArgs) - visitE(arg); - visitT(e.type); - foreach (arg; e.ctorArgs) - visitE(arg); - } - static if (is(E == NewAnonClassExpression)) - { - foreach (arg; e.newArgs) - visitE(arg); - foreach (base; e.bases) - visitT(base); - foreach (arg; e.ctorArgs) - visitE(arg); - visitD(e.decls); - } - static if (is(E == AsmBracketExpression)) - visitE(e.e); - static if (is(E == TemplateInstanceExpression)) - e.targs && visitN(e.targs); - static if (is(E == ArrayLiteralExpression)) - foreach (value; e.values) - visitE(value); - static if (is(E == AArrayLiteralExpression)) - foreach (i, key; e.keys) - visitE(key), visitE(e.values[i]); - static if (is(E == AssertExpression)) - visitE(e.expr), e.msg && visitE(e.msg); - static if (is(E == MixinExpression) || - is(E == ImportExpression)) - visitE(e.expr); - static if (is(E == TypeofExpression) || - is(E == TypeDotIdExpression) || - is(E == TypeidExpression)) - visitT(e.type); - static if (is(E == IsExpression)) - visitT(e.type), e.specType && visitT(e.specType), - e.tparams && visitN(e.tparams); - static if (is(E == FunctionLiteralExpression)) - e.returnType && visitT(e.returnType), - e.params && visitN(e.params), - visitS(e.funcBody); - static if (is(E == ParenExpression)) - visitE(e.next); - static if (is(E == TraitsExpression)) - visitN(e.targs); - // VoidInitializer has no subnodes. - static if (is(E == ArrayInitExpression)) - foreach (i, key; e.keys) - key && visitE(key), visitE(e.values[i]); - static if (is(E == StructInitExpression)) - foreach (value; e.values) - visitE(value); - } - } - else - static if (is(T : Statement)) - { - alias T S; - static if (is(S == CompoundStatement)) - foreach (stmnt; s.stmnts) - visitS(stmnt); - //IllegalStatement has no subnodes. - static if (is(S == FuncBodyStatement)) - s.funcBody && visitS(s.funcBody), - s.inBody && visitS(s.inBody), - s.outBody && visitS(s.outBody); - static if (is(S == ScopeStatement) || is(S == LabeledStatement)) - visitS(s.s); - static if (is(S == ExpressionStatement)) - visitE(s.e); - static if (is(S == DeclarationStatement)) - visitD(s.decl); - static if (is(S == IfStatement)) - { - s.variable ? cast(Node)visitS(s.variable) : visitE(s.condition); - visitS(s.ifBody), s.elseBody && visitS(s.elseBody); - } - static if (is(S == WhileStatement)) - visitE(s.condition), visitS(s.whileBody); - static if (is(S == DoWhileStatement)) - visitS(s.doBody), visitE(s.condition); - static if (is(S == ForStatement)) - s.init && visitS(s.init), - s.condition && visitE(s.condition), - s.increment && visitE(s.increment), - visitS(s.forBody); - static if (is(S == ForeachStatement)) - visitN(s.params), visitE(s.aggregate), visitS(s.forBody); - static if (is(S == ForeachRangeStatement)) - visitN(s.params), visitE(s.lower), visitE(s.upper), visitS(s.forBody); - static if (is(S == SwitchStatement)) - visitE(s.condition), visitS(s.switchBody); - static if (is(S == CaseStatement)) - { - foreach (value; s.values) - visitE(value); - visitS(s.caseBody); - } - static if (is(S == DefaultStatement)) - visitS(s.defaultBody); - //ContinueStatement, - //BreakStatement have no subnodes. - static if (is(S == ReturnStatement)) - s.e && visitE(s.e); - static if (is(S == GotoStatement)) - s.caseExpr && visitE(s.caseExpr); - static if (is(S == WithStatement)) - visitE(s.e), visitS(s.withBody); - static if (is(S == SynchronizedStatement)) - s.e && visitE(s.e), visitS(s.syncBody); - static if (is(S == TryStatement)) - { - visitS(s.tryBody); - foreach (catchBody; s.catchBodies) - visitS(catchBody); - s.finallyBody && visitS(s.finallyBody); - } - static if (is(S == CatchStatement)) - s.param && visitN(s.param), visitS(s.catchBody); - static if (is(S == FinallyStatement)) - visitS(s.finallyBody); - static if (is(S == ScopeGuardStatement)) - visitS(s.scopeBody); - static if (is(S == ThrowStatement)) - visitE(s.e); - static if (is(S == VolatileStatement)) - s.volatileBody && visitS(s.volatileBody); - static if (is(S == AsmBlockStatement)) - visitS(s.statements); - static if (is(S == AsmStatement)) - foreach (op; s.operands) - visitE(op); - //AsmAlignStatement, - //IllegalAsmStatement have no subnodes. - static if (is(S == PragmaStatement)) - { - foreach (arg; s.args) - visitE(arg); - visitS(s.pragmaBody); - } - static if (is(S == MixinStatement)) - visitE(s.templateExpr); - static if (is(S == StaticIfStatement)) - visitE(s.condition), visitS(s.ifBody), s.elseBody && visitS(s.elseBody); - static if (is(S == StaticAssertStatement)) - visitE(s.condition), s.message && visitE(s.message); - static if (is(S == DebugStatement) || is(S == VersionStatement)) - visitS(s.mainBody), s.elseBody && visitS(s.elseBody); - } - else - static if (is(T : TypeNode)) - { - //IllegalType, - //IntegralType, - //ModuleScopeType, - //IdentifierType have no subnodes. - static if (is(T == QualifiedType)) - visitT(t.lhs), visitT(t.rhs); - static if (is(T == TypeofType)) - visitE(t.e); - static if (is(T == TemplateInstanceType)) - t.targs && visitN(t.targs); - static if (is(T == PointerType)) - visitT(t.next); - static if (is(T == ArrayType)) - { - visitT(t.next); - if (t.assocType) - visitT(t.assocType); - else if (t.e1) - visitE(t.e1), t.e2 && visitE(t.e2); - } - static if (is(T == FunctionType) || is(T == DelegateType)) - visitT(t.returnType), visitN(t.params); - static if (is(T == CFuncPointerType)) - visitT(t.next), t.params && visitN(t.params); - static if (is(T == BaseClassType) || - is(T == ConstType) || - is(T == InvariantType)) - visitT(t.next); - } - else - static if (is(T == Parameter)) - { - n.type && visitT(n.type); - n.defValue && visitE(n.defValue); - } - else - static if (is(T == Parameters) || - is(T == TemplateParameters) || - is(T == TemplateArguments)) - { - foreach (node; n.children) - visitN(node); - } - else - static if (is(T : TemplateParameter)) - { - static if (is(N == TemplateAliasParameter) || - is(N == TemplateTypeParameter) || - is(N == TemplateThisParameter)) - n.specType && visitN(n.specType), - n.defType && visitN(n.defType); - static if (is(N == TemplateValueParameter)) - visitT(n.valueType), - n.specValue && visitN(n.specValue), - n.defValue && visitN(n.defValue); - //TemplateTupleParameter has no subnodes. - } - else - static assert(0, "Missing default visit method for: "~typeof(t).stringof); - return t; -} - -/// Generates the default visit methods. -/// -/// E.g: -/// --- -/// private mixin .visitDefault!(ClassDeclaration) _ClassDeclaration; -/// override returnType!("ClassDeclaration") visit(ClassDeclaration node) -/// { return _ClassDeclaration.visitDefault(node); } -/// --- -char[] generateDefaultVisitMethods() -{ - char[] text; - foreach (className; g_classNames) - text ~= "private mixin .visitDefault!("~className~") _"~className~";\n" - "override returnType!(\""~className~"\") visit("~className~" node)" - "{return _"~className~".visitDefault(node);}\n"; - return text; -} -// pragma(msg, generateDefaultVisitMethods()); - -/// This class provides default methods for -/// traversing nodes and their sub-nodes. -class DefaultVisitor : Visitor -{ - // Comment out if too many errors are shown. - mixin(generateDefaultVisitMethods()); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Expression.d --- a/trunk/src/dil/ast/Expression.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,22 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Expression; - -import dil.ast.Node; -import dil.semantic.Types; -import common; - -/// The root class of all expressions. -abstract class Expression : Node -{ - Type type; /// The semantic type of this expression. - - this() - { - super(NodeCategory.Expression); - } - - override abstract Expression copy(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Expressions.d --- a/trunk/src/dil/ast/Expressions.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1118 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Expressions; - -public import dil.ast.Expression; -import dil.ast.Node; -import dil.ast.Types; -import dil.ast.Declarations; -import dil.ast.Statements; -import dil.ast.Parameters; -import dil.ast.NodeCopier; -import dil.lexer.Identifier; -import dil.semantic.Types; -import common; - -class IllegalExpression : Expression -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -abstract class BinaryExpression : Expression -{ - Expression lhs; /// Left-hand side expression. - Expression rhs; /// Right-hand side expression. - Token* tok; - this(Expression lhs, Expression rhs, Token* tok) - { - addChildren([lhs, rhs]); - this.lhs = lhs; - this.rhs = rhs; - this.tok = tok; - } - mixin(copyMethod); -} - -class CondExpression : BinaryExpression -{ - Expression condition; - this(Expression condition, Expression left, Expression right, Token* tok) - { - addChild(condition); - super(left, right, tok); - mixin(set_kind); - this.condition = condition; - } - mixin(copyMethod); -} - -class CommaExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class OrOrExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class AndAndExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class OrExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class XorExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class AndExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -abstract class CmpExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - } -} - -class EqualExpression : CmpExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -/// Expression "!"? "is" Expression -class IdentityExpression : CmpExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class RelExpression : CmpExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class InExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class LShiftExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class RShiftExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class URShiftExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class PlusExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class MinusExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class CatExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class MulExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class DivExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class ModExpression : BinaryExpression -{ - this(Expression left, Expression right, Token* tok) - { - super(left, right, tok); - mixin(set_kind); - } -} - -class AssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class LShiftAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class RShiftAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class URShiftAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class OrAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class AndAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class PlusAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class MinusAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class DivAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class MulAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class ModAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class XorAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} -class CatAssignExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} - -/// DotExpression := Expression '.' Expression -class DotExpression : BinaryExpression -{ - this(Expression left, Expression right) - { - super(left, right, null); - mixin(set_kind); - } -} - -/*++++++++++++++++++++ -+ Unary Expressions: + -++++++++++++++++++++*/ - -abstract class UnaryExpression : Expression -{ - Expression e; - this(Expression e) - { - addChild(e); - this.e = e; - } - mixin(copyMethod); -} - -class AddressExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class PreIncrExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class PreDecrExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class PostIncrExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class PostDecrExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class DerefExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class SignExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } - - bool isPos() - { - assert(begin !is null); - return begin.kind == TOK.Plus; - } - - bool isNeg() - { - assert(begin !is null); - return begin.kind == TOK.Minus; - } -} - -class NotExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class CompExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class CallExpression : UnaryExpression -{ - Expression[] args; - this(Expression e, Expression[] args) - { - super(e); - mixin(set_kind); - addOptChildren(args); - this.args = args; - } -} - -class NewExpression : /*Unary*/Expression -{ - Expression[] newArgs; - TypeNode type; - Expression[] ctorArgs; - this(/*Expression e, */Expression[] newArgs, TypeNode type, Expression[] ctorArgs) - { - /*super(e);*/ - mixin(set_kind); - addOptChildren(newArgs); - addChild(type); - addOptChildren(ctorArgs); - this.newArgs = newArgs; - this.type = type; - this.ctorArgs = ctorArgs; - } - mixin(copyMethod); -} - -class NewAnonClassExpression : /*Unary*/Expression -{ - Expression[] newArgs; - BaseClassType[] bases; - Expression[] ctorArgs; - CompoundDeclaration decls; - this(/*Expression e, */Expression[] newArgs, BaseClassType[] bases, Expression[] ctorArgs, CompoundDeclaration decls) - { - /*super(e);*/ - mixin(set_kind); - addOptChildren(newArgs); - addOptChildren(bases); - addOptChildren(ctorArgs); - addChild(decls); - - this.newArgs = newArgs; - this.bases = bases; - this.ctorArgs = ctorArgs; - this.decls = decls; - } - mixin(copyMethod); -} - -class DeleteExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class CastExpression : UnaryExpression -{ - TypeNode type; - this(Expression e, TypeNode type) - { - addChild(type); // Add type before super(). - super(e); - mixin(set_kind); - this.type = type; - } - mixin(copyMethod); -} - -class IndexExpression : UnaryExpression -{ - Expression[] args; - this(Expression e, Expression[] args) - { - super(e); - mixin(set_kind); - addChildren(args); - this.args = args; - } - mixin(copyMethod); -} - -class SliceExpression : UnaryExpression -{ - Expression left, right; - this(Expression e, Expression left, Expression right) - { - super(e); - mixin(set_kind); - assert(left ? (right !is null) : right is null); - if (left) - addChildren([left, right]); - - this.left = left; - this.right = right; - } - mixin(copyMethod); -} - -/// Module scope operator: '.' (IdentifierExpression|TemplateInstanceExpression) -class ModuleScopeExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - assert(e.kind == NodeKind.IdentifierExpression || - e.kind == NodeKind.TemplateInstanceExpression - ); - mixin(set_kind); - } -} - -/*++++++++++++++++++++++ -+ Primary Expressions: + -++++++++++++++++++++++*/ - -class IdentifierExpression : Expression -{ - Identifier* identifier; - this(Identifier* identifier) - { - mixin(set_kind); - this.identifier = identifier; - } - mixin(copyMethod); -} - -class SpecialTokenExpression : Expression -{ - Token* specialToken; - this(Token* specialToken) - { - mixin(set_kind); - this.specialToken = specialToken; - } - - Expression value; /// The expression created in the semantic phase. - - mixin(copyMethod); -} - -class TemplateInstanceExpression : Expression -{ - Identifier* ident; - TemplateArguments targs; - this(Identifier* ident, TemplateArguments targs) - { - mixin(set_kind); - addOptChild(targs); - this.ident = ident; - this.targs = targs; - } - mixin(copyMethod); -} - -class ThisExpression : Expression -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class SuperExpression : Expression -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class NullExpression : Expression -{ - this() - { - mixin(set_kind); - } - - this(Type type) - { - this(); - this.type = type; - } - - mixin(copyMethod); -} - -class DollarExpression : Expression -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class BoolExpression : Expression -{ - this() - { - mixin(set_kind); - } - - bool toBool() - { - assert(begin !is null); - return begin.kind == TOK.True ? true : false; - } - - Expression value; /// IntExpression of type int. - - mixin(copyMethod); -} - -class IntExpression : Expression -{ - ulong number; - - this(ulong number, Type type) - { - mixin(set_kind); - this.number = number; - this.type = type; - } - - this(Token* token) - { - auto type = Types.Int; // Should be most common case. - switch (token.kind) - { - // case TOK.Int32: - // type = Types.Int; break; - case TOK.Uint32: - type = Types.Uint; break; - case TOK.Int64: - type = Types.Long; break; - case TOK.Uint64: - type = Types.Ulong; break; - default: - assert(token.kind == TOK.Int32); - } - this(token.ulong_, type); - } - - mixin(copyMethod); -} - -class RealExpression : Expression -{ - real number; - - this(real number, Type type) - { - mixin(set_kind); - this.number = number; - this.type = type; - } - - this(Token* token) - { - auto type = Types.Double; // Most common case? - switch (token.kind) - { - case TOK.Float32: - type = Types.Float; break; - // case TOK.Float64: - // type = Types.Double; break; - case TOK.Float80: - type = Types.Real; break; - case TOK.Imaginary32: - type = Types.Ifloat; break; - case TOK.Imaginary64: - type = Types.Idouble; break; - case TOK.Imaginary80: - type = Types.Ireal; break; - default: - assert(token.kind == TOK.Float64); - } - this(token.real_, type); - } - - mixin(copyMethod); -} - - -/// This expression holds a complex number. -/// It is only created in the semantic phase. -class ComplexExpression : Expression -{ - creal number; - this(creal number, Type type) - { - mixin(set_kind); - this.number = number; - this.type = type; - } - mixin(copyMethod); -} - -class CharExpression : Expression -{ - dchar character; - this(dchar character) - { - mixin(set_kind); - this.character = character; - } - mixin(copyMethod); -} - -class StringExpression : Expression -{ - ubyte[] str; /// The string data. - Type charType; /// The character type of the string. - - this(ubyte[] str, Type charType) - { - mixin(set_kind); - this.str = str; - this.charType = charType; - type = new TypeSArray(charType, str.length); - } - - this(char[] str) - { - this(cast(ubyte[])str, Types.Char); - } - - this(wchar[] str) - { - this(cast(ubyte[])str, Types.Wchar); - } - - this(dchar[] str) - { - this(cast(ubyte[])str, Types.Dchar); - } - - /// Returns the string excluding the terminating 0. - char[] getString() - { - // TODO: convert to char[] if charType !is Types.Char. - return cast(char[])str[0..$-1]; - } - - mixin(copyMethod); -} - -class ArrayLiteralExpression : Expression -{ - Expression[] values; - this(Expression[] values) - { - mixin(set_kind); - addOptChildren(values); - this.values = values; - } - mixin(copyMethod); -} - -class AArrayLiteralExpression : Expression -{ - Expression[] keys, values; - this(Expression[] keys, Expression[] values) - { - assert(keys.length == values.length); - mixin(set_kind); - foreach (i, key; keys) - addChildren([key, values[i]]); - this.keys = keys; - this.values = values; - } - mixin(copyMethod); -} - -class AssertExpression : Expression -{ - Expression expr, msg; - this(Expression expr, Expression msg) - { - mixin(set_kind); - addChild(expr); - addOptChild(msg); - this.expr = expr; - this.msg = msg; - } - mixin(copyMethod); -} - -class MixinExpression : Expression -{ - Expression expr; - this(Expression expr) - { - mixin(set_kind); - addChild(expr); - this.expr = expr; - } - mixin(copyMethod); -} - -class ImportExpression : Expression -{ - Expression expr; - this(Expression expr) - { - mixin(set_kind); - addChild(expr); - this.expr = expr; - } - mixin(copyMethod); -} - -class TypeofExpression : Expression -{ - TypeNode type; - this(TypeNode type) - { - mixin(set_kind); - addChild(type); - this.type = type; - } - mixin(copyMethod); -} - -class TypeDotIdExpression : Expression -{ - TypeNode type; - Identifier* ident; - this(TypeNode type, Identifier* ident) - { - mixin(set_kind); - addChild(type); - this.type = type; - this.ident = ident; - } - mixin(copyMethod); -} - -class TypeidExpression : Expression -{ - TypeNode type; - this(TypeNode type) - { - mixin(set_kind); - addChild(type); - this.type = type; - } - mixin(copyMethod); -} - -class IsExpression : Expression -{ - TypeNode type; - Identifier* ident; - Token* opTok, specTok; - TypeNode specType; - TemplateParameters tparams; // D 2.0 - this(TypeNode type, Identifier* ident, Token* opTok, Token* specTok, - TypeNode specType, typeof(tparams) tparams) - { - mixin(set_kind); - addChild(type); - addOptChild(specType); - version(D2) - addOptChild(tparams); - this.type = type; - this.ident = ident; - this.opTok = opTok; - this.specTok = specTok; - this.specType = specType; - this.tparams = tparams; - } - mixin(copyMethod); -} - -class FunctionLiteralExpression : Expression -{ - TypeNode returnType; - Parameters params; - FuncBodyStatement funcBody; - - this() - { - mixin(set_kind); - addOptChild(returnType); - addOptChild(params); - addChild(funcBody); - } - - this(TypeNode returnType, Parameters params, FuncBodyStatement funcBody) - { - this.returnType = returnType; - this.params = params; - this.funcBody = funcBody; - this(); - } - - this(FuncBodyStatement funcBody) - { - this.funcBody = funcBody; - this(); - } - - mixin(copyMethod); -} - -/// ParenthesisExpression := "(" Expression ")" -class ParenExpression : Expression -{ - Expression next; - this(Expression next) - { - mixin(set_kind); - addChild(next); - this.next = next; - } - mixin(copyMethod); -} - -// version(D2) -// { -class TraitsExpression : Expression -{ - Identifier* ident; - TemplateArguments targs; - this(typeof(ident) ident, typeof(targs) targs) - { - mixin(set_kind); - addOptChild(targs); - this.ident = ident; - this.targs = targs; - } - mixin(copyMethod); -} -// } - -class VoidInitExpression : Expression -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class ArrayInitExpression : Expression -{ - Expression[] keys; - Expression[] values; - this(Expression[] keys, Expression[] values) - { - assert(keys.length == values.length); - mixin(set_kind); - foreach (i, key; keys) - { - addOptChild(key); // The key is optional in ArrayInitializers. - addChild(values[i]); - } - this.keys = keys; - this.values = values; - } - mixin(copyMethod); -} - -class StructInitExpression : Expression -{ - Identifier*[] idents; - Expression[] values; - this(Identifier*[] idents, Expression[] values) - { - mixin(set_kind); - addOptChildren(values); - this.idents = idents; - this.values = values; - } - mixin(copyMethod); -} - -class AsmTypeExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class AsmOffsetExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class AsmSegExpression : UnaryExpression -{ - this(Expression e) - { - super(e); - mixin(set_kind); - } -} - -class AsmPostBracketExpression : UnaryExpression -{ - Expression e2; /// Expression in brackets: e [ e2 ] - this(Expression e, Expression e2) - { - super(e); - mixin(set_kind); - addChild(e2); - this.e2 = e2; - } - mixin(copyMethod); -} - -class AsmBracketExpression : Expression -{ - Expression e; - this(Expression e) - { - mixin(set_kind); - addChild(e); - this.e = e; - } - mixin(copyMethod); -} - -class AsmLocalSizeExpression : Expression -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class AsmRegisterExpression : Expression -{ - Identifier* register; - int number; // ST(0) - ST(7) or FS:0, FS:4, FS:8 - this(Identifier* register, int number = -1) - { - mixin(set_kind); - this.register = register; - this.number = number; - } - mixin(copyMethod); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Node.d --- a/trunk/src/dil/ast/Node.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,98 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Node; - -import common; - -public import dil.lexer.Token; -public import dil.ast.NodesEnum; - -/// The root class of all D syntax tree elements. -abstract class Node -{ - NodeCategory category; /// The category of this node. - NodeKind kind; /// The kind of this node. - Node[] children; // Will be probably removed sometime. - Token* begin, end; /// The begin and end tokens of this node. - - /// Constructs a node object. - this(NodeCategory category) - { - assert(category != NodeCategory.Undefined); - this.category = category; - } - - void setTokens(Token* begin, Token* end) - { - this.begin = begin; - this.end = end; - } - - Class setToks(Class)(Class node) - { - node.setTokens(this.begin, this.end); - return node; - } - - void addChild(Node child) - { - assert(child !is null, "failed in " ~ this.classinfo.name); - this.children ~= child; - } - - void addOptChild(Node child) - { - child is null || addChild(child); - } - - void addChildren(Node[] children) - { - assert(children !is null && delegate{ - foreach (child; children) - if (child is null) - return false; - return true; }(), - "failed in " ~ this.classinfo.name - ); - this.children ~= children; - } - - void addOptChildren(Node[] children) - { - children is null || addChildren(children); - } - - /// Returns a reference to Class if this node can be cast to it. - Class Is(Class)() - { - if (kind == mixin("NodeKind." ~ typeof(Class).stringof)) - return cast(Class)cast(void*)this; - return null; - } - - /// Casts this node to Class. - Class to(Class)() - { - return cast(Class)cast(void*)this; - } - - /// Returns a deep copy of this node. - abstract Node copy(); - - /// Returns a shallow copy of this object. - final Node dup() - { - // Find out the size of this object. - alias typeof(this.classinfo.init[0]) byte_t; - size_t size = this.classinfo.init.length; - // Copy this object's data. - byte_t[] data = (cast(byte_t*)this)[0..size].dup; - return cast(Node)data.ptr; - } - - /// This string is mixed into the constructor of a class that inherits - /// from Node. It sets the member kind. - const string set_kind = `this.kind = mixin("NodeKind." ~ typeof(this).stringof);`; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/NodeCopier.d --- a/trunk/src/dil/ast/NodeCopier.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,324 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.NodeCopier; - -import common; - -/// Mixed into the body of a class that inherits from Node. -const string copyMethod = ` - override typeof(this) copy() - { - mixin copyNode!(typeof(this)); - return copyNode(this); - }`; - -/// A helper function that generates code for copying subnodes. -string doCopy_(string obj, string[] members) -{ - char[] result; - foreach (member; members) - { - if (member.length > 2 && member[$-2..$] == "[]") // Array copy. - { - member = member[0..$-2]; - // obj.member = obj.member.dup; - // foreach (ref m_; obj.member) - // m_ = m_.copy(); - result ~= obj~"."~member~" = "~obj~"."~member~".dup;" - "foreach (ref m_; "~obj~"."~member~")" - "m_ = m_.copy();"; - } - else if (member[$-1] == '?') // Optional member copy. - { - member = member[0..$-1]; - // obj.member && (obj.member = obj.member.copy()); - result ~= obj~"."~member~" && ("~obj~"."~member~" = "~obj~"."~member~".copy());"; - } - else // Non-optional member copy. - // obj.member = obj.member.copy(); - result ~= obj~"."~member~" = "~obj~"."~member~".copy();"; - } - return result; -} - -string doCopy(string[] members) -{ - return doCopy_("x", members); -} - -string doCopy(string member) -{ - return doCopy_("x", [member]); -} - -// pragma(msg, doCopy("decls?")); - -/// Returns a deep copy of node. -T copyNode(T)(T node) -{ - assert(node !is null); - - // Firstly do a shallow copy. - T x = cast(T)cast(void*)node.dup; - - // Now copy the subnodes. - static if (is(Declaration) && is(T : Declaration)) - { - alias T D; - static if (is(D == CompoundDeclaration)) - mixin(doCopy("decls[]")); - //EmptyDeclaration, - //IllegalDeclaration, - //ModuleDeclaration have no subnodes. - static if (is(D == AliasDeclaration) || - is(D == TypedefDeclaration)) - mixin(doCopy("decl")); - static if (is(D == EnumDeclaration)) - mixin(doCopy(["baseType?", "members[]"])); - static if (is(D == EnumMemberDeclaration)) - mixin(doCopy("value")); - static if (is(D == ClassDeclaration) || is( D == InterfaceDeclaration)) - mixin(doCopy(["bases[]", "decls"])); - static if (is(D == StructDeclaration) || is(D == UnionDeclaration)) - mixin(doCopy("decls")); - static if (is(D == ConstructorDeclaration)) - mixin(doCopy(["params", "funcBody"])); - static if (is(D == StaticConstructorDeclaration) || - is(D == DestructorDeclaration) || - is(D == StaticDestructorDeclaration) || - is(D == InvariantDeclaration) || - is(D == UnittestDeclaration)) - mixin(doCopy("funcBody")); - static if (is(D == FunctionDeclaration)) - mixin(doCopy(["returnType", "params", "funcBody"])); - static if (is(D == VariablesDeclaration)) - { - mixin(doCopy("typeNode?")); - x.inits = x.inits.dup; - foreach(ref init; x.inits) - init && (init = init.copy()); - } - static if (is(D == DebugDeclaration) || is(D == VersionDeclaration)) - mixin(doCopy(["decls?","elseDecls?"])); - static if (is(D == StaticIfDeclaration)) - mixin(doCopy(["condition","ifDecls", "elseDecls?"])); - static if (is(D == StaticAssertDeclaration)) - mixin(doCopy(["condition","message?"])); - static if (is(D == TemplateDeclaration)) - mixin(doCopy(["tparams","decls"])); - static if (is(D == NewDeclaration) || is(D == DeleteDeclaration)) - mixin(doCopy(["params","funcBody"])); - static if (is(D == ProtectionDeclaration) || - is(D == StorageClassDeclaration) || - is(D == LinkageDeclaration) || - is(D == AlignDeclaration)) - mixin(doCopy("decls")); - static if (is(D == PragmaDeclaration)) - mixin(doCopy(["args[]","decls"])); - static if (is(D == MixinDeclaration)) - mixin(doCopy(["templateExpr?","argument?"])); - } - else - static if (is(Expression) && is(T : Expression)) - { - alias T E; - static if (is(E == IllegalExpression)) - {} - else - static if (is(E == CondExpression)) - mixin(doCopy(["condition", "lhs", "rhs"])); - else - static if (is(E : BinaryExpression)) - mixin(doCopy(["lhs", "rhs"])); - else - static if (is(E : UnaryExpression)) - { - static if (is(E == CastExpression)) - mixin(doCopy("type")); - mixin(doCopy("e")); // Copy member in base class UnaryExpression. - static if (is(E == IndexExpression)) - mixin(doCopy("args[]")); - static if (is(E == SliceExpression)) - mixin(doCopy(["left?", "right?"])); - static if (is(E == AsmPostBracketExpression)) - mixin(doCopy("e2")); - } - else - { - static if (is(E == NewExpression)) - mixin(doCopy(["newArgs[]", "type", "ctorArgs[]"])); - static if (is(E == NewAnonClassExpression)) - mixin(doCopy(["newArgs[]", "bases[]", "ctorArgs[]", "decls"])); - static if (is(E == AsmBracketExpression)) - mixin(doCopy("e")); - static if (is(E == TemplateInstanceExpression)) - mixin(doCopy("targs?")); - static if (is(E == ArrayLiteralExpression)) - mixin(doCopy("values[]")); - static if (is(E == AArrayLiteralExpression)) - mixin(doCopy(["keys[]", "values[]"])); - static if (is(E == AssertExpression)) - mixin(doCopy(["expr", "msg?"])); - static if (is(E == MixinExpression) || - is(E == ImportExpression)) - mixin(doCopy("expr")); - static if (is(E == TypeofExpression) || - is(E == TypeDotIdExpression) || - is(E == TypeidExpression)) - mixin(doCopy("type")); - static if (is(E == IsExpression)) - mixin(doCopy(["type", "specType?", "tparams?"])); - static if (is(E == FunctionLiteralExpression)) - mixin(doCopy(["returnType?", "params?", "funcBody"])); - static if (is(E == ParenExpression)) - mixin(doCopy("next")); - static if (is(E == TraitsExpression)) - mixin(doCopy("targs")); - // VoidInitializer has no subnodes. - static if (is(E == ArrayInitExpression)) - { - mixin(doCopy("values[]")); - x.keys = x.keys.dup; - foreach(ref key; x.keys) - key && (key = key.copy()); - } - static if (is(E == StructInitExpression)) - mixin(doCopy("values[]")); - static if (is(E == StringExpression)) - x.str = x.str.dup; - } - } - else - static if (is(Statement) && is(T : Statement)) - { - alias T S; - static if (is(S == CompoundStatement)) - mixin(doCopy("stmnts[]")); - //IllegalStatement, - //EmptyStatement have no subnodes. - static if (is(S == FuncBodyStatement)) - mixin(doCopy(["funcBody?", "inBody?", "outBody?"])); - static if (is(S == ScopeStatement) || is(S == LabeledStatement)) - mixin(doCopy("s")); - static if (is(S == ExpressionStatement)) - mixin(doCopy("e")); - static if (is(S == DeclarationStatement)) - mixin(doCopy("decl")); - static if (is(S == IfStatement)) - { - if (x.variable) - mixin(doCopy("variable")); - else - mixin(doCopy("condition")); - mixin(doCopy(["ifBody", "elseBody?"])); - } - static if (is(S == WhileStatement)) - mixin(doCopy(["condition", "whileBody"])); - static if (is(S == DoWhileStatement)) - mixin(doCopy(["doBody", "condition"])); - static if (is(S == ForStatement)) - mixin(doCopy(["init?", "condition?", "increment?", "forBody"])); - static if (is(S == ForeachStatement)) - mixin(doCopy(["params", "aggregate", "forBody"])); - static if (is(S == ForeachRangeStatement)) - mixin(doCopy(["params", "lower", "upper", "forBody"])); - static if (is(S == SwitchStatement)) - mixin(doCopy(["condition", "switchBody"])); - static if (is(S == CaseStatement)) - mixin(doCopy(["values[]", "caseBody"])); - static if (is(S == DefaultStatement)) - mixin(doCopy("defaultBody")); - //ContinueStatement, - //BreakStatement have no subnodes. - static if (is(S == ReturnStatement)) - mixin(doCopy("e?")); - static if (is(S == GotoStatement)) - mixin(doCopy("caseExpr?")); - static if (is(S == WithStatement)) - mixin(doCopy(["e", "withBody"])); - static if (is(S == SynchronizedStatement)) - mixin(doCopy(["e?", "syncBody"])); - static if (is(S == TryStatement)) - mixin(doCopy(["tryBody", "catchBodies[]", "finallyBody?"])); - static if (is(S == CatchStatement)) - mixin(doCopy(["param?", "catchBody"])); - static if (is(S == FinallyStatement)) - mixin(doCopy("finallyBody")); - static if (is(S == ScopeGuardStatement)) - mixin(doCopy("scopeBody")); - static if (is(S == ThrowStatement)) - mixin(doCopy("e")); - static if (is(S == VolatileStatement)) - mixin(doCopy("volatileBody?")); - static if (is(S == AsmBlockStatement)) - mixin(doCopy("statements")); - static if (is(S == AsmStatement)) - mixin(doCopy("operands[]")); - //AsmAlignStatement, - //IllegalAsmStatement have no subnodes. - static if (is(S == PragmaStatement)) - mixin(doCopy(["args[]", "pragmaBody"])); - static if (is(S == MixinStatement)) - mixin(doCopy("templateExpr")); - static if (is(S == StaticIfStatement)) - mixin(doCopy(["condition", "ifBody", "elseBody?"])); - static if (is(S == StaticAssertStatement)) - mixin(doCopy(["condition", "message?"])); - static if (is(S == DebugStatement) || is(S == VersionStatement)) - mixin(doCopy(["mainBody", "elseBody?"])); - } - else - static if (is(TypeNode) && is(T : TypeNode)) - { - //IllegalType, - //IntegralType, - //ModuleScopeType, - //IdentifierType have no subnodes. - static if (is(T == QualifiedType)) - mixin(doCopy(["lhs", "rhs"])); - static if (is(T == TypeofType)) - mixin(doCopy("e")); - static if (is(T == TemplateInstanceType)) - mixin(doCopy("targs?")); - static if (is(T == PointerType)) - mixin(doCopy("next")); - static if (is(T == ArrayType)) - mixin(doCopy(["next", "assocType?", "e1?", "e2?"])); - static if (is(T == FunctionType) || is(T == DelegateType)) - mixin(doCopy(["returnType", "params"])); - static if (is(T == CFuncPointerType)) - mixin(doCopy(["next", "params?"])); - static if (is(T == BaseClassType) || - is(T == ConstType) || - is(T == InvariantType)) - mixin(doCopy("next")); - } - else - static if (is(Parameter) && is(T == Parameter)) - { - mixin(doCopy(["type?", "defValue?"])); - } - else - static if (is(Parameter) && is(T == Parameters) || - is(TemplateParameter) && is(T == TemplateParameters) || - is(TemplateArguments) && is(T == TemplateArguments)) - { - mixin(doCopy("children[]")); - } - else - static if (is(TemplateParameter) && is(T : TemplateParameter)) - { - static if (is(N == TemplateAliasParameter) || - is(N == TemplateTypeParameter) || - is(N == TemplateThisParameter)) - mixin(doCopy(["specType", "defType"])); - static if (is(N == TemplateValueParameter)) - mixin(doCopy(["valueType", "specValue?", "defValue?"])); - //TemplateTupleParameter has no subnodes. - } - else - static assert(0, "copying of "~typeof(x).stringof~" is not handled"); - return x; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/NodesEnum.d --- a/trunk/src/dil/ast/NodesEnum.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,234 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.NodesEnum; - -/// Enumerates the categories of a node. -enum NodeCategory : ushort -{ - Undefined, - Declaration, - Statement, - Expression, - Type, - Other -} - -/// A list of all class names that inherit from Node. -static const char[][] g_classNames = [ - // Declarations: - "CompoundDeclaration", - "EmptyDeclaration", - "IllegalDeclaration", - "ModuleDeclaration", - "ImportDeclaration", - "AliasDeclaration", - "TypedefDeclaration", - "EnumDeclaration", - "EnumMemberDeclaration", - "ClassDeclaration", - "InterfaceDeclaration", - "StructDeclaration", - "UnionDeclaration", - "ConstructorDeclaration", - "StaticConstructorDeclaration", - "DestructorDeclaration", - "StaticDestructorDeclaration", - "FunctionDeclaration", - "VariablesDeclaration", - "InvariantDeclaration", - "UnittestDeclaration", - "DebugDeclaration", - "VersionDeclaration", - "StaticIfDeclaration", - "StaticAssertDeclaration", - "TemplateDeclaration", - "NewDeclaration", - "DeleteDeclaration", - "ProtectionDeclaration", - "StorageClassDeclaration", - "LinkageDeclaration", - "AlignDeclaration", - "PragmaDeclaration", - "MixinDeclaration", - - // Statements: - "CompoundStatement", - "IllegalStatement", - "EmptyStatement", - "FuncBodyStatement", - "ScopeStatement", - "LabeledStatement", - "ExpressionStatement", - "DeclarationStatement", - "IfStatement", - "WhileStatement", - "DoWhileStatement", - "ForStatement", - "ForeachStatement", - "ForeachRangeStatement", // D2.0 - "SwitchStatement", - "CaseStatement", - "DefaultStatement", - "ContinueStatement", - "BreakStatement", - "ReturnStatement", - "GotoStatement", - "WithStatement", - "SynchronizedStatement", - "TryStatement", - "CatchStatement", - "FinallyStatement", - "ScopeGuardStatement", - "ThrowStatement", - "VolatileStatement", - "AsmBlockStatement", - "AsmStatement", - "AsmAlignStatement", - "IllegalAsmStatement", - "PragmaStatement", - "MixinStatement", - "StaticIfStatement", - "StaticAssertStatement", - "DebugStatement", - "VersionStatement", - - // Expressions: - "IllegalExpression", - "CondExpression", - "CommaExpression", - "OrOrExpression", - "AndAndExpression", - "OrExpression", - "XorExpression", - "AndExpression", - "EqualExpression", - "IdentityExpression", - "RelExpression", - "InExpression", - "LShiftExpression", - "RShiftExpression", - "URShiftExpression", - "PlusExpression", - "MinusExpression", - "CatExpression", - "MulExpression", - "DivExpression", - "ModExpression", - "AssignExpression", - "LShiftAssignExpression", - "RShiftAssignExpression", - "URShiftAssignExpression", - "OrAssignExpression", - "AndAssignExpression", - "PlusAssignExpression", - "MinusAssignExpression", - "DivAssignExpression", - "MulAssignExpression", - "ModAssignExpression", - "XorAssignExpression", - "CatAssignExpression", - "AddressExpression", - "PreIncrExpression", - "PreDecrExpression", - "PostIncrExpression", - "PostDecrExpression", - "DerefExpression", - "SignExpression", - "NotExpression", - "CompExpression", - "CallExpression", - "NewExpression", - "NewAnonClassExpression", - "DeleteExpression", - "CastExpression", - "IndexExpression", - "SliceExpression", - "ModuleScopeExpression", - "IdentifierExpression", - "SpecialTokenExpression", - "DotExpression", - "TemplateInstanceExpression", - "ThisExpression", - "SuperExpression", - "NullExpression", - "DollarExpression", - "BoolExpression", - "IntExpression", - "RealExpression", - "ComplexExpression", - "CharExpression", - "StringExpression", - "ArrayLiteralExpression", - "AArrayLiteralExpression", - "AssertExpression", - "MixinExpression", - "ImportExpression", - "TypeofExpression", - "TypeDotIdExpression", - "TypeidExpression", - "IsExpression", - "ParenExpression", - "FunctionLiteralExpression", - "TraitsExpression", // D2.0 - "VoidInitExpression", - "ArrayInitExpression", - "StructInitExpression", - "AsmTypeExpression", - "AsmOffsetExpression", - "AsmSegExpression", - "AsmPostBracketExpression", - "AsmBracketExpression", - "AsmLocalSizeExpression", - "AsmRegisterExpression", - - // Types: - "IllegalType", - "IntegralType", - "QualifiedType", - "ModuleScopeType", - "IdentifierType", - "TypeofType", - "TemplateInstanceType", - "PointerType", - "ArrayType", - "FunctionType", - "DelegateType", - "CFuncPointerType", - "BaseClassType", - "ConstType", // D2.0 - "InvariantType", // D2.0 - - // Other: - "Parameter", - "Parameters", - "TemplateAliasParameter", - "TemplateTypeParameter", - "TemplateThisParameter", // D2.0 - "TemplateValueParameter", - "TemplateTupleParameter", - "TemplateParameters", - "TemplateArguments", -]; - -/// Generates the members of enum NodeKind. -char[] generateNodeKindMembers() -{ - char[] text; - foreach (className; g_classNames) - text ~= className ~ ","; - return text; -} -// pragma(msg, generateNodeKindMembers()); - -version(DDoc) - /// The node kind identifies every class that inherits from Node. - enum NodeKind : ushort; -else -mixin( - "enum NodeKind : ushort" - "{" - ~ generateNodeKindMembers ~ - "}" -); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Parameters.d --- a/trunk/src/dil/ast/Parameters.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,227 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Parameters; - -import dil.ast.Node; -import dil.ast.Type; -import dil.ast.Expression; -import dil.ast.NodeCopier; -import dil.lexer.Identifier; -import dil.Enums; - -/// A function or foreach parameter. -class Parameter : Node -{ - StorageClass stc; /// The storage classes of the parameter. - TypeNode type; /// The parameter's type. - Identifier* name; /// The name of the parameter. - Expression defValue; /// The default initialization value. - - this(StorageClass stc, TypeNode type, Identifier* name, Expression defValue) - { - super(NodeCategory.Other); - mixin(set_kind); - // type can be null when param in foreach statement - addOptChild(type); - addOptChild(defValue); - - this.stc = stc; - this.type = type; - this.name = name; - this.defValue = defValue; - } - - /// Returns true if this is a D-style variadic parameter. - /// E.g.: func(int[] values ...) - bool isDVariadic() - { - return isVariadic && !isCVariadic; - } - - /// Returns true if this is a C-style variadic parameter. - /// E.g.: func(...) - bool isCVariadic() - { - return stc == StorageClass.Variadic && - type is null && name is null; - } - - /// Returns true if this is a D- or C-style variadic parameter. - bool isVariadic() - { - return !!(stc & StorageClass.Variadic); - } - - mixin(copyMethod); -} - -/// Array of parameters. -class Parameters : Node -{ - this() - { - super(NodeCategory.Other); - mixin(set_kind); - } - - bool hasVariadic() - { - if (children.length != 0) - return items[$-1].isVariadic(); - return false; - } - - void opCatAssign(Parameter param) - { addChild(param); } - - Parameter[] items() - { return cast(Parameter[])children; } - - size_t length() - { return children.length; } - - mixin(copyMethod); -} - -/*~~~~~~~~~~~~~~~~~~~~~~ -~ Template parameters: ~ -~~~~~~~~~~~~~~~~~~~~~~*/ - -/// Abstract base class for all template parameters. -abstract class TemplateParameter : Node -{ - Identifier* ident; - this(Identifier* ident) - { - super(NodeCategory.Other); - this.ident = ident; - } - override abstract TemplateParameter copy(); -} - -/// E.g.: (alias T) -class TemplateAliasParameter : TemplateParameter -{ - TypeNode specType, defType; - this(Identifier* ident, TypeNode specType, TypeNode defType) - { - super(ident); - mixin(set_kind); - addOptChild(specType); - addOptChild(defType); - this.ident = ident; - this.specType = specType; - this.defType = defType; - } - mixin(copyMethod); -} - -/// E.g.: (T t) -class TemplateTypeParameter : TemplateParameter -{ - TypeNode specType, defType; - this(Identifier* ident, TypeNode specType, TypeNode defType) - { - super(ident); - mixin(set_kind); - addOptChild(specType); - addOptChild(defType); - this.ident = ident; - this.specType = specType; - this.defType = defType; - } - mixin(copyMethod); -} - -// version(D2) -// { -/// E.g.: (this T) -class TemplateThisParameter : TemplateParameter -{ - TypeNode specType, defType; - this(Identifier* ident, TypeNode specType, TypeNode defType) - { - super(ident); - mixin(set_kind); - addOptChild(specType); - addOptChild(defType); - this.ident = ident; - this.specType = specType; - this.defType = defType; - } - mixin(copyMethod); -} -// } - -/// E.g.: (T) -class TemplateValueParameter : TemplateParameter -{ - TypeNode valueType; - Expression specValue, defValue; - this(TypeNode valueType, Identifier* ident, Expression specValue, Expression defValue) - { - super(ident); - mixin(set_kind); - addChild(valueType); - addOptChild(specValue); - addOptChild(defValue); - this.valueType = valueType; - this.ident = ident; - this.specValue = specValue; - this.defValue = defValue; - } - mixin(copyMethod); -} - -/// E.g.: (T...) -class TemplateTupleParameter : TemplateParameter -{ - this(Identifier* ident) - { - super(ident); - mixin(set_kind); - this.ident = ident; - } - mixin(copyMethod); -} - -/// Array of template parameters. -class TemplateParameters : Node -{ - this() - { - super(NodeCategory.Other); - mixin(set_kind); - } - - void opCatAssign(TemplateParameter parameter) - { - addChild(parameter); - } - - TemplateParameter[] items() - { - return cast(TemplateParameter[])children; - } - - mixin(copyMethod); -} - -/// Array of template arguments. -class TemplateArguments : Node -{ - this() - { - super(NodeCategory.Other); - mixin(set_kind); - } - - void opCatAssign(Node argument) - { - addChild(argument); - } - - mixin(copyMethod); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Statement.d --- a/trunk/src/dil/ast/Statement.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,18 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Statement; - -import dil.ast.Node; - -/// The root class of all statements. -abstract class Statement : Node -{ - this() - { - super(NodeCategory.Statement); - } - - override abstract Statement copy(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Statements.d --- a/trunk/src/dil/ast/Statements.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,607 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Statements; - -public import dil.ast.Statement; -import dil.ast.Node; -import dil.ast.Expression; -import dil.ast.Declaration; -import dil.ast.Type; -import dil.ast.Parameters; -import dil.ast.NodeCopier; -import dil.lexer.IdTable; - -class CompoundStatement : Statement -{ - this() - { - mixin(set_kind); - } - - void opCatAssign(Statement s) - { - addChild(s); - } - - Statement[] stmnts() - { - return cast(Statement[])this.children; - } - - void stmnts(Statement[] stmnts) - { - this.children = stmnts; - } - - mixin(copyMethod); -} - -class IllegalStatement : Statement -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class EmptyStatement : Statement -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class FuncBodyStatement : Statement -{ - Statement funcBody, inBody, outBody; - Identifier* outIdent; - this() - { - mixin(set_kind); - } - - void finishConstruction() - { - addOptChild(funcBody); - addOptChild(inBody); - addOptChild(outBody); - } - - bool isEmpty() - { - return funcBody is null; - } - - mixin(copyMethod); -} - -class ScopeStatement : Statement -{ - Statement s; - this(Statement s) - { - mixin(set_kind); - addChild(s); - this.s = s; - } - mixin(copyMethod); -} - -class LabeledStatement : Statement -{ - Identifier* label; - Statement s; - this(Identifier* label, Statement s) - { - mixin(set_kind); - addChild(s); - this.label = label; - this.s = s; - } - mixin(copyMethod); -} - -class ExpressionStatement : Statement -{ - Expression e; - this(Expression e) - { - mixin(set_kind); - addChild(e); - this.e = e; - } - mixin(copyMethod); -} - -class DeclarationStatement : Statement -{ - Declaration decl; - this(Declaration decl) - { - mixin(set_kind); - addChild(decl); - this.decl = decl; - } - mixin(copyMethod); -} - -class IfStatement : Statement -{ - Statement variable; // AutoDeclaration or VariableDeclaration - Expression condition; - Statement ifBody; - Statement elseBody; - this(Statement variable, Expression condition, Statement ifBody, Statement elseBody) - { - mixin(set_kind); - if (variable) - addChild(variable); - else - addChild(condition); - addChild(ifBody); - addOptChild(elseBody); - - this.variable = variable; - this.condition = condition; - this.ifBody = ifBody; - this.elseBody = elseBody; - } - mixin(copyMethod); -} - -class WhileStatement : Statement -{ - Expression condition; - Statement whileBody; - this(Expression condition, Statement whileBody) - { - mixin(set_kind); - addChild(condition); - addChild(whileBody); - - this.condition = condition; - this.whileBody = whileBody; - } - mixin(copyMethod); -} - -class DoWhileStatement : Statement -{ - Statement doBody; - Expression condition; - this(Expression condition, Statement doBody) - { - mixin(set_kind); - addChild(doBody); - addChild(condition); - - this.condition = condition; - this.doBody = doBody; - } - mixin(copyMethod); -} - -class ForStatement : Statement -{ - Statement init; - Expression condition, increment; - Statement forBody; - - this(Statement init, Expression condition, Expression increment, Statement forBody) - { - mixin(set_kind); - addOptChild(init); - addOptChild(condition); - addOptChild(increment); - addChild(forBody); - - this.init = init; - this.condition = condition; - this.increment = increment; - this.forBody = forBody; - } - mixin(copyMethod); -} - -class ForeachStatement : Statement -{ - TOK tok; - Parameters params; - Expression aggregate; - Statement forBody; - - this(TOK tok, Parameters params, Expression aggregate, Statement forBody) - { - mixin(set_kind); - addChildren([cast(Node)params, aggregate, forBody]); - - this.tok = tok; - this.params = params; - this.aggregate = aggregate; - this.forBody = forBody; - } - mixin(copyMethod); -} - -// version(D2) -// { -class ForeachRangeStatement : Statement -{ - TOK tok; - Parameters params; - Expression lower, upper; - Statement forBody; - - this(TOK tok, Parameters params, Expression lower, Expression upper, Statement forBody) - { - mixin(set_kind); - addChildren([cast(Node)params, lower, upper, forBody]); - - this.tok = tok; - this.params = params; - this.lower = lower; - this.upper = upper; - this.forBody = forBody; - } - mixin(copyMethod); -} -// } - -class SwitchStatement : Statement -{ - Expression condition; - Statement switchBody; - - this(Expression condition, Statement switchBody) - { - mixin(set_kind); - addChild(condition); - addChild(switchBody); - - this.condition = condition; - this.switchBody = switchBody; - } - mixin(copyMethod); -} - -class CaseStatement : Statement -{ - Expression[] values; - Statement caseBody; - - this(Expression[] values, Statement caseBody) - { - mixin(set_kind); - addChildren(values); - addChild(caseBody); - - this.values = values; - this.caseBody = caseBody; - } - mixin(copyMethod); -} - -class DefaultStatement : Statement -{ - Statement defaultBody; - this(Statement defaultBody) - { - mixin(set_kind); - addChild(defaultBody); - - this.defaultBody = defaultBody; - } - mixin(copyMethod); -} - -class ContinueStatement : Statement -{ - Identifier* ident; - this(Identifier* ident) - { - mixin(set_kind); - this.ident = ident; - } - mixin(copyMethod); -} - -class BreakStatement : Statement -{ - Identifier* ident; - this(Identifier* ident) - { - mixin(set_kind); - this.ident = ident; - } - mixin(copyMethod); -} - -class ReturnStatement : Statement -{ - Expression e; - this(Expression e) - { - mixin(set_kind); - addOptChild(e); - this.e = e; - } - mixin(copyMethod); -} - -class GotoStatement : Statement -{ - Identifier* ident; - Expression caseExpr; - this(Identifier* ident, Expression caseExpr) - { - mixin(set_kind); - addOptChild(caseExpr); - this.ident = ident; - this.caseExpr = caseExpr; - } - mixin(copyMethod); -} - -class WithStatement : Statement -{ - Expression e; - Statement withBody; - this(Expression e, Statement withBody) - { - mixin(set_kind); - addChild(e); - addChild(withBody); - - this.e = e; - this.withBody = withBody; - } - mixin(copyMethod); -} - -class SynchronizedStatement : Statement -{ - Expression e; - Statement syncBody; - this(Expression e, Statement syncBody) - { - mixin(set_kind); - addOptChild(e); - addChild(syncBody); - - this.e = e; - this.syncBody = syncBody; - } - mixin(copyMethod); -} - -class TryStatement : Statement -{ - Statement tryBody; - CatchStatement[] catchBodies; - FinallyStatement finallyBody; - this(Statement tryBody, CatchStatement[] catchBodies, FinallyStatement finallyBody) - { - mixin(set_kind); - addChild(tryBody); - addOptChildren(catchBodies); - addOptChild(finallyBody); - - this.tryBody = tryBody; - this.catchBodies = catchBodies; - this.finallyBody = finallyBody; - } - mixin(copyMethod); -} - -class CatchStatement : Statement -{ - Parameter param; - Statement catchBody; - this(Parameter param, Statement catchBody) - { - mixin(set_kind); - addOptChild(param); - addChild(catchBody); - this.param = param; - this.catchBody = catchBody; - } - mixin(copyMethod); -} - -class FinallyStatement : Statement -{ - Statement finallyBody; - this(Statement finallyBody) - { - mixin(set_kind); - addChild(finallyBody); - this.finallyBody = finallyBody; - } - mixin(copyMethod); -} - -class ScopeGuardStatement : Statement -{ - Identifier* condition; - Statement scopeBody; - this(Identifier* condition, Statement scopeBody) - { - mixin(set_kind); - addChild(scopeBody); - this.condition = condition; - this.scopeBody = scopeBody; - } - mixin(copyMethod); -} - -class ThrowStatement : Statement -{ - Expression e; - this(Expression e) - { - mixin(set_kind); - addChild(e); - this.e = e; - } - mixin(copyMethod); -} - -class VolatileStatement : Statement -{ - Statement volatileBody; - this(Statement volatileBody) - { - mixin(set_kind); - addOptChild(volatileBody); - this.volatileBody = volatileBody; - } - mixin(copyMethod); -} - -class AsmBlockStatement : Statement -{ - CompoundStatement statements; - this(CompoundStatement statements) - { - mixin(set_kind); - addChild(statements); - this.statements = statements; - } - mixin(copyMethod); -} - -class AsmStatement : Statement -{ - Identifier* ident; - Expression[] operands; - this(Identifier* ident, Expression[] operands) - { - mixin(set_kind); - addOptChildren(operands); - this.ident = ident; - this.operands = operands; - } - mixin(copyMethod); -} - -class AsmAlignStatement : Statement -{ - int number; - this(int number) - { - mixin(set_kind); - this.number = number; - } - mixin(copyMethod); -} - -class IllegalAsmStatement : IllegalStatement -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -class PragmaStatement : Statement -{ - Identifier* ident; - Expression[] args; - Statement pragmaBody; - this(Identifier* ident, Expression[] args, Statement pragmaBody) - { - mixin(set_kind); - addOptChildren(args); - addChild(pragmaBody); - - this.ident = ident; - this.args = args; - this.pragmaBody = pragmaBody; - } - mixin(copyMethod); -} - -class MixinStatement : Statement -{ - Expression templateExpr; - Identifier* mixinIdent; - this(Expression templateExpr, Identifier* mixinIdent) - { - mixin(set_kind); - addChild(templateExpr); - this.templateExpr = templateExpr; - this.mixinIdent = mixinIdent; - } - mixin(copyMethod); -} - -class StaticIfStatement : Statement -{ - Expression condition; - Statement ifBody, elseBody; - this(Expression condition, Statement ifBody, Statement elseBody) - { - mixin(set_kind); - addChild(condition); - addChild(ifBody); - addOptChild(elseBody); - this.condition = condition; - this.ifBody = ifBody; - this.elseBody = elseBody; - } - mixin(copyMethod); -} - -class StaticAssertStatement : Statement -{ - Expression condition, message; - this(Expression condition, Expression message) - { - mixin(set_kind); - addChild(condition); - addOptChild(message); - this.condition = condition; - this.message = message; - } - mixin(copyMethod); -} - -abstract class ConditionalCompilationStatement : Statement -{ - Token* cond; - Statement mainBody, elseBody; - this(Token* cond, Statement mainBody, Statement elseBody) - { - addChild(mainBody); - addOptChild(elseBody); - this.cond = cond; - this.mainBody = mainBody; - this.elseBody = elseBody; - } -} - -class DebugStatement : ConditionalCompilationStatement -{ - this(Token* cond, Statement debugBody, Statement elseBody) - { - super(cond, debugBody, elseBody); - mixin(set_kind); - } - mixin(copyMethod); -} - -class VersionStatement : ConditionalCompilationStatement -{ - this(Token* cond, Statement versionBody, Statement elseBody) - { - super(cond, versionBody, elseBody); - mixin(set_kind); - } - mixin(copyMethod); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Type.d --- a/trunk/src/dil/ast/Type.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,38 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Type; - -import dil.ast.Node; -import dil.semantic.Types; - -/// The root class of all type nodes. -abstract class TypeNode : Node -{ - TypeNode next; /// The next type in the type chain. - Type type; /// The semantic type of this type node. - - this() - { - this(null); - } - - this(TypeNode next) - { - super(NodeCategory.Type); - addOptChild(next); - this.next = next; - } - - /// Returns the root type of the type chain. - TypeNode baseType() - { - auto type = this; - while (type.next) - type = type.next; - return type; - } - - override abstract TypeNode copy(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Types.d --- a/trunk/src/dil/ast/Types.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,262 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Types; - -public import dil.ast.Type; -import dil.ast.Node; -import dil.ast.Expression; -import dil.ast.Parameters; -import dil.ast.NodeCopier; -import dil.lexer.Identifier; -import dil.semantic.Types; -import dil.Enums; - -/// Syntax error. -class IllegalType : TypeNode -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -/// char, int, float etc. -class IntegralType : TypeNode -{ - TOK tok; - this(TOK tok) - { - mixin(set_kind); - this.tok = tok; - } - mixin(copyMethod); -} - -/// Identifier -class IdentifierType : TypeNode -{ - Identifier* ident; - this(Identifier* ident) - { - mixin(set_kind); - this.ident = ident; - } - mixin(copyMethod); -} - -/// Type "." Type -class QualifiedType : TypeNode -{ - alias next lhs; /// Left-hand side type. - TypeNode rhs; /// Right-hand side type. - this(TypeNode lhs, TypeNode rhs) - { - super(lhs); - mixin(set_kind); - addChild(rhs); - this.rhs = rhs; - } - mixin(copyMethod); -} - -/// "." Type -class ModuleScopeType : TypeNode -{ - this() - { - mixin(set_kind); - } - mixin(copyMethod); -} - -/// "typeof" "(" Expression ")" or$(BR) -/// "typeof" "(" "return" ")" (D2.0) -class TypeofType : TypeNode -{ - Expression e; - this(Expression e) - { - this(); - addChild(e); - this.e = e; - } - - // For D2.0: "typeof" "(" "return" ")" - this() - { - mixin(set_kind); - } - - bool isTypeofReturn() - { - return e is null; - } - - mixin(copyMethod); -} - -/// Identifier "!" "(" TemplateParameters? ")" -class TemplateInstanceType : TypeNode -{ - Identifier* ident; - TemplateArguments targs; - this(Identifier* ident, TemplateArguments targs) - { - mixin(set_kind); - addOptChild(targs); - this.ident = ident; - this.targs = targs; - } - mixin(copyMethod); -} - -/// Type * -class PointerType : TypeNode -{ - this(TypeNode next) - { - super(next); - mixin(set_kind); - } - mixin(copyMethod); -} - -/// Dynamic array: T[] or$(BR) -/// Static array: T[E] or$(BR) -/// Slice array (for tuples): T[E..E] or$(BR) -/// Associative array: T[T] -class ArrayType : TypeNode -{ - Expression e1, e2; - TypeNode assocType; - - this(TypeNode t) - { - super(t); - mixin(set_kind); - } - - this(TypeNode t, Expression e1, Expression e2) - { - this(t); - addChild(e1); - addOptChild(e2); - this.e1 = e1; - this.e2 = e2; - } - - this(TypeNode t, TypeNode assocType) - { - this(t); - addChild(assocType); - this.assocType = assocType; - } - - bool isDynamic() - { - return !assocType && !e1; - } - - bool isStatic() - { - return e1 && !e2; - } - - bool isSlice() - { - return e1 && e2; - } - - bool isAssociative() - { - return assocType !is null; - } - - mixin(copyMethod); -} - -/// ReturnType "function" "(" Parameters? ")" -class FunctionType : TypeNode -{ - alias next returnType; - Parameters params; - this(TypeNode returnType, Parameters params) - { - super(returnType); - mixin(set_kind); - addChild(params); - this.params = params; - } - mixin(copyMethod); -} - -/// ReturnType "delegate" "(" Parameters? ")" -class DelegateType : TypeNode -{ - alias next returnType; - Parameters params; - this(TypeNode returnType, Parameters params) - { - super(returnType); - mixin(set_kind); - addChild(params); - this.params = params; - } - mixin(copyMethod); -} - -/// Type "(" BasicType2 Identifier ")" "(" Parameters? ")" -class CFuncPointerType : TypeNode -{ - Parameters params; - this(TypeNode type, Parameters params) - { - super(type); - mixin(set_kind); - addOptChild(params); - } - mixin(copyMethod); -} - -/// "class" Identifier : BaseClasses -class BaseClassType : TypeNode -{ - Protection prot; - this(Protection prot, TypeNode type) - { - super(type); - mixin(set_kind); - this.prot = prot; - } - mixin(copyMethod); -} - -// version(D2) -// { -/// "const" "(" Type ")" -class ConstType : TypeNode -{ - this(TypeNode next) - { - // If t is null: cast(const) - super(next); - mixin(set_kind); - } - mixin(copyMethod); -} - -/// "invariant" "(" Type ")" -class InvariantType : TypeNode -{ - this(TypeNode next) - { - // If t is null: cast(invariant) - super(next); - mixin(set_kind); - } - mixin(copyMethod); -} -// } // version(D2) diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/ast/Visitor.d --- a/trunk/src/dil/ast/Visitor.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,155 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.ast.Visitor; - -import dil.ast.Node; -import dil.ast.Declarations, - dil.ast.Expressions, - dil.ast.Statements, - dil.ast.Types, - dil.ast.Parameters; - -/// Generate visit methods. -/// -/// E.g.: -/// --- -/// Declaration visit(ClassDeclaration){return null;}; -/// Expression visit(CommaExpression){return null;}; -/// --- -char[] generateVisitMethods() -{ - char[] text; - foreach (className; g_classNames) - text ~= "returnType!(\""~className~"\") visit("~className~" node){return node;}\n"; - return text; -} -// pragma(msg, generateAbstractVisitMethods()); - -/// Gets the appropriate return type for the provided class. -template returnType(char[] className) -{ - static if (is(typeof(mixin(className)) : Declaration)) - alias Declaration returnType; - else - static if (is(typeof(mixin(className)) : Statement)) - alias Statement returnType; - else - static if (is(typeof(mixin(className)) : Expression)) - alias Expression returnType; - else - static if (is(typeof(mixin(className)) : TypeNode)) - alias TypeNode returnType; - else - alias Node returnType; -} - -/// Generate functions which do the second dispatch. -/// -/// E.g.: -/// --- -/// Expression visitCommaExpression(Visitor visitor, CommaExpression c) -/// { visitor.visit(c); /* Second dispatch. */ } -/// --- -/// The equivalent in the traditional visitor pattern would be: -/// --- -/// class CommaExpression : Expression -/// { -/// void accept(Visitor visitor) -/// { visitor.visit(this); } -/// } -/// --- -char[] generateDispatchFunctions() -{ - char[] text; - foreach (className; g_classNames) - text ~= "returnType!(\""~className~"\") visit"~className~"(Visitor visitor, "~className~" c)\n" - "{ return visitor.visit(c); }\n"; - return text; -} -// pragma(msg, generateDispatchFunctions()); - -/++ - Generates an array of function pointers. - - --- - [ - cast(void*)&visitCommaExpression, - // etc. - ] - --- -+/ -char[] generateVTable() -{ - char[] text = "["; - foreach (className; g_classNames) - text ~= "cast(void*)&visit"~className~",\n"; - return text[0..$-2]~"]"; // slice away last ",\n" -} -// pragma(msg, generateVTable()); - -/// Implements a variation of the visitor pattern. -/// -/// Inherited by classes that need to traverse a D syntax tree -/// and do computations, transformations and other things on it. -abstract class Visitor -{ - mixin(generateVisitMethods()); - - static - mixin(generateDispatchFunctions()); - - /// The table holding function pointers to the second dispatch functions. - static const void*[] dispatch_vtable = mixin(generateVTable()); - static assert(dispatch_vtable.length == g_classNames.length, "vtable length doesn't match number of classes"); - - /// Looks up the second dispatch function for n and returns that. - Node function(Visitor, Node) getDispatchFunction()(Node n) - { - return cast(Node function(Visitor, Node))dispatch_vtable[n.kind]; - } - - /// The main and first dispatch function. - Node dispatch(Node n) - { // Second dispatch is done in the called function. - return getDispatchFunction(n)(this, n); - } - -final: - Declaration visit(Declaration n) - { return visitD(n); } - Statement visit(Statement n) - { return visitS(n); } - Expression visit(Expression n) - { return visitE(n); } - TypeNode visit(TypeNode n) - { return visitT(n); } - Node visit(Node n) - { return visitN(n); } - - Declaration visitD(Declaration n) - { - return cast(Declaration)cast(void*)dispatch(n); - } - - Statement visitS(Statement n) - { - return cast(Statement)cast(void*)dispatch(n); - } - - Expression visitE(Expression n) - { - return cast(Expression)cast(void*)dispatch(n); - } - - TypeNode visitT(TypeNode n) - { - return cast(TypeNode)cast(void*)dispatch(n); - } - - Node visitN(Node n) - { - return dispatch(n); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/doc/Doc.d --- a/trunk/src/dil/doc/Doc.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,463 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.doc.Doc; - -import dil.doc.Parser; -import dil.ast.Node; -import dil.lexer.Funcs; -import dil.Unicode; -import common; - -import tango.text.Ascii : icompare; - -/// Represents a sanitized and parsed DDoc comment. -class DDocComment -{ - Section[] sections; /// The sections of this comment. - Section summary; /// Optional summary section. - Section description; /// Optional description section. - - this(Section[] sections, Section summary, Section description) - { - this.sections = sections; - this.summary = summary; - this.description = description; - } - - /// Removes the first copyright section and returns it. - Section takeCopyright() - { - foreach (i, section; sections) - if (section.Is("copyright")) - { - sections = sections[0..i] ~ sections[i+1..$]; - return section; - } - return null; - } - - /// Returns true if "ditto" is the only text in this comment. - bool isDitto() - { - if (summary && sections.length == 1 && - icompare(strip(summary.text), "ditto") == 0) - return true; - return false; - } -} - -/// Returns a node's DDocComment. -DDocComment getDDocComment(Node node) -{ - DDocParser p; - auto docTokens = getDocTokens(node); - if (!docTokens.length) - return null; - p.parse(getDDocText(docTokens)); - return new DDocComment(p.sections, p.summary, p.description); -} - -/// Strips leading and trailing whitespace characters. -/// Whitespace: ' ', '\t', '\v', '\f' and '\n' -/// Returns: a slice into str. -char[] strip(char[] str) -{ - if (str.length == 0) - return null; - uint i; - for (; i < str.length; i++) - if (!isspace(str[i]) && str[i] != '\n') - break; - if (str.length == i) - return null; - str = str[i..$]; - assert(str.length); - for (i = str.length; i; i--) - if (!isspace(str[i-1]) && str[i-1] != '\n') - break; - return str[0..i]; -} - -/// Parses a DDoc comment string. -struct DDocParser -{ - char* p; /// Current character pointer. - char* textEnd; /// Points one character past the end of the text. - Section[] sections; /// Parsed sections. - Section summary; /// Optional summary section. - Section description; /// Optional description section. - - /// Parses the DDoc text into sections. - Section[] parse(string text) - { - if (!text.length) - return null; - p = text.ptr; - textEnd = p + text.length; - - char* summaryBegin; - string ident, nextIdent; - char* bodyBegin, nextBodyBegin; - - skipWhitespace(p); - summaryBegin = p; - - if (findNextIdColon(ident, bodyBegin)) - { // Check that this is not an explicit section. - if (summaryBegin != ident.ptr) - scanSummaryAndDescription(summaryBegin, ident.ptr); - } - else // There are no explicit sections. - { - scanSummaryAndDescription(summaryBegin, textEnd); - return sections; - } - - assert(ident.length); - // Continue parsing. - while (findNextIdColon(nextIdent, nextBodyBegin)) - { - sections ~= new Section(ident, textBody(bodyBegin, nextIdent.ptr)); - ident = nextIdent; - bodyBegin = nextBodyBegin; - } - // Add last section. - sections ~= new Section(ident, textBody(bodyBegin, textEnd)); - return sections; - } - - /// Returns the text body. Trailing whitespace characters are not included. - char[] textBody(char* begin, char* end) - { - // The body of A is empty, e.g.: - // A: - // B: some text - // ^- begin and end point to B (or to this.textEnd in the 2nd case.) - if (begin is end) - return ""; - // Remove trailing whitespace. - while (isspace(*--end) || *end == '\n') - {} - end++; - return makeString(begin, end); - } - - /// Separates the text between p and end - /// into a summary and description section. - void scanSummaryAndDescription(char* p, char* end) - { - assert(p <= end); - char* sectionBegin = p; - // Search for the end of the first paragraph. - end--; // Decrement end, so we can look ahead one character. - while (p < end && !(*p == '\n' && p[1] == '\n')) - { - if (isCodeSection(p, end)) - skipCodeSection(p, end); - p++; - } - end++; - if (p+1 >= end) - p = end; - assert(p == end || (*p == '\n' && p[1] == '\n')); - // The first paragraph is the summary. - summary = new Section("", makeString(sectionBegin, p)); - sections ~= summary; - // The rest is the description section. - if (p < end) - { - skipWhitespace(p); - sectionBegin = p; - if (p < end) - { - description = new Section("", makeString(sectionBegin, end)); - sections ~= description; - } - } - } - - /// Returns true if p points to "$(DDD)". - bool isCodeSection(char* p, char* end) - { - return p+2 < end && *p == '-' && p[1] == '-' && p[2] == '-'; - } - - /// Skips over a code section. - /// - /// Note that dmd apparently doesn't skip over code sections when - /// parsing DDoc sections. However, from experience it seems - /// to be a good idea to do that. - void skipCodeSection(ref char* p, char* end) - out { assert(p+1 == end || *p == '-'); } - body - { - assert(isCodeSection(p, end)); - - while (p < end && *p == '-') - p++; - p--; - while (++p < end) - if (p+2 < end && *p == '-' && p[1] == '-' && p[2] == '-') - break; - while (p < end && *p == '-') - p++; - p--; - } - - void skipWhitespace(ref char* p) - { - while (p < textEnd && (isspace(*p) || *p == '\n')) - p++; - } - - /// Find next "Identifier:". - /// Params: - /// ident = set to the Identifier. - /// bodyBegin = set to the beginning of the text body (whitespace skipped.) - /// Returns: true if found. - bool findNextIdColon(ref char[] ident, ref char* bodyBegin) - { - while (p < textEnd) - { - skipWhitespace(p); - if (p >= textEnd) - break; - if (isCodeSection(p, textEnd)) - { - skipCodeSection(p, textEnd); - p++; - continue; - } - assert(isascii(*p) || isLeadByte(*p)); - auto idBegin = p; - if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart - { - do // IdChar* - p++; - while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) - auto idEnd = p; - if (p < textEnd && *p == ':') // : - { - p++; - skipWhitespace(p); - bodyBegin = p; - ident = makeString(idBegin, idEnd); - return true; - } - } - // Skip this line. - while (p < textEnd && *p != '\n') - p++; - } - return false; - } -} - -/// Represents a DDoc section. -class Section -{ - string name; - string text; - this(string name, string text) - { - this.name = name; - this.text = text; - } - - /// Case-insensitively compares the section's name with name2. - bool Is(char[] name2) - { - return icompare(name, name2) == 0; - } -} - -class ParamsSection : Section -{ - string[] paramNames; /// Parameter names. - string[] paramDescs; /// Parameter descriptions. - this(string name, string text) - { - super(name, text); - IdentValueParser parser; - auto idvalues = parser.parse(text); - this.paramNames = new string[idvalues.length]; - this.paramDescs = new string[idvalues.length]; - foreach (i, idvalue; idvalues) - { - this.paramNames[i] = idvalue.ident; - this.paramDescs[i] = idvalue.value; - } - } -} - -class MacrosSection : Section -{ - string[] macroNames; /// Macro names. - string[] macroTexts; /// Macro texts. - this(string name, string text) - { - super(name, text); - IdentValueParser parser; - auto idvalues = parser.parse(text); - this.macroNames = new string[idvalues.length]; - this.macroTexts = new string[idvalues.length]; - foreach (i, idvalue; idvalues) - { - this.macroNames[i] = idvalue.ident; - this.macroTexts[i] = idvalue.value; - } - } -} - -/// Returns true if token is a Doxygen comment. -bool isDoxygenComment(Token* token) -{ // Doxygen: '/+!' '/*!' '//!' - return token.kind == TOK.Comment && token.start[2] == '!'; -} - -/// Returns true if token is a DDoc comment. -bool isDDocComment(Token* token) -{ // DDOC: '/++' '/**' '///' - return token.kind == TOK.Comment && token.start[1] == token.start[2]; -} - -/// Returns the surrounding documentation comment tokens. -/// Params: -/// node = the node to find doc comments for. -/// isDocComment = a function predicate that checks for doc comment tokens. -/// Note: this function works correctly only if -/// the source text is syntactically correct. -Token*[] getDocTokens(Node node, bool function(Token*) isDocComment = &isDDocComment) -{ - Token*[] comments; - auto isEnumMember = node.kind == NodeKind.EnumMemberDeclaration; - // Get preceding comments. - auto token = node.begin; - // Scan backwards until we hit another declaration. -Loop: - for (; token; token = token.prev) - { - if (token.kind == TOK.LBrace || - token.kind == TOK.RBrace || - token.kind == TOK.Semicolon || - /+token.kind == TOK.HEAD ||+/ - (isEnumMember && token.kind == TOK.Comma)) - break; - - if (token.kind == TOK.Comment) - { // Check that this comment doesn't belong to the previous declaration. - switch (token.prev.kind) - { - case TOK.Semicolon, TOK.RBrace, TOK.Comma: - break Loop; - default: - if (isDocComment(token)) - comments = [token] ~ comments; - } - } - } - // Get single comment to the right. - token = node.end.next; - if (token.kind == TOK.Comment && isDocComment(token)) - comments ~= token; - else if (isEnumMember) - { - token = node.end.nextNWS; - if (token.kind == TOK.Comma) - { - token = token.next; - if (token.kind == TOK.Comment && isDocComment(token)) - comments ~= token; - } - } - return comments; -} - -bool isLineComment(Token* t) -{ - assert(t.kind == TOK.Comment); - return t.start[1] == '/'; -} - -/// Extracts the text body of the comment tokens. -string getDDocText(Token*[] tokens) -{ - if (tokens.length == 0) - return null; - string result; - foreach (token; tokens) - { - auto n = isLineComment(token) ? 0 : 2; // 0 for "//", 2 for "+/" and "*/". - result ~= sanitize(token.srcText[3 .. $-n], token.start[1]); - assert(token.next); - if (token.next.kind == TOK.Newline) - result ~= \n; - else - result ~= ' '; - } -// Stdout.formatln("→{}←", result); - return result[0..$-1]; // Remove \n or ' ' -} - -/// Sanitizes a DDoc comment string. -/// -/// Leading "commentChar"s are removed from the lines. -/// The various newline types are converted to '\n'. -/// Params: -/// comment = the string to be sanitized. -/// commentChar = '/', '+', or '*' -string sanitize(string comment, char commentChar) -{ - alias comment result; - - bool newline = true; // True when at the beginning of a new line. - uint i, j; - auto len = result.length; - for (; i < len; i++, j++) - { - if (newline) - { // Ignore commentChars at the beginning of each new line. - newline = false; - auto begin = i; - while (i < len && isspace(result[i])) - i++; - if (i < len && result[i] == commentChar) - while (++i < len && result[i] == commentChar) - {} - else - i = begin; // Reset. No commentChar found. - if (i >= len) - break; - } - // Check for Newline. - switch (result[i]) - { - case '\r': - if (i+1 < len && result[i+1] == '\n') - i++; - case '\n': - result[j] = '\n'; // Copy Newline as '\n'. - newline = true; - continue; - default: - if (!isascii(result[i]) && i+2 < len && isUnicodeNewline(result.ptr + i)) - { - i += 2; - goto case '\n'; - } - } - // Copy character. - result[j] = result[i]; - } - result.length = j; // Adjust length. - // Lastly, strip trailing commentChars. - if (!result.length) - return null; - i = result.length; - for (; i && result[i-1] == commentChar; i--) - {} - result.length = i; - return result; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/doc/Macro.d --- a/trunk/src/dil/doc/Macro.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,348 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.doc.Macro; - -import dil.doc.Parser; -import dil.lexer.Funcs; -import dil.Unicode; -import dil.Information; -import dil.Messages; -import common; - -/// The DDoc macro class. -class Macro -{ - string name; /// The name of the macro. - string text; /// The substitution text. - uint callLevel; /// Recursive call level. - this (string name, string text) - { - this.name = name; - this.text = text; - } -} - -/// Maps macro names to Macro objects. -/// -/// MacroTables can be chained so that they build a linear hierarchy. -/// Macro definitions in the current table override the ones in the parent tables. -class MacroTable -{ - /// The parent in the hierarchy. Or null if this is the root. - MacroTable parent; - Macro[string] table; /// The associative array that holds the macro definitions. - - /// Constructs a MacroTable instance. - this(MacroTable parent = null) - { - this.parent = parent; - } - - /// Inserts the macro m into the table. - /// Overwrites the current macro if one exists. - void insert(Macro m) - { - table[m.name] = m; - } - - /// Inserts an array of macros into the table. - void insert(Macro[] macros) - { - foreach (m; macros) - insert(m); - } - - /// Creates a macro using name and text and inserts that into the table. - void insert(string name, string text) - { - insert(new Macro(name, text)); - } - - /// Creates a macro using name[n] and text[n] and inserts that into the table. - void insert(string[] names, string[] texts) - { - assert(names.length == texts.length); - foreach (i, name; names) - insert(name, texts[i]); - } - - /// Searches for a macro. - /// - /// If the macro isn't found in this table the search - /// continues upwards in the table hierarchy. - /// Returns: the macro if found, or null if not. - Macro search(string name) - { - auto pmacro = name in table; - if (pmacro) - return *pmacro; - if (!isRoot()) - return parent.search(name); - return null; - } - - /// Returns: true if this is the root of the hierarchy. - bool isRoot() - { return parent is null; } -} - -/// Parses a text with macro definitions. -struct MacroParser -{ - Macro[] parse(string text) - { - IdentValueParser parser; - auto idvalues = parser.parse(text); - auto macros = new Macro[idvalues.length]; - foreach (i, idvalue; idvalues) - macros[i] = new Macro(idvalue.ident, idvalue.value); - return macros; - } - - /// Scans for a macro invocation. E.g.: $(DDOC) - /// Returns: a pointer set to one char past the closing parenthesis, - /// or null if this isn't a macro invocation. - static char* scanMacro(char* p, char* textEnd) - { - assert(*p == '$'); - if (p+2 < textEnd && p[1] == '(') - { - p += 2; - if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart - { - do // IdChar* - p++; - while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) - MacroExpander.scanArguments(p, textEnd); - p != textEnd && p++; // Skip ')'. - return p; - } - } - return null; - } -} - -/// Expands DDoc macros in a text. -struct MacroExpander -{ - MacroTable mtable; /// Used to look up macros. - InfoManager infoMan; /// Collects warning messages. - char[] filePath; /// Used in warning messages. - - /// Starts expanding the macros. - static char[] expand(MacroTable mtable, char[] text, char[] filePath, - InfoManager infoMan = null) - { - MacroExpander me; - me.mtable = mtable; - me.infoMan = infoMan; - me.filePath = filePath; - return me.expandMacros(text); - } - - /// Reports a warning message. - void warning(char[] msg, char[] macroName) - { - msg = Format(msg, macroName); - if (infoMan) - infoMan ~= new Warning(new Location(filePath, 0), msg); - } - - /// Expands the macros from the table in the text. - char[] expandMacros(char[] text, char[] prevArg0 = null/+, uint depth = 1000+/) - { - // if (depth == 0) - // return text; - // depth--; - char[] result; - char* p = text.ptr; - char* textEnd = p + text.length; - char* macroEnd = p; - while (p+3 < textEnd) // minimum 4 chars: $(x) - { - if (*p == '$' && p[1] == '(') - { - // Copy string between macros. - if (macroEnd != p) - result ~= makeString(macroEnd, p); - p += 2; - auto idBegin = p; - if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart - { - do // IdChar* - p++; - while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) - // Create macro name. - auto macroName = makeString(idBegin, p); - // Get arguments. - auto macroArgs = scanArguments(p, textEnd); - if (p == textEnd) - { - warning(MSG.UnterminatedDDocMacro, macroName); - result ~= "$(" ~ macroName ~ " "; - } - else - p++; - macroEnd = p; // Point past ')'. - - auto macro_ = mtable.search(macroName); - if (macro_) - { // Ignore recursive macro if: - auto macroArg0 = macroArgs.length ? macroArgs[0] : null; - if (macro_.callLevel != 0 && - (macroArgs.length == 0/+ || // Macro has no arguments. - prevArg0 == macroArg0+/)) // macroArg0 equals previous arg0. - { continue; } - macro_.callLevel++; - // Expand the arguments in the macro text. - auto expandedText = expandArguments(macro_.text, macroArgs); - result ~= expandMacros(expandedText, macroArg0/+, depth+/); - macro_.callLevel--; - } - else - { - warning(MSG.UndefinedDDocMacro, macroName); - //result ~= makeString(macroName.ptr-2, macroEnd); - } - continue; - } - } - p++; - } - if (macroEnd == text.ptr) - return text; // No macros found. Return original text. - if (macroEnd < textEnd) - result ~= makeString(macroEnd, textEnd); - return result; - } - - /// Scans until the closing parenthesis is found. Sets p to one char past it. - /// Returns: [arg0, arg1, arg2 ...]. - static char[][] scanArguments(ref char* p, char* textEnd) - out(args) { assert(args.length != 1); } - body - { - // D specs: "The argument text can contain nested parentheses, - // "" or '' strings, comments, or tags." - uint level = 1; // Nesting level of the parentheses. - char[][] args; - - // Skip leading spaces. - while (p < textEnd && isspace(*p)) - p++; - - char* arg0Begin = p; // Whole argument list. - char* argBegin = p; - MainLoop: - while (p < textEnd) - { - switch (*p) - { - case ',': - if (level != 1) // Ignore comma if inside (). - break; - // Add a new argument. - args ~= makeString(argBegin, p); - while (++p < textEnd && isspace(*p)) // Skip spaces. - {} - argBegin = p; - continue; - case '(': - level++; - break; - case ')': - if (--level == 0) - break MainLoop; - break; - // Commented out: causes too many problems in the expansion pass. - // case '"', '\'': - // auto c = *p; - // while (++p < textEnd && *p != c) // Scan to next " or '. - // {} - // assert(*p == c || p == textEnd); - // if (p == textEnd) - // break MainLoop; - // break; - case '<': - p++; - if (p+2 < textEnd && *p == '!' && p[1] == '-' && p[2] == '-') // ". - while (++p < textEnd) - if (p+2 < textEnd && *p == '-' && p[1] == '-' && p[2] == '>') - p += 2; // Point to '>'. - } // or - else if (p < textEnd && (isalpha(*p) || *p == '/')) - while (++p < textEnd && *p != '>') // Skip to closing '>'. - {} - else - continue MainLoop; - if (p == textEnd) - break MainLoop; - assert(*p == '>'); - break; - default: - } - p++; - } - assert(*p == ')' && level == 0 || p == textEnd); - if (arg0Begin == p) - return null; - // arg0 spans the whole argument list. - auto arg0 = makeString(arg0Begin, p); - // Add last argument. - args ~= makeString(argBegin, p); - return arg0 ~ args; - } - - /// Expands "$+", "$0" - "$9" with args[n] in text. - /// Params: - /// text = the text to scan for argument placeholders. - /// args = the first element, args[0], is the whole argument string and - /// the following elements are slices into it.$(BR) - /// The array is empty if there are no arguments. - char[] expandArguments(char[] text, char[][] args) - in { assert(args.length != 1, "zero or more than 1 args expected"); } - body - { - char[] result; - char* p = text.ptr; - char* textEnd = p + text.length; - char* placeholderEnd = p; - - while (p+1 < textEnd) - { - if (*p == '$' && (*++p == '+' || isdigit(*p))) - { - // Copy string between argument placeholders. - if (placeholderEnd != p-1) - result ~= makeString(placeholderEnd, p-1); - placeholderEnd = p+1; // Set new placeholder end. - - if (args.length == 0) - continue; - - if (*p == '+') - { // $+ = $2 to $n - if (args.length > 2) - result ~= makeString(args[2].ptr, args[0].ptr + args[0].length); - } - else - { // 0 - 9 - uint nthArg = *p - '0'; - if (nthArg < args.length) - result ~= args[nthArg]; - } - } - p++; - } - if (placeholderEnd == text.ptr) - return text; // No placeholders found. Return original text. - if (placeholderEnd < textEnd) - result ~= makeString(placeholderEnd, textEnd); - return result; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/doc/Parser.d --- a/trunk/src/dil/doc/Parser.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,131 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.doc.Parser; - -import dil.lexer.Funcs; -import dil.Unicode; -import common; - -/// A pair of strings. -class IdentValue -{ - string ident; - string value; - this (string ident, string value) - { - this.ident = ident; - this.value = value; - } -} - -/// Parses text of the form: -/// 
    -/// ident = value
-/// ident2 = value2
-///          more text
-///
    -struct IdentValueParser -{ - char* p; /// Current pointer. - char* textEnd; - - IdentValue[] parse(string text) - { - if (!text.length) - return null; - - p = text.ptr; - textEnd = p + text.length; - - IdentValue[] idvalues; - - string ident, nextIdent; - char* bodyBegin = p, nextBodyBegin; - - // Init. - findNextIdent(ident, bodyBegin); - // Continue. - while (findNextIdent(nextIdent, nextBodyBegin)) - { - idvalues ~= new IdentValue(ident, textBody(bodyBegin, nextIdent.ptr)); - ident = nextIdent; - bodyBegin = nextBodyBegin; - } - // Add last ident value. - idvalues ~= new IdentValue(ident, textBody(bodyBegin, textEnd)); - return idvalues; - } - - /// Returns the text body. Trailing whitespace characters are not included. - char[] textBody(char* begin, char* end) - { - // The body of A is empty, e.g.: - // A = - // B = some text - // ^- begin and end point to B (or to this.textEnd in the 2nd case.) - if (begin is end) - return ""; - // Remove trailing whitespace. - while (isspace(*--end) || *end == '\n') - {} - end++; - return makeString(begin, end); - } - - /// Finds the next "Identifier =". - /// Params: - /// ident = set to Identifier. - /// bodyBegin = set to the beginning of the text body (whitespace skipped.) - /// Returns: true if found. - bool findNextIdent(ref string ident, ref char* bodyBegin) - { - while (p < textEnd) - { - skipWhitespace(); - if (p >= textEnd) - break; - auto idBegin = p; - if (isidbeg(*p) || isUnicodeAlpha(p, textEnd)) // IdStart - { - do // IdChar* - p++; - while (p < textEnd && (isident(*p) || isUnicodeAlpha(p, textEnd))) - auto idEnd = p; - - skipWhitespace(); - if (p < textEnd && *p == '=') - { - p++; - skipWhitespace(); - bodyBegin = p; - ident = makeString(idBegin, idEnd); - return true; - } - } - skipLine(); - } - return false; - } - - void skipWhitespace() - { - while (p < textEnd && (isspace(*p) || *p == '\n')) - p++; - } - - void skipLine() - { - while (p < textEnd && *p != '\n') - p++; - p++; - } -} - -/// Returns a string slice ranging from begin to end. -char[] makeString(char* begin, char* end) -{ - assert(begin && end && begin <= end); - return begin[0 .. end - begin]; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/Funcs.d --- a/trunk/src/dil/lexer/Funcs.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,174 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.Funcs; - -const char[3] LS = \u2028; /// Unicode line separator. -const dchar LSd = 0x2028; /// ditto -const char[3] PS = \u2029; /// Unicode paragraph separator. -const dchar PSd = 0x2029; /// ditto -static assert(LS[0] == PS[0] && LS[1] == PS[1]); - -const dchar _Z_ = 26; /// Control+Z. - -/// Returns: true if d is a Unicode line or paragraph separator. -bool isUnicodeNewlineChar(dchar d) -{ - return d == LSd || d == PSd; -} - -/// Returns: true if p points to a line or paragraph separator. -bool isUnicodeNewline(char* p) -{ - return *p == LS[0] && p[1] == LS[1] && (p[2] == LS[2] || p[2] == PS[2]); -} - -/// Returns: true if p points to the start of a Newline. -/// Newline: \n | \r | \r\n | LS | PS -bool isNewline(char* p) -{ - return *p == '\n' || *p == '\r' || isUnicodeNewline(p); -} - -/// Returns: true if c is a Newline character. -bool isNewline(dchar c) -{ - return c == '\n' || c == '\r' || isUnicodeNewlineChar(c); -} - -/// Returns: true if p points to an EOF character. -/// EOF: 0 | _Z_ -bool isEOF(dchar c) -{ - return c == 0 || c == _Z_; -} - -/// Returns: true if p points to the first character of an EndOfLine. -/// EndOfLine: Newline | EOF -bool isEndOfLine(char* p) -{ - return isNewline(p) || isEOF(*p); -} - -/// Scans a Newline and sets p one character past it. -/// Returns: '\n' if found or 0 otherwise. -dchar scanNewline(ref char* p) -{ - switch (*p) - { - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - ++p; - return '\n'; - default: - if (isUnicodeNewline(p)) - { - p += 3; - return '\n'; - } - } - return 0; -} - -/// ASCII character properties table. -static const int ptable[256] = [ - 0, 0, 0, 0, 0, 0, 0, 0, 0,32, 0,32,32, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -32, 0, 0x2200, 0, 0, 0, 0, 0x2700, 0, 0, 0, 0, 0, 0, 0, 0, - 7, 7, 7, 7, 7, 7, 7, 7, 6, 6, 0, 0, 0, 0, 0, 0x3f00, - 0,12,12,12,12,12,12, 8, 8, 8, 8, 8, 8, 8, 8, 8, - 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 0, 0x5c00, 0, 0,16, - 0, 0x70c, 0x80c,12,12,12, 0xc0c, 8, 8, 8, 8, 8, 8, 8, 0xa08, 8, - 8, 8, 0xd08, 8, 0x908, 8, 0xb08, 8, 8, 8, 8, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -]; - -/// Enumeration of character property flags. -enum CProperty -{ - Octal = 1, /// 0-7 - Digit = 1<<1, /// 0-9 - Hex = 1<<2, /// 0-9a-fA-F - Alpha = 1<<3, /// a-zA-Z - Underscore = 1<<4, /// _ - Whitespace = 1<<5 /// ' ' \t \v \f -} - -const uint EVMask = 0xFF00; // Bit mask for escape value. - -private alias CProperty CP; -/// Returns: true if c is an octal digit. -int isoctal(char c) { return ptable[c] & CP.Octal; } -/// Returns: true if c is a decimal digit. -int isdigit(char c) { return ptable[c] & CP.Digit; } -/// Returns: true if c is a hexadecimal digit. -int ishexad(char c) { return ptable[c] & CP.Hex; } -/// Returns: true if c is a letter. -int isalpha(char c) { return ptable[c] & CP.Alpha; } -/// Returns: true if c is an alphanumeric. -int isalnum(char c) { return ptable[c] & (CP.Alpha | CP.Digit); } -/// Returns: true if c is the beginning of a D identifier (only ASCII.) -int isidbeg(char c) { return ptable[c] & (CP.Alpha | CP.Underscore); } -/// Returns: true if c is a D identifier character (only ASCII.) -int isident(char c) { return ptable[c] & (CP.Alpha | CP.Underscore | CP.Digit); } -/// Returns: true if c is a whitespace character. -int isspace(char c) { return ptable[c] & CP.Whitespace; } -/// Returns: the escape value for c. -int char2ev(char c) { return ptable[c] >> 8; /*(ptable[c] & EVMask) >> 8;*/ } -/// Returns: true if c is an ASCII character. -int isascii(uint c) { return c < 128; } - -version(gen_ptable) -static this() -{ - alias ptable p; - assert(p.length == 256); - // Initialize character properties table. - for (int i; i < p.length; ++i) - { - p[i] = 0; // Reset - if ('0' <= i && i <= '7') - p[i] |= CP.Octal; - if ('0' <= i && i <= '9') - p[i] |= CP.Digit | CP.Hex; - if ('a' <= i && i <= 'f' || 'A' <= i && i <= 'F') - p[i] |= CP.Hex; - if ('a' <= i && i <= 'z' || 'A' <= i && i <= 'Z') - p[i] |= CP.Alpha; - if (i == '_') - p[i] |= CP.Underscore; - if (i == ' ' || i == '\t' || i == '\v' || i == '\f') - p[i] |= CP.Whitespace; - } - // Store escape sequence values in second byte. - assert(CProperty.max <= ubyte.max, "character property flags and escape value byte overlap."); - p['\''] |= 39 << 8; - p['"'] |= 34 << 8; - p['?'] |= 63 << 8; - p['\\'] |= 92 << 8; - p['a'] |= 7 << 8; - p['b'] |= 8 << 8; - p['f'] |= 12 << 8; - p['n'] |= 10 << 8; - p['r'] |= 13 << 8; - p['t'] |= 9 << 8; - p['v'] |= 11 << 8; - // Print a formatted array literal. - char[] array = "[\n"; - foreach (i, c; ptable) - { - array ~= Format((c>255?" 0x{0:x},":"{0,2},"), c) ~ (((i+1) % 16) ? "":"\n"); - } - array[$-2..$] = "\n]"; - Stdout(array).newline; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/IdTable.d --- a/trunk/src/dil/lexer/IdTable.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,162 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.IdTable; - -import dil.lexer.TokensEnum; -import dil.lexer.IdentsGenerator; -import dil.lexer.Keywords; -import common; - -public import dil.lexer.Identifier; -public import dil.lexer.IdentsEnum; - -/// A namespace for the predefined identifiers. -struct Ident -{ - const static - { - mixin(generateIdentMembers()); - } - - static Identifier*[] allIds() - { - return __allIds; - } -} - -/// Global table for hoarding and retrieving identifiers. -struct IdTable -{ -static: - /// A set of common, predefined identifiers for fast lookups. - private Identifier*[string] staticTable; - /// A table that grows with every newly found, unique identifier. - private Identifier*[string] growingTable; - - /// Loads keywords and predefined identifiers into the static table. - static this() - { - foreach (ref k; g_reservedIds) - staticTable[k.str] = &k; - foreach (id; Ident.allIds()) - staticTable[id.str] = id; - staticTable.rehash; - } - - /// Looks up idString in both tables. - Identifier* lookup(string idString) - { - auto id = inStatic(idString); - if (id) - return id; - return inGrowing(idString); - } - - /// Looks up idString in the static table. - Identifier* inStatic(string idString) - { - auto id = idString in staticTable; - return id ? *id : null; - } - - alias Identifier* function(string idString) LookupFunction; - /// Looks up idString in the growing table. - LookupFunction inGrowing = &_inGrowing_unsafe; // Default to unsafe function. - - /// Sets the thread safety mode of the growing table. - void setThreadsafe(bool b) - { - if (b) - inGrowing = &_inGrowing_safe; - else - inGrowing = &_inGrowing_unsafe; - } - - /// Returns true if access to the growing table is thread-safe. - bool isThreadsafe() - { - return inGrowing is &_inGrowing_safe; - } - - /// Looks up idString in the table. - /// - /// Adds idString to the table if not found. - private Identifier* _inGrowing_unsafe(string idString) - out(id) - { assert(id !is null); } - body - { - auto id = idString in growingTable; - if (id) - return *id; - auto newID = Identifier(idString, TOK.Identifier); - growingTable[idString] = newID; - return newID; - } - - /// Looks up idString in the table. - /// - /// Adds idString to the table if not found. - /// Access to the data structure is synchronized. - private Identifier* _inGrowing_safe(string idString) - { - synchronized - return _inGrowing_unsafe(idString); - } - - /+ - Identifier* addIdentifiers(char[][] idStrings) - { - auto ids = new Identifier*[idStrings.length]; - foreach (i, idString; idStrings) - { - Identifier** id = idString in tabulatedIds; - if (!id) - { - auto newID = Identifier(TOK.Identifier, idString); - tabulatedIds[idString] = newID; - id = &newID; - } - ids[i] = *id; - } - } - +/ - - static uint anonCount; /// Counter for anonymous identifiers. - - /// Generates an anonymous identifier. - /// - /// Concatenates prefix with anonCount. - /// The identifier is not inserted into the table. - Identifier* genAnonymousID(string prefix) - { - ++anonCount; - auto x = anonCount; - // Convert count to a string and append it to str. - char[] num; - do - num = cast(char)('0' + (x % 10)) ~ num; - while (x /= 10) - return Identifier(prefix ~ num, TOK.Identifier); - } - - /// Generates an identifier for an anonymous enum. - Identifier* genAnonEnumID() - { - return genAnonymousID("__anonenum"); - } -} - -unittest -{ - // TODO: write benchmark. - // Single table - - // Single table. synchronized - - // Two tables. - - // Two tables. synchronized -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/Identifier.d --- a/trunk/src/dil/lexer/Identifier.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,54 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.Identifier; - -import dil.lexer.TokensEnum; -import dil.lexer.IdentsEnum; -import common; - -/// Represents an identifier as defined in the D specs. -/// -///
    -///  Identifier := IdStart IdChar*
-///  IdStart := "_" | Letter
-///  IdChar := IdStart | "0"-"9"
-///  Letter := UniAlpha
-///
    -/// Unicode alphas are defined in Unicode 5.0.0. -align(1) -struct Identifier -{ - string str; /// The UTF-8 string of the identifier. - TOK kind; /// The token kind. - IDK idKind; /// Only for predefined identifiers. - - static Identifier* opCall(string str, TOK kind) - { - auto id = new Identifier; - id.str = str; - id.kind = kind; - return id; - } - - static Identifier* opCall(string str, TOK kind, IDK idKind) - { - auto id = new Identifier; - id.str = str; - id.kind = kind; - id.idKind = idKind; - return id; - } - - uint toHash() - { - uint hash; - foreach(c; str) { - hash *= 11; - hash += c; - } - return hash; - } -} -// pragma(msg, Identifier.sizeof.stringof); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/IdentsEnum.d --- a/trunk/src/dil/lexer/IdentsEnum.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,18 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.IdentsEnum; - -import dil.lexer.IdentsGenerator; - -version(DDoc) - enum IDK : ushort; /// Enumeration of predefined identifier kinds. -else -mixin( - // Enumerates predefined identifiers. - "enum IDK : ushort {" - "Null," - ~ generateIDMembers ~ - "}" -); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/IdentsGenerator.d --- a/trunk/src/dil/lexer/IdentsGenerator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,117 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.IdentsGenerator; - -struct StrPair -{ -const: - char[] str; /// Identifier string in code. - char[] idStr; /// In table. -} - -/// Table of predefined identifiers. -static const StrPair[] identPairs = [ - // Predefined version identifiers: - {"DigitalMars"}, {"X86"}, {"X86_64"}, - /*{"Windows"}, */{"Win32"}, {"Win64"}, - {"linux"}, {"LittleEndian"}, {"BigEndian"}, - {"D_Coverage"}, {"D_InlineAsm_X86"}, {"D_Version2"}, - {"none"}, {"all"}, - // Variadic parameters: - {"_arguments"}, {"_argptr"}, - // scope: - {"exit"}, {"success"}, {"failure"}, - // pragma: - {"msg"}, {"lib"}, {"startaddress"}, - // Linkage: - {"C"}, {"D"}, {"Windows"}, {"Pascal"}, {"System"}, - // Con-/Destructor: - {"__ctor"}, {"__dtor"}, - // new() and delete() methods. - {"__new"}, {"__delete"}, - // Unittest and invariant. - {"__unittest"}, {"__invariant"}, - // Operator methods: - {"opNeg"}, - {"opPos"}, - {"opComp"}, - {"opAddAssign"}, - {"opSubAssign"}, - {"opPostInc"}, - {"opPostDec"}, - {"opCall"}, - {"opCast"}, - {"opIndex"}, - {"opSlice"}, - // ASM identifiers: - {"near"}, {"far"}, {"word"}, {"dword"}, {"qword"}, - {"ptr"}, {"offset"}, {"seg"}, {"__LOCAL_SIZE"}, - {"FS"}, {"ST"}, - {"AL"}, {"AH"}, {"AX"}, {"EAX"}, - {"BL"}, {"BH"}, {"BX"}, {"EBX"}, - {"CL"}, {"CH"}, {"CX"}, {"ECX"}, - {"DL"}, {"DH"}, {"DX"}, {"EDX"}, - {"BP"}, {"EBP"}, {"SP"}, {"ESP"}, - {"DI"}, {"EDI"}, {"SI"}, {"ESI"}, - {"ES"}, {"CS"}, {"SS"}, {"DS"}, {"GS"}, - {"CR0"}, {"CR2"}, {"CR3"}, {"CR4"}, - {"DR0"}, {"DR1"}, {"DR2"}, {"DR3"}, {"DR6"}, {"DR7"}, - {"TR3"}, {"TR4"}, {"TR5"}, {"TR6"}, {"TR7"}, - {"MM0"}, {"MM1"}, {"MM2"}, {"MM3"}, - {"MM4"}, {"MM5"}, {"MM6"}, {"MM7"}, - {"XMM0"}, {"XMM1"}, {"XMM2"}, {"XMM3"}, - {"XMM4"}, {"XMM5"}, {"XMM6"}, {"XMM7"}, -]; - -/++ - CTF for generating the members of the struct Ident. - - The resulting string looks like this: - --- - private struct Ids {static const: - Identifier _str = {"str", TOK.Identifier, IDK.str}; - // more ... - } - Identifier* str = &Ids._str; - // more ... - private Identifier*[] __allIds = [ - str, - // more ... - ] - --- -+/ -char[] generateIdentMembers() -{ - char[] private_members = "private struct Ids {static const:"; - - char[] public_members = ""; - char[] array = "private Identifier*[] __allIds = ["; - foreach (pair; identPairs) - { - // N.B.: Compiler cries for some reason when trying to access pair.idStr. - // Identifier _str = {"str", TOK.Identifier, ID.str}; - private_members ~= "Identifier _"~pair.str~` = {"`~pair.str~`", TOK.Identifier, IDK.`~pair.str~"};\n"; - // Identifier* str = &_str; - public_members ~= "Identifier* "~pair.str~" = &Ids._"~pair.str~";\n"; - array ~= pair.str~","; - } - - private_members ~= "}"; // Close private { - array ~= "];"; - - return private_members ~ public_members ~ array; -} - -/// CTF for generating the members of the enum IDK. -char[] generateIDMembers() -{ - char[] members; - foreach (pair; identPairs) - members ~= pair.str ~ ",\n"; - return members; -} - -// pragma(msg, generateIdentMembers()); -// pragma(msg, generateIDMembers()); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/Keywords.d --- a/trunk/src/dil/lexer/Keywords.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,122 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.Keywords; - -import dil.lexer.Token; -import dil.lexer.Identifier; - -/// Table of reserved identifiers. -static const Identifier[] g_reservedIds = [ - {"abstract", TOK.Abstract}, - {"alias", TOK.Alias}, - {"align", TOK.Align}, - {"asm", TOK.Asm}, - {"assert", TOK.Assert}, - {"auto", TOK.Auto}, - {"body", TOK.Body}, - {"bool", TOK.Bool}, - {"break", TOK.Break}, - {"byte", TOK.Byte}, - {"case", TOK.Case}, - {"cast", TOK.Cast}, - {"catch", TOK.Catch}, - {"cdouble", TOK.Cdouble}, - {"cent", TOK.Cent}, - {"cfloat", TOK.Cfloat}, - {"char", TOK.Char}, - {"class", TOK.Class}, - {"const", TOK.Const}, - {"continue", TOK.Continue}, - {"creal", TOK.Creal}, - {"dchar", TOK.Dchar}, - {"debug", TOK.Debug}, - {"default", TOK.Default}, - {"delegate", TOK.Delegate}, - {"delete", TOK.Delete}, - {"deprecated", TOK.Deprecated}, - {"do", TOK.Do}, - {"double", TOK.Double}, - {"else", TOK.Else}, - {"enum", TOK.Enum}, - {"export", TOK.Export}, - {"extern", TOK.Extern}, - {"false", TOK.False}, - {"final", TOK.Final}, - {"finally", TOK.Finally}, - {"float", TOK.Float}, - {"for", TOK.For}, - {"foreach", TOK.Foreach}, - {"foreach_reverse", TOK.Foreach_reverse}, - {"function", TOK.Function}, - {"goto", TOK.Goto}, - {"idouble", TOK.Idouble}, - {"if", TOK.If}, - {"ifloat", TOK.Ifloat}, - {"import", TOK.Import}, - {"in", TOK.In}, - {"inout", TOK.Inout}, - {"int", TOK.Int}, - {"interface", TOK.Interface}, - {"invariant", TOK.Invariant}, - {"ireal", TOK.Ireal}, - {"is", TOK.Is}, - {"lazy", TOK.Lazy}, - {"long", TOK.Long}, - {"macro", TOK.Macro}, // D2.0 - {"mixin", TOK.Mixin}, - {"module", TOK.Module}, - {"new", TOK.New}, - {"nothrow", TOK.Nothrow}, // D2.0 - {"null", TOK.Null}, - {"out", TOK.Out}, - {"override", TOK.Override}, - {"package", TOK.Package}, - {"pragma", TOK.Pragma}, - {"private", TOK.Private}, - {"protected", TOK.Protected}, - {"public", TOK.Public}, - {"pure", TOK.Pure}, // D2.0 - {"real", TOK.Real}, - {"ref", TOK.Ref}, - {"return", TOK.Return}, - {"scope", TOK.Scope}, - {"short", TOK.Short}, - {"static", TOK.Static}, - {"struct", TOK.Struct}, - {"super", TOK.Super}, - {"switch", TOK.Switch}, - {"synchronized", TOK.Synchronized}, - {"template", TOK.Template}, - {"this", TOK.This}, - {"throw", TOK.Throw}, - {"__traits", TOK.Traits}, // D2.0 - {"true", TOK.True}, - {"try", TOK.Try}, - {"typedef", TOK.Typedef}, - {"typeid", TOK.Typeid}, - {"typeof", TOK.Typeof}, - {"ubyte", TOK.Ubyte}, - {"ucent", TOK.Ucent}, - {"uint", TOK.Uint}, - {"ulong", TOK.Ulong}, - {"union", TOK.Union}, - {"unittest", TOK.Unittest}, - {"ushort", TOK.Ushort}, - {"version", TOK.Version}, - {"void", TOK.Void}, - {"volatile", TOK.Volatile}, - {"wchar", TOK.Wchar}, - {"while", TOK.While}, - {"with", TOK.With}, - // Special tokens: - {"__FILE__", TOK.FILE}, - {"__LINE__", TOK.LINE}, - {"__DATE__", TOK.DATE}, - {"__TIME__", TOK.TIME}, - {"__TIMESTAMP__", TOK.TIMESTAMP}, - {"__VENDOR__", TOK.VENDOR}, - {"__VERSION__", TOK.VERSION}, - {"__EOF__", TOK.EOF}, // D2.0 -]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/Lexer.d --- a/trunk/src/dil/lexer/Lexer.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2900 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.Lexer; - -import dil.lexer.Token; -import dil.lexer.Keywords; -import dil.lexer.Identifier; -import dil.lexer.IdTable; -import dil.Information; -import dil.Messages; -import dil.HtmlEntities; -import dil.CompilerInfo; -import dil.Unicode; -import dil.SourceText; -import dil.Time; -import common; - -import tango.stdc.stdlib : strtof, strtod, strtold; -import tango.stdc.errno : errno, ERANGE; - -public import dil.lexer.Funcs; - -/// The Lexer analyzes the characters of a source text and -/// produces a doubly-linked list of tokens. -class Lexer -{ - SourceText srcText; /// The source text. - char* p; /// Points to the current character in the source text. - char* end; /// Points one character past the end of the source text. - - Token* head; /// The head of the doubly linked token list. - Token* tail; /// The tail of the linked list. Set in scan(). - Token* token; /// Points to the current token in the token list. - - // Members used for error messages: - InfoManager infoMan; - LexerError[] errors; - /// Always points to the first character of the current line. - char* lineBegin; -// Token* newline; /// Current newline token. - uint lineNum = 1; /// Current, actual source text line number. - uint lineNum_hline; /// Line number set by #line. - uint inTokenString; /// > 0 if inside q{ } - /// Holds the original file path and the modified one (by #line.) - NewlineData.FilePaths* filePaths; - - /// Construct a Lexer object. - /// Params: - /// srcText = the UTF-8 source code. - /// infoMan = used for collecting error messages. - this(SourceText srcText, InfoManager infoMan = null) - { - this.srcText = srcText; - this.infoMan = infoMan; - - assert(text.length && text[$-1] == 0, "source text has no sentinel character"); - this.p = text.ptr; - this.end = this.p + text.length; - this.lineBegin = this.p; - - this.head = new Token; - this.head.kind = TOK.HEAD; - this.head.start = this.head.end = this.p; - this.token = this.head; - // Initialize this.filePaths. - newFilePath(this.srcText.filePath); - // Add a newline as the first token after the head. - auto newline = new Token; - newline.kind = TOK.Newline; - newline.setWhitespaceFlag(); - newline.start = newline.end = this.p; - newline.newline.filePaths = this.filePaths; - newline.newline.oriLineNum = 1; - newline.newline.setLineNum = 0; - // Link in. - this.token.next = newline; - newline.prev = this.token; - this.token = newline; -// this.newline = newline; - scanShebang(); - } - - /// The destructor deletes the doubly-linked token list. - ~this() - { - auto token = head.next; - while (token !is null) - { - assert(token.kind == TOK.EOF ? token == tail && token.next is null : 1); - delete token.prev; - token = token.next; - } - delete tail; - } - - char[] text() - { - return srcText.data; - } - - /// The "shebang" may optionally appear once at the beginning of a file. - /// Regexp: #![^\EndOfLine]* - void scanShebang() - { - if (*p == '#' && p[1] == '!') - { - auto t = new Token; - t.kind = TOK.Shebang; - t.setWhitespaceFlag(); - t.start = p; - ++p; - while (!isEndOfLine(++p)) - isascii(*p) || decodeUTF8(); - t.end = p; - this.token.next = t; - t.prev = this.token; - } - } - - /// Sets the value of the special token. - void finalizeSpecialToken(ref Token t) - { - assert(t.srcText[0..2] == "__"); - switch (t.kind) - { - case TOK.FILE: - t.str = this.filePaths.setPath; - break; - case TOK.LINE: - t.uint_ = this.errorLineNumber(this.lineNum); - break; - case TOK.DATE, - TOK.TIME, - TOK.TIMESTAMP: - auto time_str = Time.toString(); - switch (t.kind) - { - case TOK.DATE: - time_str = Time.month_day(time_str) ~ ' ' ~ Time.year(time_str); break; - case TOK.TIME: - time_str = Time.time(time_str); break; - case TOK.TIMESTAMP: - break; // time_str is the timestamp. - default: assert(0); - } - time_str ~= '\0'; // Terminate with a zero. - t.str = time_str; - break; - case TOK.VENDOR: - t.str = VENDOR; - break; - case TOK.VERSION: - t.uint_ = VERSION_MAJOR*1000 + VERSION_MINOR; - break; - default: - assert(0); - } - } - - /// Sets a new file path. - void newFilePath(char[] newPath) - { - auto paths = new NewlineData.FilePaths; - paths.oriPath = this.srcText.filePath; - paths.setPath = newPath; - this.filePaths = paths; - } - - private void setLineBegin(char* p) - { - // Check that we can look behind one character. - assert((p-1) >= text.ptr && p < end); - // Check that previous character is a newline. - assert(isNewlineEnd(p - 1)); - this.lineBegin = p; - } - - /// Scans the next token in the source text. - /// - /// Creates a new token if t.next is null and appends it to the list. - private void scanNext(ref Token* t) - { - assert(t !is null); - if (t.next) - { - t = t.next; -// if (t.kind == TOK.Newline) -// this.newline = t; - } - else if (t != this.tail) - { - Token* new_t = new Token; - scan(*new_t); - new_t.prev = t; - t.next = new_t; - t = new_t; - } - } - - /// Advance t one token forward. - void peek(ref Token* t) - { - scanNext(t); - } - - /// Advance to the next token in the source text. - TOK nextToken() - { - scanNext(this.token); - return this.token.kind; - } - - /// Returns true if p points to the last character of a Newline. - bool isNewlineEnd(char* p) - { - if (*p == '\n' || *p == '\r') - return true; - if (*p == LS[2] || *p == PS[2]) - if ((p-2) >= text.ptr) - if (p[-1] == LS[1] && p[-2] == LS[0]) - return true; - return false; - } - - /// The main method which recognizes the characters that make up a token. - /// - /// Complicated tokens are scanned in separate methods. - public void scan(ref Token t) - in - { - assert(text.ptr <= p && p < end); - } - out - { - assert(text.ptr <= t.start && t.start < end, Token.toString(t.kind)); - assert(text.ptr <= t.end && t.end <= end, Token.toString(t.kind)); - } - body - { - // Scan whitespace. - if (isspace(*p)) - { - t.ws = p; - while (isspace(*++p)) - {} - } - - // Scan a token. - uint c = *p; - { - t.start = p; - // Newline. - switch (*p) - { - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - ++p; - ++lineNum; - setLineBegin(p); -// this.newline = &t; - t.kind = TOK.Newline; - t.setWhitespaceFlag(); - t.newline.filePaths = this.filePaths; - t.newline.oriLineNum = lineNum; - t.newline.setLineNum = lineNum_hline; - t.end = p; - return; - default: - if (isUnicodeNewline(p)) - { - ++p; ++p; - goto case '\n'; - } - } - // Identifier or string literal. - if (isidbeg(c)) - { - if (c == 'r' && p[1] == '"' && ++p) - return scanRawStringLiteral(t); - if (c == 'x' && p[1] == '"') - return scanHexStringLiteral(t); - version(D2) - { - if (c == 'q' && p[1] == '"') - return scanDelimitedStringLiteral(t); - if (c == 'q' && p[1] == '{') - return scanTokenStringLiteral(t); - } - // Scan identifier. - Lidentifier: - do - { c = *++p; } - while (isident(c) || !isascii(c) && isUnicodeAlpha()) - - t.end = p; - - auto id = IdTable.lookup(t.srcText); - t.kind = id.kind; - t.ident = id; - - if (t.kind == TOK.Identifier || t.isKeyword) - return; - else if (t.isSpecialToken) - finalizeSpecialToken(t); - else if (t.kind == TOK.EOF) - { - tail = &t; - assert(t.srcText == "__EOF__"); - } - else - assert(0, "unexpected token type: " ~ Token.toString(t.kind)); - return; - } - - if (isdigit(c)) - return scanNumber(t); - - if (c == '/') - { - c = *++p; - switch(c) - { - case '=': - ++p; - t.kind = TOK.DivAssign; - t.end = p; - return; - case '+': - return scanNestedComment(t); - case '*': - return scanBlockComment(t); - case '/': - while (!isEndOfLine(++p)) - isascii(*p) || decodeUTF8(); - t.kind = TOK.Comment; - t.setWhitespaceFlag(); - t.end = p; - return; - default: - t.kind = TOK.Div; - t.end = p; - return; - } - } - - switch (c) - { - case '\'': - return scanCharacterLiteral(t); - case '`': - return scanRawStringLiteral(t); - case '"': - return scanNormalStringLiteral(t); - case '\\': - char[] buffer; - do - { - bool isBinary; - c = scanEscapeSequence(isBinary); - if (isascii(c) || isBinary) - buffer ~= c; - else - encodeUTF8(buffer, c); - } while (*p == '\\') - buffer ~= 0; - t.kind = TOK.String; - t.str = buffer; - t.end = p; - return; - case '>': /* > >= >> >>= >>> >>>= */ - c = *++p; - switch (c) - { - case '=': - t.kind = TOK.GreaterEqual; - goto Lcommon; - case '>': - if (p[1] == '>') - { - ++p; - if (p[1] == '=') - { ++p; - t.kind = TOK.URShiftAssign; - } - else - t.kind = TOK.URShift; - } - else if (p[1] == '=') - { - ++p; - t.kind = TOK.RShiftAssign; - } - else - t.kind = TOK.RShift; - goto Lcommon; - default: - t.kind = TOK.Greater; - goto Lcommon2; - } - assert(0); - case '<': /* < <= <> <>= << <<= */ - c = *++p; - switch (c) - { - case '=': - t.kind = TOK.LessEqual; - goto Lcommon; - case '<': - if (p[1] == '=') { - ++p; - t.kind = TOK.LShiftAssign; - } - else - t.kind = TOK.LShift; - goto Lcommon; - case '>': - if (p[1] == '=') { - ++p; - t.kind = TOK.LorEorG; - } - else - t.kind = TOK.LorG; - goto Lcommon; - default: - t.kind = TOK.Less; - goto Lcommon2; - } - assert(0); - case '!': /* ! !< !> !<= !>= !<> !<>= */ - c = *++p; - switch (c) - { - case '<': - c = *++p; - if (c == '>') - { - if (p[1] == '=') { - ++p; - t.kind = TOK.Unordered; - } - else - t.kind = TOK.UorE; - } - else if (c == '=') - { - t.kind = TOK.UorG; - } - else { - t.kind = TOK.UorGorE; - goto Lcommon2; - } - goto Lcommon; - case '>': - if (p[1] == '=') - { - ++p; - t.kind = TOK.UorL; - } - else - t.kind = TOK.UorLorE; - goto Lcommon; - case '=': - t.kind = TOK.NotEqual; - goto Lcommon; - default: - t.kind = TOK.Not; - goto Lcommon2; - } - assert(0); - case '.': /* . .[0-9] .. ... */ - if (p[1] == '.') - { - ++p; - if (p[1] == '.') { - ++p; - t.kind = TOK.Ellipses; - } - else - t.kind = TOK.Slice; - } - else if (isdigit(p[1])) - { - return scanReal(t); - } - else - t.kind = TOK.Dot; - goto Lcommon; - case '|': /* | || |= */ - c = *++p; - if (c == '=') - t.kind = TOK.OrAssign; - else if (c == '|') - t.kind = TOK.OrLogical; - else { - t.kind = TOK.OrBinary; - goto Lcommon2; - } - goto Lcommon; - case '&': /* & && &= */ - c = *++p; - if (c == '=') - t.kind = TOK.AndAssign; - else if (c == '&') - t.kind = TOK.AndLogical; - else { - t.kind = TOK.AndBinary; - goto Lcommon2; - } - goto Lcommon; - case '+': /* + ++ += */ - c = *++p; - if (c == '=') - t.kind = TOK.PlusAssign; - else if (c == '+') - t.kind = TOK.PlusPlus; - else { - t.kind = TOK.Plus; - goto Lcommon2; - } - goto Lcommon; - case '-': /* - -- -= */ - c = *++p; - if (c == '=') - t.kind = TOK.MinusAssign; - else if (c == '-') - t.kind = TOK.MinusMinus; - else { - t.kind = TOK.Minus; - goto Lcommon2; - } - goto Lcommon; - case '=': /* = == */ - if (p[1] == '=') { - ++p; - t.kind = TOK.Equal; - } - else - t.kind = TOK.Assign; - goto Lcommon; - case '~': /* ~ ~= */ - if (p[1] == '=') { - ++p; - t.kind = TOK.CatAssign; - } - else - t.kind = TOK.Tilde; - goto Lcommon; - case '*': /* * *= */ - if (p[1] == '=') { - ++p; - t.kind = TOK.MulAssign; - } - else - t.kind = TOK.Mul; - goto Lcommon; - case '^': /* ^ ^= */ - if (p[1] == '=') { - ++p; - t.kind = TOK.XorAssign; - } - else - t.kind = TOK.Xor; - goto Lcommon; - case '%': /* % %= */ - if (p[1] == '=') { - ++p; - t.kind = TOK.ModAssign; - } - else - t.kind = TOK.Mod; - goto Lcommon; - // Single character tokens: - case '(': - t.kind = TOK.LParen; - goto Lcommon; - case ')': - t.kind = TOK.RParen; - goto Lcommon; - case '[': - t.kind = TOK.LBracket; - goto Lcommon; - case ']': - t.kind = TOK.RBracket; - goto Lcommon; - case '{': - t.kind = TOK.LBrace; - goto Lcommon; - case '}': - t.kind = TOK.RBrace; - goto Lcommon; - case ':': - t.kind = TOK.Colon; - goto Lcommon; - case ';': - t.kind = TOK.Semicolon; - goto Lcommon; - case '?': - t.kind = TOK.Question; - goto Lcommon; - case ',': - t.kind = TOK.Comma; - goto Lcommon; - case '$': - t.kind = TOK.Dollar; - Lcommon: - ++p; - Lcommon2: - t.end = p; - return; - case '#': - return scanSpecialTokenSequence(t); - default: - } - - // Check for EOF - if (isEOF(c)) - { - assert(isEOF(*p), ""~*p); - t.kind = TOK.EOF; - t.end = p; - tail = &t; - assert(t.start == t.end); - return; - } - - if (!isascii(c)) - { - c = decodeUTF8(); - if (isUniAlpha(c)) - goto Lidentifier; - } - - error(t.start, MID.IllegalCharacter, cast(dchar)c); - - ++p; - t.kind = TOK.Illegal; - t.setWhitespaceFlag(); - t.dchar_ = c; - t.end = p; - return; - } - } - - /// Converts a string literal to an integer. - template toUint(char[] T) - { - static assert(0 < T.length && T.length <= 4); - static if (T.length == 1) - const uint toUint = T[0]; - else - const uint toUint = (T[0] << ((T.length-1)*8)) | toUint!(T[1..$]); - } - static assert(toUint!("\xAA\xBB\xCC\xDD") == 0xAABBCCDD); - - /// Constructs case statements. E.g.: - /// --- - //// // case_!("<", "Less", "Lcommon") -> - /// case 60u: - /// t.kind = TOK.Less; - /// goto Lcommon; - /// --- - /// Note:Can't use this yet due to a $(DMDBUG 1534, bug) in DMD. - template case_(char[] str, char[] kind, char[] label) - { - const char[] case_ = - `case `~toUint!(str).stringof~`:` - `t.kind = TOK.`~kind~`;` - `goto `~label~`;`; - } - //pragma(msg, case_!("<", "Less", "Lcommon")); - - template case_L4(char[] str, TOK kind) - { - const char[] case_L4 = case_!(str, kind, "Lcommon_4"); - } - - template case_L3(char[] str, TOK kind) - { - const char[] case_L3 = case_!(str, kind, "Lcommon_3"); - } - - template case_L2(char[] str, TOK kind) - { - const char[] case_L2 = case_!(str, kind, "Lcommon_2"); - } - - template case_L1(char[] str, TOK kind) - { - const char[] case_L3 = case_!(str, kind, "Lcommon"); - } - - /// An alternative scan method. - /// Profiling shows it's a bit slower. - public void scan_(ref Token t) - in - { - assert(text.ptr <= p && p < end); - } - out - { - assert(text.ptr <= t.start && t.start < end, Token.toString(t.kind)); - assert(text.ptr <= t.end && t.end <= end, Token.toString(t.kind)); - } - body - { - // Scan whitespace. - if (isspace(*p)) - { - t.ws = p; - while (isspace(*++p)) - {} - } - - // Scan a token. - t.start = p; - // Newline. - switch (*p) - { - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - ++p; - ++lineNum; - setLineBegin(p); -// this.newline = &t; - t.kind = TOK.Newline; - t.setWhitespaceFlag(); - t.newline.filePaths = this.filePaths; - t.newline.oriLineNum = lineNum; - t.newline.setLineNum = lineNum_hline; - t.end = p; - return; - default: - if (isUnicodeNewline(p)) - { - ++p; ++p; - goto case '\n'; - } - } - - uint c = *p; - assert(end - p != 0); - switch (end - p) - { - case 1: - goto L1character; - case 2: - c <<= 8; c |= p[1]; - goto L2characters; - case 3: - c <<= 8; c |= p[1]; c <<= 8; c |= p[2]; - goto L3characters; - default: - version(BigEndian) - c = *cast(uint*)p; - else - { - c <<= 8; c |= p[1]; c <<= 8; c |= p[2]; c <<= 8; c |= p[3]; - /+ - c = *cast(uint*)p; - asm - { - mov EDX, c; - bswap EDX; - mov c, EDX; - } - +/ - } - } - - // 4 character tokens. - switch (c) - { - case toUint!(">>>="): - t.kind = TOK.RShiftAssign; - goto Lcommon_4; - case toUint!("!<>="): - t.kind = TOK.Unordered; - Lcommon_4: - p += 4; - t.end = p; - return; - default: - } - - c >>>= 8; - L3characters: - assert(p == t.start); - // 3 character tokens. - switch (c) - { - case toUint!(">>="): - t.kind = TOK.RShiftAssign; - goto Lcommon_3; - case toUint!(">>>"): - t.kind = TOK.URShift; - goto Lcommon_3; - case toUint!("<>="): - t.kind = TOK.LorEorG; - goto Lcommon_3; - case toUint!("<<="): - t.kind = TOK.LShiftAssign; - goto Lcommon_3; - case toUint!("!<="): - t.kind = TOK.UorG; - goto Lcommon_3; - case toUint!("!>="): - t.kind = TOK.UorL; - goto Lcommon_3; - case toUint!("!<>"): - t.kind = TOK.UorE; - goto Lcommon_3; - case toUint!("..."): - t.kind = TOK.Ellipses; - Lcommon_3: - p += 3; - t.end = p; - return; - default: - } - - c >>>= 8; - L2characters: - assert(p == t.start); - // 2 character tokens. - switch (c) - { - case toUint!("/+"): - ++p; // Skip / - return scanNestedComment(t); - case toUint!("/*"): - ++p; // Skip / - return scanBlockComment(t); - case toUint!("//"): - ++p; // Skip / - assert(*p == '/'); - while (!isEndOfLine(++p)) - isascii(*p) || decodeUTF8(); - t.kind = TOK.Comment; - t.setWhitespaceFlag(); - t.end = p; - return; - case toUint!(">="): - t.kind = TOK.GreaterEqual; - goto Lcommon_2; - case toUint!(">>"): - t.kind = TOK.RShift; - goto Lcommon_2; - case toUint!("<<"): - t.kind = TOK.LShift; - goto Lcommon_2; - case toUint!("<="): - t.kind = TOK.LessEqual; - goto Lcommon_2; - case toUint!("<>"): - t.kind = TOK.LorG; - goto Lcommon_2; - case toUint!("!<"): - t.kind = TOK.UorGorE; - goto Lcommon_2; - case toUint!("!>"): - t.kind = TOK.UorLorE; - goto Lcommon_2; - case toUint!("!="): - t.kind = TOK.NotEqual; - goto Lcommon_2; - case toUint!(".."): - t.kind = TOK.Slice; - goto Lcommon_2; - case toUint!("&&"): - t.kind = TOK.AndLogical; - goto Lcommon_2; - case toUint!("&="): - t.kind = TOK.AndAssign; - goto Lcommon_2; - case toUint!("||"): - t.kind = TOK.OrLogical; - goto Lcommon_2; - case toUint!("|="): - t.kind = TOK.OrAssign; - goto Lcommon_2; - case toUint!("++"): - t.kind = TOK.PlusPlus; - goto Lcommon_2; - case toUint!("+="): - t.kind = TOK.PlusAssign; - goto Lcommon_2; - case toUint!("--"): - t.kind = TOK.MinusMinus; - goto Lcommon_2; - case toUint!("-="): - t.kind = TOK.MinusAssign; - goto Lcommon_2; - case toUint!("=="): - t.kind = TOK.Equal; - goto Lcommon_2; - case toUint!("~="): - t.kind = TOK.CatAssign; - goto Lcommon_2; - case toUint!("*="): - t.kind = TOK.MulAssign; - goto Lcommon_2; - case toUint!("/="): - t.kind = TOK.DivAssign; - goto Lcommon_2; - case toUint!("^="): - t.kind = TOK.XorAssign; - goto Lcommon_2; - case toUint!("%="): - t.kind = TOK.ModAssign; - Lcommon_2: - p += 2; - t.end = p; - return; - default: - } - - c >>>= 8; - L1character: - assert(p == t.start); - assert(*p == c, Format("p={0},c={1}", *p, cast(dchar)c)); - // 1 character tokens. - // TODO: consider storing the token type in ptable. - switch (c) - { - case '\'': - return scanCharacterLiteral(t); - case '`': - return scanRawStringLiteral(t); - case '"': - return scanNormalStringLiteral(t); - case '\\': - char[] buffer; - do - { - bool isBinary; - c = scanEscapeSequence(isBinary); - if (isascii(c) || isBinary) - buffer ~= c; - else - encodeUTF8(buffer, c); - } while (*p == '\\') - buffer ~= 0; - t.kind = TOK.String; - t.str = buffer; - t.end = p; - return; - case '<': - t.kind = TOK.Greater; - goto Lcommon; - case '>': - t.kind = TOK.Less; - goto Lcommon; - case '^': - t.kind = TOK.Xor; - goto Lcommon; - case '!': - t.kind = TOK.Not; - goto Lcommon; - case '.': - if (isdigit(p[1])) - return scanReal(t); - t.kind = TOK.Dot; - goto Lcommon; - case '&': - t.kind = TOK.AndBinary; - goto Lcommon; - case '|': - t.kind = TOK.OrBinary; - goto Lcommon; - case '+': - t.kind = TOK.Plus; - goto Lcommon; - case '-': - t.kind = TOK.Minus; - goto Lcommon; - case '=': - t.kind = TOK.Assign; - goto Lcommon; - case '~': - t.kind = TOK.Tilde; - goto Lcommon; - case '*': - t.kind = TOK.Mul; - goto Lcommon; - case '/': - t.kind = TOK.Div; - goto Lcommon; - case '%': - t.kind = TOK.Mod; - goto Lcommon; - case '(': - t.kind = TOK.LParen; - goto Lcommon; - case ')': - t.kind = TOK.RParen; - goto Lcommon; - case '[': - t.kind = TOK.LBracket; - goto Lcommon; - case ']': - t.kind = TOK.RBracket; - goto Lcommon; - case '{': - t.kind = TOK.LBrace; - goto Lcommon; - case '}': - t.kind = TOK.RBrace; - goto Lcommon; - case ':': - t.kind = TOK.Colon; - goto Lcommon; - case ';': - t.kind = TOK.Semicolon; - goto Lcommon; - case '?': - t.kind = TOK.Question; - goto Lcommon; - case ',': - t.kind = TOK.Comma; - goto Lcommon; - case '$': - t.kind = TOK.Dollar; - Lcommon: - ++p; - t.end = p; - return; - case '#': - return scanSpecialTokenSequence(t); - default: - } - - assert(p == t.start); - assert(*p == c); - - // TODO: consider moving isidbeg() and isdigit() up. - if (isidbeg(c)) - { - if (c == 'r' && p[1] == '"' && ++p) - return scanRawStringLiteral(t); - if (c == 'x' && p[1] == '"') - return scanHexStringLiteral(t); - version(D2) - { - if (c == 'q' && p[1] == '"') - return scanDelimitedStringLiteral(t); - if (c == 'q' && p[1] == '{') - return scanTokenStringLiteral(t); - } - // Scan identifier. - Lidentifier: - do - { c = *++p; } - while (isident(c) || !isascii(c) && isUnicodeAlpha()) - - t.end = p; - - auto id = IdTable.lookup(t.srcText); - t.kind = id.kind; - t.ident = id; - - if (t.kind == TOK.Identifier || t.isKeyword) - return; - else if (t.isSpecialToken) - finalizeSpecialToken(t); - else if (t.kind == TOK.EOF) - { - tail = &t; - assert(t.srcText == "__EOF__"); - } - else - assert(0, "unexpected token type: " ~ Token.toString(t.kind)); - return; - } - - if (isdigit(c)) - return scanNumber(t); - - // Check for EOF - if (isEOF(c)) - { - assert(isEOF(*p), *p~""); - t.kind = TOK.EOF; - t.end = p; - tail = &t; - assert(t.start == t.end); - return; - } - - if (!isascii(c)) - { - c = decodeUTF8(); - if (isUniAlpha(c)) - goto Lidentifier; - } - - error(t.start, MID.IllegalCharacter, cast(dchar)c); - - ++p; - t.kind = TOK.Illegal; - t.setWhitespaceFlag(); - t.dchar_ = c; - t.end = p; - return; - } - - /// Scans a block comment. - /// - /// BlockComment := "/*" AnyChar* "*/" - void scanBlockComment(ref Token t) - { - assert(p[-1] == '/' && *p == '*'); - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - Loop: - while (1) - { - switch (*++p) - { - case '*': - if (p[1] != '/') - continue; - p += 2; - break Loop; - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - ++lineNum; - setLineBegin(p+1); - break; - default: - if (!isascii(*p)) - { - if (isUnicodeNewlineChar(decodeUTF8())) - goto case '\n'; - } - else if (isEOF(*p)) - { - error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedBlockComment); - break Loop; - } - } - } - t.kind = TOK.Comment; - t.setWhitespaceFlag(); - t.end = p; - return; - } - - /// Scans a nested comment. - /// - /// NestedComment := "/+" (AnyChar* | NestedComment) "+/" - void scanNestedComment(ref Token t) - { - assert(p[-1] == '/' && *p == '+'); - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - uint level = 1; - Loop: - while (1) - { - switch (*++p) - { - case '/': - if (p[1] == '+') - ++p, ++level; - continue; - case '+': - if (p[1] != '/') - continue; - ++p; - if (--level != 0) - continue; - ++p; - break Loop; - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - ++lineNum; - setLineBegin(p+1); - continue; - default: - if (!isascii(*p)) - { - if (isUnicodeNewlineChar(decodeUTF8())) - goto case '\n'; - } - else if (isEOF(*p)) - { - error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedNestedComment); - break Loop; - } - } - } - t.kind = TOK.Comment; - t.setWhitespaceFlag(); - t.end = p; - return; - } - - /// Scans the postfix character of a string literal. - /// - /// PostfixChar := "c" | "w" | "d" - char scanPostfix() - { - assert(p[-1] == '"' || p[-1] == '`' || - { version(D2) return p[-1] == '}'; - else return 0; }() - ); - switch (*p) - { - case 'c': - case 'w': - case 'd': - return *p++; - default: - return 0; - } - assert(0); - } - - /// Scans a normal string literal. - /// - /// NormalStringLiteral := "\"" Char* "\"" - void scanNormalStringLiteral(ref Token t) - { - assert(*p == '"'); - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - t.kind = TOK.String; - char[] buffer; - uint c; - while (1) - { - c = *++p; - switch (c) - { - case '"': - ++p; - t.pf = scanPostfix(); - Lreturn: - t.str = buffer ~ '\0'; - t.end = p; - return; - case '\\': - bool isBinary; - c = scanEscapeSequence(isBinary); - --p; - if (isascii(c) || isBinary) - buffer ~= c; - else - encodeUTF8(buffer, c); - continue; - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - c = '\n'; // Convert Newline to \n. - ++lineNum; - setLineBegin(p+1); - break; - case 0, _Z_: - error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedString); - goto Lreturn; - default: - if (!isascii(c)) - { - c = decodeUTF8(); - if (isUnicodeNewlineChar(c)) - goto case '\n'; - encodeUTF8(buffer, c); - continue; - } - } - assert(isascii(c)); - buffer ~= c; - } - assert(0); - } - - /// Scans a character literal. - /// - /// CharLiteral := "'" Char "'" - void scanCharacterLiteral(ref Token t) - { - assert(*p == '\''); - ++p; - t.kind = TOK.CharLiteral; - switch (*p) - { - case '\\': - bool notused; - t.dchar_ = scanEscapeSequence(notused); - break; - case '\'': - error(t.start, MID.EmptyCharacterLiteral); - break; - default: - if (isEndOfLine(p)) - break; - uint c = *p; - if (!isascii(c)) - c = decodeUTF8(); - t.dchar_ = c; - ++p; - } - - if (*p == '\'') - ++p; - else - error(t.start, MID.UnterminatedCharacterLiteral); - t.end = p; - } - - /// Scans a raw string literal. - /// - /// RawStringLiteral := "r\"" AnyChar* "\"" | "`" AnyChar* "`" - void scanRawStringLiteral(ref Token t) - { - assert(*p == '`' || *p == '"' && p[-1] == 'r'); - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - t.kind = TOK.String; - uint delim = *p; - char[] buffer; - uint c; - while (1) - { - c = *++p; - switch (c) - { - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - c = '\n'; // Convert Newline to '\n'. - ++lineNum; - setLineBegin(p+1); - break; - case '`': - case '"': - if (c == delim) - { - ++p; - t.pf = scanPostfix(); - Lreturn: - t.str = buffer ~ '\0'; - t.end = p; - return; - } - break; - case 0, _Z_: - error(tokenLineNum, tokenLineBegin, t.start, - delim == 'r' ? MID.UnterminatedRawString : MID.UnterminatedBackQuoteString); - goto Lreturn; - default: - if (!isascii(c)) - { - c = decodeUTF8(); - if (isUnicodeNewlineChar(c)) - goto case '\n'; - encodeUTF8(buffer, c); - continue; - } - } - assert(isascii(c)); - buffer ~= c; - } - assert(0); - } - - /// Scans a hexadecimal string literal. - /// - /// HexStringLiteral := "x\"" (HexChar HexChar)* "\"" - void scanHexStringLiteral(ref Token t) - { - assert(p[0] == 'x' && p[1] == '"'); - t.kind = TOK.String; - - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - - uint c; - ubyte[] buffer; - ubyte h; // hex number - uint n; // number of hex digits - - ++p; - assert(*p == '"'); - while (1) - { - c = *++p; - switch (c) - { - case '"': - if (n & 1) - error(tokenLineNum, tokenLineBegin, t.start, MID.OddNumberOfDigitsInHexString); - ++p; - t.pf = scanPostfix(); - Lreturn: - t.str = cast(string) (buffer ~= 0); - t.end = p; - return; - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - ++lineNum; - setLineBegin(p+1); - continue; - default: - if (ishexad(c)) - { - if (c <= '9') - c -= '0'; - else if (c <= 'F') - c -= 'A' - 10; - else - c -= 'a' - 10; - - if (n & 1) - { - h <<= 4; - h |= c; - buffer ~= h; - } - else - h = cast(ubyte)c; - ++n; - continue; - } - else if (isspace(c)) - continue; // Skip spaces. - else if (isEOF(c)) - { - error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedHexString); - t.pf = 0; - goto Lreturn; - } - else - { - auto errorAt = p; - if (!isascii(c)) - { - c = decodeUTF8(); - if (isUnicodeNewlineChar(c)) - goto case '\n'; - } - error(errorAt, MID.NonHexCharInHexString, cast(dchar)c); - } - } - } - assert(0); - } - -version(DDoc) -{ - /// Scans a delimited string literal. - void scanDelimitedStringLiteral(ref Token t); - /// Scans a token string literal. - /// - /// TokenStringLiteral := "q{" Token* "}" - void scanTokenStringLiteral(ref Token t); -} -else -version(D2) -{ - void scanDelimitedStringLiteral(ref Token t) - { - assert(p[0] == 'q' && p[1] == '"'); - t.kind = TOK.String; - - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - - char[] buffer; - dchar opening_delim = 0, // 0 if no nested delimiter or '[', '(', '<', '{' - closing_delim; // Will be ']', ')', '>', '}, - // the first character of an identifier or - // any other Unicode/ASCII character. - char[] str_delim; // Identifier delimiter. - uint level = 1; // Counter for nestable delimiters. - - ++p; ++p; // Skip q" - uint c = *p; - switch (c) - { - case '(': - opening_delim = c; - closing_delim = ')'; // c + 1 - break; - case '[', '<', '{': - opening_delim = c; - closing_delim = c + 2; // Get to closing counterpart. Feature of ASCII table. - break; - default: - dchar scanNewline() - { - switch (*p) - { - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - ++p; - ++lineNum; - setLineBegin(p); - return '\n'; - default: - if (isUnicodeNewline(p)) - { - ++p; ++p; - goto case '\n'; - } - } - return 0; - } - // Skip leading newlines: - while (scanNewline() != 0) - {} - assert(!isNewline(p)); - - char* begin = p; - c = *p; - closing_delim = c; - // TODO: Check for non-printable characters? - if (!isascii(c)) - { - closing_delim = decodeUTF8(); - if (!isUniAlpha(closing_delim)) - break; // Not an identifier. - } - else if (!isidbeg(c)) - break; // Not an identifier. - - // Parse Identifier + EndOfLine - do - { c = *++p; } - while (isident(c) || !isascii(c) && isUnicodeAlpha()) - // Store identifier - str_delim = begin[0..p-begin]; - // Scan newline - if (scanNewline() == '\n') - --p; // Go back one because of "c = *++p;" in main loop. - else - { - // TODO: error(p, MID.ExpectedNewlineAfterIdentDelim); - } - } - - bool checkStringDelim(char* p) - { - assert(str_delim.length != 0); - if (buffer[$-1] == '\n' && // Last character copied to buffer must be '\n'. - end-p >= str_delim.length && // Check remaining length. - p[0..str_delim.length] == str_delim) // Compare. - return true; - return false; - } - - while (1) - { - c = *++p; - switch (c) - { - case '\r': - if (p[1] == '\n') - ++p; - case '\n': - assert(isNewlineEnd(p)); - c = '\n'; // Convert Newline to '\n'. - ++lineNum; - setLineBegin(p+1); - break; - case 0, _Z_: - // TODO: error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedDelimitedString); - goto Lreturn3; - default: - if (!isascii(c)) - { - auto begin = p; - c = decodeUTF8(); - if (isUnicodeNewlineChar(c)) - goto case '\n'; - if (c == closing_delim) - { - if (str_delim.length) - { - if (checkStringDelim(begin)) - { - p = begin + str_delim.length; - goto Lreturn2; - } - } - else - { - assert(level == 1); - --level; - goto Lreturn; - } - } - encodeUTF8(buffer, c); - continue; - } - else - { - if (c == opening_delim) - ++level; - else if (c == closing_delim) - { - if (str_delim.length) - { - if (checkStringDelim(p)) - { - p += str_delim.length; - goto Lreturn2; - } - } - else if (--level == 0) - goto Lreturn; - } - } - } - assert(isascii(c)); - buffer ~= c; - } - Lreturn: // Character delimiter. - assert(c == closing_delim); - assert(level == 0); - ++p; // Skip closing delimiter. - Lreturn2: // String delimiter. - if (*p == '"') - ++p; - else - { - // TODO: error(p, MID.ExpectedDblQuoteAfterDelim, str_delim.length ? str_delim : closing_delim~""); - } - - t.pf = scanPostfix(); - Lreturn3: // Error. - t.str = buffer ~ '\0'; - t.end = p; - } - - void scanTokenStringLiteral(ref Token t) - { - assert(p[0] == 'q' && p[1] == '{'); - t.kind = TOK.String; - - auto tokenLineNum = lineNum; - auto tokenLineBegin = lineBegin; - - // A guard against changes to particular members: - // this.lineNum_hline and this.errorPath - ++inTokenString; - - uint lineNum = this.lineNum; - uint level = 1; - - ++p; ++p; // Skip q{ - - auto prev_t = &t; - Token* token; - while (1) - { - token = new Token; - scan(*token); - // Save the tokens in a doubly linked list. - // Could be useful for various tools. - token.prev = prev_t; - prev_t.next = token; - prev_t = token; - switch (token.kind) - { - case TOK.LBrace: - ++level; - continue; - case TOK.RBrace: - if (--level == 0) - { - t.tok_str = t.next; - t.next = null; - break; - } - continue; - case TOK.EOF: - // TODO: error(tokenLineNum, tokenLineBegin, t.start, MID.UnterminatedTokenString); - t.tok_str = t.next; - t.next = token; - break; - default: - continue; - } - break; // Exit loop. - } - - assert(token.kind == TOK.RBrace || token.kind == TOK.EOF); - assert(token.kind == TOK.RBrace && t.next is null || - token.kind == TOK.EOF && t.next !is null); - - char[] buffer; - // token points to } or EOF - if (token.kind == TOK.EOF) - { - t.end = token.start; - buffer = t.srcText[2..$].dup ~ '\0'; - } - else - { - // Assign to buffer before scanPostfix(). - t.end = p; - buffer = t.srcText[2..$-1].dup ~ '\0'; - t.pf = scanPostfix(); - t.end = p; // Assign again because of postfix. - } - // Convert newlines to '\n'. - if (lineNum != this.lineNum) - { - assert(buffer[$-1] == '\0'); - uint i, j; - for (; i < buffer.length; ++i) - switch (buffer[i]) - { - case '\r': - if (buffer[i+1] == '\n') - ++i; - case '\n': - assert(isNewlineEnd(buffer.ptr + i)); - buffer[j++] = '\n'; // Convert Newline to '\n'. - break; - default: - if (isUnicodeNewline(buffer.ptr + i)) - { - ++i; ++i; - goto case '\n'; - } - buffer[j++] = buffer[i]; // Copy. - } - buffer.length = j; // Adjust length. - } - assert(buffer[$-1] == '\0'); - t.str = buffer; - - --inTokenString; - } -} // version(D2) - - /// Scans an escape sequence. - /// - /// EscapeSequence := "\" (Octal{1,3} | ("x" Hex{2}) | - /// ("u" Hex{4}) | ("U" Hex{8}) | - /// "'" | "\"" | "\\" | "?" | "a" | - /// "b" | "f" | "n" | "r" | "t" | "v") - /// Params: - /// isBinary = set to true for octal and hexadecimal escapes. - /// Returns: the escape value. - dchar scanEscapeSequence(ref bool isBinary) - out(result) - { assert(isValidChar(result)); } - body - { - assert(*p == '\\'); - - auto sequenceStart = p; // Used for error reporting. - - ++p; - uint c = char2ev(*p); - if (c) - { - ++p; - return c; - } - - uint digits = 2; - - switch (*p) - { - case 'x': - isBinary = true; - case_Unicode: - assert(c == 0); - assert(digits == 2 || digits == 4 || digits == 8); - while (1) - { - ++p; - if (ishexad(*p)) - { - c *= 16; - if (*p <= '9') - c += *p - '0'; - else if (*p <= 'F') - c += *p - 'A' + 10; - else - c += *p - 'a' + 10; - - if (--digits == 0) - { - ++p; - if (isValidChar(c)) - return c; // Return valid escape value. - - error(sequenceStart, MID.InvalidUnicodeEscapeSequence, - sequenceStart[0..p-sequenceStart]); - break; - } - continue; - } - - error(sequenceStart, MID.InsufficientHexDigits, - sequenceStart[0..p-sequenceStart]); - break; - } - break; - case 'u': - digits = 4; - goto case_Unicode; - case 'U': - digits = 8; - goto case_Unicode; - default: - if (isoctal(*p)) - { - isBinary = true; - assert(c == 0); - c += *p - '0'; - ++p; - if (!isoctal(*p)) - return c; - c *= 8; - c += *p - '0'; - ++p; - if (!isoctal(*p)) - return c; - c *= 8; - c += *p - '0'; - ++p; - if (c > 0xFF) - error(sequenceStart, MSG.InvalidOctalEscapeSequence, - sequenceStart[0..p-sequenceStart]); - return c; // Return valid escape value. - } - else if(*p == '&') - { - if (isalpha(*++p)) - { - auto begin = p; - while (isalnum(*++p)) - {} - - if (*p == ';') - { - // Pass entity excluding '&' and ';'. - c = entity2Unicode(begin[0..p - begin]); - ++p; // Skip ; - if (c != 0xFFFF) - return c; // Return valid escape value. - else - error(sequenceStart, MID.UndefinedHTMLEntity, sequenceStart[0 .. p - sequenceStart]); - } - else - error(sequenceStart, MID.UnterminatedHTMLEntity, sequenceStart[0 .. p - sequenceStart]); - } - else - error(sequenceStart, MID.InvalidBeginHTMLEntity); - } - else if (isEndOfLine(p)) - error(sequenceStart, MID.UndefinedEscapeSequence, - isEOF(*p) ? `\EOF` : `\NewLine`); - else - { - char[] str = `\`; - if (isascii(c)) - str ~= *p; - else - encodeUTF8(str, decodeUTF8()); - ++p; - // TODO: check for unprintable character? - error(sequenceStart, MID.UndefinedEscapeSequence, str); - } - } - return REPLACEMENT_CHAR; // Error: return replacement character. - } - - /// Scans a number literal. - /// - /// $(PRE - /// IntegerLiteral := (Dec|Hex|Bin|Oct)Suffix? - /// Dec := (0|[1-9][0-9_]*) - /// Hex := 0[xX][_]*[0-9a-zA-Z][0-9a-zA-Z_]* - /// Bin := 0[bB][_]*[01][01_]* - /// Oct := 0[0-7_]* - /// Suffix := (L[uU]?|[uU]L?) - /// ) - /// Invalid: "0b_", "0x_", "._" etc. - void scanNumber(ref Token t) - { - ulong ulong_; - bool overflow; - bool isDecimal; - size_t digits; - - if (*p != '0') - goto LscanInteger; - ++p; // skip zero - // check for xX bB ... - switch (*p) - { - case 'x','X': - goto LscanHex; - case 'b','B': - goto LscanBinary; - case 'L': - if (p[1] == 'i') - goto LscanReal; // 0Li - break; // 0L - case '.': - if (p[1] == '.') - break; // 0.. - // 0. - case 'i','f','F', // Imaginary and float literal suffixes. - 'e', 'E': // Float exponent. - goto LscanReal; - default: - if (*p == '_') - goto LscanOctal; // 0_ - else if (isdigit(*p)) - { - if (*p == '8' || *p == '9') - goto Loctal_hasDecimalDigits; // 08 or 09 - else - goto Loctal_enter_loop; // 0[0-7] - } - } - - // Number 0 - assert(p[-1] == '0'); - assert(*p != '_' && !isdigit(*p)); - assert(ulong_ == 0); - isDecimal = true; - goto Lfinalize; - - LscanInteger: - assert(*p != 0 && isdigit(*p)); - isDecimal = true; - goto Lenter_loop_int; - while (1) - { - if (*++p == '_') - continue; - if (!isdigit(*p)) - break; - Lenter_loop_int: - if (ulong_ < ulong.max/10 || (ulong_ == ulong.max/10 && *p <= '5')) - { - ulong_ *= 10; - ulong_ += *p - '0'; - continue; - } - // Overflow: skip following digits. - overflow = true; - while (isdigit(*++p)) {} - break; - } - - // The number could be a float, so check overflow below. - switch (*p) - { - case '.': - if (p[1] != '.') - goto LscanReal; - break; - case 'L': - if (p[1] != 'i') - break; - case 'i', 'f', 'F', 'e', 'E': - goto LscanReal; - default: - } - - if (overflow) - error(t.start, MID.OverflowDecimalNumber); - - assert((isdigit(p[-1]) || p[-1] == '_') && !isdigit(*p) && *p != '_'); - goto Lfinalize; - - LscanHex: - assert(digits == 0); - assert(*p == 'x' || *p == 'X'); - while (1) - { - if (*++p == '_') - continue; - if (!ishexad(*p)) - break; - ++digits; - ulong_ *= 16; - if (*p <= '9') - ulong_ += *p - '0'; - else if (*p <= 'F') - ulong_ += *p - 'A' + 10; - else - ulong_ += *p - 'a' + 10; - } - - assert(ishexad(p[-1]) || p[-1] == '_' || p[-1] == 'x' || p[-1] == 'X'); - assert(!ishexad(*p) && *p != '_'); - - switch (*p) - { - case '.': - if (p[1] == '.') - break; - case 'p', 'P': - return scanHexReal(t); - default: - } - - if (digits == 0 || digits > 16) - error(t.start, digits == 0 ? MID.NoDigitsInHexNumber : MID.OverflowHexNumber); - - goto Lfinalize; - - LscanBinary: - assert(digits == 0); - assert(*p == 'b' || *p == 'B'); - while (1) - { - if (*++p == '0') - { - ++digits; - ulong_ *= 2; - } - else if (*p == '1') - { - ++digits; - ulong_ *= 2; - ulong_ += *p - '0'; - } - else if (*p == '_') - continue; - else - break; - } - - if (digits == 0 || digits > 64) - error(t.start, digits == 0 ? MID.NoDigitsInBinNumber : MID.OverflowBinaryNumber); - - assert(p[-1] == '0' || p[-1] == '1' || p[-1] == '_' || p[-1] == 'b' || p[-1] == 'B', p[-1] ~ ""); - assert( !(*p == '0' || *p == '1' || *p == '_') ); - goto Lfinalize; - - LscanOctal: - assert(*p == '_'); - while (1) - { - if (*++p == '_') - continue; - if (!isoctal(*p)) - break; - Loctal_enter_loop: - if (ulong_ < ulong.max/2 || (ulong_ == ulong.max/2 && *p <= '1')) - { - ulong_ *= 8; - ulong_ += *p - '0'; - continue; - } - // Overflow: skip following digits. - overflow = true; - while (isoctal(*++p)) {} - break; - } - - bool hasDecimalDigits; - if (isdigit(*p)) - { - Loctal_hasDecimalDigits: - hasDecimalDigits = true; - while (isdigit(*++p)) {} - } - - // The number could be a float, so check errors below. - switch (*p) - { - case '.': - if (p[1] != '.') - goto LscanReal; - break; - case 'L': - if (p[1] != 'i') - break; - case 'i', 'f', 'F', 'e', 'E': - goto LscanReal; - default: - } - - if (hasDecimalDigits) - error(t.start, MID.OctalNumberHasDecimals); - - if (overflow) - error(t.start, MID.OverflowOctalNumber); -// goto Lfinalize; - - Lfinalize: - enum Suffix - { - None = 0, - Unsigned = 1, - Long = 2 - } - - // Scan optional suffix: L, Lu, LU, u, uL, U or UL. - Suffix suffix; - while (1) - { - switch (*p) - { - case 'L': - if (suffix & Suffix.Long) - break; - suffix |= Suffix.Long; - ++p; - continue; - case 'u', 'U': - if (suffix & Suffix.Unsigned) - break; - suffix |= Suffix.Unsigned; - ++p; - continue; - default: - break; - } - break; - } - - // Determine type of Integer. - switch (suffix) - { - case Suffix.None: - if (ulong_ & 0x8000_0000_0000_0000) - { - if (isDecimal) - error(t.start, MID.OverflowDecimalSign); - t.kind = TOK.Uint64; - } - else if (ulong_ & 0xFFFF_FFFF_0000_0000) - t.kind = TOK.Int64; - else if (ulong_ & 0x8000_0000) - t.kind = isDecimal ? TOK.Int64 : TOK.Uint32; - else - t.kind = TOK.Int32; - break; - case Suffix.Unsigned: - if (ulong_ & 0xFFFF_FFFF_0000_0000) - t.kind = TOK.Uint64; - else - t.kind = TOK.Uint32; - break; - case Suffix.Long: - if (ulong_ & 0x8000_0000_0000_0000) - { - if (isDecimal) - error(t.start, MID.OverflowDecimalSign); - t.kind = TOK.Uint64; - } - else - t.kind = TOK.Int64; - break; - case Suffix.Unsigned | Suffix.Long: - t.kind = TOK.Uint64; - break; - default: - assert(0); - } - t.ulong_ = ulong_; - t.end = p; - return; - LscanReal: - scanReal(t); - return; - } - - /// Scans a floating point number literal. - /// - /// $(PRE - /// FloatLiteral := Float[fFL]?i? - /// Float := DecFloat | HexFloat - /// DecFloat := ([0-9][0-9_]*[.][0-9_]*DecExponent?) | - /// [.][0-9][0-9_]*DecExponent? | [0-9][0-9_]*DecExponent - /// DecExponent := [eE][+-]?[0-9][0-9_]* - /// HexFloat := 0[xX](HexDigits[.]HexDigits | - /// [.][0-9a-zA-Z]HexDigits? | - /// HexDigits)HexExponent - /// HexExponent := [pP][+-]?[0-9][0-9_]* - /// ) - void scanReal(ref Token t) - { - if (*p == '.') - { - assert(p[1] != '.'); - // This function was called by scan() or scanNumber(). - while (isdigit(*++p) || *p == '_') {} - } - else - // This function was called by scanNumber(). - assert(delegate () - { - switch (*p) - { - case 'L': - if (p[1] != 'i') - return false; - case 'i', 'f', 'F', 'e', 'E': - return true; - default: - } - return false; - }() - ); - - // Scan exponent. - if (*p == 'e' || *p == 'E') - { - ++p; - if (*p == '-' || *p == '+') - ++p; - if (isdigit(*p)) - while (isdigit(*++p) || *p == '_') {} - else - error(t.start, MID.FloatExpMustStartWithDigit); - } - - // Copy whole number and remove underscores from buffer. - char[] buffer = t.start[0..p-t.start].dup; - uint j; - foreach (c; buffer) - if (c != '_') - buffer[j++] = c; - buffer.length = j; // Adjust length. - buffer ~= 0; // Terminate for C functions. - - finalizeFloat(t, buffer); - } - - /// Scans a hexadecimal floating point number literal. - void scanHexReal(ref Token t) - { - assert(*p == '.' || *p == 'p' || *p == 'P'); - MID mid; - if (*p == '.') - while (ishexad(*++p) || *p == '_') - {} - // Decimal exponent is required. - if (*p != 'p' && *p != 'P') - { - mid = MID.HexFloatExponentRequired; - goto Lerr; - } - // Scan exponent - assert(*p == 'p' || *p == 'P'); - ++p; - if (*p == '+' || *p == '-') - ++p; - if (!isdigit(*p)) - { - mid = MID.HexFloatExpMustStartWithDigit; - goto Lerr; - } - while (isdigit(*++p) || *p == '_') - {} - // Copy whole number and remove underscores from buffer. - char[] buffer = t.start[0..p-t.start].dup; - uint j; - foreach (c; buffer) - if (c != '_') - buffer[j++] = c; - buffer.length = j; // Adjust length. - buffer ~= 0; // Terminate for C functions. - finalizeFloat(t, buffer); - return; - Lerr: - t.kind = TOK.Float32; - t.end = p; - error(t.start, mid); - } - - /// Sets the value of the token. - /// Params: - /// t = receives the value. - /// buffer = the well-formed float number. - void finalizeFloat(ref Token t, string buffer) - { - assert(buffer[$-1] == 0); - // Float number is well-formed. Check suffixes and do conversion. - switch (*p) - { - case 'f', 'F': - t.kind = TOK.Float32; - t.float_ = strtof(buffer.ptr, null); - ++p; - break; - case 'L': - t.kind = TOK.Float80; - t.real_ = strtold(buffer.ptr, null); - ++p; - break; - default: - t.kind = TOK.Float64; - t.double_ = strtod(buffer.ptr, null); - } - if (*p == 'i') - { - ++p; - t.kind += 3; // Switch to imaginary counterpart. - assert(t.kind == TOK.Imaginary32 || - t.kind == TOK.Imaginary64 || - t.kind == TOK.Imaginary80); - } - if (errno() == ERANGE) - error(t.start, MID.OverflowFloatNumber); - t.end = p; - } - - /// Scans a special token sequence. - /// - /// SpecialTokenSequence := "#line" Integer Filespec? EndOfLine - void scanSpecialTokenSequence(ref Token t) - { - assert(*p == '#'); - t.kind = TOK.HashLine; - t.setWhitespaceFlag(); - - MID mid; - char* errorAtColumn = p; - char* tokenEnd = ++p; - - if (!(p[0] == 'l' && p[1] == 'i' && p[2] == 'n' && p[3] == 'e')) - { - mid = MID.ExpectedIdentifierSTLine; - goto Lerr; - } - p += 3; - tokenEnd = p + 1; - - // TODO: #line58"path/file" is legal. Require spaces? - // State.Space could be used for that purpose. - enum State - { /+Space,+/ Integer, Filespec, End } - - State state = State.Integer; - - while (!isEndOfLine(++p)) - { - if (isspace(*p)) - continue; - if (state == State.Integer) - { - if (!isdigit(*p)) - { - errorAtColumn = p; - mid = MID.ExpectedIntegerAfterSTLine; - goto Lerr; - } - t.tokLineNum = new Token; - scan(*t.tokLineNum); - tokenEnd = p; - if (t.tokLineNum.kind != TOK.Int32 && t.tokLineNum.kind != TOK.Uint32) - { - errorAtColumn = t.tokLineNum.start; - mid = MID.ExpectedIntegerAfterSTLine; - goto Lerr; - } - --p; // Go one back because scan() advanced p past the integer. - state = State.Filespec; - } - else if (state == State.Filespec && *p == '"') - { // MID.ExpectedFilespec is deprecated. - // if (*p != '"') - // { - // errorAtColumn = p; - // mid = MID.ExpectedFilespec; - // goto Lerr; - // } - t.tokLineFilespec = new Token; - t.tokLineFilespec.start = p; - t.tokLineFilespec.kind = TOK.Filespec; - t.tokLineFilespec.setWhitespaceFlag(); - while (*++p != '"') - { - if (isEndOfLine(p)) - { - errorAtColumn = t.tokLineFilespec.start; - mid = MID.UnterminatedFilespec; - t.tokLineFilespec.end = p; - tokenEnd = p; - goto Lerr; - } - isascii(*p) || decodeUTF8(); - } - auto start = t.tokLineFilespec.start +1; // +1 skips '"' - t.tokLineFilespec.str = start[0 .. p - start]; - t.tokLineFilespec.end = p + 1; - tokenEnd = p + 1; - state = State.End; - } - else/+ if (state == State.End)+/ - { - mid = MID.UnterminatedSpecialToken; - goto Lerr; - } - } - assert(isEndOfLine(p)); - - if (state == State.Integer) - { - errorAtColumn = p; - mid = MID.ExpectedIntegerAfterSTLine; - goto Lerr; - } - - // Evaluate #line only when not in token string. - if (!inTokenString && t.tokLineNum) - { - this.lineNum_hline = this.lineNum - t.tokLineNum.uint_ + 1; - if (t.tokLineFilespec) - newFilePath(t.tokLineFilespec.str); - } - p = tokenEnd; - t.end = tokenEnd; - - return; - Lerr: - p = tokenEnd; - t.end = tokenEnd; - error(errorAtColumn, mid); - } - - /// Inserts an empty dummy token (TOK.Empty) before t. - /// - /// Useful in the parsing phase for representing a node in the AST - /// that doesn't consume an actual token from the source text. - Token* insertEmptyTokenBefore(Token* t) - { - assert(t !is null && t.prev !is null); - assert(text.ptr <= t.start && t.start < end, Token.toString(t.kind)); - assert(text.ptr <= t.end && t.end <= end, Token.toString(t.kind)); - - auto prev_t = t.prev; - auto new_t = new Token; - new_t.kind = TOK.Empty; - new_t.start = new_t.end = prev_t.end; - // Link in new token. - prev_t.next = new_t; - new_t.prev = prev_t; - new_t.next = t; - t.prev = new_t; - return new_t; - } - - /// Returns the error line number. - uint errorLineNumber(uint lineNum) - { - return lineNum - this.lineNum_hline; - } - - /// Forwards error parameters. - void error(char* columnPos, char[] msg, ...) - { - error_(this.lineNum, this.lineBegin, columnPos, msg, _arguments, _argptr); - } - - /// ditto - void error(char* columnPos, MID mid, ...) - { - error_(this.lineNum, this.lineBegin, columnPos, GetMsg(mid), _arguments, _argptr); - } - - /// ditto - void error(uint lineNum, char* lineBegin, char* columnPos, MID mid, ...) - { - error_(lineNum, lineBegin, columnPos, GetMsg(mid), _arguments, _argptr); - } - - /// Creates an error report and appends it to a list. - /// Params: - /// lineNum = the line number. - /// lineBegin = points to the first character of the current line. - /// columnPos = points to the character where the error is located. - /// msg = the message. - void error_(uint lineNum, char* lineBegin, char* columnPos, char[] msg, - TypeInfo[] _arguments, Arg _argptr) - { - lineNum = this.errorLineNumber(lineNum); - auto errorPath = this.filePaths.setPath; - auto location = new Location(errorPath, lineNum, lineBegin, columnPos); - msg = Format(_arguments, _argptr, msg); - auto error = new LexerError(location, msg); - errors ~= error; - if (infoMan !is null) - infoMan ~= error; - } - - /// Scans the whole source text until EOF is encountered. - void scanAll() - { - while (nextToken() != TOK.EOF) - {} - } - - /// Returns the first token of the source text. - /// This can be the EOF token. - /// Structure: HEAD -> Newline -> First Token - Token* firstToken() - { - return this.head.next.next; - } - - /// Returns true if str is a valid D identifier. - static bool isIdentifierString(char[] str) - { - if (str.length == 0 || isdigit(str[0])) - return false; - size_t idx; - do - { - auto c = dil.Unicode.decode(str, idx); - if (c == ERROR_CHAR || !(isident(c) || !isascii(c) && isUniAlpha(c))) - return false; - } while (idx < str.length) - return true; - } - - /// Returns true if str is a keyword or a special token (__FILE__, __LINE__ etc.) - static bool isReservedIdentifier(char[] str) - { - if (!isIdentifierString(str)) - return false; // str is not a valid identifier. - - auto id = IdTable.inStatic(str); - if (id is null || id.kind == TOK.Identifier) - return false; // str is not in the table or a normal identifier. - - return true; - } - - /// Returns true if the current character to be decoded is - /// a Unicode alpha character. - /// - /// The current pointer 'p' is not advanced if false is returned. - bool isUnicodeAlpha() - { - assert(!isascii(*p), "check for ASCII char before calling decodeUTF8()."); - char* p = this.p; - dchar d = *p; - ++p; // Move to second byte. - // Error if second byte is not a trail byte. - if (!isTrailByte(*p)) - return false; - // Check for overlong sequences. - switch (d) - { - case 0xE0, 0xF0, 0xF8, 0xFC: - if ((*p & d) == 0x80) - return false; - default: - if ((d & 0xFE) == 0xC0) // 1100000x - return false; - } - const char[] checkNextByte = "if (!isTrailByte(*++p))" - " return false;"; - const char[] appendSixBits = "d = (d << 6) | *p & 0b0011_1111;"; - // Decode - if ((d & 0b1110_0000) == 0b1100_0000) - { - d &= 0b0001_1111; - mixin(appendSixBits); - } - else if ((d & 0b1111_0000) == 0b1110_0000) - { - d &= 0b0000_1111; - mixin(appendSixBits ~ - checkNextByte ~ appendSixBits); - } - else if ((d & 0b1111_1000) == 0b1111_0000) - { - d &= 0b0000_0111; - mixin(appendSixBits ~ - checkNextByte ~ appendSixBits ~ - checkNextByte ~ appendSixBits); - } - else - return false; - - assert(isTrailByte(*p)); - if (!isValidChar(d) || !isUniAlpha(d)) - return false; - // Only advance pointer if this is a Unicode alpha character. - this.p = p; - return true; - } - - /// Decodes the next UTF-8 sequence. - dchar decodeUTF8() - { - assert(!isascii(*p), "check for ASCII char before calling decodeUTF8()."); - char* p = this.p; - dchar d = *p; - - ++p; // Move to second byte. - // Error if second byte is not a trail byte. - if (!isTrailByte(*p)) - goto Lerr2; - - // Check for overlong sequences. - switch (d) - { - case 0xE0, // 11100000 100xxxxx - 0xF0, // 11110000 1000xxxx - 0xF8, // 11111000 10000xxx - 0xFC: // 11111100 100000xx - if ((*p & d) == 0x80) - goto Lerr; - default: - if ((d & 0xFE) == 0xC0) // 1100000x - goto Lerr; - } - - const char[] checkNextByte = "if (!isTrailByte(*++p))" - " goto Lerr2;"; - const char[] appendSixBits = "d = (d << 6) | *p & 0b0011_1111;"; - - // Decode - if ((d & 0b1110_0000) == 0b1100_0000) - { // 110xxxxx 10xxxxxx - d &= 0b0001_1111; - mixin(appendSixBits); - } - else if ((d & 0b1111_0000) == 0b1110_0000) - { // 1110xxxx 10xxxxxx 10xxxxxx - d &= 0b0000_1111; - mixin(appendSixBits ~ - checkNextByte ~ appendSixBits); - } - else if ((d & 0b1111_1000) == 0b1111_0000) - { // 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx - d &= 0b0000_0111; - mixin(appendSixBits ~ - checkNextByte ~ appendSixBits ~ - checkNextByte ~ appendSixBits); - } - else - // 5 and 6 byte UTF-8 sequences are not allowed yet. - // 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx - // 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx - goto Lerr; - - assert(isTrailByte(*p)); - - if (!isValidChar(d)) - { - Lerr: - // Three cases: - // *) the UTF-8 sequence was successfully decoded but the resulting - // character is invalid. - // p points to last trail byte in the sequence. - // *) the UTF-8 sequence is overlong. - // p points to second byte in the sequence. - // *) the UTF-8 sequence has more than 4 bytes or starts with - // a trail byte. - // p points to second byte in the sequence. - assert(isTrailByte(*p)); - // Move to next ASCII character or lead byte of a UTF-8 sequence. - while (p < (end-1) && isTrailByte(*p)) - ++p; - --p; - assert(!isTrailByte(p[1])); - Lerr2: - d = REPLACEMENT_CHAR; - error(this.p, MID.InvalidUTF8Sequence, formatBytes(this.p, p)); - } - - this.p = p; - return d; - } - - /// Encodes the character d and appends it to str. - static void encodeUTF8(ref char[] str, dchar d) - { - assert(!isascii(d), "check for ASCII char before calling encodeUTF8()."); - assert(isValidChar(d), "check if character is valid before calling encodeUTF8()."); - - char[6] b = void; - if (d < 0x800) - { - b[0] = 0xC0 | (d >> 6); - b[1] = 0x80 | (d & 0x3F); - str ~= b[0..2]; - } - else if (d < 0x10000) - { - b[0] = 0xE0 | (d >> 12); - b[1] = 0x80 | ((d >> 6) & 0x3F); - b[2] = 0x80 | (d & 0x3F); - str ~= b[0..3]; - } - else if (d < 0x200000) - { - b[0] = 0xF0 | (d >> 18); - b[1] = 0x80 | ((d >> 12) & 0x3F); - b[2] = 0x80 | ((d >> 6) & 0x3F); - b[3] = 0x80 | (d & 0x3F); - str ~= b[0..4]; - } - /+ // There are no 5 and 6 byte UTF-8 sequences yet. - else if (d < 0x4000000) - { - b[0] = 0xF8 | (d >> 24); - b[1] = 0x80 | ((d >> 18) & 0x3F); - b[2] = 0x80 | ((d >> 12) & 0x3F); - b[3] = 0x80 | ((d >> 6) & 0x3F); - b[4] = 0x80 | (d & 0x3F); - str ~= b[0..5]; - } - else if (d < 0x80000000) - { - b[0] = 0xFC | (d >> 30); - b[1] = 0x80 | ((d >> 24) & 0x3F); - b[2] = 0x80 | ((d >> 18) & 0x3F); - b[3] = 0x80 | ((d >> 12) & 0x3F); - b[4] = 0x80 | ((d >> 6) & 0x3F); - b[5] = 0x80 | (d & 0x3F); - str ~= b[0..6]; - } - +/ - else - assert(0); - } - - /// Formats the bytes between start and end. - /// Returns: e.g.: abc -> \x61\x62\x63 - static char[] formatBytes(char* start, char* end) - { - auto strLen = end-start; - const formatLen = `\xXX`.length; - char[] result = new char[strLen*formatLen]; // Reserve space. - result.length = 0; - foreach (c; cast(ubyte[])start[0..strLen]) - result ~= Format("\\x{:X}", c); - return result; - } - - /// Searches for an invalid UTF-8 sequence in str. - /// Returns: a formatted string of the invalid sequence (e.g. \xC0\x80). - static string findInvalidUTF8Sequence(string str) - { - char* p = str.ptr, end = p + str.length; - while (p < end) - { - if (decode(p, end) == ERROR_CHAR) - { - auto begin = p; - // Skip trail-bytes. - while (++p < end && isTrailByte(*p)) - {} - return Lexer.formatBytes(begin, p); - } - } - assert(p == end); - return ""; - } -} - -/// Tests the lexer with a list of tokens. -unittest -{ - Stdout("Testing Lexer.\n"); - struct Pair - { - char[] tokenText; - TOK kind; - } - static Pair[] pairs = [ - {"#!äöüß", TOK.Shebang}, {"\n", TOK.Newline}, - {"//çay", TOK.Comment}, {"\n", TOK.Newline}, - {"&", TOK.AndBinary}, - {"/*çağ*/", TOK.Comment}, {"&&", TOK.AndLogical}, - {"/+çak+/", TOK.Comment}, {"&=", TOK.AndAssign}, - {">", TOK.Greater}, {"+", TOK.Plus}, - {">=", TOK.GreaterEqual}, {"++", TOK.PlusPlus}, - {">>", TOK.RShift}, {"+=", TOK.PlusAssign}, - {">>=", TOK.RShiftAssign}, {"-", TOK.Minus}, - {">>>", TOK.URShift}, {"--", TOK.MinusMinus}, - {">>>=", TOK.URShiftAssign}, {"-=", TOK.MinusAssign}, - {"<", TOK.Less}, {"=", TOK.Assign}, - {"<=", TOK.LessEqual}, {"==", TOK.Equal}, - {"<>", TOK.LorG}, {"~", TOK.Tilde}, - {"<>=", TOK.LorEorG}, {"~=", TOK.CatAssign}, - {"<<", TOK.LShift}, {"*", TOK.Mul}, - {"<<=", TOK.LShiftAssign}, {"*=", TOK.MulAssign}, - {"!", TOK.Not}, {"/", TOK.Div}, - {"!=", TOK.NotEqual}, {"/=", TOK.DivAssign}, - {"!<", TOK.UorGorE}, {"^", TOK.Xor}, - {"!>", TOK.UorLorE}, {"^=", TOK.XorAssign}, - {"!<=", TOK.UorG}, {"%", TOK.Mod}, - {"!>=", TOK.UorL}, {"%=", TOK.ModAssign}, - {"!<>", TOK.UorE}, {"(", TOK.LParen}, - {"!<>=", TOK.Unordered}, {")", TOK.RParen}, - {".", TOK.Dot}, {"[", TOK.LBracket}, - {"..", TOK.Slice}, {"]", TOK.RBracket}, - {"...", TOK.Ellipses}, {"{", TOK.LBrace}, - {"|", TOK.OrBinary}, {"}", TOK.RBrace}, - {"||", TOK.OrLogical}, {":", TOK.Colon}, - {"|=", TOK.OrAssign}, {";", TOK.Semicolon}, - {"?", TOK.Question}, {",", TOK.Comma}, - {"$", TOK.Dollar}, {"cam", TOK.Identifier}, - {"çay", TOK.Identifier}, {".0", TOK.Float64}, - {"0", TOK.Int32}, {"\n", TOK.Newline}, - {"\r", TOK.Newline}, {"\r\n", TOK.Newline}, - {"\u2028", TOK.Newline}, {"\u2029", TOK.Newline} - ]; - - char[] src; - - // Join all token texts into a single string. - foreach (i, pair; pairs) - if (pair.kind == TOK.Comment && pair.tokenText[1] == '/' || // Line comment. - pair.kind == TOK.Shebang) - { - assert(pairs[i+1].kind == TOK.Newline); // Must be followed by a newline. - src ~= pair.tokenText; - } - else - src ~= pair.tokenText ~ " "; - - auto lx = new Lexer(new SourceText("", src)); - auto token = lx.getTokens(); - - uint i; - assert(token == lx.head); - assert(token.next.kind == TOK.Newline); - token = token.next.next; - do - { - assert(i < pairs.length); - assert(token.srcText == pairs[i].tokenText, Format("Scanned '{0}' but expected '{1}'", token.srcText, pairs[i].tokenText)); - ++i; - token = token.next; - } while (token.kind != TOK.EOF) -} - -/// Tests the Lexer's peek() method. -unittest -{ - Stdout("Testing method Lexer.peek()\n"); - auto sourceText = new SourceText("", "unittest { }"); - auto lx = new Lexer(sourceText, null); - - auto next = lx.head; - lx.peek(next); - assert(next.kind == TOK.Newline); - lx.peek(next); - assert(next.kind == TOK.Unittest); - lx.peek(next); - assert(next.kind == TOK.LBrace); - lx.peek(next); - assert(next.kind == TOK.RBrace); - lx.peek(next); - assert(next.kind == TOK.EOF); - - lx = new Lexer(new SourceText("", "")); - next = lx.head; - lx.peek(next); - assert(next.kind == TOK.Newline); - lx.peek(next); - assert(next.kind == TOK.EOF); -} - -unittest -{ - // Numbers unittest - // 0L 0ULi 0_L 0_UL 0x0U 0x0p2 0_Fi 0_e2 0_F 0_i - // 0u 0U 0uL 0UL 0L 0LU 0Lu - // 0Li 0f 0F 0fi 0Fi 0i - // 0b_1_LU 0b1000u - // 0x232Lu -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/Token.d --- a/trunk/src/dil/lexer/Token.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,400 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.Token; - -import dil.lexer.Identifier; -import dil.lexer.Funcs; -import dil.Location; -import tango.stdc.stdlib : malloc, free; -import tango.core.Exception; -import common; - -public import dil.lexer.TokensEnum; - -/// A Token is a sequence of characters formed by the lexical analyzer. -struct Token -{ /// Flags set by the Lexer. - enum Flags : ushort - { - None, - Whitespace = 1, /// Tokens with this flag are ignored by the Parser. - } - - TOK kind; /// The token kind. - Flags flags; /// The flags of the token. - /// Pointers to the next and previous tokens (doubly-linked list.) - Token* next, prev; - - /// Start of whitespace characters before token. Null if no WS. - /// TODO: remove to save space; can be replaced by 'prev.end'. - char* ws; - char* start; /// Points to the first character of the token. - char* end; /// Points one character past the end of the token. - - /// Data associated with this token. - /// TODO: move data structures out; use only pointers here to keep Token.sizeof small. - union - { - /// For newline tokens. - NewlineData newline; - /// For #line tokens. - struct - { - Token* tokLineNum; /// #line number - Token* tokLineFilespec; /// #line number filespec - } - /// The value of a string token. - struct - { - string str; /// Zero-terminated string. (The zero is included in the length.) - char pf; /// Postfix 'c', 'w', 'd' or 0 for none. - version(D2) - Token* tok_str; /++ Points to the contents of a token string stored as a - doubly linked list. The last token is always '}' or - EOF in case end of source text is "q{" EOF. - +/ - } - Identifier* ident; /// For keywords and identifiers. - dchar dchar_; /// A character value. - long long_; /// A long integer value. - ulong ulong_; /// An unsigned long integer value. - int int_; /// An integer value. - uint uint_; /// An unsigned integer value. - float float_; /// A float value. - double double_; /// A double value. - real real_; /// A real value. - } - - /// Returns the text of the token. - string srcText() - { - assert(start && end); - return start[0 .. end - start]; - } - - /// Returns the preceding whitespace of the token. - string wsChars() - { - assert(ws && start); - return ws[0 .. start - ws]; - } - - /// Finds the next non-whitespace token. - /// Returns: 'this' token if the next token is TOK.EOF or null. - Token* nextNWS() - out(token) - { - assert(token !is null); - } - body - { - auto token = next; - while (token !is null && token.isWhitespace) - token = token.next; - if (token is null || token.kind == TOK.EOF) - return this; - return token; - } - - /// Finds the previous non-whitespace token. - /// Returns: 'this' token if the previous token is TOK.HEAD or null. - Token* prevNWS() - out(token) - { - assert(token !is null); - } - body - { - auto token = prev; - while (token !is null && token.isWhitespace) - token = token.prev; - if (token is null || token.kind == TOK.HEAD) - return this; - return token; - } - - /// Returns the string for a token kind. - static string toString(TOK kind) - { - return tokToString[kind]; - } - - /// Adds Flags.Whitespace to this.flags. - void setWhitespaceFlag() - { - this.flags |= Flags.Whitespace; - } - - /// Returns true if this is a token that can have newlines in it. - /// - /// These can be block and nested comments and any string literal - /// except for escape string literals. - bool isMultiline() - { - return kind == TOK.String && start[0] != '\\' || - kind == TOK.Comment && start[1] != '/'; - } - - /// Returns true if this is a keyword token. - bool isKeyword() - { - return KeywordsBegin <= kind && kind <= KeywordsEnd; - } - - /// Returns true if this is an integral type token. - bool isIntegralType() - { - return IntegralTypeBegin <= kind && kind <= IntegralTypeEnd; - } - - /// Returns true if this is a whitespace token. - bool isWhitespace() - { - return !!(flags & Flags.Whitespace); - } - - /// Returns true if this is a special token. - bool isSpecialToken() - { - return SpecialTokensBegin <= kind && kind <= SpecialTokensEnd; - } - -version(D2) -{ - /// Returns true if this is a token string literal. - bool isTokenStringLiteral() - { - return kind == TOK.String && tok_str !is null; - } -} - - /// Returns true if this token starts a DeclarationDefinition. - bool isDeclDefStart() - { - return isDeclDefStartToken(kind); - } - - /// Returns true if this token starts a Statement. - bool isStatementStart() - { - return isStatementStartToken(kind); - } - - /// Returns true if this token starts an AsmStatement. - bool isAsmStatementStart() - { - return isAsmStatementStartToken(kind); - } - - int opEquals(TOK kind2) - { - return kind == kind2; - } - - int opCmp(Token* rhs) - { - return start < rhs.start; - } - - /// Returns the Location of this token. - Location getLocation(bool realLocation)() - { - auto search_t = this.prev; - // Find previous newline token. - while (search_t.kind != TOK.Newline) - search_t = search_t.prev; - static if (realLocation) - { - auto filePath = search_t.newline.filePaths.oriPath; - auto lineNum = search_t.newline.oriLineNum; - } - else - { - auto filePath = search_t.newline.filePaths.setPath; - auto lineNum = search_t.newline.oriLineNum - search_t.newline.setLineNum; - } - auto lineBegin = search_t.end; - // Determine actual line begin and line number. - while (1) - { - search_t = search_t.next; - if (search_t == this) - break; - // Multiline tokens must be rescanned for newlines. - if (search_t.isMultiline) - { - auto p = search_t.start, end = search_t.end; - while (p != end) - { - if (scanNewline(p) == '\n') - { - lineBegin = p; - ++lineNum; - } - else - ++p; - } - } - } - return new Location(filePath, lineNum, lineBegin, this.start); - } - - alias getLocation!(true) getRealLocation; - alias getLocation!(false) getErrorLocation; - - uint lineCount() - { - uint count = 1; - if (this.isMultiline) - { - auto p = this.start, end = this.end; - while (p != end) - { - if (scanNewline(p) == '\n') - ++count; - else - ++p; - } - } - return count; - } - - /// Return the source text enclosed by the left and right token. - static char[] textSpan(Token* left, Token* right) - { - assert(left.end <= right.start || left is right ); - return left.start[0 .. right.end - left.start]; - } - - /// Uses malloc() to allocate memory for a token. - new(size_t size) - { - void* p = malloc(size); - if (p is null) - throw new OutOfMemoryException(__FILE__, __LINE__); - // TODO: Token.init should be all zeros. - // Maybe use calloc() to avoid this line? - *cast(Token*)p = Token.init; - return p; - } - - /// Deletes a token using free(). - delete(void* p) - { - auto token = cast(Token*)p; - if (token) - { - if(token.kind == TOK.HashLine) - token.destructHashLineToken(); - else - { - version(D2) - if (token.isTokenStringLiteral) - token.destructTokenStringLiteral(); - } - } - free(p); - } - - void destructHashLineToken() - { - assert(kind == TOK.HashLine); - delete tokLineNum; - delete tokLineFilespec; - } - -version(D2) -{ - void destructTokenStringLiteral() - { - assert(kind == TOK.String); - assert(start && *start == 'q' && start[1] == '{'); - assert(tok_str !is null); - auto tok_it = tok_str; - auto tok_del = tok_str; - while (tok_it && tok_it.kind != TOK.EOF) - { - tok_it = tok_it.next; - assert(tok_del && tok_del.kind != TOK.EOF); - delete tok_del; - tok_del = tok_it; - } - } -} -} - -/// Data associated with newline tokens. -struct NewlineData -{ - struct FilePaths - { - char[] oriPath; /// Original path to the source text. - char[] setPath; /// Path set by #line. - } - FilePaths* filePaths; - uint oriLineNum; /// Actual line number in the source text. - uint setLineNum; /// Delta line number set by #line. -} - -/// Returns true if this token starts a DeclarationDefinition. -bool isDeclDefStartToken(TOK tok) -{ - switch (tok) - { - alias TOK T; - case T.Align, T.Pragma, T.Export, T.Private, T.Package, T.Protected, - T.Public, T.Extern, T.Deprecated, T.Override, T.Abstract, - T.Synchronized, T.Static, T.Final, T.Const, T.Invariant/*D 2.0*/, - T.Auto, T.Scope, T.Alias, T.Typedef, T.Import, T.Enum, T.Class, - T.Interface, T.Struct, T.Union, T.This, T.Tilde, T.Unittest, T.Debug, - T.Version, T.Template, T.New, T.Delete, T.Mixin, T.Semicolon, - T.Identifier, T.Dot, T.Typeof: - return true; - default: - if (IntegralTypeBegin <= tok && tok <= IntegralTypeEnd) - return true; - } - return false; -} - -/// Returns true if this token starts a Statement. -bool isStatementStartToken(TOK tok) -{ - switch (tok) - { - alias TOK T; - case T.Align, T.Extern, T.Final, T.Const, T.Auto, T.Identifier, T.Dot, - T.Typeof, T.If, T.While, T.Do, T.For, T.Foreach, T.Foreach_reverse, - T.Switch, T.Case, T.Default, T.Continue, T.Break, T.Return, T.Goto, - T.With, T.Synchronized, T.Try, T.Throw, T.Scope, T.Volatile, T.Asm, - T.Pragma, T.Mixin, T.Static, T.Debug, T.Version, T.Alias, T.Semicolon, - T.Enum, T.Class, T.Interface, T.Struct, T.Union, T.LBrace, T.Typedef, - T.This, T.Super, T.Null, T.True, T.False, T.Int32, T.Int64, T.Uint32, - T.Uint64, T.Float32, T.Float64, T.Float80, T.Imaginary32, - T.Imaginary64, T.Imaginary80, T.CharLiteral, T.String, T.LBracket, - T.Function, T.Delegate, T.Assert, T.Import, T.Typeid, T.Is, T.LParen, - T.Traits/*D2.0*/, T.AndBinary, T.PlusPlus, T.MinusMinus, T.Mul, - T.Minus, T.Plus, T.Not, T.Tilde, T.New, T.Delete, T.Cast: - return true; - default: - if (IntegralTypeBegin <= tok && tok <= IntegralTypeEnd || - SpecialTokensBegin <= tok && tok <= SpecialTokensEnd) - return true; - } - return false; -} - -/// Returns true if this token starts an AsmStatement. -bool isAsmStatementStartToken(TOK tok) -{ - switch(tok) - { - alias TOK T; - case T.In, T.Int, T.Out, T.Identifier, T.Align, T.Semicolon: - return true; - default: - } - return false; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/lexer/TokensEnum.d --- a/trunk/src/dil/lexer/TokensEnum.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,219 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.lexer.TokensEnum; - -import common; - -/// Enumeration of token kinds. -enum TOK : ushort -{ - Invalid, - - Illegal, - Comment, - Shebang, - HashLine, - Filespec, - Newline, - Empty, - - Identifier, - String, - CharLiteral, - - // Special tokens - FILE, - LINE, - DATE, - TIME, - TIMESTAMP, - VENDOR, - VERSION, - - // Number literals - Int32, Int64, Uint32, Uint64, - // Floating point number scanner relies on this order. (FloatXY + 3 == ImaginaryXY) - Float32, Float64, Float80, - Imaginary32, Imaginary64, Imaginary80, - - - // Brackets - LParen, - RParen, - LBracket, - RBracket, - LBrace, - RBrace, - - Dot, Slice, Ellipses, - - // Floating point number operators - Unordered, - UorE, - UorG, - UorGorE, - UorL, - UorLorE, - LorEorG, - LorG, - - // Normal operators - Assign, Equal, NotEqual, Not, - LessEqual, Less, - GreaterEqual, Greater, - LShiftAssign, LShift, - RShiftAssign,RShift, - URShiftAssign, URShift, - OrAssign, OrLogical, OrBinary, - AndAssign, AndLogical, AndBinary, - PlusAssign, PlusPlus, Plus, - MinusAssign, MinusMinus, Minus, - DivAssign, Div, - MulAssign, Mul, - ModAssign, Mod, - XorAssign, Xor, - CatAssign, - Tilde, - - Colon, - Semicolon, - Question, - Comma, - Dollar, - - /* Keywords: - NB.: Token.isKeyword() depends on this list being contiguous. - */ - Abstract, Alias, Align, Asm, Assert, Auto, Body, - Break, Case, Cast, Catch, - Class, Const, Continue, - Debug, Default, Delegate, Delete, Deprecated, Do, - Else, Enum, Export, Extern, False, Final, - Finally, For, Foreach, Foreach_reverse, Function, Goto, - If, Import, In, Inout, - Interface, Invariant, Is, Lazy, Macro/+D2.0+/, - Mixin, Module, New, Nothrow/+D2.0+/, Null, Out, Override, Package, - Pragma, Private, Protected, Public, Pure/+D2.0+/, Ref, Return, - Scope, Static, Struct, Super, Switch, Synchronized, - Template, This, Throw, Traits/+D2.0+/, True, Try, Typedef, Typeid, - Typeof, Union, Unittest, - Version, Volatile, While, With, - // Integral types. - Char, Wchar, Dchar, Bool, - Byte, Ubyte, Short, Ushort, - Int, Uint, Long, Ulong, - Cent, Ucent, - Float, Double, Real, - Ifloat, Idouble, Ireal, - Cfloat, Cdouble, Creal, Void, - - HEAD, // start of linked list - EOF, - MAX -} - -alias TOK.Abstract KeywordsBegin; -alias TOK.Void KeywordsEnd; -alias TOK.Char IntegralTypeBegin; -alias TOK.Void IntegralTypeEnd; -alias TOK.FILE SpecialTokensBegin; -alias TOK.VERSION SpecialTokensEnd; - -/// A table that maps each token kind to a string. -const string[TOK.MAX] tokToString = [ - "Invalid", - - "Illegal", - "Comment", - "#! /shebang/", - "#line", - `"filespec"`, - "Newline", - "Empty", - - "Identifier", - "String", - "CharLiteral", - - "__FILE__", - "__LINE__", - "__DATE__", - "__TIME__", - "__TIMESTAMP__", - "__VENDOR__", - "__VERSION__", - - "Int32", "Int64", "Uint32", "Uint64", - "Float32", "Float64", "Float80", - "Imaginary32", "Imaginary64", "Imaginary80", - - "(", - ")", - "[", - "]", - "{", - "}", - - ".", "..", "...", - - "!<>=", // Unordered - "!<>", // UorE - "!<=", // UorG - "!<", // UorGorE - "!>=", // UorL - "!>", // UorLorE - "<>=", // LorEorG - "<>", // LorG - - "=", "==", "!=", "!", - "<=", "<", - ">=", ">", - "<<=", "<<", - ">>=",">>", - ">>>=", ">>>", - "|=", "||", "|", - "&=", "&&", "&", - "+=", "++", "+", - "-=", "--", "-", - "/=", "/", - "*=", "*", - "%=", "%", - "^=", "^", - "~=", - "~", - - ":", - ";", - "?", - ",", - "$", - - "abstract","alias","align","asm","assert","auto","body", - "break","case","cast","catch", - "class","const","continue", - "debug","default","delegate","delete","deprecated","do", - "else","enum","export","extern","false","final", - "finally","for","foreach","foreach_reverse","function","goto", - "if","import","in","inout", - "interface","invariant","is","lazy","macro", - "mixin","module","new","nothrow","null","out","override","package", - "pragma","private","protected","public","pure","ref","return", - "scope","static","struct","super","switch","synchronized", - "template","this","throw","__traits","true","try","typedef","typeid", - "typeof","union","unittest", - "version","volatile","while","with", - // Integral types. - "char", "wchar", "dchar", "bool", - "byte", "ubyte", "short", "ushort", - "int", "uint", "long", "ulong", - "cent", "ucent", - "float", "double", "real", - "ifloat", "idouble", "ireal", - "cfloat", "cdouble", "creal", "void", - - "HEAD", - "EOF" -]; -static assert(tokToString.length == TOK.EOF+1); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/parser/ImportParser.d --- a/trunk/src/dil/parser/ImportParser.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,373 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.parser.ImportParser; - -import dil.parser.Parser; -import dil.ast.Node; -import dil.ast.Declarations; -import dil.ast.Statements; -import dil.SourceText; -import dil.Enums; -import common; - -private alias TOK T; - -/// A light-weight parser which looks only for import statements -/// in the source text. -class ImportParser : Parser -{ - this(SourceText srcText) - { - super(srcText); - } - - override CompoundDeclaration start() - { - auto decls = new CompoundDeclaration; - super.init(); - if (token.kind == T.Module) - decls ~= parseModuleDeclaration(); - while (token.kind != T.EOF) - parseDeclarationDefinition(Protection.None); - return decls; - } - - void parseDeclarationDefinitionsBlock(Protection prot) - { - skip(T.LBrace); - while (token.kind != T.RBrace && token.kind != T.EOF) - parseDeclarationDefinition(prot); - skip(T.RBrace); - } - - void parseDeclarationsBlock(Protection prot) - { - switch (token.kind) - { - case T.LBrace: - parseDeclarationDefinitionsBlock(prot); - break; - case T.Colon: - nT(); - while (token.kind != T.RBrace && token.kind != T.EOF) - parseDeclarationDefinition(prot); - break; - default: - parseDeclarationDefinition(prot); - } - } - - bool skipToClosing(T opening, T closing) - { - alias token next; - uint level = 1; - while (1) - { - lexer.peek(next); - if (next.kind == opening) - ++level; - else if (next.kind == closing && --level == 0) - return true; - else if (next.kind == T.EOF) - break; - } - return false; - } - - void skipToTokenAfterClosingParen() - { - skipToClosing(T.LParen, T.RParen); - nT(); - } - - void skipToTokenAfterClosingBrace() - { - skipToClosing(T.LBrace, T.RBrace); - nT(); - } - - void skip(TOK tok) - { - token.kind == tok && nT(); - } - - void parseProtectionAttribute() - { - Protection prot; - switch (token.kind) - { - case T.Private: - prot = Protection.Private; break; - case T.Package: - prot = Protection.Package; break; - case T.Protected: - prot = Protection.Protected; break; - case T.Public: - prot = Protection.Public; break; - case T.Export: - prot = Protection.Export; break; - default: - assert(0); - } - nT(); - parseDeclarationsBlock(prot); - } - - void parseDeclarationDefinition(Protection prot) - { - switch (token.kind) - { - case T.Align: - nT(); - if (token.kind == T.LParen) - nT(), nT(), nT(); // ( Integer ) - parseDeclarationsBlock(prot); - break; - case T.Pragma: - nT(); - skipToTokenAfterClosingParen(); - parseDeclarationsBlock(prot); - break; - case T.Export, - T.Private, - T.Package, - T.Protected, - T.Public: - parseProtectionAttribute(); - break; - // Storage classes - case T.Extern: - nT(); - token.kind == T.LParen && skipToTokenAfterClosingParen(); - parseDeclarationsBlock(prot); - break; - case T.Const: - version(D2) - { - if (peekNext() == T.LParen) - goto case_Declaration; - } - case T.Override, - T.Deprecated, - T.Abstract, - T.Synchronized, - // T.Static, - T.Final, - T.Auto, - T.Scope: - case_StaticAttribute: - case_InvariantAttribute: - nT(); - parseDeclarationsBlock(prot); - break; - // End of storage classes. - case T.Alias, T.Typedef: - nT(); - goto case_Declaration; - case T.Static: - switch (peekNext()) - { - case T.Import: - goto case_Import; - case T.This: - nT(), nT(); // static this - skipToTokenAfterClosingParen(); - parseFunctionBody(); - break; - case T.Tilde: - nT(), nT(), nT(), nT(), nT(); // static ~ this ( ) - parseFunctionBody(); - break; - case T.If: - nT(), nT(); - skipToTokenAfterClosingParen(); - parseDeclarationsBlock(prot); - if (token.kind == T.Else) - nT(), parseDeclarationsBlock(prot); - break; - case T.Assert: - nT(), nT(); // static assert - skipToTokenAfterClosingParen(); - skip(T.Semicolon); - break; - default: - goto case_StaticAttribute; - } - break; - case T.Import: - case_Import: - auto decl = parseImportDeclaration(); - decl.setProtection(prot); // Set the protection attribute. - imports ~= decl.to!(ImportDeclaration); - break; - case T.Enum: - nT(); - token.kind == T.Identifier && nT(); - if (token.kind == T.Colon) - { - nT(); - while (token.kind != T.LBrace && token.kind != T.EOF) - nT(); - } - if (token.kind == T.Semicolon) - nT(); - else - skipToTokenAfterClosingBrace(); - break; - case T.Class: - case T.Interface: - nT(), skip(T.Identifier); // class Identifier - token.kind == T.LParen && skipToTokenAfterClosingParen(); // Skip template params. - if (token.kind == T.Colon) - { // BaseClasses - nT(); - while (token.kind != T.LBrace && token.kind != T.EOF) - if (token.kind == T.LParen) // Skip ( tokens... ) - skipToTokenAfterClosingParen(); - else - nT(); - } - if (token.kind == T.Semicolon) - nT(); - else - parseDeclarationDefinitionsBlock(Protection.None); - break; - case T.Struct, T.Union: - nT(); skip(T.Identifier); - token.kind == T.LParen && skipToTokenAfterClosingParen(); - if (token.kind == T.Semicolon) - nT(); - else - parseDeclarationDefinitionsBlock(Protection.None); - break; - case T.Tilde: - nT(); // ~ - case T.This: - nT(); nT(); nT(); // this ( ) - parseFunctionBody(); - break; - case T.Invariant: - version(D2) - { - auto next = token; - if (peekAfter(next) == T.LParen) - { - if (peekAfter(next) != T.RParen) - goto case_Declaration; - } - else - goto case_InvariantAttribute; - } - nT(); - token.kind == T.LParen && skipToTokenAfterClosingParen(); - parseFunctionBody(); - break; - case T.Unittest: - nT(); - parseFunctionBody(); - break; - case T.Debug: - nT(); - if (token.kind == T.Assign) - { - nT(), nT(), nT(); // = Condition ; - break; - } - if (token.kind == T.LParen) - nT(), nT(), nT(); // ( Condition ) - parseDeclarationsBlock(prot); - if (token.kind == T.Else) - nT(), parseDeclarationsBlock(prot); - break; - case T.Version: - nT(); - if (token.kind == T.Assign) - { - nT(), nT(), nT(); // = Condition ; - break; - } - nT(), nT(), nT(); // ( Condition ) - parseDeclarationsBlock(prot); - if (token.kind == T.Else) - nT(), parseDeclarationsBlock(prot); - break; - case T.Template: - nT(); - skip(T.Identifier); - skipToTokenAfterClosingParen(); - parseDeclarationDefinitionsBlock(Protection.None); - break; - case T.New: - nT(); - skipToTokenAfterClosingParen(); - parseFunctionBody(); - break; - case T.Delete: - nT(); - skipToTokenAfterClosingParen(); - parseFunctionBody(); - break; - case T.Mixin: - while (token.kind != T.Semicolon && token.kind != T.EOF) - if (token.kind == T.LParen) - skipToTokenAfterClosingParen(); - else - nT(); - skip(T.Semicolon); - break; - case T.Semicolon: - nT(); - break; - // Declaration - case T.Identifier, T.Dot, T.Typeof: - case_Declaration: - while (token.kind != T.Semicolon && token.kind != T.EOF) - if (token.kind == T.LParen) - skipToTokenAfterClosingParen(); - else if (token.kind == T.LBrace) - skipToTokenAfterClosingBrace(); - else - nT(); - skip(T.Semicolon); - break; - default: - if (token.isIntegralType) - goto case_Declaration; - nT(); - } - } - - FuncBodyStatement parseFunctionBody() - { - while (1) - { - switch (token.kind) - { - case T.LBrace: - skipToTokenAfterClosingBrace(); - break; - case T.Semicolon: - nT(); - break; - case T.In: - nT(); - skipToTokenAfterClosingBrace(); - continue; - case T.Out: - nT(); - if (token.kind == T.LParen) - nT(), nT(), nT(); // ( Identifier ) - skipToTokenAfterClosingBrace(); - continue; - case T.Body: - nT(); - goto case T.LBrace; - default: - } - break; // Exit loop. - } - return null; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/parser/Parser.d --- a/trunk/src/dil/parser/Parser.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,4127 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.parser.Parser; - -import dil.lexer.Lexer; -import dil.ast.Node; -import dil.ast.Declarations; -import dil.ast.Statements; -import dil.ast.Expressions; -import dil.ast.Types; -import dil.ast.Parameters; -import dil.lexer.IdTable; -import dil.Messages; -import dil.Information; -import dil.Enums; -import dil.CompilerInfo; -import dil.SourceText; -import dil.Unicode; -import common; - -/// The Parser produces a full parse tree by examining -/// the list of tokens provided by the Lexer. -class Parser -{ - Lexer lexer; /// Used to lex the source code. - Token* token; /// Current non-whitespace token. - Token* prevToken; /// Previous non-whitespace token. - - InfoManager infoMan; - ParserError[] errors; - - ImportDeclaration[] imports; /// ImportDeclarations in the source text. - - /// Attributes are evaluated in the parsing phase. - /// TODO: will be removed. SemanticPass1 takes care of attributes. - LinkageType linkageType; - Protection protection; /// ditto - StorageClass storageClass; /// ditto - uint alignSize = DEFAULT_ALIGN_SIZE; /// ditto - - private alias TOK T; /// Used often in this class. - private alias TypeNode Type; - - /// Constructs a Parser object. - /// Params: - /// text = the UTF-8 source code. - /// infoMan = used for collecting error messages. - this(SourceText srcText, InfoManager infoMan = null) - { - this.infoMan = infoMan; - lexer = new Lexer(srcText, infoMan); - } - - /// Moves to the first token. - protected void init() - { - nT(); - prevToken = token; - } - - /// Moves to the next token. - void nT() - { - prevToken = token; - do - { - lexer.nextToken(); - token = lexer.token; - } while (token.isWhitespace) // Skip whitespace - } - - /// Start the parser and return the parsed Declarations. - CompoundDeclaration start() - { - init(); - auto begin = token; - auto decls = new CompoundDeclaration; - if (token.kind == T.Module) - decls ~= parseModuleDeclaration(); - decls.addOptChildren(parseDeclarationDefinitions()); - set(decls, begin); - return decls; - } - - /// Start the parser and return the parsed Expression. - Expression start2() - { - init(); - return parseExpression(); - } - - // Members related to the method try_(). - uint trying; /// Greater than 0 if Parser is in try_(). - uint errorCount; /// Used to track nr. of errors while being in try_(). - - /// This method executes the delegate parseMethod and when an error occurred - /// the state of the lexer and parser are restored. - /// Returns: the return value of parseMethod(). - ReturnType try_(ReturnType)(ReturnType delegate() parseMethod, out bool success) - { - // Save members. - auto oldToken = this.token; - auto oldPrevToken = this.prevToken; - auto oldCount = this.errorCount; - - ++trying; - auto result = parseMethod(); - --trying; - // Check if an error occurred. - if (errorCount != oldCount) - { // Restore members. - token = oldToken; - prevToken = oldPrevToken; - lexer.token = oldToken; - errorCount = oldCount; - success = false; - } - else - success = true; - return result; - } - - /// Sets the begin and end tokens of a syntax tree node. - Class set(Class)(Class node, Token* begin) - { - node.setTokens(begin, this.prevToken); - return node; - } - - /// Sets the begin and end tokens of a syntax tree node. - Class set(Class)(Class node, Token* begin, Token* end) - { - node.setTokens(begin, end); - return node; - } - - /// Returns true if set() has been called on a node. - static bool isNodeSet(Node node) - { - return node.begin !is null && node.end !is null; - } - - /// Returns the token kind of the next token. - TOK peekNext() - { - Token* next = token; - do - lexer.peek(next); - while (next.isWhitespace) // Skip whitespace - return next.kind; - } - - /// Returns the token kind of the token that comes after t. - TOK peekAfter(ref Token* t) - { - assert(t !is null); - do - lexer.peek(t); - while (t.isWhitespace) // Skip whitespace - return t.kind; - } - - /// Consumes the current token if its kind matches k and returns true. - bool consumed()(TOK k) // Templatized, so it's inlined. - { - return token.kind == k ? (nT(), true) : false; - } - - /// Asserts that the current token is of kind expectedKind, - /// and then moves to the next token. - void skip()(TOK expectedKind) - { - assert(token.kind == expectedKind /+|| *(int*).init+/, token.srcText()); - nT(); - } - - /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - | Declaration parsing methods | - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ - - Declaration parseModuleDeclaration() - { - skip(T.Module); - auto begin = token; - ModuleFQN moduleFQN; - do - moduleFQN ~= requireIdentifier(MSG.ExpectedModuleIdentifier); - while (consumed(T.Dot)) - require(T.Semicolon); - return set(new ModuleDeclaration(moduleFQN), begin); - } - - /// Parses DeclarationDefinitions until the end of file is hit. - /// $(PRE - /// DeclDefs := - /// DeclDef - /// DeclDefs - /// ) - Declaration[] parseDeclarationDefinitions() - { - Declaration[] decls; - while (token.kind != T.EOF) - decls ~= parseDeclarationDefinition(); - return decls; - } - - /// Parse the body of a template, class, interface, struct or union. - /// $(PRE - /// DeclDefsBlock := - /// { } - /// { DeclDefs } - /// ) - CompoundDeclaration parseDeclarationDefinitionsBody() - { - // Save attributes. - auto linkageType = this.linkageType; - auto protection = this.protection; - auto storageClass = this.storageClass; - // Clear attributes. - this.linkageType = LinkageType.None; - this.protection = Protection.None; - this.storageClass = StorageClass.None; - - // Parse body. - auto begin = token; - auto decls = new CompoundDeclaration; - require(T.LBrace); - while (token.kind != T.RBrace && token.kind != T.EOF) - decls ~= parseDeclarationDefinition(); - require(T.RBrace); - set(decls, begin); - - // Restore original values. - this.linkageType = linkageType; - this.protection = protection; - this.storageClass = storageClass; - - return decls; - } - - /// Parses a DeclarationDefinition. - Declaration parseDeclarationDefinition() - out(decl) - { assert(isNodeSet(decl)); } - body - { - auto begin = token; - Declaration decl; - switch (token.kind) - { - case T.Align, - T.Pragma, - // Protection attributes - T.Export, - T.Private, - T.Package, - T.Protected, - T.Public: - decl = parseAttributeSpecifier(); - break; - // Storage classes - case T.Extern, - T.Deprecated, - T.Override, - T.Abstract, - T.Synchronized, - //T.Static, - T.Final, - T.Const, - //T.Invariant, // D 2.0 - T.Auto, - T.Scope: - case_StaticAttribute: - case_InvariantAttribute: // D 2.0 - return parseStorageAttribute(); - case T.Alias: - nT(); - decl = new AliasDeclaration(parseVariableOrFunction()); - break; - case T.Typedef: - nT(); - decl = new TypedefDeclaration(parseVariableOrFunction()); - break; - case T.Static: - switch (peekNext()) - { - case T.Import: - goto case_Import; - case T.This: - decl = parseStaticConstructorDeclaration(); - break; - case T.Tilde: - decl = parseStaticDestructorDeclaration(); - break; - case T.If: - decl = parseStaticIfDeclaration(); - break; - case T.Assert: - decl = parseStaticAssertDeclaration(); - break; - default: - goto case_StaticAttribute; - } - break; - case T.Import: - case_Import: - decl = parseImportDeclaration(); - imports ~= decl.to!(ImportDeclaration); - // Handle specially. StorageClass mustn't be set. - decl.setProtection(this.protection); - return set(decl, begin); - case T.Enum: - decl = parseEnumDeclaration(); - break; - case T.Class: - decl = parseClassDeclaration(); - break; - case T.Interface: - decl = parseInterfaceDeclaration(); - break; - case T.Struct, T.Union: - decl = parseStructOrUnionDeclaration(); - break; - case T.This: - decl = parseConstructorDeclaration(); - break; - case T.Tilde: - decl = parseDestructorDeclaration(); - break; - case T.Invariant: - version(D2) - { - auto next = token; - if (peekAfter(next) == T.LParen) - { - if (peekAfter(next) != T.RParen) - goto case_Declaration; - } - else - goto case_InvariantAttribute; - } - decl = parseInvariantDeclaration(); - break; - case T.Unittest: - decl = parseUnittestDeclaration(); - break; - case T.Debug: - decl = parseDebugDeclaration(); - break; - case T.Version: - decl = parseVersionDeclaration(); - break; - case T.Template: - decl = parseTemplateDeclaration(); - break; - case T.New: - decl = parseNewDeclaration(); - break; - case T.Delete: - decl = parseDeleteDeclaration(); - break; - case T.Mixin: - decl = parseMixin!(MixinDeclaration)(); - break; - case T.Semicolon: - nT(); - decl = new EmptyDeclaration(); - break; - // Declaration - case T.Identifier, T.Dot, T.Typeof: - case_Declaration: - return parseVariableOrFunction(this.storageClass, this.protection, this.linkageType); - default: - if (token.isIntegralType) - goto case_Declaration; - else if (token.kind == T.Module) - { - decl = parseModuleDeclaration(); - error(begin, MSG.ModuleDeclarationNotFirst); - return decl; - } - - decl = new IllegalDeclaration(); - // Skip to next valid token. - do - nT(); - while (!token.isDeclDefStart && - token.kind != T.RBrace && - token.kind != T.EOF) - auto text = Token.textSpan(begin, this.prevToken); - error(begin, MSG.IllegalDeclaration, text); - } - decl.setProtection(this.protection); - decl.setStorageClass(this.storageClass); - assert(!isNodeSet(decl)); - set(decl, begin); - return decl; - } - - /// Parses a DeclarationsBlock. - /// $(PRE - /// DeclarationsBlock := - /// : DeclDefs - /// { } - /// { DeclDefs } - /// DeclDef - /// ) - Declaration parseDeclarationsBlock(/+bool noColon = false+/) - { - Declaration d; - switch (token.kind) - { - case T.LBrace: - auto begin = token; - nT(); - auto decls = new CompoundDeclaration; - while (token.kind != T.RBrace && token.kind != T.EOF) - decls ~= parseDeclarationDefinition(); - require(T.RBrace); - d = set(decls, begin); - break; - case T.Colon: - // if (noColon == true) - // goto default; - nT(); - auto begin = token; - auto decls = new CompoundDeclaration; - while (token.kind != T.RBrace && token.kind != T.EOF) - decls ~= parseDeclarationDefinition(); - d = set(decls, begin); - break; - default: - d = parseDeclarationDefinition(); - } - assert(isNodeSet(d)); - return d; - } - - // Declaration parseDeclarationsBlockNoColon() - // { - // return parseDeclarationsBlock(true); - // } - - /// Parses either a VariableDeclaration or a FunctionDeclaration. - /// Params: - /// stc = previously parsed storage classes - /// protection = previously parsed protection attribute - /// linkType = previously parsed linkage type - /// testAutoDeclaration = whether to check for an AutoDeclaration - /// optionalParameterList = a hint for how to parse C-style function pointers - Declaration parseVariableOrFunction(StorageClass stc = StorageClass.None, - Protection protection = Protection.None, - LinkageType linkType = LinkageType.None, - bool testAutoDeclaration = false, - bool optionalParameterList = true) - { - auto begin = token; - Type type; - Identifier* name; - - // Check for AutoDeclaration: StorageClasses Identifier = - if (testAutoDeclaration && - token.kind == T.Identifier && - peekNext() == T.Assign) - { - name = token.ident; - skip(T.Identifier); - } - else - { - type = parseType(); // VariableType or ReturnType - if (token.kind == T.LParen) - { - // C-style function pointers make the grammar ambiguous. - // We have to treat them specially at function scope. - // Example: - // void foo() { - // // A pointer to a function taking an integer and returning 'some_type'. - // some_type (*p_func)(int); - // // In the following case precedence is given to a CallExpression. - // something(*p); // 'something' may be a function/method or an object having opCall overloaded. - // } - // // A pointer to a function taking no parameters and returning 'something'. - // something(*p); - type = parseCFunctionPointerType(type, name, optionalParameterList); - } - else if (peekNext() == T.LParen) - { // Type FunctionName ( ParameterList ) FunctionBody - name = requireIdentifier(MSG.ExpectedFunctionName); - name || nT(); // Skip non-identifier token. - assert(token.kind == T.LParen); - // It's a function declaration - TemplateParameters tparams; - if (tokenAfterParenIs(T.LParen)) - // ( TemplateParameterList ) ( ParameterList ) - tparams = parseTemplateParameterList(); - - auto params = parseParameterList(); - version(D2) - { - switch (token.kind) - { - case T.Const: - stc |= StorageClass.Const; - nT(); - break; - case T.Invariant: - stc |= StorageClass.Invariant; - nT(); - break; - default: - } - } - // ReturnType FunctionName ( ParameterList ) - auto funcBody = parseFunctionBody(); - auto fd = new FunctionDeclaration(type, name,/+ tparams,+/ params, funcBody); - fd.setStorageClass(stc); - fd.setLinkageType(linkType); - fd.setProtection(protection); - if (tparams) - { - auto d = putInsideTemplateDeclaration(begin, name, fd, tparams); - d.setStorageClass(stc); - d.setProtection(protection); - return set(d, begin); - } - return set(fd, begin); - } - else - { // Type VariableName DeclaratorSuffix - name = requireIdentifier(MSG.ExpectedVariableName); - type = parseDeclaratorSuffix(type); - } - } - - // It's a variables declaration. - Identifier*[] names = [name]; // One identifier has been parsed already. - Expression[] values; - goto LenterLoop; // Enter the loop and check for an initializer. - while (consumed(T.Comma)) - { - names ~= requireIdentifier(MSG.ExpectedVariableName); - LenterLoop: - if (consumed(T.Assign)) - values ~= parseInitializer(); - else - values ~= null; - } - require(T.Semicolon); - auto d = new VariablesDeclaration(type, names, values); - d.setStorageClass(stc); - d.setLinkageType(linkType); - d.setProtection(protection); - return set(d, begin); - } - - /// Parses a variable initializer. - Expression parseInitializer() - { - if (token.kind == T.Void) - { - auto begin = token; - auto next = peekNext(); - if (next == T.Comma || next == T.Semicolon) - { - skip(T.Void); - return set(new VoidInitExpression(), begin); - } - } - return parseNonVoidInitializer(); - } - - Expression parseNonVoidInitializer() - { - auto begin = token; - Expression init; - switch (token.kind) - { - case T.LBracket: - // ArrayInitializer: - // [ ] - // [ ArrayMemberInitializations ] - Expression[] keys; - Expression[] values; - - skip(T.LBracket); - while (token.kind != T.RBracket) - { - auto e = parseNonVoidInitializer(); - if (consumed(T.Colon)) - { - keys ~= e; - values ~= parseNonVoidInitializer(); - } - else - { - keys ~= null; - values ~= e; - } - - if (!consumed(T.Comma)) - break; - } - require(T.RBracket); - init = new ArrayInitExpression(keys, values); - break; - case T.LBrace: - // StructInitializer: - // { } - // { StructMemberInitializers } - Expression parseStructInitializer() - { - Identifier*[] idents; - Expression[] values; - - skip(T.LBrace); - while (token.kind != T.RBrace) - { - if (token.kind == T.Identifier && - // Peek for colon to see if this is a member identifier. - peekNext() == T.Colon) - { - idents ~= token.ident; - skip(T.Identifier), skip(T.Colon); - } - else - idents ~= null; - - // NonVoidInitializer - values ~= parseNonVoidInitializer(); - - if (!consumed(T.Comma)) - break; - } - require(T.RBrace); - return new StructInitExpression(idents, values); - } - - bool success; - auto si = try_(&parseStructInitializer, success); - if (success) - { - init = si; - break; - } - assert(token.kind == T.LBrace); - //goto default; - default: - init = parseAssignExpression(); - } - set(init, begin); - return init; - } - - FuncBodyStatement parseFunctionBody() - { - auto begin = token; - auto func = new FuncBodyStatement; - while (1) - { - switch (token.kind) - { - case T.LBrace: - func.funcBody = parseStatements(); - break; - case T.Semicolon: - nT(); - break; - case T.In: - if (func.inBody) - error(MID.InContract); - nT(); - func.inBody = parseStatements(); - continue; - case T.Out: - if (func.outBody) - error(MID.OutContract); - nT(); - if (consumed(T.LParen)) - { - func.outIdent = requireIdentifier(MSG.ExpectedAnIdentifier); - require(T.RParen); - } - func.outBody = parseStatements(); - continue; - case T.Body: - nT(); - goto case T.LBrace; - default: - error(token, MSG.ExpectedFunctionBody, token.srcText); - } - break; // Exit loop. - } - set(func, begin); - func.finishConstruction(); - return func; - } - - LinkageType parseLinkageType() - { - LinkageType linkageType; - - if (!consumed(T.LParen)) - return linkageType; - - if (consumed(T.RParen)) - { // extern() - error(MID.MissingLinkageType); - return linkageType; - } - - auto identTok = requireId(); - - IDK idKind = identTok ? identTok.ident.idKind : IDK.Null; - - switch (idKind) - { - case IDK.C: - if (consumed(T.PlusPlus)) - { - linkageType = LinkageType.Cpp; - break; - } - linkageType = LinkageType.C; - break; - case IDK.D: - linkageType = LinkageType.D; - break; - case IDK.Windows: - linkageType = LinkageType.Windows; - break; - case IDK.Pascal: - linkageType = LinkageType.Pascal; - break; - case IDK.System: - linkageType = LinkageType.System; - break; - default: - error(MID.UnrecognizedLinkageType, token.srcText); - } - require(T.RParen); - return linkageType; - } - - void checkLinkageType(ref LinkageType prev_lt, LinkageType lt, Token* begin) - { - if (prev_lt == LinkageType.None) - prev_lt = lt; - else - error(begin, MSG.RedundantLinkageType, Token.textSpan(begin, this.prevToken)); - } - - Declaration parseStorageAttribute() - { - StorageClass stc, stc_tmp; - LinkageType prev_linkageType; - - auto saved_storageClass = this.storageClass; // Save. - // Nested function. - Declaration parse() - { - Declaration decl; - auto begin = token; - switch (token.kind) - { - case T.Extern: - if (peekNext() != T.LParen) - { - stc_tmp = StorageClass.Extern; - goto Lcommon; - } - - nT(); - auto linkageType = parseLinkageType(); - checkLinkageType(prev_linkageType, linkageType, begin); - - auto saved = this.linkageType; // Save. - this.linkageType = linkageType; // Set. - decl = new LinkageDeclaration(linkageType, parse()); - set(decl, begin); - this.linkageType = saved; // Restore. - break; - case T.Override: - stc_tmp = StorageClass.Override; - goto Lcommon; - case T.Deprecated: - stc_tmp = StorageClass.Deprecated; - goto Lcommon; - case T.Abstract: - stc_tmp = StorageClass.Abstract; - goto Lcommon; - case T.Synchronized: - stc_tmp = StorageClass.Synchronized; - goto Lcommon; - case T.Static: - stc_tmp = StorageClass.Static; - goto Lcommon; - case T.Final: - stc_tmp = StorageClass.Final; - goto Lcommon; - case T.Const: - version(D2) - { - if (peekNext() == T.LParen) - goto case_Declaration; - } - stc_tmp = StorageClass.Const; - goto Lcommon; - version(D2) - { - case T.Invariant: // D 2.0 - auto next = token; - if (peekAfter(next) == T.LParen) - { - if (peekAfter(next) != T.RParen) - goto case_Declaration; // invariant ( Type ) - decl = parseDeclarationDefinition(); // invariant ( ) - decl.setStorageClass(stc); - break; - } - // invariant as StorageClass. - stc_tmp = StorageClass.Invariant; - goto Lcommon; - } - case T.Auto: - stc_tmp = StorageClass.Auto; - goto Lcommon; - case T.Scope: - stc_tmp = StorageClass.Scope; - goto Lcommon; - Lcommon: - // Issue error if redundant. - if (stc & stc_tmp) - error(MID.RedundantStorageClass, token.srcText); - else - stc |= stc_tmp; - - nT(); - decl = new StorageClassDeclaration(stc_tmp, parse()); - set(decl, begin); - break; - case T.Identifier: - case_Declaration: - // This could be a normal Declaration or an AutoDeclaration - decl = parseVariableOrFunction(stc, this.protection, prev_linkageType, true); - break; - default: - this.storageClass = stc; // Set. - decl = parseDeclarationsBlock(); - this.storageClass = saved_storageClass; // Reset. - } - assert(isNodeSet(decl)); - return decl; - } - return parse(); - } - - uint parseAlignAttribute() - { - skip(T.Align); - uint size = DEFAULT_ALIGN_SIZE; // Global default. - if (consumed(T.LParen)) - { - if (token.kind == T.Int32) - (size = token.int_), skip(T.Int32); - else - expected(T.Int32); - require(T.RParen); - } - return size; - } - - Declaration parseAttributeSpecifier() - { - Declaration decl; - - switch (token.kind) - { - case T.Align: - uint alignSize = parseAlignAttribute(); - auto saved = this.alignSize; // Save. - this.alignSize = alignSize; // Set. - decl = new AlignDeclaration(alignSize, parseDeclarationsBlock()); - this.alignSize = saved; // Restore. - break; - case T.Pragma: - // Pragma: - // pragma ( Identifier ) - // pragma ( Identifier , ExpressionList ) - nT(); - Identifier* ident; - Expression[] args; - - require(T.LParen); - ident = requireIdentifier(MSG.ExpectedPragmaIdentifier); - - if (consumed(T.Comma)) - args = parseExpressionList(); - require(T.RParen); - - decl = new PragmaDeclaration(ident, args, parseDeclarationsBlock()); - break; - default: - // Protection attributes - Protection prot; - switch (token.kind) - { - case T.Private: - prot = Protection.Private; break; - case T.Package: - prot = Protection.Package; break; - case T.Protected: - prot = Protection.Protected; break; - case T.Public: - prot = Protection.Public; break; - case T.Export: - prot = Protection.Export; break; - default: - assert(0); - } - nT(); - auto saved = this.protection; // Save. - this.protection = prot; // Set. - decl = new ProtectionDeclaration(prot, parseDeclarationsBlock()); - this.protection = saved; // Restore. - } - return decl; - } - - Declaration parseImportDeclaration() - { - bool isStatic = consumed(T.Static); - skip(T.Import); - - ModuleFQN[] moduleFQNs; - Identifier*[] moduleAliases; - Identifier*[] bindNames; - Identifier*[] bindAliases; - - do - { - ModuleFQN moduleFQN; - Identifier* moduleAlias; - // AliasName = ModuleName - if (peekNext() == T.Assign) - { - moduleAlias = requireIdentifier(MSG.ExpectedAliasModuleName); - skip(T.Assign); - } - // Identifier ("." Identifier)* - do - moduleFQN ~= requireIdentifier(MSG.ExpectedModuleIdentifier); - while (consumed(T.Dot)) - // Push identifiers. - moduleFQNs ~= moduleFQN; - moduleAliases ~= moduleAlias; - } while (consumed(T.Comma)) - - if (consumed(T.Colon)) - { // BindAlias "=" BindName ("," BindAlias "=" BindName)*; - // BindName ("," BindName)*; - do - { - Identifier* bindAlias; - // BindAlias = BindName - if (peekNext() == T.Assign) - { - bindAlias = requireIdentifier(MSG.ExpectedAliasImportName); - skip(T.Assign); - } - // Push identifiers. - bindNames ~= requireIdentifier(MSG.ExpectedImportName); - bindAliases ~= bindAlias; - } while (consumed(T.Comma)) - } - require(T.Semicolon); - - return new ImportDeclaration(moduleFQNs, moduleAliases, bindNames, bindAliases, isStatic); - } - - Declaration parseEnumDeclaration() - { - skip(T.Enum); - - Identifier* enumName; - Type baseType; - EnumMemberDeclaration[] members; - bool hasBody; - - enumName = optionalIdentifier(); - - if (consumed(T.Colon)) - baseType = parseBasicType(); - - if (enumName && consumed(T.Semicolon)) - {} - else if (consumed(T.LBrace)) - { - hasBody = true; - while (token.kind != T.RBrace) - { - auto begin = token; - auto name = requireIdentifier(MSG.ExpectedEnumMember); - Expression value; - - if (consumed(T.Assign)) - value = parseAssignExpression(); - else - value = null; - - members ~= set(new EnumMemberDeclaration(name, value), begin); - - if (!consumed(T.Comma)) - break; - } - require(T.RBrace); - } - else - error(token, MSG.ExpectedEnumBody, token.srcText); - - return new EnumDeclaration(enumName, baseType, members, hasBody); - } - - /// Wraps a declaration inside a template declaration. - /// Params: - /// begin = begin token of decl. - /// name = name of decl. - /// decl = the declaration to be wrapped. - /// tparams = the template parameters. - TemplateDeclaration putInsideTemplateDeclaration(Token* begin, - Identifier* name, - Declaration decl, - TemplateParameters tparams) - { - set(decl, begin); - auto cd = new CompoundDeclaration; - cd ~= decl; - set(cd, begin); - return new TemplateDeclaration(name, tparams, cd); - } - - Declaration parseClassDeclaration() - { - auto begin = token; - skip(T.Class); - - Identifier* className; - TemplateParameters tparams; - BaseClassType[] bases; - CompoundDeclaration decls; - - className = requireIdentifier(MSG.ExpectedClassName); - - if (token.kind == T.LParen) - tparams = parseTemplateParameterList(); - - if (token.kind == T.Colon) - bases = parseBaseClasses(); - - if (bases.length == 0 && consumed(T.Semicolon)) - {} - else if (token.kind == T.LBrace) - decls = parseDeclarationDefinitionsBody(); - else - error(token, MSG.ExpectedClassBody, token.srcText); - - Declaration d = new ClassDeclaration(className, /+tparams, +/bases, decls); - if (tparams) - d = putInsideTemplateDeclaration(begin, className, d, tparams); - return d; - } - - BaseClassType[] parseBaseClasses(bool colonLeadsOff = true) - { - colonLeadsOff && skip(T.Colon); - - BaseClassType[] bases; - do - { - Protection prot = Protection.Public; - switch (token.kind) - { - case T.Identifier, T.Dot, T.Typeof: goto LparseBasicType; - case T.Private: prot = Protection.Private; break; - case T.Protected: prot = Protection.Protected; break; - case T.Package: prot = Protection.Package; break; - case T.Public: /*prot = Protection.Public;*/ break; - default: - error(MID.ExpectedBaseClasses, token.srcText); - return bases; - } - nT(); // Skip protection attribute. - LparseBasicType: - auto begin = token; - auto type = parseBasicType(); - bases ~= set(new BaseClassType(prot, type), begin); - } while (consumed(T.Comma)) - return bases; - } - - Declaration parseInterfaceDeclaration() - { - auto begin = token; - skip(T.Interface); - - Identifier* name; - TemplateParameters tparams; - BaseClassType[] bases; - CompoundDeclaration decls; - - name = requireIdentifier(MSG.ExpectedInterfaceName); - - if (token.kind == T.LParen) - tparams = parseTemplateParameterList(); - - if (token.kind == T.Colon) - bases = parseBaseClasses(); - - if (bases.length == 0 && consumed(T.Semicolon)) - {} - else if (token.kind == T.LBrace) - decls = parseDeclarationDefinitionsBody(); - else - error(token, MSG.ExpectedInterfaceBody, token.srcText); - - Declaration d = new InterfaceDeclaration(name, /+tparams, +/bases, decls); - if (tparams) - d = putInsideTemplateDeclaration(begin, name, d, tparams); - return d; - } - - Declaration parseStructOrUnionDeclaration() - { - assert(token.kind == T.Struct || token.kind == T.Union); - auto begin = token; - skip(token.kind); - - Identifier* name; - TemplateParameters tparams; - CompoundDeclaration decls; - - name = optionalIdentifier(); - - if (name && token.kind == T.LParen) - tparams = parseTemplateParameterList(); - - if (name && consumed(T.Semicolon)) - {} - else if (token.kind == T.LBrace) - decls = parseDeclarationDefinitionsBody(); - else - error(token, begin.kind == T.Struct ? - MSG.ExpectedStructBody : - MSG.ExpectedUnionBody, token.srcText); - - Declaration d; - if (begin.kind == T.Struct) - { - auto sd = new StructDeclaration(name, /+tparams, +/decls); - sd.setAlignSize(this.alignSize); - d = sd; - } - else - d = new UnionDeclaration(name, /+tparams, +/decls); - - if (tparams) - d = putInsideTemplateDeclaration(begin, name, d, tparams); - return d; - } - - Declaration parseConstructorDeclaration() - { - skip(T.This); - auto parameters = parseParameterList(); - auto funcBody = parseFunctionBody(); - return new ConstructorDeclaration(parameters, funcBody); - } - - Declaration parseDestructorDeclaration() - { - skip(T.Tilde); - require(T.This); - require(T.LParen); - require(T.RParen); - auto funcBody = parseFunctionBody(); - return new DestructorDeclaration(funcBody); - } - - Declaration parseStaticConstructorDeclaration() - { - skip(T.Static); - skip(T.This); - require(T.LParen); - require(T.RParen); - auto funcBody = parseFunctionBody(); - return new StaticConstructorDeclaration(funcBody); - } - - Declaration parseStaticDestructorDeclaration() - { - skip(T.Static); - skip(T.Tilde); - require(T.This); - require(T.LParen); - require(T.RParen); - auto funcBody = parseFunctionBody(); - return new StaticDestructorDeclaration(funcBody); - } - - Declaration parseInvariantDeclaration() - { - skip(T.Invariant); - // Optional () for getting ready porting to D 2.0 - if (consumed(T.LParen)) - require(T.RParen); - auto funcBody = parseFunctionBody(); - return new InvariantDeclaration(funcBody); - } - - Declaration parseUnittestDeclaration() - { - skip(T.Unittest); - auto funcBody = parseFunctionBody(); - return new UnittestDeclaration(funcBody); - } - - Token* parseIdentOrInt() - { - if (consumed(T.Int32) || consumed(T.Identifier)) - return this.prevToken; - error(token, MSG.ExpectedIdentOrInt, token.srcText); - return null; - } - - Declaration parseDebugDeclaration() - { - skip(T.Debug); - - Token* spec; - Token* cond; - Declaration decls, elseDecls; - - if (consumed(T.Assign)) - { // debug = Integer ; - // debug = Identifier ; - spec = parseIdentOrInt(); - require(T.Semicolon); - } - else - { // ( Condition ) - if (consumed(T.LParen)) - { - cond = parseIdentOrInt(); - require(T.RParen); - } - // debug DeclarationsBlock - // debug ( Condition ) DeclarationsBlock - decls = parseDeclarationsBlock(); - // else DeclarationsBlock - if (consumed(T.Else)) - elseDecls = parseDeclarationsBlock(); - } - - return new DebugDeclaration(spec, cond, decls, elseDecls); - } - - Declaration parseVersionDeclaration() - { - skip(T.Version); - - Token* spec; - Token* cond; - Declaration decls, elseDecls; - - if (consumed(T.Assign)) - { // version = Integer ; - // version = Identifier ; - spec = parseIdentOrInt(); - require(T.Semicolon); - } - else - { // ( Condition ) - require(T.LParen); - cond = parseIdentOrInt(); - require(T.RParen); - // version ( Condition ) DeclarationsBlock - decls = parseDeclarationsBlock(); - // else DeclarationsBlock - if (consumed(T.Else)) - elseDecls = parseDeclarationsBlock(); - } - - return new VersionDeclaration(spec, cond, decls, elseDecls); - } - - Declaration parseStaticIfDeclaration() - { - skip(T.Static); - skip(T.If); - - Expression condition; - Declaration ifDecls, elseDecls; - - require(T.LParen); - condition = parseAssignExpression(); - require(T.RParen); - - ifDecls = parseDeclarationsBlock(); - - if (consumed(T.Else)) - elseDecls = parseDeclarationsBlock(); - - return new StaticIfDeclaration(condition, ifDecls, elseDecls); - } - - Declaration parseStaticAssertDeclaration() - { - skip(T.Static); - skip(T.Assert); - Expression condition, message; - require(T.LParen); - condition = parseAssignExpression(); - if (consumed(T.Comma)) - message = parseAssignExpression(); - require(T.RParen); - require(T.Semicolon); - return new StaticAssertDeclaration(condition, message); - } - - Declaration parseTemplateDeclaration() - { - skip(T.Template); - auto templateName = requireIdentifier(MSG.ExpectedTemplateName); - auto templateParams = parseTemplateParameterList(); - auto decls = parseDeclarationDefinitionsBody(); - return new TemplateDeclaration(templateName, templateParams, decls); - } - - Declaration parseNewDeclaration() - { - skip(T.New); - auto parameters = parseParameterList(); - auto funcBody = parseFunctionBody(); - return new NewDeclaration(parameters, funcBody); - } - - Declaration parseDeleteDeclaration() - { - skip(T.Delete); - auto parameters = parseParameterList(); - auto funcBody = parseFunctionBody(); - return new DeleteDeclaration(parameters, funcBody); - } - - Type parseTypeofType() - { - auto begin = token; - skip(T.Typeof); - require(T.LParen); - Type type; - switch (token.kind) - { - version(D2) - { - case T.Return: - nT(); - type = new TypeofType(); - break; - } - default: - type = new TypeofType(parseExpression()); - } - require(T.RParen); - set(type, begin); - return type; - } - - /// Parses a MixinDeclaration or MixinStatement. - /// $(PRE - /// TemplateMixin := - /// mixin ( AssignExpression ) ; - /// mixin TemplateIdentifier ; - /// mixin TemplateIdentifier MixinIdentifier ; - /// mixin TemplateIdentifier !( TemplateArguments ) ; - /// mixin TemplateIdentifier !( TemplateArguments ) MixinIdentifier ; - /// ) - Class parseMixin(Class)() - { - static assert(is(Class == MixinDeclaration) || is(Class == MixinStatement)); - skip(T.Mixin); - - static if (is(Class == MixinDeclaration)) - { - if (consumed(T.LParen)) - { - auto e = parseAssignExpression(); - require(T.RParen); - require(T.Semicolon); - return new MixinDeclaration(e); - } - } - - auto begin = token; - Expression e; - Identifier* mixinIdent; - - if (consumed(T.Dot)) - e = set(new ModuleScopeExpression(parseIdentifierExpression()), begin); - else - e = parseIdentifierExpression(); - - while (consumed(T.Dot)) - e = set(new DotExpression(e, parseIdentifierExpression()), begin); - - mixinIdent = optionalIdentifier(); - require(T.Semicolon); - - return new Class(e, mixinIdent); - } - - /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - | Statement parsing methods | - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ - - CompoundStatement parseStatements() - { - auto begin = token; - require(T.LBrace); - auto statements = new CompoundStatement(); - while (token.kind != T.RBrace && token.kind != T.EOF) - statements ~= parseStatement(); - require(T.RBrace); - return set(statements, begin); - } - - /// Parses a Statement. - Statement parseStatement() - { - auto begin = token; - Statement s; - Declaration d; - - if (token.isIntegralType) - { - d = parseVariableOrFunction(); - goto LreturnDeclarationStatement; - } - - switch (token.kind) - { - case T.Align: - uint size = parseAlignAttribute(); - // Restrict align attribute to structs in parsing phase. - StructDeclaration structDecl; - if (token.kind == T.Struct) - { - auto begin2 = token; - structDecl = parseStructOrUnionDeclaration().to!(StructDeclaration); - structDecl.setAlignSize(size); - set(structDecl, begin2); - } - else - expected(T.Struct); - - d = new AlignDeclaration(size, structDecl ? cast(Declaration)structDecl : new CompoundDeclaration); - goto LreturnDeclarationStatement; - /+ Not applicable for statements. - T.Private, T.Package, T.Protected, T.Public, T.Export, - T.Deprecated, T.Override, T.Abstract,+/ - case T.Extern, - T.Final, - T.Const, - T.Auto: - //T.Scope - //T.Static - case_parseAttribute: - s = parseAttributeStatement(); - return s; - case T.Identifier: - if (peekNext() == T.Colon) - { - auto ident = token.ident; - skip(T.Identifier); skip(T.Colon); - s = new LabeledStatement(ident, parseNoScopeOrEmptyStatement()); - break; - } - goto case T.Dot; - case T.Dot, T.Typeof: - bool success; - d = try_(delegate { - return parseVariableOrFunction(StorageClass.None, - Protection.None, - LinkageType.None, false, false); - }, success - ); - if (success) - goto LreturnDeclarationStatement; // Declaration - else - goto case_parseExpressionStatement; // Expression - - case T.If: - s = parseIfStatement(); - break; - case T.While: - s = parseWhileStatement(); - break; - case T.Do: - s = parseDoWhileStatement(); - break; - case T.For: - s = parseForStatement(); - break; - case T.Foreach, T.Foreach_reverse: - s = parseForeachStatement(); - break; - case T.Switch: - s = parseSwitchStatement(); - break; - case T.Case: - s = parseCaseStatement(); - break; - case T.Default: - s = parseDefaultStatement(); - break; - case T.Continue: - s = parseContinueStatement(); - break; - case T.Break: - s = parseBreakStatement(); - break; - case T.Return: - s = parseReturnStatement(); - break; - case T.Goto: - s = parseGotoStatement(); - break; - case T.With: - s = parseWithStatement(); - break; - case T.Synchronized: - s = parseSynchronizedStatement(); - break; - case T.Try: - s = parseTryStatement(); - break; - case T.Throw: - s = parseThrowStatement(); - break; - case T.Scope: - if (peekNext() != T.LParen) - goto case_parseAttribute; - s = parseScopeGuardStatement(); - break; - case T.Volatile: - s = parseVolatileStatement(); - break; - case T.Asm: - s = parseAsmBlockStatement(); - break; - case T.Pragma: - s = parsePragmaStatement(); - break; - case T.Mixin: - if (peekNext() == T.LParen) - goto case_parseExpressionStatement; // Parse as expression. - s = parseMixin!(MixinStatement)(); - break; - case T.Static: - switch (peekNext()) - { - case T.If: - s = parseStaticIfStatement(); - break; - case T.Assert: - s = parseStaticAssertStatement(); - break; - default: - goto case_parseAttribute; - } - break; - case T.Debug: - s = parseDebugStatement(); - break; - case T.Version: - s = parseVersionStatement(); - break; - // DeclDef - case T.Alias, T.Typedef: - d = parseDeclarationDefinition(); - goto LreturnDeclarationStatement; - case T.Enum: - d = parseEnumDeclaration(); - goto LreturnDeclarationStatement; - case T.Class: - d = parseClassDeclaration(); - goto LreturnDeclarationStatement; - case T.Interface: - d = parseInterfaceDeclaration(); - goto LreturnDeclarationStatement; - case T.Struct, T.Union: - d = parseStructOrUnionDeclaration(); - // goto LreturnDeclarationStatement; - LreturnDeclarationStatement: - set(d, begin); - s = new DeclarationStatement(d); - break; - case T.LBrace: - s = parseScopeStatement(); - break; - case T.Semicolon: - nT(); - s = new EmptyStatement(); - break; - // Parse an ExpressionStatement: - // Tokens that start a PrimaryExpression. - // case T.Identifier, T.Dot, T.Typeof: - case T.This: - case T.Super: - case T.Null: - case T.True, T.False: - // case T.Dollar: - case T.Int32, T.Int64, T.Uint32, T.Uint64: - case T.Float32, T.Float64, T.Float80, - T.Imaginary32, T.Imaginary64, T.Imaginary80: - case T.CharLiteral: - case T.String: - case T.LBracket: - // case T.LBrace: - case T.Function, T.Delegate: - case T.Assert: - // case T.Mixin: - case T.Import: - case T.Typeid: - case T.Is: - case T.LParen: - case T.Traits: // D2.0 - // Tokens that can start a UnaryExpression: - case T.AndBinary, T.PlusPlus, T.MinusMinus, T.Mul, T.Minus, - T.Plus, T.Not, T.Tilde, T.New, T.Delete, T.Cast: - case_parseExpressionStatement: - s = new ExpressionStatement(parseExpression()); - require(T.Semicolon); - break; - default: - if (token.isSpecialToken) - goto case_parseExpressionStatement; - - if (token.kind != T.Dollar) - // Assert that this isn't a valid expression. - assert(delegate bool(){ - bool success; - auto expression = try_(&parseExpression, success); - return success; - }() == false, "Didn't expect valid expression." - ); - - // Report error: it's an illegal statement. - s = new IllegalStatement(); - // Skip to next valid token. - do - nT(); - while (!token.isStatementStart && - token.kind != T.RBrace && - token.kind != T.EOF) - auto text = Token.textSpan(begin, this.prevToken); - error(begin, MSG.IllegalStatement, text); - } - assert(s !is null); - set(s, begin); - return s; - } - - /// $(PRE - /// Parses a ScopeStatement. - /// ScopeStatement := - /// NoScopeStatement - /// ) - Statement parseScopeStatement() - { - return new ScopeStatement(parseNoScopeStatement()); - } - - /// $(PRE - /// NoScopeStatement := - /// NonEmptyStatement - /// BlockStatement - /// BlockStatement := - /// { } - /// { StatementList } - /// ) - Statement parseNoScopeStatement() - { - auto begin = token; - Statement s; - if (consumed(T.LBrace)) - { - auto ss = new CompoundStatement(); - while (token.kind != T.RBrace && token.kind != T.EOF) - ss ~= parseStatement(); - require(T.RBrace); - s = set(ss, begin); - } - else if (token.kind == T.Semicolon) - { - error(token, MSG.ExpectedNonEmptyStatement); - nT(); - s = set(new EmptyStatement(), begin); - } - else - s = parseStatement(); - return s; - } - - /// $(PRE - /// NoScopeOrEmptyStatement := - /// ; - /// NoScopeStatement - /// ) - Statement parseNoScopeOrEmptyStatement() - { - if (consumed(T.Semicolon)) - return set(new EmptyStatement(), this.prevToken); - else - return parseNoScopeStatement(); - } - - Statement parseAttributeStatement() - { - StorageClass stc, stc_tmp; - LinkageType prev_linkageType; - - Declaration parse() // Nested function. - { - auto begin = token; - Declaration d; - switch (token.kind) - { - case T.Extern: - if (peekNext() != T.LParen) - { - stc_tmp = StorageClass.Extern; - goto Lcommon; - } - - nT(); - auto linkageType = parseLinkageType(); - checkLinkageType(prev_linkageType, linkageType, begin); - - d = new LinkageDeclaration(linkageType, parse()); - break; - case T.Static: - stc_tmp = StorageClass.Static; - goto Lcommon; - case T.Final: - stc_tmp = StorageClass.Final; - goto Lcommon; - case T.Const: - version(D2) - { - if (peekNext() == T.LParen) - goto case_Declaration; - } - stc_tmp = StorageClass.Const; - goto Lcommon; - version(D2) - { - case T.Invariant: // D 2.0 - if (peekNext() == T.LParen) - goto case_Declaration; - stc_tmp = StorageClass.Invariant; - goto Lcommon; - } - case T.Auto: - stc_tmp = StorageClass.Auto; - goto Lcommon; - case T.Scope: - stc_tmp = StorageClass.Scope; - goto Lcommon; - Lcommon: - // Issue error if redundant. - if (stc & stc_tmp) - error(MID.RedundantStorageClass, token.srcText); - else - stc |= stc_tmp; - - nT(); - d = new StorageClassDeclaration(stc_tmp, parse()); - break; - // TODO: allow "scope class", "abstract scope class" in function bodies? - //case T.Class: - default: - case_Declaration: - return parseVariableOrFunction(stc, Protection.None, prev_linkageType, true); - } - return set(d, begin); - } - return new DeclarationStatement(parse()); - } - - Statement parseIfStatement() - { - skip(T.If); - - Statement variable; - Expression condition; - Statement ifBody, elseBody; - - require(T.LParen); - - Identifier* ident; - auto begin = token; // For start of AutoDeclaration or normal Declaration. - // auto Identifier = Expression - if (consumed(T.Auto)) - { - ident = requireIdentifier(MSG.ExpectedVariableName); - require(T.Assign); - auto init = parseExpression(); - auto v = new VariablesDeclaration(null, [ident], [init]); - set(v, begin.nextNWS); - auto d = new StorageClassDeclaration(StorageClass.Auto, v); - set(d, begin); - variable = new DeclarationStatement(d); - set(variable, begin); - } - else - { // Declarator = Expression - Type parseDeclaratorAssign() - { - auto type = parseDeclarator(ident); - require(T.Assign); - return type; - } - bool success; - auto type = try_(&parseDeclaratorAssign, success); - if (success) - { - auto init = parseExpression(); - auto v = new VariablesDeclaration(type, [ident], [init]); - set(v, begin); - variable = new DeclarationStatement(v); - set(variable, begin); - } - else - condition = parseExpression(); - } - require(T.RParen); - ifBody = parseScopeStatement(); - if (consumed(T.Else)) - elseBody = parseScopeStatement(); - return new IfStatement(variable, condition, ifBody, elseBody); - } - - Statement parseWhileStatement() - { - skip(T.While); - require(T.LParen); - auto condition = parseExpression(); - require(T.RParen); - return new WhileStatement(condition, parseScopeStatement()); - } - - Statement parseDoWhileStatement() - { - skip(T.Do); - auto doBody = parseScopeStatement(); - require(T.While); - require(T.LParen); - auto condition = parseExpression(); - require(T.RParen); - return new DoWhileStatement(condition, doBody); - } - - Statement parseForStatement() - { - skip(T.For); - - Statement init, forBody; - Expression condition, increment; - - require(T.LParen); - if (!consumed(T.Semicolon)) - init = parseNoScopeStatement(); - if (token.kind != T.Semicolon) - condition = parseExpression(); - require(T.Semicolon); - if (token.kind != T.RParen) - increment = parseExpression(); - require(T.RParen); - forBody = parseScopeStatement(); - return new ForStatement(init, condition, increment, forBody); - } - - Statement parseForeachStatement() - { - assert(token.kind == T.Foreach || token.kind == T.Foreach_reverse); - TOK tok = token.kind; - nT(); - - auto params = new Parameters; - Expression e; // Aggregate or LwrExpression - - require(T.LParen); - auto paramsBegin = token; - do - { - auto paramBegin = token; - StorageClass stc; - Type type; - Identifier* ident; - - switch (token.kind) - { - case T.Ref, T.Inout: - stc = StorageClass.Ref; - nT(); - // fall through - case T.Identifier: - auto next = peekNext(); - if (next == T.Comma || next == T.Semicolon || next == T.RParen) - { - ident = requireIdentifier(MSG.ExpectedVariableName); - break; - } - // fall through - default: - type = parseDeclarator(ident); - } - - params ~= set(new Parameter(stc, type, ident, null), paramBegin); - } while (consumed(T.Comma)) - set(params, paramsBegin); - require(T.Semicolon); - e = parseExpression(); - version(D2) - { //Foreach (ForeachType; LwrExpression .. UprExpression ) ScopeStatement - if (consumed(T.Slice)) - { - // if (params.length != 1) - // error(MID.XYZ); // TODO: issue error msg - auto upper = parseExpression(); - require(T.RParen); - auto forBody = parseScopeStatement(); - return new ForeachRangeStatement(tok, params, e, upper, forBody); - } - } - // Foreach (ForeachTypeList; Aggregate) ScopeStatement - require(T.RParen); - auto forBody = parseScopeStatement(); - return new ForeachStatement(tok, params, e, forBody); - } - - Statement parseSwitchStatement() - { - skip(T.Switch); - require(T.LParen); - auto condition = parseExpression(); - require(T.RParen); - auto switchBody = parseScopeStatement(); - return new SwitchStatement(condition, switchBody); - } - - /// Helper function for parsing the body of a default or case statement. - Statement parseCaseOrDefaultBody() - { - // This function is similar to parseNoScopeStatement() - auto begin = token; - auto s = new CompoundStatement(); - while (token.kind != T.Case && - token.kind != T.Default && - token.kind != T.RBrace && - token.kind != T.EOF) - s ~= parseStatement(); - set(s, begin); - return set(new ScopeStatement(s), begin); - } - - Statement parseCaseStatement() - { - skip(T.Case); - auto values = parseExpressionList(); - require(T.Colon); - auto caseBody = parseCaseOrDefaultBody(); - return new CaseStatement(values, caseBody); - } - - Statement parseDefaultStatement() - { - skip(T.Default); - require(T.Colon); - auto defaultBody = parseCaseOrDefaultBody(); - return new DefaultStatement(defaultBody); - } - - Statement parseContinueStatement() - { - skip(T.Continue); - auto ident = optionalIdentifier(); - require(T.Semicolon); - return new ContinueStatement(ident); - } - - Statement parseBreakStatement() - { - skip(T.Break); - auto ident = optionalIdentifier(); - require(T.Semicolon); - return new BreakStatement(ident); - } - - Statement parseReturnStatement() - { - skip(T.Return); - Expression expr; - if (token.kind != T.Semicolon) - expr = parseExpression(); - require(T.Semicolon); - return new ReturnStatement(expr); - } - - Statement parseGotoStatement() - { - skip(T.Goto); - Identifier* ident; - Expression caseExpr; - switch (token.kind) - { - case T.Case: - ident = token.ident; - nT(); - if (token.kind == T.Semicolon) - break; - caseExpr = parseExpression(); - break; - case T.Default: - ident = token.ident; - nT(); - break; - default: - ident = requireIdentifier(MSG.ExpectedAnIdentifier); - } - require(T.Semicolon); - return new GotoStatement(ident, caseExpr); - } - - Statement parseWithStatement() - { - skip(T.With); - require(T.LParen); - auto expr = parseExpression(); - require(T.RParen); - return new WithStatement(expr, parseScopeStatement()); - } - - Statement parseSynchronizedStatement() - { - skip(T.Synchronized); - Expression expr; - if (consumed(T.LParen)) - { - expr = parseExpression(); - require(T.RParen); - } - return new SynchronizedStatement(expr, parseScopeStatement()); - } - - Statement parseTryStatement() - { - auto begin = token; - skip(T.Try); - - auto tryBody = parseScopeStatement(); - CatchStatement[] catchBodies; - FinallyStatement finBody; - - while (consumed(T.Catch)) - { - Parameter param; - if (consumed(T.LParen)) - { - auto begin2 = token; - Identifier* ident; - auto type = parseDeclarator(ident, true); - param = new Parameter(StorageClass.None, type, ident, null); - set(param, begin2); - require(T.RParen); - } - catchBodies ~= set(new CatchStatement(param, parseNoScopeStatement()), begin); - if (param is null) - break; // This is a LastCatch - begin = token; - } - - if (consumed(T.Finally)) - finBody = set(new FinallyStatement(parseNoScopeStatement()), prevToken); - - if (catchBodies.length == 0 && finBody is null) - assert(begin.kind == T.Try), error(begin, MSG.MissingCatchOrFinally); - - return new TryStatement(tryBody, catchBodies, finBody); - } - - Statement parseThrowStatement() - { - skip(T.Throw); - auto expr = parseExpression(); - require(T.Semicolon); - return new ThrowStatement(expr); - } - - Statement parseScopeGuardStatement() - { - skip(T.Scope); - skip(T.LParen); - auto condition = requireIdentifier(MSG.ExpectedScopeIdentifier); - if (condition) - switch (condition.idKind) - { - case IDK.exit, IDK.success, IDK.failure: - break; - default: - error(this.prevToken, MSG.InvalidScopeIdentifier, this.prevToken.srcText); - } - require(T.RParen); - Statement scopeBody; - if (token.kind == T.LBrace) - scopeBody = parseScopeStatement(); - else - scopeBody = parseNoScopeStatement(); - return new ScopeGuardStatement(condition, scopeBody); - } - - Statement parseVolatileStatement() - { - skip(T.Volatile); - Statement volatileBody; - if (token.kind == T.Semicolon) - nT(); - else if (token.kind == T.LBrace) - volatileBody = parseScopeStatement(); - else - volatileBody = parseStatement(); - return new VolatileStatement(volatileBody); - } - - Statement parsePragmaStatement() - { - skip(T.Pragma); - - Identifier* ident; - Expression[] args; - Statement pragmaBody; - - require(T.LParen); - ident = requireIdentifier(MSG.ExpectedPragmaIdentifier); - - if (consumed(T.Comma)) - args = parseExpressionList(); - require(T.RParen); - - pragmaBody = parseNoScopeOrEmptyStatement(); - - return new PragmaStatement(ident, args, pragmaBody); - } - - Statement parseStaticIfStatement() - { - skip(T.Static); - skip(T.If); - Expression condition; - Statement ifBody, elseBody; - - require(T.LParen); - condition = parseExpression(); - require(T.RParen); - ifBody = parseNoScopeStatement(); - if (consumed(T.Else)) - elseBody = parseNoScopeStatement(); - return new StaticIfStatement(condition, ifBody, elseBody); - } - - Statement parseStaticAssertStatement() - { - skip(T.Static); - skip(T.Assert); - Expression condition, message; - - require(T.LParen); - condition = parseAssignExpression(); // Condition. - if (consumed(T.Comma)) - message = parseAssignExpression(); // Error message. - require(T.RParen); - require(T.Semicolon); - return new StaticAssertStatement(condition, message); - } - - Statement parseDebugStatement() - { - skip(T.Debug); - Token* cond; - Statement debugBody, elseBody; - - // ( Condition ) - if (consumed(T.LParen)) - { - cond = parseIdentOrInt(); - require(T.RParen); - } - // debug Statement - // debug ( Condition ) Statement - debugBody = parseNoScopeStatement(); - // else Statement - if (consumed(T.Else)) - elseBody = parseNoScopeStatement(); - - return new DebugStatement(cond, debugBody, elseBody); - } - - Statement parseVersionStatement() - { - skip(T.Version); - Token* cond; - Statement versionBody, elseBody; - - // ( Condition ) - require(T.LParen); - cond = parseIdentOrInt(); - require(T.RParen); - // version ( Condition ) Statement - versionBody = parseNoScopeStatement(); - // else Statement - if (consumed(T.Else)) - elseBody = parseNoScopeStatement(); - - return new VersionStatement(cond, versionBody, elseBody); - } - - /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - | Assembler parsing methods | - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ - - /// Parses an AsmBlockStatement. - Statement parseAsmBlockStatement() - { - skip(T.Asm); - require(T.LBrace); - auto ss = new CompoundStatement; - while (token.kind != T.RBrace && token.kind != T.EOF) - ss ~= parseAsmStatement(); - require(T.RBrace); - return new AsmBlockStatement(ss); - } - - Statement parseAsmStatement() - { - auto begin = token; - Statement s; - Identifier* ident; - switch (token.kind) - { - // Keywords that are valid opcodes. - case T.In, T.Int, T.Out: - ident = token.ident; - nT(); - goto LOpcode; - case T.Identifier: - ident = token.ident; - nT(); - if (consumed(T.Colon)) - { // Identifier : AsmStatement - s = new LabeledStatement(ident, parseAsmStatement()); - break; - } - - LOpcode: - // Opcode ; - // Opcode Operands ; - // Opcode - // Identifier - Expression[] es; - if (token.kind != T.Semicolon) - do - es ~= parseAsmExpression(); - while (consumed(T.Comma)) - require(T.Semicolon); - s = new AsmStatement(ident, es); - break; - case T.Align: - // align Integer; - nT(); - int number = -1; - if (token.kind == T.Int32) - (number = token.int_), skip(T.Int32); - else - error(token, MSG.ExpectedIntegerAfterAlign, token.srcText); - require(T.Semicolon); - s = new AsmAlignStatement(number); - break; - case T.Semicolon: - s = new EmptyStatement(); - nT(); - break; - default: - s = new IllegalAsmStatement(); - // Skip to next valid token. - do - nT(); - while (!token.isAsmStatementStart && - token.kind != T.RBrace && - token.kind != T.EOF) - auto text = Token.textSpan(begin, this.prevToken); - error(begin, MSG.IllegalAsmStatement, text); - } - set(s, begin); - return s; - } - - Expression parseAsmExpression() - { - auto begin = token; - auto e = parseAsmOrOrExpression(); - if (consumed(T.Question)) - { - auto tok = this.prevToken; - auto iftrue = parseAsmExpression(); - require(T.Colon); - auto iffalse = parseAsmExpression(); - e = new CondExpression(e, iftrue, iffalse, tok); - set(e, begin); - } - // TODO: create AsmExpression that contains e? - return e; - } - - Expression parseAsmOrOrExpression() - { - alias parseAsmAndAndExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.OrLogical) - { - auto tok = token; - nT(); - e = new OrOrExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAsmAndAndExpression() - { - alias parseAsmOrExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.AndLogical) - { - auto tok = token; - nT(); - e = new AndAndExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAsmOrExpression() - { - alias parseAsmXorExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.OrBinary) - { - auto tok = token; - nT(); - e = new OrExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAsmXorExpression() - { - alias parseAsmAndExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.Xor) - { - auto tok = token; - nT(); - e = new XorExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAsmAndExpression() - { - alias parseAsmCmpExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.AndBinary) - { - auto tok = token; - nT(); - e = new AndExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAsmCmpExpression() - { - alias parseAsmShiftExpression parseNext; - auto begin = token; - auto e = parseNext(); - - auto operator = token; - switch (operator.kind) - { - case T.Equal, T.NotEqual: - nT(); - e = new EqualExpression(e, parseNext(), operator); - break; - case T.LessEqual, T.Less, T.GreaterEqual, T.Greater: - nT(); - e = new RelExpression(e, parseNext(), operator); - break; - default: - return e; - } - set(e, begin); - return e; - } - - Expression parseAsmShiftExpression() - { - alias parseAsmAddExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (1) - { - auto operator = token; - switch (operator.kind) - { - case T.LShift: nT(); e = new LShiftExpression(e, parseNext(), operator); break; - case T.RShift: nT(); e = new RShiftExpression(e, parseNext(), operator); break; - case T.URShift: nT(); e = new URShiftExpression(e, parseNext(), operator); break; - default: - return e; - } - set(e, begin); - } - assert(0); - } - - Expression parseAsmAddExpression() - { - alias parseAsmMulExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (1) - { - auto operator = token; - switch (operator.kind) - { - case T.Plus: nT(); e = new PlusExpression(e, parseNext(), operator); break; - case T.Minus: nT(); e = new MinusExpression(e, parseNext(), operator); break; - // Not allowed in asm - //case T.Tilde: nT(); e = new CatExpression(e, parseNext(), operator); break; - default: - return e; - } - set(e, begin); - } - assert(0); - } - - Expression parseAsmMulExpression() - { - alias parseAsmPostExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (1) - { - auto operator = token; - switch (operator.kind) - { - case T.Mul: nT(); e = new MulExpression(e, parseNext(), operator); break; - case T.Div: nT(); e = new DivExpression(e, parseNext(), operator); break; - case T.Mod: nT(); e = new ModExpression(e, parseNext(), operator); break; - default: - return e; - } - set(e, begin); - } - assert(0); - } - - Expression parseAsmPostExpression() - { - auto begin = token; - auto e = parseAsmUnaryExpression(); - while (consumed(T.LBracket)) - { - e = new AsmPostBracketExpression(e, parseAsmExpression()); - require(T.RBracket); - set(e, begin); - } - return e; - } - - Expression parseAsmUnaryExpression() - { - auto begin = token; - Expression e; - switch (token.kind) - { - case T.Byte, T.Short, T.Int, - T.Float, T.Double, T.Real: - goto LAsmTypePrefix; - case T.Identifier: - switch (token.ident.idKind) - { - case IDK.near, IDK.far,/* "byte", "short", "int",*/ - IDK.word, IDK.dword, IDK.qword/*, "float", "double", "real"*/: - LAsmTypePrefix: - nT(); - if (token.kind == T.Identifier && token.ident is Ident.ptr) - skip(T.Identifier); - else - error(MID.ExpectedButFound, "ptr", token.srcText); - e = new AsmTypeExpression(parseAsmExpression()); - break; - case IDK.offset: - nT(); - e = new AsmOffsetExpression(parseAsmExpression()); - break; - case IDK.seg: - nT(); - e = new AsmSegExpression(parseAsmExpression()); - break; - default: - goto LparseAsmPrimaryExpression; - } - break; - case T.Minus: - case T.Plus: - nT(); - e = new SignExpression(parseAsmUnaryExpression()); - break; - case T.Not: - nT(); - e = new NotExpression(parseAsmUnaryExpression()); - break; - case T.Tilde: - nT(); - e = new CompExpression(parseAsmUnaryExpression()); - break; - case T.Dot: - nT(); - e = new ModuleScopeExpression(parseIdentifierExpression()); - while (consumed(TOK.Dot)) - { - e = new DotExpression(e, parseIdentifierExpression()); - set(e, begin); - } - break; - default: - LparseAsmPrimaryExpression: - e = parseAsmPrimaryExpression(); - return e; - } - set(e, begin); - return e; - } - - Expression parseAsmPrimaryExpression() - { - auto begin = token; - Expression e; - switch (token.kind) - { - case T.Int32, T.Int64, T.Uint32, T.Uint64: - e = new IntExpression(token); - nT(); - break; - case T.Float32, T.Float64, T.Float80, - T.Imaginary32, T.Imaginary64, T.Imaginary80: - e = new RealExpression(token); - nT(); - break; - case T.Dollar: - e = new DollarExpression(); - nT(); - break; - case T.LBracket: - // [ AsmExpression ] - nT(); - e = parseAsmExpression(); - require(T.RBracket); - e = new AsmBracketExpression(e); - break; - case T.Identifier: - auto register = token.ident; - switch (register.idKind) - { - // __LOCAL_SIZE - case IDK.__LOCAL_SIZE: - nT(); - e = new AsmLocalSizeExpression(); - break; - // Register - case IDK.ST: - nT(); - // (1) - (7) - int number = -1; - if (consumed(T.LParen)) - { - if (token.kind == T.Int32) - (number = token.int_), skip(T.Int32); - else - expected(T.Int32); - require(T.RParen); - } - e = new AsmRegisterExpression(register, number); - break; - case IDK.FS: - nT(); - // TODO: is the colon-number part optional? - int number = -1; - if (consumed(T.Colon)) - { - // :0, :4, :8 - if (token.kind == T.Int32) - (number = token.int_), skip(T.Int32); - if (number != 0 && number != 4 && number != 8) - error(MID.ExpectedButFound, "0, 4 or 8", token.srcText); - } - e = new AsmRegisterExpression(register, number); - break; - case IDK.AL, IDK.AH, IDK.AX, IDK.EAX, - IDK.BL, IDK.BH, IDK.BX, IDK.EBX, - IDK.CL, IDK.CH, IDK.CX, IDK.ECX, - IDK.DL, IDK.DH, IDK.DX, IDK.EDX, - IDK.BP, IDK.EBP, IDK.SP, IDK.ESP, - IDK.DI, IDK.EDI, IDK.SI, IDK.ESI, - IDK.ES, IDK.CS, IDK.SS, IDK.DS, IDK.GS, - IDK.CR0, IDK.CR2, IDK.CR3, IDK.CR4, - IDK.DR0, IDK.DR1, IDK.DR2, IDK.DR3, IDK.DR6, IDK.DR7, - IDK.TR3, IDK.TR4, IDK.TR5, IDK.TR6, IDK.TR7, - IDK.MM0, IDK.MM1, IDK.MM2, IDK.MM3, - IDK.MM4, IDK.MM5, IDK.MM6, IDK.MM7, - IDK.XMM0, IDK.XMM1, IDK.XMM2, IDK.XMM3, - IDK.XMM4, IDK.XMM5, IDK.XMM6, IDK.XMM7: - nT(); - e = new AsmRegisterExpression(register); - break; - default: - e = parseIdentifierExpression(); - while (consumed(TOK.Dot)) - { - e = new DotExpression(e, parseIdentifierExpression()); - set(e, begin); - } - } // end of switch - break; - default: - error(MID.ExpectedButFound, "Expression", token.srcText); - e = new IllegalExpression(); - if (!trying) - { // Insert a dummy token and don't consume current one. - begin = lexer.insertEmptyTokenBefore(token); - this.prevToken = begin; - } - } - set(e, begin); - return e; - } - - /+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - | Expression parsing methods | - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+/ - - /// Parses an Expression. - Expression parseExpression() - { - alias parseAssignExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.Comma) - { - auto comma = token; - nT(); - e = new CommaExpression(e, parseNext(), comma); - set(e, begin); - } - return e; - } - - Expression parseAssignExpression() - { - alias parseAssignExpression parseNext; - auto begin = token; - auto e = parseCondExpression(); - switch (token.kind) - { - case T.Assign: - nT(); e = new AssignExpression(e, parseNext()); break; - case T.LShiftAssign: - nT(); e = new LShiftAssignExpression(e, parseNext()); break; - case T.RShiftAssign: - nT(); e = new RShiftAssignExpression(e, parseNext()); break; - case T.URShiftAssign: - nT(); e = new URShiftAssignExpression(e, parseNext()); break; - case T.OrAssign: - nT(); e = new OrAssignExpression(e, parseNext()); break; - case T.AndAssign: - nT(); e = new AndAssignExpression(e, parseNext()); break; - case T.PlusAssign: - nT(); e = new PlusAssignExpression(e, parseNext()); break; - case T.MinusAssign: - nT(); e = new MinusAssignExpression(e, parseNext()); break; - case T.DivAssign: - nT(); e = new DivAssignExpression(e, parseNext()); break; - case T.MulAssign: - nT(); e = new MulAssignExpression(e, parseNext()); break; - case T.ModAssign: - nT(); e = new ModAssignExpression(e, parseNext()); break; - case T.XorAssign: - nT(); e = new XorAssignExpression(e, parseNext()); break; - case T.CatAssign: - nT(); e = new CatAssignExpression(e, parseNext()); break; - default: - return e; - } - set(e, begin); - return e; - } - - Expression parseCondExpression() - { - auto begin = token; - auto e = parseOrOrExpression(); - if (token.kind == T.Question) - { - auto tok = token; - nT(); - auto iftrue = parseExpression(); - require(T.Colon); - auto iffalse = parseCondExpression(); - e = new CondExpression(e, iftrue, iffalse, tok); - set(e, begin); - } - return e; - } - - Expression parseOrOrExpression() - { - alias parseAndAndExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.OrLogical) - { - auto tok = token; - nT(); - e = new OrOrExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAndAndExpression() - { - alias parseOrExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.AndLogical) - { - auto tok = token; - nT(); - e = new AndAndExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseOrExpression() - { - alias parseXorExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.OrBinary) - { - auto tok = token; - nT(); - e = new OrExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseXorExpression() - { - alias parseAndExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.Xor) - { - auto tok = token; - nT(); - e = new XorExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseAndExpression() - { - alias parseCmpExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (token.kind == T.AndBinary) - { - auto tok = token; - nT(); - e = new AndExpression(e, parseNext(), tok); - set(e, begin); - } - return e; - } - - Expression parseCmpExpression() - { - alias parseShiftExpression parseNext; - auto begin = token; - auto e = parseShiftExpression(); - - auto operator = token; - switch (operator.kind) - { - case T.Equal, T.NotEqual: - nT(); - e = new EqualExpression(e, parseNext(), operator); - break; - case T.Not: - if (peekNext() != T.Is) - break; - nT(); - // fall through - case T.Is: - nT(); - e = new IdentityExpression(e, parseNext(), operator); - break; - case T.LessEqual, T.Less, T.GreaterEqual, T.Greater, - T.Unordered, T.UorE, T.UorG, T.UorGorE, - T.UorL, T.UorLorE, T.LorEorG, T.LorG: - nT(); - e = new RelExpression(e, parseNext(), operator); - break; - case T.In: - nT(); - e = new InExpression(e, parseNext(), operator); - break; - default: - return e; - } - set(e, begin); - return e; - } - - Expression parseShiftExpression() - { - alias parseAddExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (1) - { - auto operator = token; - switch (operator.kind) - { - case T.LShift: nT(); e = new LShiftExpression(e, parseNext(), operator); break; - case T.RShift: nT(); e = new RShiftExpression(e, parseNext(), operator); break; - case T.URShift: nT(); e = new URShiftExpression(e, parseNext(), operator); break; - default: - return e; - } - set(e, begin); - } - assert(0); - } - - Expression parseAddExpression() - { - alias parseMulExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (1) - { - auto operator = token; - switch (operator.kind) - { - case T.Plus: nT(); e = new PlusExpression(e, parseNext(), operator); break; - case T.Minus: nT(); e = new MinusExpression(e, parseNext(), operator); break; - case T.Tilde: nT(); e = new CatExpression(e, parseNext(), operator); break; - default: - return e; - } - set(e, begin); - } - assert(0); - } - - Expression parseMulExpression() - { - alias parsePostExpression parseNext; - auto begin = token; - auto e = parseNext(); - while (1) - { - auto operator = token; - switch (operator.kind) - { - case T.Mul: nT(); e = new MulExpression(e, parseNext(), operator); break; - case T.Div: nT(); e = new DivExpression(e, parseNext(), operator); break; - case T.Mod: nT(); e = new ModExpression(e, parseNext(), operator); break; - default: - return e; - } - set(e, begin); - } - assert(0); - } - - Expression parsePostExpression() - { - auto begin = token; - auto e = parseUnaryExpression(); - while (1) - { - while (consumed(T.Dot)) - { - e = new DotExpression(e, parseNewOrIdentifierExpression()); - set(e, begin); - } - - switch (token.kind) - { - case T.PlusPlus: - e = new PostIncrExpression(e); - break; - case T.MinusMinus: - e = new PostDecrExpression(e); - break; - case T.LParen: - e = new CallExpression(e, parseArguments()); - goto Lset; - case T.LBracket: - // parse Slice- and IndexExpression - nT(); - // [] is a SliceExpression - if (token.kind == T.RBracket) - { - e = new SliceExpression(e, null, null); - break; - } - - Expression[] es = [parseAssignExpression()]; - - // [ AssignExpression .. AssignExpression ] - if (consumed(T.Slice)) - { - e = new SliceExpression(e, es[0], parseAssignExpression()); - require(T.RBracket); - goto Lset; - } - - // [ ExpressionList ] - if (consumed(T.Comma)) - es ~= parseExpressionList(); - require(T.RBracket); - - e = new IndexExpression(e, es); - goto Lset; - default: - return e; - } - nT(); - Lset: // Jumped here to skip nT(). - set(e, begin); - } - assert(0); - } - - Expression parseUnaryExpression() - { - auto begin = token; - Expression e; - switch (token.kind) - { - case T.AndBinary: - nT(); - e = new AddressExpression(parseUnaryExpression()); - break; - case T.PlusPlus: - nT(); - e = new PreIncrExpression(parseUnaryExpression()); - break; - case T.MinusMinus: - nT(); - e = new PreDecrExpression(parseUnaryExpression()); - break; - case T.Mul: - nT(); - e = new DerefExpression(parseUnaryExpression()); - break; - case T.Minus: - case T.Plus: - nT(); - e = new SignExpression(parseUnaryExpression()); - break; - case T.Not: - nT(); - e = new NotExpression(parseUnaryExpression()); - break; - case T.Tilde: - nT(); - e = new CompExpression(parseUnaryExpression()); - break; - case T.New: - e = parseNewExpression(); - return e; - case T.Delete: - nT(); - e = new DeleteExpression(parseUnaryExpression()); - break; - case T.Cast: - requireNext(T.LParen); - Type type; - switch (token.kind) - { - version(D2) - { - auto begin2 = token; - case T.Const: - type = new ConstType(null); - goto case_break; - case T.Invariant: - type = new InvariantType(null); - case_break: - nT(); - set(type, begin2); - break; - } - default: - type = parseType(); - } - require(T.RParen); - e = new CastExpression(parseUnaryExpression(), type); - break; - case T.LParen: - // ( Type ) . Identifier - Type parseType_() - { - skip(T.LParen); - auto type = parseType(); - require(T.RParen); - require(T.Dot); - return type; - } - bool success; - auto type = try_(&parseType_, success); - if (success) - { - auto ident = requireIdentifier(MSG.ExpectedIdAfterTypeDot); - e = new TypeDotIdExpression(type, ident); - break; - } - goto default; - case T.Dot: - nT(); - e = new ModuleScopeExpression(parseIdentifierExpression()); - break; - default: - e = parsePrimaryExpression(); - return e; - } - assert(e !is null); - set(e, begin); - return e; - } - - /// $(PRE - /// IdentifierExpression := - /// Identifier - /// TemplateInstance - /// TemplateInstance := - /// Identifier !( TemplateArguments ) - /// ) - Expression parseIdentifierExpression() - { - auto begin = token; - auto ident = requireIdentifier(MSG.ExpectedAnIdentifier); - Expression e; - // Peek for '(' to avoid matching: id !is id - if (token.kind == T.Not && peekNext() == T.LParen) - { // Identifier !( TemplateArguments ) - skip(T.Not); - auto tparams = parseTemplateArguments(); - e = new TemplateInstanceExpression(ident, tparams); - } - else // Identifier - e = new IdentifierExpression(ident); - return set(e, begin); - } - - Expression parseNewOrIdentifierExpression() - { - return token.kind == T.New ? parseNewExpression() : parseIdentifierExpression(); - } - - Expression parsePrimaryExpression() - { - auto begin = token; - Expression e; - switch (token.kind) - { - case T.Identifier: - e = parseIdentifierExpression(); - return e; - case T.Typeof: - e = new TypeofExpression(parseTypeofType()); - break; - case T.This: - nT(); - e = new ThisExpression(); - break; - case T.Super: - nT(); - e = new SuperExpression(); - break; - case T.Null: - nT(); - e = new NullExpression(); - break; - case T.True, T.False: - nT(); - e = new BoolExpression(); - break; - case T.Dollar: - nT(); - e = new DollarExpression(); - break; - case T.Int32, T.Int64, T.Uint32, T.Uint64: - e = new IntExpression(token); - nT(); - break; - case T.Float32, T.Float64, T.Float80, - T.Imaginary32, T.Imaginary64, T.Imaginary80: - e = new RealExpression(token); - nT(); - break; - case T.CharLiteral: - e = new CharExpression(token.dchar_); - nT(); - break; - case T.String: - char[] str = token.str; - char postfix = token.pf; - nT(); - while (token.kind == T.String) - { - /+if (postfix == 0) - postfix = token.pf; - else+/ - if (token.pf && token.pf != postfix) - error(token, MSG.StringPostfixMismatch); - str.length = str.length - 1; // Exclude '\0'. - str ~= token.str; - nT(); - } - switch (postfix) - { - case 'w': - if (checkString(begin, str)) - goto default; - e = new StringExpression(dil.Unicode.toUTF16(str)); break; - case 'd': - if (checkString(begin, str)) - goto default; - e = new StringExpression(dil.Unicode.toUTF32(str)); break; - case 'c': - default: - // No checking done to allow for binary data. - e = new StringExpression(str); break; - } - break; - case T.LBracket: - Expression[] values; - - nT(); - if (!consumed(T.RBracket)) - { - e = parseAssignExpression(); - if (consumed(T.Colon)) - goto LparseAssocArray; - if (consumed(T.Comma)) - values = [e] ~ parseExpressionList(); - require(T.RBracket); - } - - e = new ArrayLiteralExpression(values); - break; - - LparseAssocArray: - Expression[] keys = [e]; - - goto LenterLoop; - do - { - keys ~= parseAssignExpression(); - require(T.Colon); - LenterLoop: - values ~= parseAssignExpression(); - } while (consumed(T.Comma)) - require(T.RBracket); - e = new AArrayLiteralExpression(keys, values); - break; - case T.LBrace: - // DelegateLiteral := { Statements } - auto funcBody = parseFunctionBody(); - e = new FunctionLiteralExpression(funcBody); - break; - case T.Function, T.Delegate: - // FunctionLiteral := ("function"|"delegate") Type? "(" ArgumentList ")" FunctionBody - nT(); // Skip function or delegate keyword. - Type returnType; - Parameters parameters; - if (token.kind != T.LBrace) - { - if (token.kind != T.LParen) // Optional return type - returnType = parseType(); - parameters = parseParameterList(); - } - auto funcBody = parseFunctionBody(); - e = new FunctionLiteralExpression(returnType, parameters, funcBody); - break; - case T.Assert: - Expression msg; - requireNext(T.LParen); - e = parseAssignExpression(); - if (consumed(T.Comma)) - msg = parseAssignExpression(); - require(T.RParen); - e = new AssertExpression(e, msg); - break; - case T.Mixin: - requireNext(T.LParen); - e = parseAssignExpression(); - require(T.RParen); - e = new MixinExpression(e); - break; - case T.Import: - requireNext(T.LParen); - e = parseAssignExpression(); - require(T.RParen); - e = new ImportExpression(e); - break; - case T.Typeid: - requireNext(T.LParen); - auto type = parseType(); - require(T.RParen); - e = new TypeidExpression(type); - break; - case T.Is: - requireNext(T.LParen); - - Type type, specType; - Identifier* ident; // optional Identifier - Token* opTok, specTok; - - type = parseDeclarator(ident, true); - - switch (token.kind) - { - case T.Colon, T.Equal: - opTok = token; - nT(); - switch (token.kind) - { - case T.Typedef, - T.Struct, - T.Union, - T.Class, - T.Interface, - T.Enum, - T.Function, - T.Delegate, - T.Super, - T.Return: - case_Const_Invariant: - specTok = token; - nT(); - break; - case T.Const, T.Invariant: - if (peekNext() != T.LParen) - goto case_Const_Invariant; - // Fall through. It's a type. - default: - specType = parseType(); - } - default: - } - - TemplateParameters tparams; - version(D2) - { - // is ( Type Identifier : TypeSpecialization , TemplateParameterList ) - // is ( Type Identifier == TypeSpecialization , TemplateParameterList ) - if (ident && specType && token.kind == T.Comma) - tparams = parseTemplateParameterList2(); - } - require(T.RParen); - e = new IsExpression(type, ident, opTok, specTok, specType, tparams); - break; - case T.LParen: - if (tokenAfterParenIs(T.LBrace)) // Check for "(...) {" - { // ( ParameterList ) FunctionBody - auto parameters = parseParameterList(); - auto funcBody = parseFunctionBody(); - e = new FunctionLiteralExpression(null, parameters, funcBody); - } - else - { // ( Expression ) - skip(T.LParen); - e = parseExpression(); - require(T.RParen); - e = new ParenExpression(e); - } - break; - version(D2) - { - case T.Traits: - requireNext(T.LParen); - auto id = requireIdentifier(MSG.ExpectedAnIdentifier); - TemplateArguments args; - if (token.kind == T.Comma) - args = parseTemplateArguments2(); - else - require(T.RParen); - e = new TraitsExpression(id, args); - break; - } - default: - if (token.isIntegralType) - { // IntegralType . Identifier - auto type = new IntegralType(token.kind); - nT(); - set(type, begin); - require(T.Dot); - auto ident = requireIdentifier(MSG.ExpectedIdAfterTypeDot); - e = new TypeDotIdExpression(type, ident); - } - else if (token.isSpecialToken) - { - e = new SpecialTokenExpression(token); - nT(); - } - else - { - error(MID.ExpectedButFound, "Expression", token.srcText); - e = new IllegalExpression(); - if (!trying) - { // Insert a dummy token and don't consume current one. - begin = lexer.insertEmptyTokenBefore(token); - this.prevToken = begin; - } - } - } - set(e, begin); - return e; - } - - Expression parseNewExpression(/*Expression e*/) - { - auto begin = token; - skip(T.New); - - Expression[] newArguments; - Expression[] ctorArguments; - - if (token.kind == T.LParen) - newArguments = parseArguments(); - - // NewAnonClassExpression: - // new (ArgumentList)opt class (ArgumentList)opt SuperClassopt InterfaceClassesopt ClassBody - if (consumed(T.Class)) - { - if (token.kind == T.LParen) - ctorArguments = parseArguments(); - - BaseClassType[] bases = token.kind != T.LBrace ? parseBaseClasses(false) : null ; - - auto decls = parseDeclarationDefinitionsBody(); - return set(new NewAnonClassExpression(/*e, */newArguments, bases, ctorArguments, decls), begin); - } - - // NewExpression: - // NewArguments Type [ AssignExpression ] - // NewArguments Type ( ArgumentList ) - // NewArguments Type - auto type = parseType(); - - if (token.kind == T.LParen) - ctorArguments = parseArguments(); - - return set(new NewExpression(/*e, */newArguments, type, ctorArguments), begin); - } - - /// Parses a Type. - Type parseType() - { - return parseBasicType2(parseBasicType()); - } - - Type parseIdentifierType() - { - auto begin = token; - auto ident = requireIdentifier(MSG.ExpectedAnIdentifier); - Type t; - if (consumed(T.Not)) // Identifier !( TemplateArguments ) - t = new TemplateInstanceType(ident, parseTemplateArguments()); - else // Identifier - t = new IdentifierType(ident); - return set(t, begin); - } - - Type parseQualifiedType() - { - auto begin = token; - Type type; - if (token.kind == T.Dot) - type = set(new ModuleScopeType(), begin, begin); - else if (token.kind == T.Typeof) - type = parseTypeofType(); - else - type = parseIdentifierType(); - - while (consumed(T.Dot)) - type = set(new QualifiedType(type, parseIdentifierType()), begin); - return type; - } - - Type parseBasicType() - { - auto begin = token; - Type t; - - if (token.isIntegralType) - { - t = new IntegralType(token.kind); - nT(); - } - else - switch (token.kind) - { - case T.Identifier, T.Typeof, T.Dot: - t = parseQualifiedType(); - return t; - version(D2) - { - case T.Const: - // const ( Type ) - requireNext(T.LParen); - t = parseType(); - require(T.RParen); - t = new ConstType(t); - break; - case T.Invariant: - // invariant ( Type ) - requireNext(T.LParen); - t = parseType(); - require(T.RParen); - t = new InvariantType(t); - break; - } // version(D2) - default: - error(MID.ExpectedButFound, "BasicType", token.srcText); - t = new IllegalType(); - nT(); - } - return set(t, begin); - } - - Type parseBasicType2(Type t) - { - while (1) - { - auto begin = token; - switch (token.kind) - { - case T.Mul: - t = new PointerType(t); - nT(); - break; - case T.LBracket: - t = parseArrayType(t); - continue; - case T.Function, T.Delegate: - TOK tok = token.kind; - nT(); - auto parameters = parseParameterList(); - if (tok == T.Function) - t = new FunctionType(t, parameters); - else - t = new DelegateType(t, parameters); - break; - default: - return t; - } - set(t, begin); - } - assert(0); - } - - /// Returns true if the token after the closing parenthesis - /// is of kind tok. - bool tokenAfterParenIs(TOK tok) - { - // We count nested parentheses tokens because template types - // may appear inside parameter lists. E.g.: (int x, Foo!(int) y) - assert(token.kind == T.LParen); - Token* next = token; - uint level = 1; - Loop: - while (1) - { - lexer.peek(next); - switch (next.kind) - { - case T.RParen: - if (--level == 0) - { // Last, closing parentheses found. - do - lexer.peek(next); - while (next.isWhitespace) - break Loop; - } - break; - case T.LParen: - ++level; - break; - case T.EOF: - break Loop; - default: - } - } - return next.kind == tok; - } - - /// Parse the array types after the declarator (C-style.) E.g.: int a[] - Type parseDeclaratorSuffix(Type lhsType) - { - // The Type chain should be as follows: - // int[3]* Identifier [][32] - // <- <- -> -. - // ^-----------------´ - // Resulting chain: [][32]*[3]int - Type parseNext() // Nested function required to accomplish this. - { - if (token.kind != T.LBracket) - return lhsType; // Break recursion; return Type on the left hand side of the Identifier. - - auto begin = token; - Type t; - skip(T.LBracket); - if (consumed(T.RBracket)) - t = new ArrayType(parseNext()); // [ ] - else - { - bool success; - Type parseAAType() - { - auto type = parseType(); - require(T.RBracket); - return type; - } - auto assocType = try_(&parseAAType, success); - if (success) - t = new ArrayType(parseNext(), assocType); // [ Type ] - else - { - Expression e = parseExpression(), e2; - if (consumed(T.Slice)) - e2 = parseExpression(); - require(T.RBracket); - t = new ArrayType(parseNext(), e, e2); // [ Expression .. Expression ] - } - } - set(t, begin); - return t; - } - return parseNext(); - } - - Type parseArrayType(Type t) - { - auto begin = token; - skip(T.LBracket); - if (consumed(T.RBracket)) - t = new ArrayType(t); - else - { - bool success; - Type parseAAType() - { - auto type = parseType(); - require(T.RBracket); - return type; - } - auto assocType = try_(&parseAAType, success); - if (success) - t = new ArrayType(t, assocType); - else - { - Expression e = parseExpression(), e2; - if (consumed(T.Slice)) - e2 = parseExpression(); - require(T.RBracket); - t = new ArrayType(t, e, e2); - } - } - set(t, begin); - return t; - } - - Type parseCFunctionPointerType(Type type, ref Identifier* ident, bool optionalParamList) - { - assert(type !is null); - auto begin = token; - skip(T.LParen); - - type = parseBasicType2(type); - if (token.kind == T.LParen) - { // Can be nested. - type = parseCFunctionPointerType(type, ident, true); - } - else if (token.kind == T.Identifier) - { // The identifier of the function pointer and the declaration. - ident = token.ident; - nT(); - type = parseDeclaratorSuffix(type); - } - require(T.RParen); - - Parameters params; - if (optionalParamList) - params = token.kind == T.LParen ? parseParameterList() : null; - else - params = parseParameterList(); - - type = new CFuncPointerType(type, params); - return set(type, begin); - } - - Type parseDeclarator(ref Identifier* ident, bool identOptional = false) - { - auto t = parseType(); - - if (token.kind == T.LParen) - t = parseCFunctionPointerType(t, ident, true); - else if (token.kind == T.Identifier) - { - ident = token.ident; - nT(); - t = parseDeclaratorSuffix(t); - } - - if (ident is null && !identOptional) - error(token, MSG.ExpectedDeclaratorIdentifier, token.srcText); - - return t; - } - - /// Parses a list of AssignExpressions. - /// $(PRE - /// ExpressionList := - /// AssignExpression - /// AssignExpression , ExpressionList - /// ) - Expression[] parseExpressionList() - { - Expression[] expressions; - do - expressions ~= parseAssignExpression(); - while(consumed(T.Comma)) - return expressions; - } - - /// Parses a list of Arguments. - /// $(PRE - /// Arguments := - /// ( ) - /// ( ExpressionList ) - /// ) - Expression[] parseArguments() - { - skip(T.LParen); - Expression[] args; - if (token.kind != T.RParen) - args = parseExpressionList(); - require(T.RParen); - return args; - } - - /// Parses a ParameterList. - Parameters parseParameterList() - out(params) - { - if (params.length > 1) - foreach (param; params.items[0..$-1]) - { - if (param.isVariadic()) - assert(0, "variadic arguments can only appear at the end of the parameter list."); - } - } - body - { - auto begin = token; - require(T.LParen); - - auto params = new Parameters(); - - if (consumed(T.RParen)) - return set(params, begin); - - do - { - auto paramBegin = token; - StorageClass stc, stc_; - Type type; - Identifier* ident; - Expression defValue; - - void pushParameter() - { - params ~= set(new Parameter(stc, type, ident, defValue), paramBegin); - } - - if (consumed(T.Ellipses)) - { - stc = StorageClass.Variadic; - pushParameter(); // type, ident and defValue will be null. - break; - } - - while (1) - { // Parse storage classes. - switch (token.kind) - { - version(D2) - { - case T.Invariant: // D2.0 - if (peekNext() == T.LParen) - break; - stc_ = StorageClass.Invariant; - goto Lcommon; - case T.Const: // D2.0 - if (peekNext() == T.LParen) - break; - stc_ = StorageClass.Const; - goto Lcommon; - case T.Final: // D2.0 - stc_ = StorageClass.Final; - goto Lcommon; - case T.Scope: // D2.0 - stc_ = StorageClass.Scope; - goto Lcommon; - case T.Static: // D2.0 - stc_ = StorageClass.Static; - goto Lcommon; - } - case T.In: - stc_ = StorageClass.In; - goto Lcommon; - case T.Out: - stc_ = StorageClass.Out; - goto Lcommon; - case T.Inout, T.Ref: - stc_ = StorageClass.Ref; - goto Lcommon; - case T.Lazy: - stc_ = StorageClass.Lazy; - goto Lcommon; - Lcommon: - // Check for redundancy. - if (stc & stc_) - error(MID.RedundantStorageClass, token.srcText); - else - stc |= stc_; - nT(); - version(D2) - continue; - else - break; // In D1.0 the grammar only allows one storage class. - default: - } - break; // Break out of inner loop. - } - type = parseDeclarator(ident, true); - - if (consumed(T.Assign)) - defValue = parseAssignExpression(); - - if (consumed(T.Ellipses)) - { - stc |= StorageClass.Variadic; - pushParameter(); - break; - } - pushParameter(); - - } while (consumed(T.Comma)) - require(T.RParen); - return set(params, begin); - } - - TemplateArguments parseTemplateArguments() - { - TemplateArguments targs; - require(T.LParen); - if (token.kind != T.RParen) - targs = parseTemplateArguments_(); - require(T.RParen); - return targs; - } - -version(D2) -{ - TemplateArguments parseTemplateArguments2() - { - skip(T.Comma); - TemplateArguments targs; - if (token.kind != T.RParen) - targs = parseTemplateArguments_(); - else - error(token, MSG.ExpectedTypeOrExpression); - require(T.RParen); - return targs; - } -} // version(D2) - - TemplateArguments parseTemplateArguments_() - { - auto begin = token; - auto targs = new TemplateArguments; - do - { - Type parseType_() - { - auto type = parseType(); - if (token.kind == T.Comma || token.kind == T.RParen) - return type; - errorCount++; // Cause try_() to fail. - return null; - } - bool success; - auto typeArgument = try_(&parseType_, success); - if (success) - // TemplateArgument: - // Type - // Symbol - targs ~= typeArgument; - else - // TemplateArgument: - // AssignExpression - targs ~= parseAssignExpression(); - } while (consumed(T.Comma)) - set(targs, begin); - return targs; - } - - TemplateParameters parseTemplateParameterList() - { - auto begin = token; - auto tparams = new TemplateParameters; - require(T.LParen); - if (token.kind != T.RParen) - parseTemplateParameterList_(tparams); - require(T.RParen); - return set(tparams, begin); - } - -version(D2) -{ - TemplateParameters parseTemplateParameterList2() - { - skip(T.Comma); - auto begin = token; - auto tparams = new TemplateParameters; - if (token.kind != T.RParen) - parseTemplateParameterList_(tparams); - else - error(token, MSG.ExpectedTemplateParameters); - return set(tparams, begin); - } -} // version(D2) - - /// Parses template parameters. - void parseTemplateParameterList_(TemplateParameters tparams) - { - do - { - auto paramBegin = token; - TemplateParameter tp; - Identifier* ident; - Type specType, defType; - - void parseSpecAndOrDefaultType() - { - // : SpecializationType - if (consumed(T.Colon)) - specType = parseType(); - // = DefaultType - if (consumed(T.Assign)) - defType = parseType(); - } - - switch (token.kind) - { - case T.Alias: - // TemplateAliasParameter: - // alias Identifier - skip(T.Alias); - ident = requireIdentifier(MSG.ExpectedAliasTemplateParam); - parseSpecAndOrDefaultType(); - tp = new TemplateAliasParameter(ident, specType, defType); - break; - case T.Identifier: - ident = token.ident; - switch (peekNext()) - { - case T.Ellipses: - // TemplateTupleParameter: - // Identifier ... - skip(T.Identifier); skip(T.Ellipses); - if (token.kind == T.Comma) - error(MID.TemplateTupleParameter); - tp = new TemplateTupleParameter(ident); - break; - case T.Comma, T.RParen, T.Colon, T.Assign: - // TemplateTypeParameter: - // Identifier - skip(T.Identifier); - parseSpecAndOrDefaultType(); - tp = new TemplateTypeParameter(ident, specType, defType); - break; - default: - // TemplateValueParameter: - // Declarator - ident = null; - goto LTemplateValueParameter; - } - break; - version(D2) - { - case T.This: - // TemplateThisParameter - // this TemplateTypeParameter - skip(T.This); - ident = requireIdentifier(MSG.ExpectedNameForThisTempParam); - parseSpecAndOrDefaultType(); - tp = new TemplateThisParameter(ident, specType, defType); - break; - } - default: - LTemplateValueParameter: - // TemplateValueParameter: - // Declarator - Expression specValue, defValue; - auto valueType = parseDeclarator(ident); - // : SpecializationValue - if (consumed(T.Colon)) - specValue = parseCondExpression(); - // = DefaultValue - if (consumed(T.Assign)) - defValue = parseCondExpression(); - tp = new TemplateValueParameter(valueType, ident, specValue, defValue); - } - - // Push template parameter. - tparams ~= set(tp, paramBegin); - - } while (consumed(T.Comma)) - } - - alias require expected; - - /// Requires a token of kind tok. - void require(TOK tok) - { - if (token.kind == tok) - nT(); - else - error(MID.ExpectedButFound, Token.toString(tok), token.srcText); - } - - /// Requires the next token to be of kind tok. - void requireNext(TOK tok) - { - nT(); - require(tok); - } - - /// Optionally parses an identifier. - /// Returns: null or the identifier. - Identifier* optionalIdentifier() - { - Identifier* id; - if (token.kind == T.Identifier) - (id = token.ident), skip(T.Identifier); - return id; - } - - Identifier* requireIdentifier() - { - Identifier* id; - if (token.kind == T.Identifier) - (id = token.ident), skip(T.Identifier); - else - error(MID.ExpectedButFound, "Identifier", token.srcText); - return id; - } - - /// Reports an error if the current token is not an identifier. - /// Params: - /// errorMsg = the error message to be used. - /// Returns: null or the identifier. - Identifier* requireIdentifier(char[] errorMsg) - { - Identifier* id; - if (token.kind == T.Identifier) - (id = token.ident), skip(T.Identifier); - else - error(token, errorMsg, token.srcText); - return id; - } - - /// Reports an error if the current token is not an identifier. - /// Params: - /// mid = the error message ID to be used. - /// Returns: null or the identifier. - Identifier* requireIdentifier(MID mid) - { - Identifier* id; - if (token.kind == T.Identifier) - (id = token.ident), skip(T.Identifier); - else - error(mid, token.srcText); - return id; - } - - /// Reports an error if the current token is not an identifier. - /// Returns: null or the token. - Token* requireId() - { - Token* idtok; - if (token.kind == T.Identifier) - (idtok = token), skip(T.Identifier); - else - error(MID.ExpectedButFound, "Identifier", token.srcText); - return idtok; - } - - Token* requireIdToken(char[] errorMsg) - { - Token* idtok; - if (token.kind == T.Identifier) - (idtok = token), skip(T.Identifier); - else - { - error(token, errorMsg, token.srcText); - idtok = lexer.insertEmptyTokenBefore(token); - this.prevToken = idtok; - } - return idtok; - } - - /// Returns true if the string str has an invalid UTF-8 sequence. - bool checkString(Token* begin, string str) - { - auto utf8Seq = Lexer.findInvalidUTF8Sequence(str); - if (utf8Seq.length) - error(begin, MSG.InvalidUTF8SequenceInString, utf8Seq); - return utf8Seq.length != 0; - } - - /// Forwards error parameters. - void error(Token* token, char[] formatMsg, ...) - { - error_(token, formatMsg, _arguments, _argptr); - } - - /// ditto - void error(MID mid, ...) - { - error_(this.token, GetMsg(mid), _arguments, _argptr); - } - - /// Creates an error report and appends it to a list. - /// Params: - /// token = used to get the location of where the error is. - /// formatMsg = the compiler error message. - void error_(Token* token, char[] formatMsg, TypeInfo[] _arguments, Arg _argptr) - { - if (trying) - { - ++errorCount; - return; - } - auto location = token.getErrorLocation(); - auto msg = Format(_arguments, _argptr, formatMsg); - auto error = new ParserError(location, msg); - errors ~= error; - if (infoMan !is null) - infoMan ~= error; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Analysis.d --- a/trunk/src/dil/semantic/Analysis.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,106 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Analysis; - -import dil.ast.Node; -import dil.ast.Expressions; -import dil.semantic.Scope; -import dil.lexer.IdTable; -import dil.Compilation; -import common; - -/// Common semantics for pragma declarations and statements. -void pragmaSemantic(Scope scop, Token* pragmaLoc, - Identifier* ident, - Expression[] args) -{ - if (ident is Ident.msg) - pragma_msg(scop, pragmaLoc, args); - else if (ident is Ident.lib) - pragma_lib(scop, pragmaLoc, args); - // else - // scop.error(begin, "unrecognized pragma"); -} - -/// Evaluates a msg pragma. -void pragma_msg(Scope scop, Token* pragmaLoc, Expression[] args) -{ - if (args.length == 0) - return /*scop.error(pragmaLoc, "expected expression arguments to pragma")*/; - - foreach (arg; args) - { - auto e = arg/+.evaluate()+/; - if (e is null) - { - // scop.error(e.begin, "expression is not evaluatable at compile time"); - } - else if (auto stringExpr = e.Is!(StringExpression)) - // Print string to standard output. - Stdout(stringExpr.getString()); - else - { - // scop.error(e.begin, "expression must evaluate to a string"); - } - } - // Print a newline at the end. - Stdout('\n'); -} - -/// Evaluates a lib pragma. -void pragma_lib(Scope scop, Token* pragmaLoc, Expression[] args) -{ - if (args.length != 1) - return /*scop.error(pragmaLoc, "expected one expression argument to pragma")*/; - - auto e = args[0]/+.evaluate()+/; - if (e is null) - { - // scop.error(e.begin, "expression is not evaluatable at compile time"); - } - else if (auto stringExpr = e.Is!(StringExpression)) - { - // TODO: collect library paths in Module? - // scop.modul.addLibrary(stringExpr.getString()); - } - else - { - // scop.error(e.begin, "expression must evaluate to a string"); - } -} - -/// Returns true if the first branch (of a debug declaration/statement) or -/// false if the else-branch should be compiled in. -bool debugBranchChoice(Token* cond, CompilationContext context) -{ - if (cond) - { - if (cond.kind == TOK.Identifier) - { - if (context.findDebugId(cond.ident.str)) - return true; - } - else if (cond.uint_ <= context.debugLevel) - return true; - } - else if (1 <= context.debugLevel) - return true; - return false; -} - -/// Returns true if the first branch (of a version declaration/statement) or -/// false if the else-branch should be compiled in. -bool versionBranchChoice(Token* cond, CompilationContext context) -{ - assert(cond); - if (cond.kind == TOK.Identifier) - { - if (context.findVersionId(cond.ident.str)) - return true; - } - else if (cond.uint_ >= context.versionLevel) - return true; - return false; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Interpreter.d --- a/trunk/src/dil/semantic/Interpreter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,74 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Interpreter; - -import dil.ast.Visitor; -import dil.ast.Node, - dil.ast.Declarations, - dil.ast.Expressions, - dil.ast.Statements, - dil.ast.Types, - dil.ast.Parameters; - -import dil.semantic.Symbol, - dil.semantic.Symbols, - dil.semantic.Scope, - dil.semantic.Types; -import dil.Information; - -/// Used for compile-time evaluation of expressions. -class Interpreter : Visitor -{ - // Scope scop; - InfoManager infoMan; - - static class Result : Expression - { - override Result copy(){return null;} - } - - static const Result NAR; /// Not a Result. Similar to NAN in floating point arithmetics. - - static this() - { - NAR = new Result; - NAR.type = Types.Error; - } - - /// Evaluates the expression e. - /// Returns: NAR or a value. - static Expression interpret(Expression e, InfoManager infoMan/+, Scope scop+/) - { - return (new Interpreter(/+scop,+/ infoMan)).eval(e); - } - - /// Constructs an Interpreter object. - this(/+Scope scop, +/InfoManager infoMan) - { - // this.scop = scop; - this.infoMan = infoMan; - } - - /// Start evaluation. - Expression eval(Expression e) - { - return e; - } - - /// Returns true if e is immutable. - bool isImmutable(Expression e) - { - switch (e.kind) - { - alias NodeKind NK; - case NK.IntExpression, NK.RealExpression, - NK.ComplexExpression, NK.CharExpression, - NK.BoolExpression, NK.StringExpression: - return true; - default: - } - return false; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Module.d --- a/trunk/src/dil/semantic/Module.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,158 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Module; - -import dil.ast.Node; -import dil.ast.Declarations; -import dil.parser.Parser; -import dil.lexer.Lexer; -import dil.semantic.Symbol; -import dil.semantic.Symbols; -import dil.Information; -import dil.SourceText; -import common; - -import tango.io.FilePath; -import tango.io.FileConst; - -alias FileConst.PathSeparatorChar dirSep; - -/// Represents a semantic D module and a source file. -class Module : ScopeSymbol -{ - SourceText sourceText; /// The source file of this module. - string moduleFQN; /// Fully qualified name of the module. E.g.: dil.ast.Node - string packageName; /// E.g.: dil.ast - string moduleName; /// E.g.: Node - - CompoundDeclaration root; /// The root of the parse tree. - ImportDeclaration[] imports; /// ImportDeclarations found in this file. - ModuleDeclaration moduleDecl; /// The optional ModuleDeclaration in this file. - Parser parser; /// The parser used to parse this file. - - // Module[] modules; - - InfoManager infoMan; /// Collects error messages. - - this() - { - super(SYM.Module, null, null); - } - - /// Constructs a Module object. - /// Params: - /// filePath = file path to the source text; loaded in the constructor. - /// infoMan = used for collecting error messages. - this(string filePath, InfoManager infoMan = null) - { - this(); - this.sourceText = new SourceText(filePath); - this.infoMan = infoMan; - this.sourceText.load(infoMan); - } - - /// Returns the file path of the source text. - string filePath() - { - return sourceText.filePath; - } - - /// Returns the file extension: "d" or "di". - string fileExtension() - { - foreach_reverse(i, c; filePath) - if (c == '.') - return filePath[i+1..$]; - return ""; - } - - /// Sets the parser to be used for parsing the source text. - void setParser(Parser parser) - { - this.parser = parser; - } - - /// Parses the module. - /// Throws: - /// An Exception if the there's no ModuleDeclaration and - /// the file name is an invalid or reserved D identifier. - void parse() - { - if (this.parser is null) - this.parser = new Parser(sourceText, infoMan); - - this.root = parser.start(); - this.imports = parser.imports; - - if (root.children.length) - { // moduleDecl will be null if first node isn't a ModuleDeclaration. - this.moduleDecl = root.children[0].Is!(ModuleDeclaration); - if (this.moduleDecl) - this.setFQN(moduleDecl.getFQN()); - } - - if (!this.moduleFQN.length) - { // Take base name of file path as module name. - auto str = (new FilePath(filePath)).name(); - if (Lexer.isReservedIdentifier(str)) - throw new Exception("'"~str~"' is not a valid module name; it's a reserved or invalid D identifier."); - this.moduleFQN = this.moduleName = str; - } - } - - /// Returns the first token of the module's source text. - Token* firstToken() - { - return parser.lexer.firstToken(); - } - - /// Returns true if there are errors in the source file. - bool hasErrors() - { - return parser.errors.length || parser.lexer.errors.length; - } - - /// Returns a list of import paths. - /// E.g.: ["dil/ast/Node", "dil/semantic/Module"] - string[] getImportPaths() - { - string[] result; - foreach (import_; imports) - result ~= import_.getModuleFQNs(dirSep); - return result; - } - - /// Returns the fully qualified name of this module. - /// E.g.: dil.ast.Node - string getFQN() - { - return moduleFQN; - } - - /// Set's the module's FQN. - void setFQN(string moduleFQN) - { - uint i = moduleFQN.length; - if (i != 0) // Don't decrement if string has zero length. - i--; - // Find last dot. - for (; i != 0 && moduleFQN[i] != '.'; i--) - {} - this.moduleFQN = moduleFQN; - this.packageName = moduleFQN[0..i]; - this.moduleName = moduleFQN[(i == 0 ? 0 : i+1) .. $]; - } - - /// Returns the module's FQN with slashes instead of dots. - /// E.g.: dil/ast/Node - string getFQNPath() - { - string FQNPath = moduleFQN.dup; - foreach (i, c; FQNPath) - if (c == '.') - FQNPath[i] = dirSep; - return FQNPath; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Package.d --- a/trunk/src/dil/semantic/Package.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,12 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Package; - -import dil.semantic.Symbol; - -class Package : Symbol -{ - -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Pass1.d --- a/trunk/src/dil/semantic/Pass1.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,504 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Pass1; - -import dil.ast.Visitor; -import dil.ast.Node, - dil.ast.Declarations, - dil.ast.Expressions, - dil.ast.Statements, - dil.ast.Types, - dil.ast.Parameters; -import dil.lexer.IdTable; -import dil.semantic.Symbol, - dil.semantic.Symbols, - dil.semantic.Types, - dil.semantic.Scope, - dil.semantic.Module, - dil.semantic.Analysis; -import dil.Compilation; -import dil.Location; -import dil.Information; -import dil.Messages; -import dil.Enums; -import dil.CompilerInfo; -import common; - -/// The first pass is the declaration pass. -/// -/// The basic task of this class is to traverse the parse tree, -/// find all kinds of declarations and add them -/// to the symbol tables of their respective scopes. -class SemanticPass1 : Visitor -{ - Scope scop; /// The current scope. - Module modul; /// The module to be semantically checked. - CompilationContext context; /// The compilation context. - - // Attributes: - LinkageType linkageType; /// Current linkage type. - Protection protection; /// Current protection attribute. - StorageClass storageClass; /// Current storage classes. - uint alignSize; /// Current align size. - - /// Constructs a SemanticPass1 object. - /// Params: - /// modul = the module to be processed. - /// context = the compilation context. - this(Module modul, CompilationContext context) - { - this.modul = modul; - this.context = new CompilationContext(context); - this.alignSize = context.structAlign; - } - - /// Starts processing the module. - void start() - { - assert(modul.root !is null); - // Create module scope. - scop = new Scope(null, modul); - visit(modul.root); - } - - /// Enters a new scope. - void enterScope(ScopeSymbol s) - { - scop = scop.enter(s); - } - - /// Exits the current scope. - void exitScope() - { - scop = scop.exit(); - } - - /// Returns true if this is the module scope. - bool isModuleScope() - { - return scop.symbol.isModule(); - } - - /// Inserts a symbol into the current scope. - void insert(Symbol symbol, Identifier* name) - { - auto symX = scop.symbol.lookup(name); - if (symX) - reportSymbolConflict(symbol, symX, name); - else - scop.symbol.insert(symbol, name); - // Set the current scope symbol as the parent. - symbol.parent = scop.symbol; - } - - /// Inserts a symbol into scopeSym. - void insert(Symbol symbol, ScopeSymbol scopeSym) - { - auto symX = scopeSym.lookup(symbol.name); - if (symX) - reportSymbolConflict(symbol, symX, symbol.name); - else - scopeSym.insert(symbol, symbol.name); - // Set the current scope symbol as the parent. - symbol.parent = scopeSym; - } - - /// Inserts a symbol, overloading on the name, into the current scope. - void insertOverload(Symbol sym, Identifier* name) - { - auto sym2 = scop.symbol.lookup(name); - if (sym2) - { - if (sym2.isOverloadSet) - (cast(OverloadSet)cast(void*)sym2).add(sym); - else - reportSymbolConflict(sym, sym2, name); - } - else - // Create a new overload set. - scop.symbol.insert(new OverloadSet(name, sym.node), name); - // Set the current scope symbol as the parent. - sym.parent = scop.symbol; - } - - /// Reports an error: new symbol s1 conflicts with existing symbol s2. - void reportSymbolConflict(Symbol s1, Symbol s2, Identifier* name) - { - auto loc = s2.node.begin.getErrorLocation(); - auto locString = Format("{}({},{})", loc.filePath, loc.lineNum, loc.colNum); - error(s1.node.begin, MSG.DeclConflictsWithDecl, name.str, locString); - } - - /// Creates an error report. - void error(Token* token, char[] formatMsg, ...) - { - if (!modul.infoMan) - return; - auto location = token.getErrorLocation(); - auto msg = Format(_arguments, _argptr, formatMsg); - modul.infoMan ~= new SemanticError(location, msg); - } - - - /// Collects info about nodes which have to be evaluated later. - static class Deferred - { - Node node; - ScopeSymbol symbol; - // Saved attributes. - LinkageType linkageType; - Protection protection; - StorageClass storageClass; - uint alignSize; - } - - /// List of mixin, static if, static assert and pragma(msg,...) declarations. - /// - /// Their analysis must be deferred because they entail - /// evaluation of expressions. - Deferred[] deferred; - - /// Adds a deferred node to the list. - void addDeferred(Node node) - { - auto d = new Deferred; - d.node = node; - d.symbol = scop.symbol; - d.linkageType = linkageType; - d.protection = protection; - d.storageClass = storageClass; - d.alignSize = alignSize; - deferred ~= d; - } - - private alias Declaration D; /// A handy alias. Saves typing. - -override -{ - D visit(CompoundDeclaration d) - { - foreach (decl; d.decls) - visitD(decl); - return d; - } - - D visit(IllegalDeclaration) - { assert(0, "semantic pass on invalid AST"); return null; } - - D visit(EmptyDeclaration ed) - { return ed; } - - D visit(ModuleDeclaration) - { return null; } - - D visit(ImportDeclaration d) - { - return d; - } - - D visit(AliasDeclaration ad) - { - return ad; - } - - D visit(TypedefDeclaration td) - { - return td; - } - - D visit(EnumDeclaration d) - { - // Create the symbol. - d.symbol = new Enum(d.name, d); - auto isAnonymous = d.name is null; - if (isAnonymous) - d.symbol.name = IdTable.genAnonEnumID(); - insert(d.symbol, d.symbol.name); - auto parentScopeSymbol = scop.symbol; - enterScope(d.symbol); - // Declare members. - foreach (member; d.members) - { - visitD(member); - if (isAnonymous) // Also insert into parent scope if enum is anonymous. - insert(member.symbol, parentScopeSymbol); - member.symbol.parent = d.symbol; - } - exitScope(); - return d; - } - - D visit(EnumMemberDeclaration d) - { - d.symbol = new EnumMember(d.name, protection, storageClass, linkageType, d); - insert(d.symbol, d.symbol.name); - return d; - } - - D visit(ClassDeclaration d) - { - if (d.symbol) - return d; - d.symbol = new Class(d.name, d); - // Insert into current scope. - insert(d.symbol, d.name); - enterScope(d.symbol); - // Continue semantic analysis. - d.decls && visitD(d.decls); - exitScope(); - return d; - } - - D visit(InterfaceDeclaration d) - { - if (d.symbol) - return d; - d.symbol = new dil.semantic.Symbols.Interface(d.name, d); - // Insert into current scope. - insert(d.symbol, d.name); - enterScope(d.symbol); - // Continue semantic analysis. - d.decls && visitD(d.decls); - exitScope(); - return d; - } - - D visit(StructDeclaration d) - { - if (d.symbol) - return d; - d.symbol = new Struct(d.name, d); - // Insert into current scope. - if (d.name) - insert(d.symbol, d.name); - enterScope(d.symbol); - // Continue semantic analysis. - d.decls && visitD(d.decls); - exitScope(); - return d; - } - - D visit(UnionDeclaration d) - { - if (d.symbol) - return d; - d.symbol = new Union(d.name, d); - // Insert into current scope. - if (d.name) - insert(d.symbol, d.name); - enterScope(d.symbol); - // Continue semantic analysis. - d.decls && visitD(d.decls); - exitScope(); - return d; - } - - D visit(ConstructorDeclaration d) - { - auto func = new Function(Ident.__ctor, d); - insertOverload(func, func.name); - return d; - } - - D visit(StaticConstructorDeclaration d) - { - auto func = new Function(Ident.__ctor, d); - insertOverload(func, func.name); - return d; - } - - D visit(DestructorDeclaration d) - { - auto func = new Function(Ident.__dtor, d); - insertOverload(func, func.name); - return d; - } - - D visit(StaticDestructorDeclaration d) - { - auto func = new Function(Ident.__dtor, d); - insertOverload(func, func.name); - return d; - } - - D visit(FunctionDeclaration d) - { - auto func = new Function(d.name, d); - insertOverload(func, func.name); - return d; - } - - D visit(VariablesDeclaration vd) - { - // Error if we are in an interface. - if (scop.symbol.isInterface && !vd.isStatic) - return error(vd.begin, MSG.InterfaceCantHaveVariables), vd; - - // Insert variable symbols in this declaration into the symbol table. - foreach (i, name; vd.names) - { - auto variable = new Variable(name, protection, storageClass, linkageType, vd); - variable.value = vd.inits[i]; - vd.variables ~= variable; - insert(variable, name); - } - return vd; - } - - D visit(InvariantDeclaration d) - { - auto func = new Function(Ident.__invariant, d); - insert(func, func.name); - return d; - } - - D visit(UnittestDeclaration d) - { - auto func = new Function(Ident.__unittest, d); - insertOverload(func, func.name); - return d; - } - - D visit(DebugDeclaration d) - { - if (d.isSpecification) - { - if (!isModuleScope()) - error(d.begin, MSG.DebugSpecModuleLevel, d.spec.srcText); - else if (d.spec.kind == TOK.Identifier) - context.addDebugId(d.spec.ident.str); - else - context.debugLevel = d.spec.uint_; - } - else - { - if (debugBranchChoice(d.cond, context)) - d.compiledDecls = d.decls; - else - d.compiledDecls = d.elseDecls; - d.compiledDecls && visitD(d.compiledDecls); - } - return d; - } - - D visit(VersionDeclaration d) - { - if (d.isSpecification) - { - if (!isModuleScope()) - error(d.begin, MSG.VersionSpecModuleLevel, d.spec.srcText); - else if (d.spec.kind == TOK.Identifier) - context.addVersionId(d.spec.ident.str); - else - context.versionLevel = d.spec.uint_; - } - else - { - if (versionBranchChoice(d.cond, context)) - d.compiledDecls = d.decls; - else - d.compiledDecls = d.elseDecls; - d.compiledDecls && visitD(d.compiledDecls); - } - return d; - } - - D visit(TemplateDeclaration d) - { - if (d.symbol) - return d; - d.symbol = new Template(d.name, d); - // Insert into current scope. - insertOverload(d.symbol, d.name); - enterScope(d.symbol); - // Continue semantic analysis. - visitD(d.decls); - exitScope(); - return d; - } - - D visit(NewDeclaration d) - { - auto func = new Function(Ident.__new, d); - insert(func, func.name); - return d; - } - - D visit(DeleteDeclaration d) - { - auto func = new Function(Ident.__delete, d); - insert(func, func.name); - return d; - } - - D visit(ProtectionDeclaration d) - { - auto saved = protection; // Save. - protection = d.prot; // Set. - visitD(d.decls); - protection = saved; // Restore. - return d; - } - - D visit(StorageClassDeclaration d) - { - auto saved = storageClass; // Save. - storageClass = d.storageClass; // Set. - visitD(d.decls); - storageClass = saved; // Restore. - return d; - } - - D visit(LinkageDeclaration d) - { - auto saved = linkageType; // Save. - linkageType = d.linkageType; // Set. - visitD(d.decls); - linkageType = saved; // Restore. - return d; - } - - D visit(AlignDeclaration d) - { - auto saved = alignSize; // Save. - alignSize = d.size; // Set. - visitD(d.decls); - alignSize = saved; // Restore. - return d; - } - - // Deferred declarations: - - D visit(StaticAssertDeclaration d) - { - addDeferred(d); - return d; - } - - D visit(StaticIfDeclaration d) - { - addDeferred(d); - return d; - } - - D visit(MixinDeclaration d) - { - addDeferred(d); - return d; - } - - D visit(PragmaDeclaration d) - { - if (d.ident is Ident.msg) - addDeferred(d); - else - { - pragmaSemantic(scop, d.begin, d.ident, d.args); - visitD(d.decls); - } - return d; - } -} // override -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Pass2.d --- a/trunk/src/dil/semantic/Pass2.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,436 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Pass2; - -import dil.ast.DefaultVisitor; -import dil.ast.Node, - dil.ast.Declarations, - dil.ast.Expressions, - dil.ast.Statements, - dil.ast.Types, - dil.ast.Parameters; -import dil.lexer.Identifier; -import dil.semantic.Symbol, - dil.semantic.Symbols, - dil.semantic.Types, - dil.semantic.Scope, - dil.semantic.Module, - dil.semantic.Analysis, - dil.semantic.Interpreter; -import dil.parser.Parser; -import dil.SourceText; -import dil.Location; -import dil.Information; -import dil.Messages; -import dil.Enums; -import dil.CompilerInfo; -import common; - -/// The second pass determines the types of symbols and the types -/// of expressions and also evaluates them. -class SemanticPass2 : DefaultVisitor -{ - Scope scop; /// The current scope. - Module modul; /// The module to be semantically checked. - - /// Constructs a SemanticPass2 object. - /// Params: - /// modul = the module to be checked. - this(Module modul) - { - this.modul = modul; - } - - /// Start semantic analysis. - void start() - { - assert(modul.root !is null); - // Create module scope. - scop = new Scope(null, modul); - visit(modul.root); - } - - /// Enters a new scope. - void enterScope(ScopeSymbol s) - { - scop = scop.enter(s); - } - - /// Exits the current scope. - void exitScope() - { - scop = scop.exit(); - } - - /// Evaluates e and returns the result. - Expression interpret(Expression e) - { - return Interpreter.interpret(e, modul.infoMan/+, scop+/); - } - - /// Creates an error report. - void error(Token* token, char[] formatMsg, ...) - { - auto location = token.getErrorLocation(); - auto msg = Format(_arguments, _argptr, formatMsg); - modul.infoMan ~= new SemanticError(location, msg); - } - - /// Some handy aliases. - private alias Declaration D; - private alias Expression E; /// ditto - private alias Statement S; /// ditto - private alias TypeNode T; /// ditto - - /// The current scope symbol to use for looking up identifiers. - /// E.g.: - /// --- - /// object.method(); // *) object is looked up in the current scope. - /// // *) idScope is set if object is a ScopeSymbol. - /// // *) method will be looked up in idScope. - /// dil.ast.Node.Node node; // A fully qualified type. - /// --- - ScopeSymbol idScope; - - /// Searches for a symbol. - Symbol search(Token* idTok) - { - assert(idTok.kind == TOK.Identifier); - auto id = idTok.ident; - Symbol symbol; - - if (idScope is null) - for (auto sc = scop; sc; sc = sc.parent) - { - symbol = sc.lookup(id); - if (symbol) - return symbol; - } - else - symbol = idScope.lookup(id); - - if (symbol is null) - error(idTok, MSG.UndefinedIdentifier, id.str); - else if (auto scopSymbol = cast(ScopeSymbol)symbol) - idScope = scopSymbol; - - return symbol; - } - -override -{ - D visit(CompoundDeclaration d) - { - return super.visit(d); - } - - D visit(EnumDeclaration d) - { - d.symbol.setCompleting(); - - Type type = Types.Int; // Default to int. - if (d.baseType) - type = visitT(d.baseType).type; - d.symbol.type = new TypeEnum(d.symbol, type); - - enterScope(d.symbol); - - foreach (member; d.members) - { - Expression finalValue; - member.symbol.setCompleting(); - if (member.value) - { - member.value = visitE(member.value); - finalValue = interpret(member.value); - if (finalValue is Interpreter.NAR) - finalValue = new IntExpression(0, d.symbol.type); - } - //else - // TODO: increment a number variable and assign that to value. - member.symbol.type = d.symbol.type; // Assign TypeEnum. - member.symbol.value = finalValue; - member.symbol.setComplete(); - } - - exitScope(); - d.symbol.setComplete(); - return d; - } - - D visit(MixinDeclaration md) - { - if (md.decls) - return md.decls; - if (md.isMixinExpression) - { - md.argument = visitE(md.argument); - auto expr = interpret(md.argument); - if (expr is Interpreter.NAR) - return md; - auto stringExpr = expr.Is!(StringExpression); - if (stringExpr is null) - { - error(md.begin, MSG.MixinArgumentMustBeString); - return md; - } - else - { // Parse the declarations in the string. - auto loc = md.begin.getErrorLocation(); - auto filePath = loc.filePath; - auto sourceText = new SourceText(filePath, stringExpr.getString()); - auto parser = new Parser(sourceText, modul.infoMan); - md.decls = parser.start(); - } - } - else - { - // TODO: implement template mixin. - } - return md.decls; - } - - // Type nodes: - - T visit(TypeofType t) - { - t.e = visitE(t.e); - t.type = t.e.type; - return t; - } - - T visit(ArrayType t) - { - auto baseType = visitT(t.next).type; - if (t.isAssociative) - t.type = baseType.arrayOf(visitT(t.assocType).type); - else if (t.isDynamic) - t.type = baseType.arrayOf(); - else if (t.isStatic) - {} - else - assert(t.isSlice); - return t; - } - - T visit(PointerType t) - { - t.type = visitT(t.next).type.ptrTo(); - return t; - } - - T visit(QualifiedType t) - { - if (t.lhs.Is!(QualifiedType) is null) - idScope = null; // Reset at left-most type. - visitT(t.lhs); - visitT(t.rhs); - t.type = t.rhs.type; - return t; - } - - T visit(IdentifierType t) - { - auto idToken = t.begin; - auto symbol = search(idToken); - // TODO: save symbol or its type in t. - return t; - } - - T visit(TemplateInstanceType t) - { - auto idToken = t.begin; - auto symbol = search(idToken); - // TODO: save symbol or its type in t. - return t; - } - - T visit(ModuleScopeType t) - { - idScope = modul; - return t; - } - - T visit(IntegralType t) - { - // A table mapping the kind of a token to its corresponding semantic Type. - TypeBasic[TOK] tok2Type = [ - TOK.Char : Types.Char, TOK.Wchar : Types.Wchar, TOK.Dchar : Types.Dchar, TOK.Bool : Types.Bool, - TOK.Byte : Types.Byte, TOK.Ubyte : Types.Ubyte, TOK.Short : Types.Short, TOK.Ushort : Types.Ushort, - TOK.Int : Types.Int, TOK.Uint : Types.Uint, TOK.Long : Types.Long, TOK.Ulong : Types.Ulong, - TOK.Cent : Types.Cent, TOK.Ucent : Types.Ucent, - TOK.Float : Types.Float, TOK.Double : Types.Double, TOK.Real : Types.Real, - TOK.Ifloat : Types.Ifloat, TOK.Idouble : Types.Idouble, TOK.Ireal : Types.Ireal, - TOK.Cfloat : Types.Cfloat, TOK.Cdouble : Types.Cdouble, TOK.Creal : Types.Creal, TOK.Void : Types.Void - ]; - assert(t.tok in tok2Type); - t.type = tok2Type[t.tok]; - return t; - } - - // Expression nodes: - - E visit(ParenExpression e) - { - if (!e.type) - { - e.next = visitE(e.next); - e.type = e.next.type; - } - return e; - } - - E visit(CommaExpression e) - { - if (!e.type) - { - e.lhs = visitE(e.lhs); - e.rhs = visitE(e.rhs); - e.type = e.rhs.type; - } - return e; - } - - E visit(OrOrExpression) - { return null; } - - E visit(AndAndExpression) - { return null; } - - E visit(SpecialTokenExpression e) - { - if (e.type) - return e.value; - switch (e.specialToken.kind) - { - case TOK.LINE, TOK.VERSION: - e.value = new IntExpression(e.specialToken.uint_, Types.Uint); - break; - case TOK.FILE, TOK.DATE, TOK.TIME, TOK.TIMESTAMP, TOK.VENDOR: - e.value = new StringExpression(e.specialToken.str); - break; - default: - assert(0); - } - e.type = e.value.type; - return e.value; - } - - E visit(DollarExpression e) - { - if (e.type) - return e; - e.type = Types.Size_t; - // if (!inArraySubscript) - // error("$ can only be in an array subscript."); - return e; - } - - E visit(NullExpression e) - { - if (!e.type) - e.type = Types.Void_ptr; - return e; - } - - E visit(BoolExpression e) - { - if (e.type) - return e; - e.value = new IntExpression(e.toBool(), Types.Bool); - e.type = Types.Bool; - return e; - } - - E visit(IntExpression e) - { - if (e.type) - return e; - - if (e.number & 0x8000_0000_0000_0000) - e.type = Types.Ulong; // 0xFFFF_FFFF_FFFF_FFFF - else if (e.number & 0xFFFF_FFFF_0000_0000) - e.type = Types.Long; // 0x7FFF_FFFF_FFFF_FFFF - else if (e.number & 0x8000_0000) - e.type = Types.Uint; // 0xFFFF_FFFF - else - e.type = Types.Int; // 0x7FFF_FFFF - return e; - } - - E visit(RealExpression e) - { - if (!e.type) - e.type = Types.Double; - return e; - } - - E visit(ComplexExpression e) - { - if (!e.type) - e.type = Types.Cdouble; - return e; - } - - E visit(CharExpression e) - { - if (e.type) - return e; - if (e.character <= 0xFF) - e.type = Types.Char; - else if (e.character <= 0xFFFF) - e.type = Types.Wchar; - else - e.type = Types.Dchar; - return e; - } - - E visit(StringExpression e) - { - return e; - } - - E visit(MixinExpression me) - { - if (me.type) - return me.expr; - me.expr = visitE(me.expr); - auto expr = interpret(me.expr); - if (expr is Interpreter.NAR) - return me; - auto stringExpr = expr.Is!(StringExpression); - if (stringExpr is null) - error(me.begin, MSG.MixinArgumentMustBeString); - else - { - auto loc = me.begin.getErrorLocation(); - auto filePath = loc.filePath; - auto sourceText = new SourceText(filePath, stringExpr.getString()); - auto parser = new Parser(sourceText, modul.infoMan); - expr = parser.start2(); - expr = visitE(expr); // Check expression. - } - me.expr = expr; - me.type = expr.type; - return me.expr; - } - - E visit(ImportExpression ie) - { - if (ie.type) - return ie.expr; - ie.expr = visitE(ie.expr); - auto expr = interpret(ie.expr); - if (expr is Interpreter.NAR) - return ie; - auto stringExpr = expr.Is!(StringExpression); - //if (stringExpr is null) - // error(me.begin, MSG.ImportArgumentMustBeString); - // TODO: load file - //ie.expr = new StringExpression(loadImportFile(stringExpr.getString())); - return ie.expr; - } -} -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Scope.d --- a/trunk/src/dil/semantic/Scope.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,72 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Scope; - -import dil.semantic.Symbol; -import dil.semantic.Symbols; -import dil.lexer.Identifier; -import common; - -/// Builds a hierarchy of environments. -class Scope -{ - Scope parent; /// The surrounding scope, or null if this is the root scope. - - ScopeSymbol symbol; /// The current symbol with the symbol table. - - this(Scope parent, ScopeSymbol symbol) - { - this.parent = parent; - this.symbol = symbol; - } - - /// Find a symbol in this scope. - /// Params: - /// name = the name of the symbol. - Symbol lookup(Identifier* name) - { - return symbol.lookup(name); - } - - /// Create a new inner scope and return that. - Scope enter(ScopeSymbol symbol) - { - return new Scope(this, symbol); - } - - /// Destroy this scope and return the outer scope. - Scope exit() - { - auto sc = parent; - // delete this; - return sc; - } - - /// Search for the enclosing Class scope. - Scope classScope() - { - auto scop = this; - do - { - if (scop.symbol.isClass) - return scop; - scop = scop.parent; - } while (scop) - return null; - } - - /// Search for the enclosing Module scope. - Scope moduleScope() - { - auto scop = this; - do - { - if (scop.symbol.isModule) - return scop; - scop = scop.parent; - } while (scop) - return null; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Symbol.d --- a/trunk/src/dil/semantic/Symbol.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,111 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Symbol; - -import dil.ast.Node; -import dil.lexer.Identifier; -import common; - -/// Enumeration of Symbol IDs. -enum SYM -{ - Module, - Class, - Interface, - Struct, - Union, - Enum, - EnumMember, - Template, - Variable, - Function, - Alias, - OverloadSet, -// Type, -} - -/// A symbol represents an object with semantic code information. -class Symbol -{ /// Enumeration of symbol statuses. - enum Status : ushort - { - Declared, /// The symbol has been declared. - Completing, /// The symbol is being processed. - Complete /// The symbol is complete. - } - - SYM sid; /// The ID of this symbol. - Status status; /// The semantic status of this symbol. - Symbol parent; /// The parent this symbol belongs to. - Identifier* name; /// The name of this symbol. - /// The syntax tree node that produced this symbol. - /// Useful for source code location info and retrieval of doc comments. - Node node; - - /// Constructs a Symbol object. - /// Params: - /// sid = the symbol's ID. - /// name = the symbol's name. - /// node = the symbol's node. - this(SYM sid, Identifier* name, Node node) - { - this.sid = sid; - this.name = name; - this.node = node; - } - - /// Change the status to Status.Completing. - void setCompleting() - { status = Status.Completing; } - - /// Change the status to Status.Complete. - void setComplete() - { status = Status.Complete; } - - /// Returns true if the symbol is being completed. - bool isCompleting() - { return status == Status.Completing; } - - /// Returns true if the symbols is complete. - bool isComplete() - { return status == Status.Complete; } - - /// A template macro for building isXYZ() methods. - private template isX(char[] kind) - { - const char[] isX = `bool is`~kind~`(){ return sid == SYM.`~kind~`; }`; - } - mixin(isX!("Module")); - mixin(isX!("Class")); - mixin(isX!("Interface")); - mixin(isX!("Struct")); - mixin(isX!("Union")); - mixin(isX!("Enum")); - mixin(isX!("EnumMember")); - mixin(isX!("Template")); - mixin(isX!("Variable")); - mixin(isX!("Function")); - mixin(isX!("Alias")); - mixin(isX!("OverloadSet")); -// mixin(isX!("Type")); - - /// Casts the symbol to Class. - Class to(Class)() - { - assert(mixin(`this.sid == mixin("SYM." ~ typeof(Class).stringof)`)); - return cast(Class)cast(void*)this; - } - - /// Returns: the fully qualified name of this symbol. - /// E.g.: dil.semantic.Symbol.Symbol.getFQN - char[] getFQN() - { - if (!name) - return parent ? parent.getFQN() : ""; - if (parent) - return parent.getFQN() ~ '.' ~ name.str; - return name.str; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/SymbolTable.d --- a/trunk/src/dil/semantic/SymbolTable.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,30 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.SymbolTable; - -import dil.semantic.Symbol; -import dil.lexer.Identifier; -import common; - -/// Maps an identifier string to a Symbol. -struct SymbolTable -{ - Symbol[char[]] table; /// The table data structure. - - /// Looks up ident in the table. - /// Returns: the symbol if there, otherwise null. - Symbol lookup(Identifier* ident) - { - assert(ident !is null); - auto psym = ident.str in table; - return psym ? *psym : null; - } - - /// Inserts a symbol into the table. - void insert(Symbol symbol, Identifier* ident) - { - table[ident.str] = symbol; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Symbols.d --- a/trunk/src/dil/semantic/Symbols.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,206 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Symbols; - -import dil.ast.Expression; -import dil.semantic.Symbol; -import dil.semantic.SymbolTable; -import dil.semantic.Types; -import dil.ast.Node; -import dil.lexer.IdTable; -import dil.Enums; -import common; - -/// A symbol that has its own scope with a symbol table. -class ScopeSymbol : Symbol -{ - SymbolTable symbolTable; /// The symbol table. - Symbol[] members; /// The member symbols (in lexical order.) - - this(SYM sid, Identifier* name, Node node) - { - super(sid, name, node); - } - - /// Look up name in the table. - Symbol lookup(Identifier* name) - { - return symbolTable.lookup(name); - } - - /// Look up name in the table. - Symbol lookup(string name) - { - auto id = IdTable.lookup(name); - return id ? symbolTable.lookup(id) : null; - } - - /// Insert a symbol into the table. - void insert(Symbol s, Identifier* name) - { - symbolTable.insert(s, name); - members ~= s; - } -} - -/// Aggregates have function and field members. -abstract class Aggregate : ScopeSymbol -{ - Function[] funcs; - Variable[] fields; - - this(SYM sid, Identifier* name, Node node) - { - super(sid, name, node); - } - - override void insert(Symbol s, Identifier* ident) - { - if (s.isVariable) - // Append variable to fields. - fields ~= cast(Variable)cast(void*)s; - else if (s.isFunction) - // Append function to funcs. - funcs ~= cast(Function)cast(void*)s; - super.insert(s, ident); - } -} - -/// A class symbol. -class Class : Aggregate -{ - this(Identifier* name, Node classNode) - { - super(SYM.Class, name, classNode); - } -} - -/// An interface symbol. -class Interface : Aggregate -{ - this(Identifier* name, Node interfaceNode) - { - super(SYM.Interface, name, interfaceNode); - } -} - -/// A union symbol. -class Union : Aggregate -{ - this(Identifier* name, Node unionNode) - { - super(SYM.Union, name, unionNode); - } -} - -/// A struct symbol. -class Struct : Aggregate -{ - this(Identifier* name, Node structNode) - { - super(SYM.Struct, name, structNode); - } -} - -/// An enum symbol. -class Enum : ScopeSymbol -{ - TypeEnum type; - this(Identifier* name, Node enumNode) - { - super(SYM.Enum, name, enumNode); - } - - void setType(TypeEnum type) - { - this.type = type; - } -} - -/// A template symbol. -class Template : ScopeSymbol -{ - this(Identifier* name, Node templateNode) - { - super(SYM.Template, name, templateNode); - } -} - -/// A function symbol. -class Function : ScopeSymbol -{ - Protection prot; /// The protection. - StorageClass stc; /// The storage classes. - LinkageType linkType; /// The linkage type. - - Type returnType; - Variable[] params; - - this(Identifier* name, Node functionNode) - { - super(SYM.Function, name, functionNode); - } -} - -/// A variable symbol. -class Variable : Symbol -{ - Protection prot; /// The protection. - StorageClass stc; /// The storage classes. - LinkageType linkType; /// The linkage type. - - Type type; /// The type of this variable. - Expression value; /// The value of this variable. - - this(Identifier* name, - Protection prot, StorageClass stc, LinkageType linkType, - Node variableNode) - { - super(SYM.Variable, name, variableNode); - - this.prot = prot; - this.stc = stc; - this.linkType = linkType; - } -} - -/// An enum member symbol. -class EnumMember : Variable -{ - this(Identifier* name, - Protection prot, StorageClass stc, LinkageType linkType, - Node enumMemberNode) - { - super(name, prot, stc, linkType, enumMemberNode); - this.sid = SYM.EnumMember; - } -} - -/// An alias symbol. -class Alias : Symbol -{ - this(Identifier* name, Node aliasNode) - { - super(SYM.Alias, name, aliasNode); - } -} - -/// A list of symbols that share the same identifier. -/// -/// These can be functions, templates and aggregates with template parameter lists. -class OverloadSet : Symbol -{ - Symbol[] symbols; - - this(Identifier* name, Node node) - { - super(SYM.OverloadSet, name, node); - } - - void add(Symbol s) - { - symbols ~= s; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/Types.d --- a/trunk/src/dil/semantic/Types.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,403 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.Types; - -import dil.semantic.Symbol; -import dil.semantic.TypesEnum; -import dil.lexer.Identifier; -import dil.CompilerInfo; - -/// The base type for all type structures. -abstract class Type/* : Symbol*/ -{ - Type next; /// The next type in the type structure. - TYP tid; /// The ID of the type. - Symbol symbol; /// Not null if this type has a symbol. - - this(){} - - /// Constructs a Type object. - /// Params: - /// next = the type's next type. - /// tid = the type's ID. - this(Type next, TYP tid) - { -// this.sid = SYM.Type; - - this.next = next; - this.tid = tid; - } - - /// Returns a pointer type to this type. - TypePointer ptrTo() - { - return new TypePointer(this); - } - - /// Returns a dynamic array type using this type as its base. - TypeDArray arrayOf() - { - return new TypeDArray(this); - } - - /// Returns an associative array type using this type as its base. - /// Params: - /// key = the key type. - TypeAArray arrayOf(Type key) - { - return new TypeAArray(this, key); - } - - /// Returns the byte size of this type. - final size_t sizeOf() - { - return MITable.getSize(this); - } - - /// Size is not in MITable. Find out via virtual method. - size_t sizeOf_() - { - return sizeOf(); - } - - /// Returns true if this type has a symbol. - bool hasSymbol() - { - return symbol !is null; - } -} - -/// All basic types. E.g.: int, char, real etc. -class TypeBasic : Type -{ - this(TYP typ) - { - super(null, typ); - } -} - -/// Dynamic array type. -class TypeDArray : Type -{ - this(Type next) - { - super(next, TYP.DArray); - } -} - -/// Associative array type. -class TypeAArray : Type -{ - Type keyType; - this(Type next, Type keyType) - { - super(next, TYP.AArray); - this.keyType = keyType; - } -} - -/// Static array type. -class TypeSArray : Type -{ - size_t dimension; - this(Type next, size_t dimension) - { - super(next, TYP.SArray); - this.dimension = dimension; - } -} - -/// Pointer type. -class TypePointer : Type -{ - this(Type next) - { - super(next, TYP.Pointer); - } -} - -/// Reference type. -class TypeReference : Type -{ - this(Type next) - { - super(next, TYP.Reference); - } -} - -/// Enum type. -class TypeEnum : Type -{ - this(Symbol symbol, Type baseType) - { - super(baseType, TYP.Enum); - this.symbol = symbol; - } - - Type baseType() - { - return next; - } -} - -/// Struct type. -class TypeStruct : Type -{ - this(Symbol symbol) - { - super(null, TYP.Struct); - this.symbol = symbol; - } -} - -/// Class type. -class TypeClass : Type -{ - this(Symbol symbol) - { - super(null, TYP.Class); - this.symbol = symbol; - } -} - -/// Typedef type. -class TypeTypedef : Type -{ - this(Type next) - { - super(next, TYP.Typedef); - } -} - -/// Function type. -class TypeFunction : Type -{ - this(Type next) - { - super(next, TYP.Function); - } -} - -/// Delegate type. -class TypeDelegate : Type -{ - this(Type next) - { - super(next, TYP.Delegate); - } -} - -/// Identifier type. -class TypeIdentifier : Type -{ - Identifier* ident; - this(Identifier* ident) - { - super(null, TYP.Identifier); - } -} - -/// Template instantiation type. -class TypeTemplInstance : Type -{ - this() - { - super(null, TYP.TInstance); - } -} - -/// Template tuple type. -class TypeTuple : Type -{ - this(Type next) - { - super(next, TYP.Tuple); - } -} - -/// Constant type. D2.0 -class TypeConst : Type -{ - this(Type next) - { - super(next, TYP.Const); - } -} - -/// Invariant type. D2.0 -class TypeInvariant : Type -{ - this(Type next) - { - super(next, TYP.Const); - } -} - -/// Represents a value related to a Type. -union Value -{ - void* pvoid; - bool bool_; - dchar dchar_; - long long_; - ulong ulong_; - int int_; - uint uint_; - float float_; - double double_; - real real_; - creal creal_; -} - -/// Information related to a Type. -struct TypeMetaInfo -{ - char mangle; /// Mangle character of the type. - ushort size; /// Byte size of the type. - Value* defaultInit; /// Default initialization value. -} - -/// Namespace for the meta info table. -struct MITable -{ -static: - const ushort SIZE_NOT_AVAILABLE = 0; /// Size not available. - const Value VZERO = {int_:0}; /// Value 0. - const Value VNULL = {pvoid:null}; /// Value null. - const Value V0xFF = {dchar_:0xFF}; /// Value 0xFF. - const Value V0xFFFF = {dchar_:0xFFFF}; /// Value 0xFFFF. - const Value VFALSE = {bool_:false}; /// Value false. - const Value VNAN = {float_:float.nan}; /// Value NAN. - const Value VCNAN = {creal_:creal.nan}; /// Value complex NAN. - private alias SIZE_NOT_AVAILABLE SNA; - private alias PTR_SIZE PS; - /// The meta info table. - private const TypeMetaInfo metaInfoTable[] = [ - {'?', SNA}, // Error - - {'a', 1, &V0xFF}, // Char - {'u', 2, &V0xFFFF}, // Wchar - {'w', 4, &V0xFFFF}, // Dchar - {'b', 1, &VFALSE}, // Bool - {'g', 1, &VZERO}, // Byte - {'h', 1, &VZERO}, // Ubyte - {'s', 2, &VZERO}, // Short - {'t', 2, &VZERO}, // Ushort - {'i', 4, &VZERO}, // Int - {'k', 4, &VZERO}, // Uint - {'l', 8, &VZERO}, // Long - {'m', 8, &VZERO}, // Ulong - {'?', 16, &VZERO}, // Cent - {'?', 16, &VZERO}, // Ucent - {'f', 4, &VNAN}, // Float - {'d', 8, &VNAN}, // Double - {'e', 12, &VNAN}, // Real - {'o', 4, &VNAN}, // Ifloat - {'p', 8, &VNAN}, // Idouble - {'j', 12, &VNAN}, // Ireal - {'q', 8, &VCNAN}, // Cfloat - {'r', 16, &VCNAN}, // Cdouble - {'c', 24, &VCNAN}, // Creal - {'v', 1}, // void - - {'n', SNA}, // None - - {'A', PS*2, &VNULL}, // Dynamic array - {'G', PS*2, &VNULL}, // Static array - {'H', PS*2, &VNULL}, // Associative array - - {'E', SNA}, // Enum - {'S', SNA}, // Struct - {'C', PS, &VNULL}, // Class - {'T', SNA}, // Typedef - {'F', PS}, // Function - {'D', PS*2, &VNULL}, // Delegate - {'P', PS, &VNULL}, // Pointer - {'R', PS, &VNULL}, // Reference - {'I', SNA}, // Identifier - {'?', SNA}, // Template instance - {'B', SNA}, // Tuple - {'x', SNA}, // Const, D2 - {'y', SNA}, // Invariant, D2 - ]; - static assert(metaInfoTable.length == TYP.max+1); - - /// Returns the size of a type. - size_t getSize(Type type) - { - auto size = metaInfoTable[type.tid].size; - if (size == SIZE_NOT_AVAILABLE) - return type.sizeOf_(); - return size; - } -} - -/// Namespace for a set of predefined types. -struct Types -{ -static: - /// Predefined basic types. - TypeBasic Char, Wchar, Dchar, Bool, - Byte, Ubyte, Short, Ushort, - Int, Uint, Long, Ulong, - Cent, Ucent, - Float, Double, Real, - Ifloat, Idouble, Ireal, - Cfloat, Cdouble, Creal, Void; - - TypeBasic Size_t; /// The size type. - TypeBasic Ptrdiff_t; /// The pointer difference type. - TypePointer Void_ptr; /// The void pointer type. - TypeBasic Error; /// The error type. - TypeBasic Undefined; /// The undefined type. - - /// Allocates an instance of TypeBasic and assigns it to typeName. - template newTB(char[] typeName) - { - const newTB = mixin(typeName~" = new TypeBasic(TYP."~typeName~")"); - } - - /// Initializes predefined types. - static this() - { - newTB!("Char"); - newTB!("Wchar"); - newTB!("Dchar"); - newTB!("Bool"); - newTB!("Byte"); - newTB!("Ubyte"); - newTB!("Short"); - newTB!("Ushort"); - newTB!("Int"); - newTB!("Uint"); - newTB!("Long"); - newTB!("Ulong"); - newTB!("Cent"); - newTB!("Ucent"); - newTB!("Float"); - newTB!("Double"); - newTB!("Real"); - newTB!("Ifloat"); - newTB!("Idouble"); - newTB!("Ireal"); - newTB!("Cfloat"); - newTB!("Cdouble"); - newTB!("Creal"); - newTB!("Void"); - version(X86_64) - { - Size_t = Ulong; - Ptrdiff_t = Long; - } - else - { - Size_t = Uint; - Ptrdiff_t = Int; - } - Void_ptr = Void.ptrTo; - Error = new TypeBasic(TYP.Error); - Undefined = new TypeBasic(TYP.Error); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/semantic/TypesEnum.d --- a/trunk/src/dil/semantic/TypesEnum.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,56 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.semantic.TypesEnum; - -/// Enumeration of Type IDs. -enum TYP -{ - Error, - // Basic types. - Char, /// char - Wchar, /// wchar - Dchar, /// dchar - Bool, /// bool - Byte, /// int8 - Ubyte, /// uint8 - Short, /// int16 - Ushort, /// uint16 - Int, /// int32 - Uint, /// uint32 - Long, /// int64 - Ulong, /// uint64 - Cent, /// int128 - Ucent, /// uint128 - Float, /// float32 - Double, /// float64 - Real, /// float80 - Ifloat, /// imaginary float32 - Idouble, /// imaginary float64 - Ireal, /// imaginary float80 - Cfloat, /// complex float32 - Cdouble, /// complex float64 - Creal, /// complex float80 - Void, /// void - - None, /// TypeNone in the specs. Why? - - DArray, /// Dynamic array. - SArray, /// Static array. - AArray, /// Associative array. - - Enum, /// An enum. - Struct, /// A struct. - Class, /// A class. - Typedef, /// A typedef. - Function, /// A function. - Delegate, /// A delegate. - Pointer, /// A pointer. - Reference, /// A reference. - Identifier, /// An identifier. - TInstance, /// Template instance. - Tuple, /// A template tuple. - Const, /// A constant type. D2.0 - Invariant, /// An invariant type. D2.0 -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/dil/translator/German.d --- a/trunk/src/dil/translator/German.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,340 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module dil.translator.German; - -import dil.ast.DefaultVisitor; -import dil.ast.Node; -import dil.ast.Declarations, - dil.ast.Statements, - dil.ast.Types, - dil.ast.Parameters; -import tango.io.Print; - -private alias Declaration D; - -/// Translates a syntax tree into German. -class GermanTranslator : DefaultVisitor -{ - Print!(char) put; /// Output buffer. - - char[] indent; /// Current indendation string. - char[] indentStep; /// Appended to indent at each indendation level. - - Declaration inAggregate; /// Current aggregate. - Declaration inFunc; /// Current function. - - bool pluralize; /// Whether to use the plural when printing the next types. - bool pointer; /// Whether next types should consider the previous pointer. - - /// Construct a GermanTranslator. - /// Params: - /// put = buffer to print to. - /// indentStep = added at every indendation step. - this(Print!(char) put, char[] indentStep) - { - this.put = put; - this.indentStep = indentStep; - } - - /// Start translation. - void translate(Node root) - { - visitN(root); - } - - /// Increases the indentation when instantiated. - /// The indentation is restored when the instance goes out of scope. - scope class Indent - { - char[] old_indent; - this() - { - old_indent = this.outer.indent; - this.outer.indent ~= this.outer.indentStep; - } - - ~this() - { this.outer.indent = old_indent; } - - char[] toString() - { return this.outer.indent; } - } - - /// Saves an outer member when instantiated. - /// It is restored when the instance goes out of scope. - scope class Enter(T) - { - T t_save; - this(T t) - { - auto t_save = t; - static if (is(T == ClassDeclaration) || - is(T == InterfaceDeclaration) || - is(T == StructDeclaration) || - is(T == UnionDeclaration)) - this.outer.inAggregate = t; - static if (is(T == FunctionDeclaration) || - is(T == ConstructorDeclaration)) - this.outer.inFunc = t; - } - - ~this() - { - static if (is(T == ClassDeclaration) || - is(T == InterfaceDeclaration) || - is(T == StructDeclaration) || - is(T == UnionDeclaration)) - this.outer.inAggregate = t_save; - static if (is(T == FunctionDeclaration) || - is(T == ConstructorDeclaration)) - this.outer.inFunc = t_save; - } - } - - alias Enter!(ClassDeclaration) EnteredClass; - alias Enter!(InterfaceDeclaration) EnteredInterface; - alias Enter!(StructDeclaration) EnteredStruct; - alias Enter!(UnionDeclaration) EnteredUnion; - alias Enter!(FunctionDeclaration) EnteredFunction; - alias Enter!(ConstructorDeclaration) EnteredConstructor; - - /// Prints the location of a node: @(lin,col) - void printLoc(Node node) - { - auto loc = node.begin.getRealLocation(); - put(indent).formatln("@({},{})",/+ loc.filePath,+/ loc.lineNum, loc.colNum); - } - -override: - D visit(ModuleDeclaration n) - { - printLoc(n); - put.format("Dies ist das Modul '{}'", n.moduleName.str); - if (n.packages.length) - put.format(" im Paket '{}'", n.getPackageName('.')); - put(".").newline; - return n; - } - - D visit(ImportDeclaration n) - { - printLoc(n); - put("Importiert Symbole aus einem anderen Modul bzw. Module.").newline; - return n; - } - - D visit(ClassDeclaration n) - { - printLoc(n); - scope E = new EnteredClass(n); - put(indent).formatln("'{}' is eine Klasse mit den Eigenschaften:", n.name.str); - scope I = new Indent(); - n.decls && visitD(n.decls); - return n; - } - - D visit(InterfaceDeclaration n) - { - printLoc(n); - scope E = new EnteredInterface(n); - put(indent).formatln("'{}' is ein Interface mit den Eigenschaften:", n.name.str); - scope I = new Indent(); - n.decls && visitD(n.decls); - return n; - } - - D visit(StructDeclaration n) - { - printLoc(n); - scope E = new EnteredStruct(n); - put(indent).formatln("'{}' is eine Datenstruktur mit den Eigenschaften:", n.name.str); - scope I = new Indent(); - n.decls && visitD(n.decls); - return n; - } - - D visit(UnionDeclaration n) - { - printLoc(n); - scope E = new EnteredUnion(n); - put(indent).formatln("'{}' is eine Datenunion mit den Eigenschaften:", n.name.str); - scope I = new Indent(); - n.decls && visitD(n.decls); - return n; - } - - D visit(VariablesDeclaration n) - { - printLoc(n); - char[] was; - if (inAggregate) - was = "Membervariable"; - else if (inFunc) - was = "lokale Variable"; - else - was = "globale Variable"; - foreach (name; n.names) - { - put(indent).format("'{}' ist eine {} des Typs: ", name.str, was); - if (n.typeNode) - visitT(n.typeNode); - else - put("auto"); - put.newline; - } - return n; - } - - D visit(FunctionDeclaration n) - { - printLoc(n); - char[] was; - if (inAggregate) - was = "Methode"; - else if(inFunc) - was = "geschachtelte Funktion"; - else - was = "Funktion"; - scope E = new EnteredFunction(n); - put(indent).format("'{}' ist eine {} ", n.name.str, was); - if (n.params.length == 1) - put("mit dem Argument "), visitN(n.params); - else if (n.params.length > 1) - put("mit den Argumenten "), visitN(n.params); - else - put("ohne Argumente"); - put(".").newline; - scope I = new Indent(); - return n; - } - - D visit(ConstructorDeclaration n) - { - printLoc(n); - scope E = new EnteredConstructor(n); - put(indent)("Ein Konstruktor "); - if (n.params.length == 1) - put("mit dem Argument "), visitN(n.params); - else if (n.params.length > 1) - put("mit den Argumenten "), visitN(n.params); - else - put("ohne Argumente"); - put(".").newline; - return n; - } - - D visit(StaticConstructorDeclaration n) - { - printLoc(n); - put(indent)("Ein statischer Konstruktor.").newline; - return n; - } - - D visit(DestructorDeclaration n) - { - printLoc(n); - put(indent)("Ein Destruktor.").newline; - return n; - } - - D visit(StaticDestructorDeclaration n) - { - printLoc(n); - put(indent)("Ein statischer Destruktor.").newline; - return n; - } - - D visit(InvariantDeclaration n) - { - printLoc(n); - put(indent)("Eine Unveränderliche.").newline; - return n; - } - - D visit(UnittestDeclaration n) - { - printLoc(n); - put(indent)("Ein Komponententest.").newline; - return n; - } - - Node visit(Parameter n) - { - put.format("'{}' des Typs \"", n.name ? n.name.str : "unbenannt"); - n.type && visitN(n.type); - put(\"); - return n; - } - - Node visit(Parameters n) - { - if (n.length > 1) - { - visitN(n.children[0]); - foreach (node; n.children[1..$]) - put(", "), visitN(node); - } - else - super.visit(n); - return n; - } - - TypeNode visit(ArrayType n) - { - char[] c1 = "s", c2 = ""; - if (pluralize) - (c1 = pointer ? ""[] : "n"), (c2 = "s"); - pointer = false; - if (n.assocType) - put.format("assoziative{} Array{} von ", c1, c2); -// visitT(n.assocType); - else if (n.e1) - { - if (n.e2) - put.format("gescheibte{} Array{} von ", c1, c2); - else - put.format("statische{} Array{} von ", c1, c2); -// visitE(n.e), n.e2 && visitE(n.e2); - } - else - put.format("dynamische{} Array{} von ", c1, c2); - // Types following arrays should be in plural. - pluralize = true; - visitT(n.next); - pluralize = false; - return n; - } - - TypeNode visit(PointerType n) - { - char[] c = pluralize ? (pointer ? ""[] : "n") : ""; - pointer = true; - put.format("Zeiger{} auf ", c), visitT(n.next); - return n; - } - - TypeNode visit(QualifiedType n) - { - visitT(n.lhs); - put("."); - visitT(n.rhs); - return n; - } - - TypeNode visit(IdentifierType n) - { - put(n.ident.str); - return n; - } - - TypeNode visit(IntegralType n) - { - char[] c = pluralize ? "s"[] : ""; - if (n.tok == TOK.Void) // Avoid pluralizing "void" - c = ""; - put.format("{}{}", n.begin.srcText, c); - return n; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/archdoc.xmi --- a/trunk/src/docgen/archdoc.xmi Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1066 +0,0 @@ - - - - - umbrello uml modeller http://uml.sf.net - 1.5.8 - UnicodeUTF8 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/config/configurator.d --- a/trunk/src/docgen/config/configurator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,81 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.config.configurator; - -import docgen.config.reader; -import docgen.config.reflection; -import docgen.misc.options; - -import Integer = tango.text.convert.Integer; -import tango.io.stream.FileStream; -import tango.io.Stdout; - -/** - * Class for handling and merging doc generator options. - */ -interface Configurator { - /** - * Merges configuration options from the given file. - */ - void mergeConfiguration(char[] cfgFile); - - /** - * Returns a hierarchical structure of configuration options. - */ - DocGeneratorOptions *getConfiguration(); -} - -private DocGeneratorOptions options; -private Struct!(options) opt; -private const cases = makeTypeStringForStruct!(opt); - -class DefaultConfigurator : Configurator { - private: - - DocGeneratorOptions options; - - public: - - const defaultProfileLocation = "docgen/config/default.cfg"; - - this() { - mergeConfiguration(defaultProfileLocation); - } - - this(char[] cfgFile) { - this(); - mergeConfiguration(cfgFile); - } - - void mergeConfiguration(char[] cfgFile) { - - auto inputStream = new FileInput(cfgFile); - auto content = new char[inputStream.length]; - auto bytesRead = inputStream.read (content); - - assert(bytesRead == inputStream.length, "Error reading configuration file"); - - auto tokens = lex(content); - auto configuration = parse(tokens); - - foreach(key, val; configuration) { - bool err() { - throw new Exception( - "Configurator: Invalid key-val pair " ~ key ~ - "=" ~ (val.length ? val[0] : "null")); - } - - // cuteness, lul - mixin(_switch( - mixin(cases) ~ - `default: throw new Exception("Illegal configuration key " ~ key);` - )); - } - } - - DocGeneratorOptions *getConfiguration() { - return &options; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/config/default.cfg --- a/trunk/src/docgen/config/default.cfg Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,49 +0,0 @@ -# -# This file contains the default configuration. You don't need to -# worry about this, the system will load the settings automatically. -# -# If you need to customize settings, just take a copy of this or -# write one from scratch and pass the new file location as a -# command line parameter. -# - - -(options - (graph - (imageFormat PNG) - (depth -1) - (nodeColor white) - (cyclicNodeColor tomato) - (unlocatableNodeColor gray) - (depColor black) - (cyclicDepColor red) - (publicDepColor blue) - (clusterColor blue) - (includeUnlocatableModules true) - (highlightCyclicEdges true) - (highlightCyclicVertices true) - (groupByPackageNames true) - (groupByFullPackageName false) - ) - (listing - (literateStyle true) - (enableListings true) - ) - (templates - (title "Test project") - (versionString 1.0) - (copyright "Crashtest dummy") - (paperSize a4paper) - (shortFileNames false) - (templateStyle default) - ) - (parser - (importPaths) - (rootPaths) - (strRegexps) - (commentFormat Doxygen) - (depth -1) - ) - (outputFormats LaTeX HTML PlainText) - (outputDir tmp/) -) diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/config/reader.d --- a/trunk/src/docgen/config/reader.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,144 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.config.reader; - -debug import tango.io.Stdout; - -/** - * Lexes a s-exp like input - */ -char[][] lex(char[] input) { - char[][] tokens; - - uint state = 0, level = 0; - size_t sidx = 0; - - void err(size_t i, int type = 0) { - auto b = input[i<20 ? 0 : i-20..i]; - auto e = input[i+1..i+21>$ ? $ : i+21]; - - throw new Exception("Lex: " ~ - (type == 0 ? "Illegal character" : - type == 1 ? "Wrong number of parenthesis" : - "Unexpected end of input") ~ - ": " ~ b ~ " >>>" ~ input[i] ~ "<<< " ~ e ~ "." - ); - } - void begin(size_t i) { sidx = i; } - void end(size_t i) { tokens ~= input[sidx..i]; } - - foreach(size_t i, c; input) { - if (sidx > i) continue; - switch(c) { // states space, token, textEnd - case '"': // starts a ""-string - switch(state) { - case 0: - char[] buf; - bool escape; - char d; - sidx = i; - while(!((d = input[++sidx]) == '"' && !escape) && sidx 0) end(input.length); - - debug { - foreach(i, tok; tokens) - Stdout.format("{}{}", tok, (i != tokens.length-1 ? " " : "")); - Stdout.newline; - } - - return tokens; -} - -/** - * Parser a s-exp like input used by the config files. - */ -char[][][char[]] parse(char[][] tokens) { - char[][][char[]] values; - size_t i = 1; - - void err(size_t j) { - auto b = tokens[j < 5 ? 0 : j-5..j]; - auto e = tokens[j+1..j+6>$ ? $ : j+6]; - char[] tmp; - foreach(t; b) tmp ~= t ~ " "; - tmp ~= ">>>" ~ tokens[j] ~ "<<< "; - foreach(t; e) tmp ~= t ~ " "; - - throw new Exception( - "Parse: Illegal token: " ~ tmp ~ "." - ); - } - - if (tokens[0] != "(") err(0); - - void doParse(char[] prefix = null) { - if (tokens[i] == "(" || - tokens[i] == ")") err(i); - if (prefix) prefix ~= "."; - auto v = prefix ~ tokens[i++]; - //values[v] = null; - while (tokens[i] != ")") - if (tokens[i++] == "(") - doParse(v); - else - values[v] ~= tokens[i-1]; - i++; - } - - doParse(); - - return values; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/config/reflection.d --- a/trunk/src/docgen/config/reflection.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,275 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.config.reflection; - -import docgen.misc.meta; -import docgen.misc.options; - -//// -// -// Macros for reading input -// -//// - -char[] _wrong(char[] key) { - return `if (val.length != 1) throw new Exception( - "Wrong number of arguments for `~key~`");`; -} - -char[] _switch(char[] stuff) { - return "switch(key) {" ~ stuff ~ "}"; -} - -char[] _parseI(char[] key) { - return `case "` ~ key ~ `":` ~ _wrong(key) ~ key ~ - `= Integer.parse(val[0]); continue;`; -} - -char[] _parseS(char[] key) { - return `case "` ~ key ~ `":` ~ _wrong(key) ~ key ~ - `= val[0]; continue;`; -} - -char[] _parseB(char[] key) { - return `case "` ~ key ~ `":` ~ _wrong(key) ~ key ~ - `= val[0] == "true" ? true : val[0] == "false" ? false : err(); continue;`; -} - -char[] _parseList(char[] key) { - return `case "` ~ key ~ `":foreach(v; val) ` ~ - key ~ `~= v; continue;`; -} - -template _parseEnum_(bool list, char[] key, V...) { - static if (V.length>1) - const char[] _parseEnum_ = - `case "` ~ V[0] ~ `":` ~ key ~ (list ? "~" : "") ~ `=` ~ V[1] ~ `; continue;` \n ~ - _parseEnum_!(list, key, V[2..$]); - else - const char[] _parseEnum_ = ""; -} - -template _parseEnum(char[] key, V...) { - const char[] _parseEnum = `case "` ~ key ~ - `":` ~ _wrong(key) ~ `switch(val[0]) {` ~ - _parseEnum_!(false, key, V) ~ - `default: err(); } continue;`; -} - -template _parseEnumList(char[] key, V...) { - const char[] _parseEnumList = `case "` ~ key ~ - `":` ~ `foreach (item; val) switch(item) {` ~ - _parseEnum_!(true, key, V) ~ - `default: err(); } continue;`; -} - -//// -// -// Reflection for properties. This code will hopefully get better when the dmdfe bugs get fixed. -// -//// - -// dmdfe bug -- Seriously WTF? -char[] fixType(char[] type) { - return type[$-1] == ' ' ? type[0..$-1] : type; -} - -// take the last part of field name -char[] getLastName(char[] field) { - if (field.length == 0) return ""; - int t = 0; - // dmdfe bug: a while loop causes index out of bounds error - for (int i=field.length-1; i >= 0; i--) - if (field[i] == '.') { t = i+1; break; } - return field[t..$]; -} - -// dmdfe bug: cannot return evalType alias -template _evalType(char[] type) { - mixin("alias "~type~" value;"); -} - -// Note: stuple wrappers needed for tuples, otherwise: -// dmdfe bug: cc1d: ../.././gcc/d/dmd/expression.c:4865: virtual Expression* DotIdExp::semantic(Scope*): Assertion `0' failed. -template evalType(char[] type) { - alias _evalType!(type).value evalType; -} - -// wraps the reflected struct and enum data inside struct because otherwise the handling becomes impossibly hard -template getType(T) { - static if(is(T == struct)) - alias Struct!(T) getType; - else static if(is(T == enum)) - alias Enum!(T) getType; - else static if(isEnumList!(T)) - alias Enum!(enumListType!(T), true) getType; - else - alias T getType; -} - -template getTypes(alias S, int idx = S.tupleof.length) { - static if(idx) - alias Tuple!(getTypes!(S, idx-1), getType!(typeof(S.tupleof[idx-1]))) getTypes; - else - alias Tuple!() getTypes; -} - -/** - * Extracts the comma separated struct field names using .tupleof.stringof. - * This is needed since the struct.tupleof[n].stringof is broken for enums and tuples in dmdfe. - * - * Bugs: handling of tuples - */ -char[] __getNames(char[] type) { - char[] tmp; - bool end = false; - - foreach(c; type[5..$]) { - if (c != ' ' && c != '(' && c != ')' && end) tmp ~= c; - if (c == ',') end = false; - if (c == '.') end = true; - } - - return tmp; -} - -template _getNames(char[] str, T...) { - static if (str.length) { - static if (str[0] == ',') - alias _getNames!(str[1..$], T, "") _getNames; - else - alias _getNames!(str[1..$], T[0..$-1], T[$-1] ~ str[0]) _getNames; - } else - alias T _getNames; -} - -template getNames(char[] str) { - alias _getNames!(__getNames(str), "") getNames; -} - -struct Struct(alias T) { - const type = "struct"; // used for pattern matching... apparently there's no other way - const name = fixType(T.stringof); // dmdfe bug: trailing space - alias STuple!(getNames!(T.tupleof.stringof)) names; - alias STuple!(getTypes!(T)) types; -} - -struct Enum(alias T, bool list = false) { - const type = list ? "enumlist" : "enum"; // used for pattern matching... apparently there's no other way - const name = T.stringof[1..$]; // dmdfe bug: returns enum base type instead enum type - alias evalType!("___"~name).tuple elements; -} - -// determines the enumtype[] type -template isEnumList(T : T[]) { - const isEnumList = T.stringof[0] == '_'; -} - -template isEnumList(T) { - const isEnumList = false; -} - -template enumListType(T : T[]) { - alias T enumListType; -} - -template enumListType(T) { - static assert(false, "Not enum list type!"); -} - -char[] createIParser(char[] field) { - return `_parseI("` ~ field ~ `") ~` \n; -} - -char[] createBParser(char[] field) { - return `_parseB("` ~ field ~ `") ~` \n; -} - -char[] createSParser(char[] field) { - return `_parseS("` ~ field ~ `") ~` \n; -} - -char[] createLParser(char[] field) { - return `_parseList("` ~ field ~ `") ~` \n; -} - -char[] createEParser(char[] field, char[] contents) { - return `_parseEnum!("` ~ field ~ `",` ~ contents ~ `) ~` \n; -} - -char[] createELParser(char[] field, char[] contents) { - return `_parseEnumList!("` ~ field ~ `",` ~ contents ~ `) ~` \n; -} - -template _makeEnumString(char[] t, E...) { - static if (E.length) - const _makeEnumString = `"` ~ E[0] ~ `", "` ~ t ~ "." ~ E[0] ~ `",` ~ - _makeEnumString!(t, E[1..$]); - else - const _makeEnumString = ""; -} - -// avoids the following dmdfe bugs: -// - Error: elements is not a type (typeof(T).elements) -// - Error: tuple is not a valid template value argument (T.elements where T is the complex type def) -template makeEnumString(char[] t, T) { - const makeEnumString = _makeEnumString!(t, T.elements); -} - -/** - * Generates code for parsing data from the configuration data structure. - */ -template makeTypeString(int i, N, T, char[] prefix) { - static assert(N.tuple.length == T.tuple.length); - static if (i < N.tuple.length) { - static if (is(T.tuple[i] == bool)) - const makeTypeString = createBParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); - else static if (is(T.tuple[i] : int)) - const makeTypeString = createIParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); - else static if (is(T.tuple[i] == char[])) - const makeTypeString = createSParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); - else static if (is(T.tuple[i] == char[][])) - const makeTypeString = createLParser(prefix ~ N.tuple[i]) ~ makeTypeString!(i+1, N, T, prefix); - else static if (is(T.tuple[i] == struct)) { - static if (T.tuple[i].type == "struct") - const makeTypeString = makeTypeString!(0, typeof(T.tuple[i].names), - typeof(T.tuple[i].types), prefix~N.tuple[i]~".") ~ - makeTypeString!(i+1, N, T, prefix); - else static if (T.tuple[i].type == "enum") - const makeTypeString = createEParser(prefix ~ N.tuple[i], - makeEnumString!(T.tuple[i].name, T.tuple[i])[0..$-1]) ~ - makeTypeString!(i+1, N, T, prefix); - else static if (T.tuple[i].type == "enumlist") - const makeTypeString = createELParser(prefix ~ N.tuple[i], - makeEnumString!(T.tuple[i].name, T.tuple[i])[0..$-1]) ~ - makeTypeString!(i+1, N, T, prefix); - else { - const makeTypeString = "?" ~ makeTypeString!(i+1, N, T, prefix); - static assert(false, "Unknown type"); - } - } else { - const makeTypeString = "?" ~ makeTypeString!(i+1, N, T, prefix); - static assert(false, "Unknown type"); - } - } else - const makeTypeString = ""; -} - -template makeTypeStringForStruct(alias opt) { - const makeTypeStringForStruct = makeTypeString!(0, opt.names, opt.types, opt.name~".")[0..$-2]; -} - -/* some leftovers -template handleType(T, char[] prefix="") { - static if(is(typeof(T) == struct)) { - static if(T.type == "enum") - // another dmdfe weirdness: T.stringof == "Enum!(Type)" here, but if do - // alias T handleType;, the result.stringof is "struct Enum". - alias T handleType; - } else - alias T handleType; -} -*/ - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/docgen.d --- a/trunk/src/docgen/docgen.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,131 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.docgen; - -import docgen.graphutils.writers; -import docgen.config.configurator; -import docgen.document.latexgenerator; -import docgen.document.htmlgenerator; -import docgen.document.xmlgenerator; -import docgen.document.plaintextgenerator; - -//import dil.Settings; -import dil.SettingsLoader; - -import tango.core.Array; -import tango.text.Text; -import tango.io.Stdout; - -void usage() { - Stdout( - "Usage: docgen rootpath importpath_1 ... importpath_n outputdir" - ).newline; -} - -void main(char[][] args) { - dil.SettingsLoader.loadSettings(); - - Stdout(docgen_version).newline.newline; - - if (args.length<3) { - usage(); - return; - } - - Configurator config = new DefaultConfigurator(); - - auto options = config.getConfiguration(); - options.parser.rootPaths = [ args[1] ]; - options.parser.importPaths = args[2..$-1]; - options.outputDir = args[$-1]; - - alias DepGraph.Vertex Vertex; - alias DepGraph.Edge Edge; - - Module[] cachedModules; - DepGraph cachedGraph; - - void parser(ref Module[] modules, ref DepGraph depGraph) { - Edge[] edges; - Vertex[char[]] vertices; - - if (cachedGraph !is null) { - modules = cachedModules; - depGraph = cachedGraph; - return; - } - - int id = 1; - - Parser.loadModules( - options.parser.rootPaths, - options.parser.importPaths, - options.parser.strRegexps, - options.graph.includeUnlocatableModules, - options.parser.depth, - (char[] fqn, char[] path, Module m) { - if (m is null) { - if (fqn in vertices) { - debug Stdout.format("{} already set.\n", fqn); - return; - } - auto vertex = new Vertex(fqn, path, id++); - vertices[fqn] = vertex; - debug Stdout.format("Setting {} = {}.\n", fqn, path); - } else { - vertices[m.moduleFQN] = new Vertex(m.moduleFQN, m.filePath, id++); - debug Stdout.format("Setting {} = {}.\n", m.moduleFQN, m.filePath); - } - }, - (Module imported, Module importer, bool isPublic, bool isStatic) { - debug Stdout.format("Connecting {} - {}.\n", imported.moduleFQN, importer.moduleFQN); - auto edge = vertices[imported.moduleFQN].addChild(vertices[importer.moduleFQN]); - edge.isPublic = isPublic; - edge.isStatic = isStatic; - edges ~= edge; - }, - modules - ); - - modules.sort( - (Module a, Module b) { return ((new Text!(char)(a.moduleFQN)).compare(b.moduleFQN)) < 0; } - ); - - depGraph.edges = edges; - depGraph.vertices = vertices.values; - - cachedGraph = depGraph; - cachedModules = modules; - } - - GraphCache graphcache = new DefaultGraphCache(); - - foreach(format; options.outputFormats) { - DocGenerator generator; - - switch(format) { - case DocFormat.LaTeX: - Stdout("Generating LaTeX docs.."); - generator = new LaTeXDocGenerator(*options, &parser, graphcache); - break; - case DocFormat.HTML: - Stdout("Generating HTML docs.."); - generator = new HTMLDocGenerator(*options, &parser, graphcache); - break; - case DocFormat.XML: - Stdout("Generating XML docs.."); - generator = new XMLDocGenerator(*options, &parser); - break; - case DocFormat.PlainText: - Stdout("Generating plain text docs.."); - generator = new PlainTextDocGenerator(*options, &parser, graphcache); - break; - default: throw new Exception("Format not supported"); - } - - generator.generate(); - Stdout("done.").newline; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/document/generator.d --- a/trunk/src/docgen/document/generator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,147 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.document.generator; - -import docgen.misc.misc; -import docgen.misc.parser; -public import docgen.misc.options; -import docgen.page.writers; -import docgen.moduledoc.writers; -import docgen.graphutils.writers; -import docgen.sourcelisting.writers; -import docgen.config.configurator; -import tango.io.stream.FileStream; -import tango.io.FilePath; -import tango.io.FileScan; -debug import tango.io.Stdout; - - -alias void delegate(ref Module[], ref DepGraph) ParserDg; - -abstract class DefaultDocGenerator : DocGenerator { - protected: - - DocFormat docFormat; - auto makeFile = "make.sh"; - char[] genDir; - - DocGeneratorOptions m_options; - ParserDg m_parser; - PageWriter docWriter; - - GraphWriterFactory graphFactory; - PageWriterFactory pageFactory; - ListingWriterFactory listingFactory; - ModuleDocWriterFactory moduleDocFactory; - - Module[] modules; - DepGraph depGraph; - - public: - - this(DocGeneratorOptions options, ParserDg parser) { - m_options = options; - m_parser = parser; - - createGraphWriterFactory(); - createPageWriterFactory(); - createListingWriterFactory(); - createModuleDocWriterFactory(); - - // create output dir - (new FilePath(options.outputDir ~ "/" ~ genDir)).create(); - - // copy static files - copyStaticContent(); - } - - DocGeneratorOptions *options() { - return &m_options; - } - - protected: - - void createGraphWriterFactory() { - graphFactory = new DefaultGraphWriterFactory(this); - } - - void createPageWriterFactory() { - pageFactory = new DefaultPageWriterFactory(this); - } - - void createListingWriterFactory() { - listingFactory = new DefaultListingWriterFactory(this); - } - - void createModuleDocWriterFactory() { - moduleDocFactory = new DefaultModuleDocWriterFactory(this); - } - - char[] outPath(char[] file) { - return options.outputDir ~ "/" ~ genDir ~ "/" ~ file; - } - - void copyStaticContent() { - auto scan = new FileScan(); - scan(templateDir~options.templates.templateStyle~"/"~formatDirs[docFormat]~"/static/"); - - foreach(filePath; scan.files) { - (new FilePath(outPath(filePath.file))).copy(filePath.toString()); - } - - debug Stdout(scan.files.length)(" static files copied.\n"); - } - - FileOutput outputFile(char[] fname) { - return new FileOutput(outPath(fname)); - } - - void parseSources() { - depGraph = new DepGraph(); - m_parser(modules, depGraph); - } - - //--- - - void writeSimpleFile(char[] fname, void delegate() dg) { - auto docFile = outputFile(fname); - - docWriter.setOutput([docFile]); - dg(); - - docFile.close(); - } - - /** - * Generates "makefile" for processing e.g. .dot files. - */ - void generateMakeFile(char[][] args ...) { - writeSimpleFile(makeFile, { docWriter.generateCustomPage("makefile", args); } ); - } - -} - -abstract class DefaultCachingDocGenerator : DefaultDocGenerator, CachingDocGenerator { - private: - - GraphCache m_graphCache; - - public: - - this(DocGeneratorOptions options, ParserDg parser, GraphCache graphCache) { - super(options, parser); - m_graphCache = graphCache; - } - - GraphCache graphCache() { - return m_graphCache; - } - - protected: - - void createGraphWriterFactory() { - graphFactory = new DefaultCachingGraphWriterFactory(this); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/document/htmlgenerator.d --- a/trunk/src/docgen/document/htmlgenerator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,156 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.document.htmlgenerator; - -import docgen.document.generator; -import docgen.misc.misc; -import tango.io.stream.FileStream; -import tango.text.Util : replace; - -class HTMLDocGenerator : DefaultCachingDocGenerator { - private: - - auto docFileNames = [ - "index.html"[], "toc.html"[], "classes.html"[], - "modules.html"[], "files.html"[] - ]; - - auto depGraphFile = "depgraph.dot"; - auto depGraphDocFile = "depgraph.html"; - auto styleSheetFile = "default.css"; - - public: - - this(DocGeneratorOptions options, ParserDg parser, GraphCache graphcache) { - genDir = "html"; - docFormat = DocFormat.HTML; - super(options, parser, graphcache); - } - - /** - * Generates the documentation. - */ - void generate() { - parseSources(); - - docWriter = pageFactory.createPageWriter( null, docFormat ); - - // stylesheet needs to be created first to propagate the css file name - generateStyleSheet(); - - generateDoc(); - - if (options.listing.enableListings) - generateListings(); - - generateClasses(); - generateModules(); - generateDependencies(); - generateMakeFile(imageFormatExts[options.graph.imageFormat]); - } - - protected: - - /** - * Generates document skeleton. - */ - void generateDoc() { - writeSimpleFile(docFileNames[0], { - docWriter.generateFirstPage(); - }); - /* - writeSimpleFile(docFileNames[1], { - docWriter.generateTOC(modules); - docWriter.generateCustomPage("pagetemplate2", docgen_version); - });*/ - } - - /** - * Generates a global style sheet. - */ - void generateStyleSheet() { - writeSimpleFile(styleSheetFile, { - docWriter.generateCustomPage("stylesheet"); - }); - } - - /** - * Generates documentation for classes. - */ - void generateClasses() { - writeSimpleFile(docFileNames[2], { - docWriter.generateClassSection(); - docWriter.generateCustomPage("pagetemplate2", docgen_version); - }); - } - - /** - * Generates documentation for modules. - */ - void generateModules() { - writeSimpleFile(docFileNames[3], { - docWriter.generateModuleSection(modules); - docWriter.generateCustomPage("pagetemplate2", docgen_version); - }); - -// auto mdw = moduleDocFactory.createModuleDocWriter(docWriter, docFormat); - - } - - /** - * Generates source file listings. - */ - void generateListings() { - writeSimpleFile(docFileNames[4], { - docWriter.generateListingSection(modules); - - char[][] contents; - - contents ~= "("; - - foreach(mod; modules) { - auto FQN = mod.moduleFQN; - auto dstFname = replace(mod.moduleFQN.dup, '.', '_') ~ ".html"; - contents ~= `` ~ FQN ~ ""; - } - - contents ~= ")"; - - docWriter.addList(contents, false); - - docWriter.generateCustomPage("pagetemplate2", docgen_version); - }); - - auto writer = listingFactory.createListingWriter(docWriter, docFormat); - - foreach(mod; modules) { - auto dstFname = replace(mod.moduleFQN.dup, '.', '_') ~ ".html"; - - writeSimpleFile(dstFname, { - auto srcFile = new FileInput(mod.filePath); - writer.generateListing(srcFile, null, mod.moduleFQN); - srcFile.close(); - }); - } - } - - /** - * Generates dependency graphs. - */ - void generateDependencies() { - writeSimpleFile(depGraphDocFile, { - docWriter.generateDepGraphSection(); - - auto imgFile = outputFile(depGraphFile); - - auto writer = graphFactory.createGraphWriter( docWriter, GraphFormat.Dot ); - writer.generateDepGraph(depGraph, imgFile); - - imgFile.close(); - - docWriter.generateCustomPage("pagetemplate2", docgen_version); - }); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/document/latexgenerator.d --- a/trunk/src/docgen/document/latexgenerator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,121 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.document.latexgenerator; - -import docgen.document.generator; -import docgen.misc.misc; -import tango.io.stream.FileStream; -import tango.text.Util : replace; - -/** - * Main routine for LaTeX doc generation. - */ -class LaTeXDocGenerator : DefaultCachingDocGenerator { - private: - - auto docFileName = "document.tex"; - auto depGraphDocFile = "dependencies.tex"; - auto depGraphFile = "depgraph.dot"; - auto listingFile = "files.tex"; - auto modulesFile = "modules.tex"; - - public: - - this(DocGeneratorOptions options, ParserDg parser, GraphCache graphcache) { - genDir = "latex"; - docFormat = DocFormat.LaTeX; - - super(options, parser, graphcache); - } - - /** - * Generates the documentation. - */ - void generate() { - parseSources(); - - generateDoc(); - - if (options.listing.enableListings) - generateListings(); - - generateClasses(); - generateModules(); - generateDependencies(); - generateMakeFile(docFileName, "pdf"); - } - - protected: - - /** - * Generates document skeleton. - */ - void generateDoc() { - auto docFile = outputFile(docFileName); - docWriter = pageFactory.createPageWriter( [ docFile ], docFormat ); - - docWriter.generateFirstPage(); - docWriter.generateTOC(modules); - docWriter.generateClassSection(); - docWriter.generateModuleSection(modules); - docWriter.generateListingSection(modules); - docWriter.generateDepGraphSection(); - docWriter.generateIndexSection(); - docWriter.generateLastPage(); - - docFile.close(); - } - - /** - * Generates documentation for classes. - */ - void generateClasses() { - auto docFile = outputFile(modulesFile); - docFile.close(); - } - - /** - * Generates documentation for modules. - */ - void generateModules() { - auto docFile = outputFile(modulesFile); - docFile.close(); - } - - /** - * Generates source file listings. - */ - void generateListings() { - writeSimpleFile(listingFile, { - auto writer = listingFactory.createListingWriter(docWriter, docFormat); - - foreach(mod; modules) { - auto dstFname = replace(mod.moduleFQN.dup, '.', '_') ~ ".d"; - - auto srcFile = new FileInput(mod.filePath); - auto dstFile = outputFile(dstFname); - - writer.generateListing(srcFile, dstFile, mod.moduleFQN); - - srcFile.close(); - dstFile.close(); - } - }); - } - - /** - * Generates dependency graphs. - */ - void generateDependencies() { - writeSimpleFile(depGraphDocFile, { - auto imgFile = outputFile(depGraphFile); - - auto writer = graphFactory.createGraphWriter( docWriter, GraphFormat.Dot ); - writer.generateDepGraph(depGraph, imgFile); - - imgFile.close(); - }); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/document/plaintextgenerator.d --- a/trunk/src/docgen/document/plaintextgenerator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,121 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.document.plaintextgenerator; - -import docgen.document.generator; -import docgen.misc.misc; -import tango.io.stream.FileStream; -import tango.io.FilePath; -import tango.text.Util : replace; - -class PlainTextDocGenerator : DefaultCachingDocGenerator { - private: - - auto docFileNames = [ - "index.txt"[], "toc.txt"[], "classes.txt"[], - "modules.txt"[], "files.txt"[] - ]; - - auto depGraphFile = "depgraph.dot"; - auto depGraphDocFile = "depgraph.txt"; - - public: - - this(DocGeneratorOptions options, ParserDg parser, GraphCache graphcache) { - genDir = "txt"; - docFormat = DocFormat.PlainText; - - super(options, parser, graphcache); - } - - /** - * Generates the documentation. - */ - void generate() { - parseSources(); - - docWriter = pageFactory.createPageWriter( null, docFormat ); - - generateDoc(); - - if (options.listing.enableListings) - generateListings(); - - generateClasses(); - generateModules(); - generateDependencies(); - generateMakeFile(imageFormatExts[options.graph.imageFormat]); - } - - protected: - - /** - * Generates document skeleton. - */ - void generateDoc() { - writeSimpleFile(docFileNames[0], { - docWriter.generateFirstPage(); - }); - - writeSimpleFile(docFileNames[1], { - docWriter.generateTOC(modules); - }); - } - - /** - * Generates documentation for classes. - */ - void generateClasses() { - writeSimpleFile(docFileNames[2], { - docWriter.generateClassSection(); - }); - } - - /** - * Generates documentation for modules. - */ - void generateModules() { - writeSimpleFile(docFileNames[3], { - docWriter.generateModuleSection(modules); - }); - } - - /** - * Generates source file listings. - */ - void generateListings() { - writeSimpleFile(docFileNames[4], { - docWriter.generateListingSection(modules); - - char[][] contents; - - foreach(mod; modules) { - auto FQN = mod.moduleFQN; - contents ~= FQN ~ " (see " ~ replace(FQN.dup, '.', '_') ~ ".d)"; - } - - docWriter.addList(contents, false); - }); - - foreach(mod; modules) - (new FilePath(outPath(replace(mod.moduleFQN.dup, '.', '_') ~ ".d"))).copy(mod.filePath); - } - - /** - * Generates dependency graphs. - */ - void generateDependencies() { - writeSimpleFile(depGraphDocFile, { - docWriter.generateDepGraphSection(); - - auto imgFile = outputFile(depGraphFile); - - auto writer = graphFactory.createGraphWriter( docWriter, GraphFormat.Dot ); - writer.generateDepGraph(depGraph, imgFile); - - imgFile.close(); - }); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/document/xmlgenerator.d --- a/trunk/src/docgen/document/xmlgenerator.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,23 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.document.xmlgenerator; - -import docgen.document.generator; -import docgen.misc.misc; -import tango.io.stream.FileStream; -import tango.text.Util : replace; - -class XMLDocGenerator : DefaultDocGenerator { - public: - - this(DocGeneratorOptions options, ParserDg parser) { - genDir = "xml"; - docFormat = DocFormat.XML; - - super(options, parser); - } - - void generate() { /* TODO */ } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/graphutils/dotwriter.d --- a/trunk/src/docgen/graphutils/dotwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,170 +0,0 @@ -/** - * Author: Aziz Köksal & Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.graphutils.dotwriter; -import docgen.graphutils.writer; - -import tango.io.Print: Print; -import tango.text.convert.Layout : Layout; -import tango.io.FilePath; -import tango.text.Util; -import tango.text.convert.Sprint; -debug import tango.io.Stdout; - -/** - * Creates a graph rule file for the dot utility. - */ -class DotWriter : AbstractGraphWriter { - public: - - this(GraphWriterFactory factory, PageWriter writer) { - super(factory, writer); - } - - void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { - generateImageTag(imageFile); - - auto image = generateDepImageFile(depGraph); - auto printer = new Print!(char)(new Layout!(char), imageFile); - printer(image); - } - - protected: - - char[] generateDepImageFile(DepGraph depGraph) { - char[] image; - auto sprint = new Sprint!(char); - - auto edges = depGraph.edges; - auto vertices = depGraph.vertices; - - DepGraph.Vertex[][char[]] verticesByPckgName; - if (factory.options.graph.groupByFullPackageName || - factory.options.graph.groupByPackageNames) { - foreach (mod; vertices) { - auto parts = mod.name.delimit("."); - - if (parts.length>1) { - auto pkg = parts[0..$-1].join("."); - verticesByPckgName[pkg] ~= mod; - } - } - } - - if (factory.options.graph.highlightCyclicVertices || - factory.options.graph.highlightCyclicEdges) - depGraph.markCycles(); - - image ~= "Digraph ModuleDependencies {\n"; - - foreach (module_; vertices) { - auto nodeName = - factory.options.graph.groupByPackageNames ? - module_.name.split(".")[$-1] : - module_.name; - - image ~= sprint.format( - ` n{} [label="{}",style=filled,fillcolor={}];`\n, - module_.id, - nodeName, - module_.cyclic && factory.options.graph.highlightCyclicVertices ? - factory.options.graph.cyclicNodeColor : - module_.incoming.length == 0 && module_.outgoing.length == 0 ? - factory.options.graph.unlocatableNodeColor : - factory.options.graph.nodeColor - ); - } - - foreach (edge; edges) - image ~= sprint.format( - ` n{} -> n{}[color={}];`\n, - edge.outgoing.id, - edge.incoming.id, - edge.cyclic ? - factory.options.graph.cyclicDepColor : - edge.isPublic ? - factory.options.graph.publicDepColor ~ ",style=bold" : - factory.options.graph.depColor - ); - - if (factory.options.graph.groupByPackageNames) - - if (!factory.options.graph.groupByFullPackageName) { - foreach (packageName, vertices; verticesByPckgName) { - auto name = packageName.split("."); - - if (name.length > 1) { - char[] pkg; - foreach(part; name) { - pkg ~= part ~ "."; - image ~= sprint.format( - `subgraph "cluster_{0}" {{`\n` label="{0}"`\n, - pkg[0..$-1], - pkg[0..$-1] - ); - } - for (int i=0; i< name.length; i++) { - image ~= "}\n"; - } - } - } - } - foreach (packageName, vertices; verticesByPckgName) { - image ~= sprint.format( - ` subgraph "cluster_{0}" {{`\n` label="{0}";color=` - ~ factory.options.graph.clusterColor ~ `;`\n` `, - packageName, - packageName - ); - - foreach (module_; vertices) - image ~= sprint.format(`n{0};`, module_.id); - image ~= "\n }\n"; - } - - image ~= "}"; - - return image; - } - - void generateImageTag(OutputStream imageFile) { - // name of the .dot file - char[] fn = (cast(Object)imageFile.conduit).toString(); - fn = FilePath(fn).file; - - fn = fn[0..$-3] ~ imageFormatExts[factory.options.graph.imageFormat]; - - writer.addGraphics(fn); - } -} - -class CachingDotWriter : DotWriter { - private: - - CachingGraphWriterFactory factory; - - public: - - this(CachingGraphWriterFactory factory, PageWriter writer) { - super(factory, writer); - this.factory = factory; - } - - override void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { - generateImageTag(imageFile); - - auto cached = factory.graphCache.getCachedGraph(depGraph, GraphFormat.Dot); - - auto printer = new Print!(char)(new Layout!(char), imageFile); - - if (cached) { - printer(cached); - } else { - auto image = generateDepImageFile(depGraph); - factory.graphCache.setCachedGraph(depGraph, GraphFormat.Dot, image); - printer(image); - } - } -} - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/graphutils/modulenamewriter.d --- a/trunk/src/docgen/graphutils/modulenamewriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,42 +0,0 @@ -/** - * Author: Aziz Köksal & Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.graphutils.modulenamewriter; -import docgen.graphutils.writer; - -import tango.io.Print: Print; -import tango.text.convert.Layout : Layout; - -class ModuleNameWriter : AbstractGraphWriter { - public: - - this(GraphWriterFactory factory, PageWriter writer) { - super(factory, writer); - } - - void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { - char[][] contents; - - auto edges = depGraph.edges; - auto vertices = depGraph.vertices; - - void doList(DepGraph.Vertex[] v, uint level) { - if (!level) return; - - contents ~= "("; - - foreach (vertex; v) { - contents ~= vertex.name; - if (vertex.outgoing.length) - doList(vertex.outgoing, level-1); - } - - contents ~= ")"; - } - - doList(vertices, factory.options.graph.depth); - - writer.addList(contents, false); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/graphutils/modulepathwriter.d --- a/trunk/src/docgen/graphutils/modulepathwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,42 +0,0 @@ -/** - * Author: Aziz Köksal & Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.graphutils.modulepathwriter; -import docgen.graphutils.writer; - -import tango.io.Print: Print; -import tango.text.convert.Layout : Layout; - -class ModulePathWriter : AbstractGraphWriter { - public: - - this(GraphWriterFactory factory, PageWriter writer) { - super(factory, writer); - } - - void generateDepGraph(DepGraph depGraph, OutputStream imageFile) { - char[][] contents; - - auto edges = depGraph.edges; - auto vertices = depGraph.vertices; - - void doList(DepGraph.Vertex[] v, uint level) { - if (!level) return; - - contents ~= "("; - - foreach (vertex; v) { - contents ~= vertex.location; - if (vertex.outgoing.length) - doList(vertex.outgoing, level-1); - } - - contents ~= ")"; - } - - doList(vertices, factory.options.graph.depth); - - writer.addList(contents, false); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/graphutils/primitives.d --- a/trunk/src/docgen/graphutils/primitives.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,142 +0,0 @@ -/** - * Author: Aziz Köksal & Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.graphutils.primitives; - -/** - * Extendible graph class. Should support separation of concerns now better with mixins. - * Provides a method for cycle marking. - */ -class Graph(alias V, alias E, int capacity = 1) { - static class Edge { - Vertex outgoing, incoming; - //bool cyclic = true; - - this(Vertex o, Vertex i) { - outgoing = o; - incoming = i; - o.outgoingEdges ~= this; - i.incomingEdges ~= this; - } - - bool cyclic() { - return outgoing.cyclic && incoming.cyclic; - } - - mixin E; - } - - static class Vertex { - Edge[] incomingEdges; - Edge[] outgoingEdges; - bool cyclic = true; - - Edge addChild(Vertex v) { - return new Edge(v, this); - } - - Edge addParent(Vertex v) { - return v.addChild(this); - } - - Vertex[] incoming() { - Vertex[] tmp; - - foreach(edge; incomingEdges) - tmp ~= edge.outgoing; - - return tmp; - } - - Vertex[] outgoing() { - Vertex[] tmp; - - foreach(edge; outgoingEdges) - tmp ~= edge.incoming; - - return tmp; - } - - mixin V; - } - - Vertex[] vertices; - Edge[] edges; - - this() { - vertices.length = capacity; - vertices.length = 0; - edges.length = capacity; - edges.length = 0; - } - - void add(Vertex vertex) { vertices ~= vertex; } - - void add(Edge edge) { edges ~= edge; } - - void connect(Vertex from, Vertex to) { edges ~= from.addParent(to); } - - void connect(int from, int to) { connect(vertices[from], vertices[to]); } - - /** - * Starts from non-cyclic nodes and propagates two both directions. - * Bugs: marks non-cyclic imports between two cycles as cyclic. Could be fixed later if it's really needed (slow) - */ - void markCycles() { - void mark(Vertex v) { - v.cyclic = false; - foreach(o; v.outgoing) { - if (!o.cyclic) continue; - - // propagate - bool cyclic = false; - foreach(p; o.incoming) if (p.cyclic) { cyclic = true; break; } - if (!cyclic) mark(o); - } - } - - void mark2(Vertex v) { - v.cyclic = false; - foreach(o; v.incoming) { - if (!o.cyclic) continue; - - // propagate - bool cyclic = false; - foreach(p; o.outgoing) if (p.cyclic) { cyclic = true; break; } - if (!cyclic) mark2(o); - } - } - - foreach(e; vertices) - if (e.cyclic) { - if (!e.incoming.length) mark(e); - if (!e.outgoing.length) mark2(e); - } - } -} - -template Empty() {} - - -// graph elements used in dep graphs - - -template DepEdge() { - bool isPublic; /// Public import. - bool isStatic; /// Static import. -} - -template DepVertex() { - char[] name; - char[] location; - uint id; - - this(char[] name, char[] location, uint id = 0) { - this.name = name; - this.location = location; - this.id = id; - } -} - -alias Graph!(DepVertex, DepEdge, 100) DepGraph; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/graphutils/writer.d --- a/trunk/src/docgen/graphutils/writer.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,113 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.graphutils.writer; - -public import docgen.misc.misc; -public import docgen.graphutils.primitives; -public import docgen.page.writer; -debug import tango.io.Stdout; - -interface GraphWriter { - void generateDepGraph(DepGraph depGraph, OutputStream imageFile); -} - -interface GraphWriterFactory : WriterFactory { - GraphWriter createGraphWriter(PageWriter writer, GraphFormat outputFormat); -} - -interface CachingGraphWriterFactory : GraphWriterFactory { - GraphCache graphCache(); -} -/+ -/** - * Marks all cycles in the graph. - * - * May have bugs, but is a bit simpler than the previous version. - */ -void findCycles(Vertex[] vertices, Edge[] edges) { - debug void p() { - foreach(e; edges) Stderr(e.type)(" "c); - Stderr.newline; - } - - bool visit(Edge edge) { - if (edge.cycleType == CycleType.Reserved) { - edge.cycleType = CycleType.Cyclic; - version(VerboseDebug) p(); - return true; - } - - bool wasCyclic = edge.isCyclic(); - edge.cycleType = CycleType.Reserved; - version(VerboseDebug) p(); - - foreach(edge2; edge.incoming.outgoingEdges) - if (visit(edge2)) { - if (edge.isCyclic()) { - edge.cycleType = CycleType.Reserved; - wasCyclic = true; - version(VerboseDebug) p(); - continue; - } - edge.cycleType = CycleType.Cyclic; - return true; - } - - edge.cycleType = wasCyclic ? CycleType.Cyclic : CycleType.Cyclefree; - version(VerboseDebug) p(); - return false; - } - - foreach(vertex; vertices) - foreach(edge; vertex.outgoingEdges) - if (edge.cycleType == CycleType.Unspecified) { - visit(edge); - debug Stderr("*\n"); - } -} -+/ - -abstract class AbstractGraphWriter : AbstractWriter!(GraphWriterFactory), GraphWriter { - protected: - - PageWriter writer; - - public: - - this(GraphWriterFactory factory, PageWriter writer) { - super(factory); - this.writer = writer; - } -} - -class DefaultGraphCache : GraphCache { - private: - - char[][Object][GraphFormat] m_graphCache; - - public: - - char[] getCachedGraph(Object graph, GraphFormat format) { - debug Stdout("Starting graph lookup\n"); - debug Stdout(&graph, format).newline; - debug Stdout(&m_graphCache).newline; - - auto lookup1 = format in m_graphCache; - if (lookup1) { - auto lookup2 = graph in *lookup1; - if (lookup2) { - return *lookup2; - } - } - debug Stdout("Graph cache miss!\n"); - return null; - } - - void setCachedGraph(Object graph, GraphFormat format, char[] - contents) { - m_graphCache[format][graph] = contents; - debug Stdout("Graph cache updated!\n"); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/graphutils/writers.d --- a/trunk/src/docgen/graphutils/writers.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,59 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.graphutils.writers; - -public import docgen.graphutils.writer; -import docgen.graphutils.dotwriter; -import docgen.graphutils.modulepathwriter; -import docgen.graphutils.modulenamewriter; - -class DefaultGraphWriterFactory : AbstractWriterFactory, GraphWriterFactory { - public: - - this(DocGenerator generator) { - super(generator); - } - - GraphWriter createGraphWriter(PageWriter writer, GraphFormat outputFormat) { - switch (outputFormat) { - case GraphFormat.Dot: - return new DotWriter(this, writer); - case GraphFormat.ModuleNames: - return new ModuleNameWriter(this, writer); - case GraphFormat.ModulePaths: - return new ModulePathWriter(this, writer); - default: - throw new Exception("Graph writer type does not exist!"); - } - } -} - -class DefaultCachingGraphWriterFactory : AbstractWriterFactory, CachingGraphWriterFactory { - public: - - CachingDocGenerator generator; - - this(CachingDocGenerator generator) { - super(generator); - this.generator = generator; - } - - GraphCache graphCache() { - return generator.graphCache; - } - - override GraphWriter createGraphWriter(PageWriter writer, GraphFormat outputFormat) { - switch (outputFormat) { - case GraphFormat.Dot: - return new CachingDotWriter(this, writer); - case GraphFormat.ModuleNames: - return new ModuleNameWriter(this, writer); - case GraphFormat.ModulePaths: - return new ModulePathWriter(this, writer); - default: - throw new Exception("Graph writer type does not exist!"); - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/lstlang0.sty --- a/trunk/src/docgen/lstlang0.sty Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,21 +0,0 @@ -%% -%% D definition (c) 2007 Jari-Matti Mäkelä -%% -\lst@definelanguage{D}% - {morekeywords={abstract,alias,align,asm,assert,auto,body,bool,break,% - byte,case,cast,catch,cdouble,cent,cfloat,char,class,const,continue,% - creal,dchar,debug,default,delegate,delete,deprecated,do,double,% - else,enum,export,extern,false,final,finally,float,for,foreach,% - foreach_reverse,function,goto,idouble,if,ifloat,import,in,inout,% - int,interface,invariant,ireal,is,lazy,long,macro,mixin,module,new,% - null,out,override,package,pragma,private,protected,public,real,ref,% - return,scope,short,static,struct,super,switch,synchronized,template,% - this,throw,true,try,typedef,typeid,typeof,ubyte,ucent,uint,ulong,% - union,unittest,ushort,version,void,volatile,wchar,while,with},% - sensitive,% - morecomment=[s]{/*}{*/},% - morecomment=[n]{/+}{+/},% - morecomment=[l]//,% - morestring=[b]",% - morestring=[b]`% - }[keywords,comments,strings]% \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/misc/meta.d --- a/trunk/src/docgen/misc/meta.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,98 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.misc.meta; - -/// tuple literal workaround -template Tuple(T...) { alias T Tuple; } - -/// another tuple literal workaround (can be nested & avoids at least one dmdfe bug) -struct STuple(T...) { alias T tuple; } - - -// (a -> b), [a] -> [b] -template map(alias S, T...) { - static if (T.length) - alias Tuple!(S!(T[0]), map!(S, T[1..$])) map; - else - alias T map; -} - -/// (a -> Bool), [a] -> [a] -template filter(alias S, T...) { - static if (!T.length) - alias Tuple!() filter; - else static if (S!(T[0])) - alias Tuple!(T[0], filter!(S, T[1..$])) filter; - else - alias filter!(S, T[1..$]) filter; -} - -/// Int -> Bool -template odd(int T) { - const odd = T%2 == 1; -} - -/// Int -> Bool -template even(int T) { - const even = !odd!(T); -} - -/// a [a] -> a -- max x y = max2 x (max y) -T max(T, U...)(T a, U b) { - static if (b.length) - return a > max(b) ? a : max(b); - else - return a; -} - -/// a [a] -> a -- min x y = min2 x (min y) -T min(T, U...)(T a, U b) { - static if (b.length) - return a < min(b) ? a : min(b); - else - return a; -} - -/// Upcasts derivatives of B to B -template UpCast(B, T) { alias T UpCast; } -template UpCast(B, T : B) { alias B UpCast; } - -/// converts integer to ascii, base 10 -char[] itoa(int i) { - char[] ret; - auto numbers = "0123456789ABCDEF"; - - do { - ret = numbers[i%10] ~ ret; - i /= 10; - } while (i) - - return ret; -} - -/// Enum stuff - -template _genList(char[] pre, char[] post, T...) { - static if (T.length) - const _genList = pre ~ T[0] ~ post ~ (T.length>1 ? "," : "") ~ - _genList!(pre, post, T[1..$]); - else - const _genList = ``; -} - -/** - * Creates - * - a typedef for enum (workaround for .tupleof.stringof) - * - the enum structure - * - string array of enum items (for runtime programming) - * - string tuple of enum items (for metaprogramming - char[][] doesn't work) - */ -template createEnum(char[] tName, char[] eName, char[] arName, char[] alName, T...) { - const createEnum = - "typedef int " ~ tName ~ ";" ~ - "enum " ~ eName ~ ":" ~ tName ~ "{" ~ _genList!("", "", T) ~ "};" ~ - "char[][] " ~ arName ~ "=[" ~ _genList!(`"`, `"[]`, T) ~ "];" ~ - "alias STuple!(" ~ _genList!(`"`, `"`, T) ~ ") " ~ alName ~ ";"; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/misc/misc.d --- a/trunk/src/docgen/misc/misc.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,61 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.misc.misc; -import docgen.misc.options; -import tango.io.model.IConduit : OutputStream; - -char[] docgen_version = "Dil document generator 0.1"; - -interface DocGenerator { - DocGeneratorOptions *options(); - void generate(); -} - -interface GraphCache { - char[] getCachedGraph(Object graph, GraphFormat format); - void setCachedGraph(Object graph, GraphFormat format, char[] contents); -} - -interface CachingDocGenerator : DocGenerator { - GraphCache graphCache(); -} - -interface WriterFactory { - DocGeneratorOptions *options(); -} - -abstract class AbstractWriterFactory : WriterFactory { - protected DocGenerator generator; - - public: - - DocGeneratorOptions *options() { - return generator.options; - } - - this(DocGenerator generator) { - this.generator = generator; - } -} - - -template AbstractWriter(T, int n = 0) { - abstract class AbstractWriter { - protected T factory; - protected OutputStream[] outputs; - - static if (n > 0) { - this(T factory, OutputStream[] outputs) { - this.factory = factory; - this.outputs = outputs; - assert(outputs.length == n, "Incorrect number of outputs"); - } - } - - this(T factory) { - this.factory = factory; - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/misc/options.d --- a/trunk/src/docgen/misc/options.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,112 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.misc.options; - -import docgen.misc.meta; - -/** creates reflective enums, syntax: enum name + list of elements */ -template optionEnum(char[] name, T...) { - const optionEnum = createEnum!("_" ~ name, name, "__" ~ name, "___" ~ name, T); -} - -/** Supported document output formats. */ -mixin(optionEnum!("DocFormat", "LaTeX", "XML", "HTML", "PlainText")); - -/** - * Supported comment formats. - * - * http://www.stack.nl/~dimitri/doxygen/docblocks.html - * http://www.digitalmars.com/d/ddoc.html - */ -mixin(optionEnum!("CommentFormat", "Ddoc", "Doxygen")); - -/** Supported image formats. */ -mixin(optionEnum!("ImageFormat", "PNG", "SVG", "GIF", "PDF")); - -/** Image format extensions. */ -const imageFormatExts = [ "png", "svg", "gif", "pdf" ]; - -/** Supported graph writers. */ -mixin(optionEnum!("GraphFormat", "Dot", "ModuleNames", "ModulePaths")); - -struct GraphOptions { - /// image format to use for graphs - ImageFormat imageFormat; - /// maximum depth of dependencies in graphs - uint depth; - /// color of normal modules - char[] nodeColor; - /// color of the modules in cyclic dep relation - char[] cyclicNodeColor; - /// unlocatable module color - char[] unlocatableNodeColor; - /// color of the dependencies - char[] depColor; - /// color of the dependencies in cyclic dep relation - char[] cyclicDepColor; - /// color of the public dependencies - char[] publicDepColor; - /// package color - char[] clusterColor; - /// include unlocatable modules to the dep graph - bool includeUnlocatableModules; - /// highlight imports in cyclic dep relation - bool highlightCyclicEdges; - /// highlight modules in cyclic dep relation - bool highlightCyclicVertices; - /// group modules by package names in dep graph - bool groupByPackageNames; - /// group modules hierarchically or by full package name - bool groupByFullPackageName; -} - -struct ListingOptions { - /// use literate programming symbols [LaTeX] - bool literateStyle; - /// enable source code listings - bool enableListings; -} - -struct TemplateOptions { - /// project title - char[] title; - /// project version - char[] versionString; - /// copyright notice - char[] copyright; - /// paper size [LaTeX] - char[] paperSize; - /// use short file names [HTML] - bool shortFileNames; - /// page template style to use, customizable via docgen/templates - char[] templateStyle; -} - -struct ParserOptions { - /// paths to search for imports - char[][] importPaths; - /// paths to "root files" - char[][] rootPaths; - /// regexps for excluding modules - char[][] strRegexps; - /// comment format [comment parser] - CommentFormat commentFormat; - /// maximum depth of dependencies - uint depth; -} - -struct DocGeneratorOptions { - /// location for the generated output - char[] outputDir; - - /// list of document formats to be generated - DocFormat[] outputFormats; - - GraphOptions graph; - ListingOptions listing; - TemplateOptions templates; - ParserOptions parser; -} - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/misc/parser.d --- a/trunk/src/docgen/misc/parser.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,210 +0,0 @@ -/** - * Author: Aziz Köksal & Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.misc.parser; - -import dil.parser.Parser; -import dil.parser.ImportParser; -import dil.File; -import dil.Settings; -public import dil.semantic.Module; -import tango.text.Regex : RegExp = Regex; -import tango.io.FilePath; -import tango.io.FileSystem; -import tango.core.Array : remove; -import tango.text.Util; -import docgen.misc.meta; -debug import tango.io.Stdout; - -alias void delegate (char[] fqn, char[] path, Module module_) modDg; -alias void delegate (Module imported, Module importer, bool isPublic, bool isStatic) importDg; - -class Parser { - protected: - -// ParserOptions m_options; - - - static char[] findModuleFilePath(char[] moduleFQNPath, char[][] importPaths) { - auto filePath = new FilePath(); - foreach (importPath; importPaths) { - filePath.set(importPath); - filePath.append(moduleFQNPath); - - foreach (moduleSuffix; [".d", ".di"/*interface file*/]) - { - filePath.suffix(moduleSuffix); - debug Stdout("Trying ")(filePath.toString()).newline; - if (filePath.exists()) - return filePath.toString(); - } - } - - debug Stdout(" * ")(moduleFQNPath)(" does not exist in imports\n")(); - return null; - } - - public: - - /** - * Imports the transitive closure of imports starting from "filePath", - * limited by recursionDepth. - * - * The search can be filtered by providing a list of regexps that match the - * FQNs of modules to be ignored. - * - * Params: - * filePath = Path of the file to parse - * importPaths = Directories to look for imports - * strRegexps = Filter regexps - * IncludeUnlocatableModules = Call the delegate also for unlocatable files - * recursionDepth = How many levels of imports to follow (-1 = no limit) - * mdg = Delegate that gets called for every module found - * idg = Delegate that gets called for every import found - * modules = List of parsed modules - */ - static void loadModules(char[] filePath, char[][] importPaths, char[][] strRegexps, - bool IncludeUnlocatableModules, int recursionDepth, - modDg mdg, importDg idg, out Module[] modules) { - - loadModules([filePath], importPaths, strRegexps, IncludeUnlocatableModules, - recursionDepth, mdg, idg, modules); - } - - /** - * Imports the transitive closure of imports starting from "filePath", - * limited by recursionDepth. - * - * The search can be filtered by providing a list of regexps that match the - * FQNs of modules to be ignored. - * - * Params: - * filePaths = Paths of the files to parse - * importPaths = Directories to look for imports - * strRegexps = Filter regexps - * IncludeUnlocatableModules = Call the delegate also for unlocatable files - * recursionDepth = How many levels of imports to follow (-1 = no limit) - * mdg = Delegate that gets called for every module found - * idg = Delegate that gets called for every import found - * modules = List of parsed modules - */ - static void loadModules(char[][] filePaths, char[][] importPaths, char[][] strRegexps, - bool IncludeUnlocatableModules, int recursionDepth, - modDg mdg, importDg idg, out Module[] modules) { - - // Initialize regular expressions. - RegExp[] regexps; - foreach (strRegexp; strRegexps) - regexps ~= new RegExp(strRegexp); - - importPaths ~= "."; - - // Add directory of file and global directories to import paths. - foreach(filePath; filePaths) { - auto fileDir = (new FilePath(filePath)).folder(); - if (fileDir.length) - importPaths ~= fileDir; - } - -// importPaths ~= GlobalSettings.importPaths; - - debug foreach(path; importPaths) { - Stdout("Import path: ")(path).newline; - } - - Module[char[]] loadedModules; - - Module loadModule(char[] moduleFQNPath, int depth) { - if (depth == 0) return null; - - debug Stdout("Loading ")(moduleFQNPath).newline; - - // Return already loaded module. - auto mod_ = moduleFQNPath in loadedModules; - if (mod_ !is null) { - debug Stdout(" Already loaded.")(moduleFQNPath).newline; - return *mod_; - } - - auto FQN = replace(moduleFQNPath.dup, dirSep, '.'); - - // Ignore module names matching regular expressions. - foreach (rx; regexps) - if (rx.test(FQN)) return null; - - auto moduleFilePath = findModuleFilePath(moduleFQNPath, importPaths); - - debug Stdout(" FQN ")(FQN).newline; - debug Stdout(" Module path ")(moduleFilePath).newline; - - Module mod = null; - - if (moduleFilePath is null) { - if (IncludeUnlocatableModules) - mdg(FQN, moduleFQNPath, null); - } else { - mod = new Module(moduleFilePath); - - // Use lightweight ImportParser. -// mod.parser = new ImportParser(loadFile(moduleFilePath), moduleFilePath); - mod.parse(); - - debug Stdout(" Parsed FQN ")(mod.getFQN()).newline; - - // autoinclude dirs (similar to Java) - // running docgen in foo/bar/abc/ also finds foo/xyz/zzz.d if module names are right - { - // foo.bar.mod -> [ "foo", "bar" ] - auto modPackage = split(mod.getFQN, ".")[0..$-1]; - auto modDir = split(FileSystem.toAbsolute(new FilePath(moduleFilePath)).standard().folder(), "/"); - auto modLocation = modDir[0..modDir.remove(".")]; - - bool matches = false; - int i; - - for(i = 1; i <= min(modPackage.length, modLocation.length); i++) { - matches = true; - debug Stdout(" Decl: ")(modPackage[$-i]).newline; - debug Stdout(" Path: ")(modLocation[$-i]).newline; - if (modPackage[$-i] != modLocation[$-i]) { - matches = false; - break; - } - } - if (matches) { - auto loc = modLocation[0..$-i+1].join("/"); - debug Stdout(" Autoadding import: ")(loc).newline; - importPaths ~= loc; - } - } - - mdg(FQN, moduleFQNPath, mod); - loadedModules[moduleFQNPath] = mod; - - foreach (importDecl; mod.imports) - foreach(moduleFQN_; importDecl.getModuleFQNs(dirSep)) { - auto loaded_mod = loadModule(moduleFQN_, depth == -1 ? depth : depth-1); - - if (loaded_mod !is null) { - idg(loaded_mod, mod, importDecl.isPublic(), importDecl.isStatic()); - } else if (IncludeUnlocatableModules) { - auto tmp = new Module(null); - tmp.setFQN(replace(moduleFQN_.dup, dirSep, '.')); - idg(tmp, mod, importDecl.isPublic(), importDecl.isStatic()); - } - } - } - - return mod; - } // loadModule - - foreach(filePath; filePaths) - loadModule(filePath, recursionDepth); - - // Finished loading modules. - - // Ordered list of loaded modules. - modules = loadedModules.values; - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/misc/textutils.d --- a/trunk/src/docgen/misc/textutils.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,38 +0,0 @@ -/** - * Author: Aziz Köksal & Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.misc.textutils; - -// copied from Generate.d -char[] xml_escape(char[] text) -{ - char[] result; - result.length = text.length; - result.length = 0; - foreach(c; text) - switch(c) - { - case '<': result ~= "<"; break; - case '>': result ~= ">"; break; - case '&': result ~= "&"; break; - default: result ~= c; - } - return result; -} - -char[] plainTextHeading(char[] s) { - char[] line; - line.length = 80; - line[] = '='; - - return s ~ \n ~ line[0..s.length].dup ~ \n ~ \n; -} - -char[] plainTextHorizLine(int l = 80) { - char[] line; - line.length = 80; - line[] = '-'; - - return line[0..l].dup ~ \n; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/moduledoc/htmlwriter.d --- a/trunk/src/docgen/moduledoc/htmlwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,38 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.moduledoc.htmlwriter; - -import docgen.moduledoc.writer; -import docgen.misc.textutils; - - -/** - * TODO - */ -class HTMLWriter : AbstractWriter!(ModuleDocWriterFactory), ModuleDocWriter { - PageWriter writer; - - this(ModuleDocWriterFactory factory, PageWriter writer) { - super(factory); - this.writer = writer; - } - - void generateModuleDoc(Module mod, OutputStream output) { - - /* - auto inputStream = cast(FileInput)input; - auto content = new char[inputStream.length]; - auto bytesRead = inputStream.read (content); - - assert(bytesRead == inputStream.length, "Error reading source file"); - assert(output == null); - - writer.addListing( - moduleName, - xml_escape(content) - );*/ - } -} - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/moduledoc/writer.d --- a/trunk/src/docgen/moduledoc/writer.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,17 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.moduledoc.writer; - -import docgen.misc.misc; -public import docgen.page.writer; -import tango.io.model.IConduit : OutputStream, InputStream; - -interface ModuleDocWriter { - void generateModuleDoc(Module mod, OutputStream output); -} - -interface ModuleDocWriterFactory : WriterFactory { - ModuleDocWriter createModuleDocWriter(PageWriter writer, DocFormat outputFormat); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/moduledoc/writers.d --- a/trunk/src/docgen/moduledoc/writers.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,30 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.moduledoc.writers; - -public import docgen.moduledoc.writer; -//import docgen.moduledoc.latexwriter; -import docgen.moduledoc.htmlwriter; -//import docgen.moduledoc.xmlwriter; - -class DefaultModuleDocWriterFactory : AbstractWriterFactory, ModuleDocWriterFactory { - this(DocGenerator generator) { - super(generator); - } - - ModuleDocWriter createModuleDocWriter(PageWriter writer, DocFormat outputFormat) { - switch (outputFormat) {/* - case DocFormat.LaTeX: - return new LaTeXWriter(this, writer); - case DocFormat.XML: - return new XMLWriter(this, writer);*/ - case DocFormat.HTML: - return new HTMLWriter(this, writer); - default: - throw new Exception("Moduledoc writer type does not exist!"); - } - } -} - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/page/htmlwriter.d --- a/trunk/src/docgen/page/htmlwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,121 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.page.htmlwriter; - -import docgen.page.writer; -import docgen.misc.textutils; -import tango.io.FileConduit : FileConduit; -import tango.text.convert.Sprint; -import tango.io.FilePath; - -/** - * Writes a HTML document skeleton. - */ -class HTMLWriter : AbstractPageWriter!("html") { - private: - - char[] styleSheetFile; - - public: - - this(PageWriterFactory factory, OutputStream[] outputs) { - super(factory); - } - - override void generateClassSection() { - print.format(getTemplate("classes"), factory.options.templates.title); - } - - override void generateModuleSection(Module[] modules) { - print.format(getTemplate("modules"), factory.options.templates.title); - } - - override void generateListingSection(Module[] modules) { - print.format(getTemplate("listings"), factory.options.templates.title); - } - - override void generateDepGraphSection() { - print.format(getTemplate("dependencies"), factory.options.templates.title); - } - - void generateFirstPage() { - print.format( - getTemplate("firstpage"), - factory.options.templates.title, - factory.options.templates.versionString, - factory.options.templates.copyright - ); - - footer(); - } - - /** - * A hack for figuring out the stylesheet file name. - */ - override void generateCustomPage(char[] name, char[][] args ...) { - super.generateCustomPage(name, args); - - if (name == "stylesheet") { - styleSheetFile = (new FilePath( - (cast(Object)outputs[0].conduit).toString())).file(); - } - } - - /** - * Overrides the default template fetcher in order to - * provide a consistent layout for all pages. - */ - override char[] getTemplate(char[] name) { - auto content = super.getTemplate(name); - - foreach(pageName; [ - "firstpage"[], "toc"[], "classes"[], "modules"[], "listing"[], - "listings"[], "dependencies"[], "lastpage"[] ]) { - if (name == pageName) { - auto sprint = new Sprint!(char)(5120); - char[] title = factory.options.templates.title ~ " "; - switch(name) { - case "firstpage": title ~= "Documentation"; break; - case "toc": title ~= "TOC"; break; - case "classes": title ~= "Class index"; break; - case "modules": title ~= "Module index"; break; - case "listing": title ~= "File contents"; break; - case "listings": title ~= "File index"; break; - case "dependencies": title ~="Dependencies"; break; - } - return - sprint.format(super.getTemplate("pagetemplate"), styleSheetFile, title) ~ - content; - } - } - - return content; - } - - void addList(char[][] contents, bool ordered) { - foreach(item; contents) { - switch(item) { - case "(": print(ordered ? "
      " : "
        "); continue; - case ")": print(ordered ? "
    " : ""); continue; - default: print("
  • ")(item)("
    • "); - } - } - } - - override void addListing(char[] moduleName, char[] contents, bool inline) { - print.format(getTemplate("listing"), moduleName, contents); - - footer(); - } - - protected: - - /** - * Writes the page footer. - */ - void footer() { - print.format(getTemplate("pagetemplate2"), docgen_version); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/page/latexwriter.d --- a/trunk/src/docgen/page/latexwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,39 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.page.latexwriter; - -import docgen.page.writer; -import tango.io.FileConduit : FileConduit; - -/** - * Writes a LaTeX document skeleton. - */ -class LaTeXWriter : AbstractPageWriter!("latex", 1) { - this(PageWriterFactory factory, OutputStream[] outputs) { - super(factory, outputs); - } - - void generateFirstPage() { - print.format( - getTemplate("firstpage"), - factory.options.templates.paperSize, - factory.options.templates.title, - factory.options.templates.versionString, - docgen_version, - timeNow(), - factory.options.listing.literateStyle ? "" : "%" - ); - } - - void addList(char[][] contents, bool ordered) { - foreach(item; contents) { - switch(item) { - case "(": print(ordered ? "\\begin{enumerate}" : "\\begin{itemize}"); continue; - case ")": print(ordered ? "\\end{enumerate}" : "\\end{itemize}"); continue; - default: print("\\item")(item)(\n); - } - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/page/plaintextwriter.d --- a/trunk/src/docgen/page/plaintextwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,61 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.page.plaintextwriter; - -import docgen.page.writer; -import docgen.misc.textutils; -import tango.io.FileConduit : FileConduit; - -/** - * Writes a plain text document skeleton. - */ -class PlainTextWriter : AbstractPageWriter!("plaintext") { - this(PageWriterFactory factory, OutputStream[] outputs) { - super(factory); - } - - override void generateTOC(Module[] modules) { - print.format(getTemplate("toc")); - } - - override void generateModuleSection(Module[] modules) { - print.format(getTemplate("modules")); - } - - override void generateListingSection(Module[] modules) { - print.format(getTemplate("listings")); - } - - void generateDepGraphSection() { - print.format(getTemplate("dependencies")); - } - - void generateFirstPage() { - print.format(getTemplate("firstpage"), - plainTextHeading(factory.options.templates.title ~ " Reference Manual"), - factory.options.templates.versionString, - docgen_version, - timeNow() - ); - } - - void addList(char[][] contents, bool ordered) { - uint[] counters; - foreach(item; contents) { - switch(item) { - case "(": counters ~= 1; continue; - case ")": counters.length = counters.length - 1; continue; - default: - if (counters.length>0) - for (int i=0; i <= counters.length; i++) - print(" "); - if (ordered) - print(++counters[$-1])(". ")(item)(\n); - else - print("* ")(item)(\n); - } - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/page/writer.d --- a/trunk/src/docgen/page/writer.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,231 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.page.writer; - -public import docgen.misc.misc; -public import docgen.misc.options; -public import docgen.misc.parser; -import tango.io.model.IConduit : OutputStream; -import tango.time.chrono.Gregorian; -import tango.text.locale.Core; -import tango.time.WallClock; -import tango.text.convert.Sprint; -import tango.io.stream.FileStream; -import tango.io.Stdout; -import tango.io.Print: Print; -import tango.text.convert.Layout : Layout; -import tango.io.FilePath; -import tango.io.FileScan; - -const templateDir = "docgen/templates/"; - -const formatDirs = [ "latex"[], "xml"[], "html"[], "plaintext"[] ]; - -/** - * Writes the logical subcomponents of a document, - * e.g. sections, embedded graphics, lists - */ -interface PageWriter { - /** - * Updates the outputstreams. - */ - void setOutput(OutputStream[] outputs); - - /** - * Generates the first page(s). - */ - void generateFirstPage(); - - /** - * Generates table of contents. - */ - void generateTOC(Module[] modules); - - /** - * Generates class documentation section. - */ - void generateClassSection(); - - /** - * Generates module documentation section. - */ - void generateModuleSection(Module[] modules); - - /** - * Generates source code listing section. - */ - void generateListingSection(Module[] modules); - - /** - * Generates dependency graph section. - */ - void generateDepGraphSection(); - - /** - * Generates an index section. - */ - void generateIndexSection(); - - /** - * Generates the last page(s). - */ - void generateLastPage(); - - /** - * Generates a page using a custom template file. - * - * Some examples: style sheet, DTD files, makefiles. - */ - void generateCustomPage(char[] name, char[][] args ...); - - // --- page components - // - /* - * Adds an external graphics file. - */ - void addGraphics(char[] imageFile); - - /** - * Adds a source code listing. - */ - void addListing(char[] moduleName, char[] contents, bool inline = true); - - /** - * Adds a list of items. - */ - void addList(char[][] contents, bool ordered); -} - -interface PageWriterFactory : WriterFactory { - PageWriter createPageWriter(OutputStream[] outputs, DocFormat outputFormat); -} - -template AbstractPageWriter(char[] format, int n = 0) { - abstract class AbstractPageWriter : AbstractWriter!(PageWriterFactory, n), PageWriter { - protected: - - char[][char[]] m_templates; - Print!(char) print; - - public: - - this(PageWriterFactory factory, OutputStream[] outputs) { - this(factory); - setOutput(outputs); - } - - void setOutput(OutputStream[] outputs) { - this.outputs = outputs; - static if (n > 0) - assert(outputs.length == n, "Incorrect number of outputs"); - - print = new Print!(char)(new Layout!(char), outputs[0]); - } - - void generateTOC(Module[] modules) { - print.format(getTemplate("toc")); - } - - void generateClassSection() { - print.format(getTemplate("classes")); - } - - void generateModuleSection(Module[] modules) { - print.format(getTemplate("modules")); - } - - void generateListingSection(Module[] modules) { - print.format(getTemplate("listings")); - } - - void generateDepGraphSection() { - print.format(getTemplate("dependencies")); - } - - void generateIndexSection() { - print.format(getTemplate("index")); - } - - void generateLastPage() { - print.format(getTemplate("lastpage")); - } - - void generateCustomPage(char[] name, char[][] args ...) { - switch(args.length) { - case 0: print.format(getTemplate(name)); break; - case 1: print.format(getTemplate(name), args[0]); break; - case 2: print.format(getTemplate(name), args[0], args[1]); break; - case 3: print.format(getTemplate(name), args[0], args[1], args[2]); break; - default: throw new Exception("Too many arguments"); - } - } - - //--- - - void addGraphics(char[] imageFile) { - print.format(getTemplate("graphics"), imageFile); - } - - void addListing(char[] moduleName, char[] contents, bool inline) { - print.format(getTemplate("listing"), moduleName, contents); - } - - protected: - - this(PageWriterFactory factory) { - super(factory); - - auto scan = new FileScan(); - scan(templateDir~factory.options.templates.templateStyle~"/"~format~"/", ".tpl"); - - debug Stdout(scan.files.length)(" template files loaded.\n"); - - foreach(tpl; scan.files) { - m_templates[tpl.name] = loadTemplate(tpl.toString()); - } - } - - char[] getTemplate(char[] name) { - auto tpl = name in m_templates; - assert(tpl, "Error: template ["~format~"/"~name~"] not found!"); - return *tpl; - } - - char[] loadTemplate(char[] fn) { - scope(failure) { - Stderr("Warning: error opening template "~fn~"."); - return null; - } - - auto file = new FileInput(fn); - auto content = new char[file.length]; - auto bytesRead = file.read(content); - - assert(bytesRead == file.length, "Error reading template"); - - file.close(); - - return content; - } - - char[] timeNow() { - auto n = WallClock.now; - auto c = Gregorian.generic; - auto d = c.toDate(n); - auto sprint = new Sprint!(char); - - auto culture = new Culture("en-GB"); - auto dateTimeFormat = culture.dateTimeFormat(); - - return sprint.format("{} {} {} {}", - dateTimeFormat.getAbbreviatedDayName(c.getDayOfWeek(n)), - 1,//d.day(), - //dateTimeFormat.getAbbreviatedMonthName(d.month()), - 2,//d.month(), - 3//d.year()) //FIXME: something is broken here (Error: function expected before (), not *(&d + 8u) of type uint) - ).dup; - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/page/writers.d --- a/trunk/src/docgen/page/writers.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,32 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.page.writers; - -public import docgen.page.writer; -import docgen.page.htmlwriter; -import docgen.page.xmlwriter; -import docgen.page.plaintextwriter; -import docgen.page.latexwriter; - -class DefaultPageWriterFactory : AbstractWriterFactory, PageWriterFactory { - this(DocGenerator generator) { - super(generator); - } - - PageWriter createPageWriter(OutputStream[] outputs, DocFormat outputFormat) { - switch (outputFormat) { - case DocFormat.LaTeX: - return new LaTeXWriter(this, outputs); - case DocFormat.XML: - return new XMLWriter(this, outputs); - case DocFormat.HTML: - return new HTMLWriter(this, outputs); - case DocFormat.PlainText: - return new PlainTextWriter(this, outputs); - default: - throw new Exception("Document writer type does not exist!"); - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/page/xmlwriter.d --- a/trunk/src/docgen/page/xmlwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,56 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.page.xmlwriter; - -import docgen.page.writer; -import docgen.misc.textutils; -import tango.io.FileConduit : FileConduit; - -//TODO: this is mostly broken now - -/** - * TODO - */ -class XMLWriter : AbstractPageWriter!("xml", 1) { - this(PageWriterFactory factory, OutputStream[] outputs) { - super(factory, outputs); - } - - void generateTOC(Module[] modules) { - // TODO - print.format(getTemplate("toc")); - } - - void generateModuleSection() { - // TODO - print.format(getTemplate("modules")); - } - - void generateListingSection() { - // TODO - print.format(getTemplate("listings")); - } - - void generateDepGraphSection() { - // TODO - print.format(getTemplate("dependencies")); - } - - void generateIndexSection() { } - - void generateLastPage() { } - - void generateFirstPage() { } - - void addList(char[][] contents, bool ordered) { - foreach(item; contents) { - switch(item) { - case "(": print(ordered ? "
      " : "
        "); continue; - case ")": print(ordered ? "
    " : ""); continue; - default: print("
  • ")(xml_escape(item))("
    • "); - } - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/sourcelisting/htmlwriter.d --- a/trunk/src/docgen/sourcelisting/htmlwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,39 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.sourcelisting.htmlwriter; - -import docgen.sourcelisting.writer; -import docgen.misc.textutils; -//import dil.Parser; -import tango.io.stream.FileStream; - - -/** - * TODO - */ -class HTMLWriter : AbstractWriter!(ListingWriterFactory), ListingWriter { - PageWriter writer; - - this(ListingWriterFactory factory, PageWriter writer) { - super(factory); - this.writer = writer; - } - - //void generateListing(Parser parser) { /* TODO */ } - - void generateListing(InputStream input, OutputStream output, char[] moduleName) { - auto inputStream = cast(FileInput)input; - auto content = new char[inputStream.length]; - auto bytesRead = inputStream.read (content); - - assert(bytesRead == inputStream.length, "Error reading source file"); - assert(output == null); - - writer.addListing( - moduleName, - xml_escape(content) - ); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/sourcelisting/latexwriter.d --- a/trunk/src/docgen/sourcelisting/latexwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,32 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.sourcelisting.latexwriter; - -import docgen.sourcelisting.writer; -//import dil.Parser; -import tango.io.FilePath; - -/** - * Adds a code listing section for the given file. - */ -class LaTeXWriter : AbstractWriter!(ListingWriterFactory), ListingWriter { - PageWriter writer; - - this(ListingWriterFactory factory, PageWriter writer) { - super(factory); - this.writer = writer; - } - - //void generateListing(Parser parser) { /* TODO */ } - - void generateListing(InputStream input, OutputStream output, char[] moduleName) { - output.copy(input); - - writer.addListing( - moduleName, - FilePath((cast(Object)output.conduit).toString()).file - ); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/sourcelisting/writer.d --- a/trunk/src/docgen/sourcelisting/writer.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,19 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.sourcelisting.writer; - -import docgen.misc.misc; -public import docgen.page.writer; -//import dil.Parser; -import tango.io.model.IConduit : OutputStream, InputStream; - -interface ListingWriter { - //void generateListing(Parser parser); - void generateListing(InputStream input, OutputStream output, char[] moduleName); -} - -interface ListingWriterFactory : WriterFactory { - ListingWriter createListingWriter(PageWriter writer, DocFormat outputFormat); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/sourcelisting/writers.d --- a/trunk/src/docgen/sourcelisting/writers.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,29 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.sourcelisting.writers; - -public import docgen.sourcelisting.writer; -import docgen.sourcelisting.latexwriter; -import docgen.sourcelisting.htmlwriter; -import docgen.sourcelisting.xmlwriter; - -class DefaultListingWriterFactory : AbstractWriterFactory, ListingWriterFactory { - this(DocGenerator generator) { - super(generator); - } - - ListingWriter createListingWriter(PageWriter writer, DocFormat outputFormat) { - switch (outputFormat) { - case DocFormat.LaTeX: - return new LaTeXWriter(this, writer); - case DocFormat.XML: - return new XMLWriter(this, writer); - case DocFormat.HTML: - return new HTMLWriter(this, writer); - default: - throw new Exception("Listing writer type does not exist!"); - } - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/sourcelisting/xmlwriter.d --- a/trunk/src/docgen/sourcelisting/xmlwriter.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,23 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.sourcelisting.xmlwriter; - -import docgen.sourcelisting.writer; -//import dil.Parser; - -/** - * TODO - */ -class XMLWriter : AbstractWriter!(ListingWriterFactory), ListingWriter { - PageWriter writer; - - this(ListingWriterFactory factory, PageWriter writer) { - super(factory); - this.writer = writer; - } - - //void generateListing(Parser parser) { /* TODO */ } - void generateListing(InputStream input, OutputStream output, char[] moduleName) { /* TODO */ } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/README --- a/trunk/src/docgen/templates/README Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,14 +0,0 @@ -The docgen uses the following template file names by default - -firstpage -toc -classes -modules -listings -dependencies -index -lastpage -langdef -makefile -graphics -listing diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/classes.tpl --- a/trunk/src/docgen/templates/default/html/classes.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -

    {0} Class List

    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/dependencies.tpl --- a/trunk/src/docgen/templates/default/html/dependencies.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -

    {} Dependencies

    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/firstpage.tpl --- a/trunk/src/docgen/templates/default/html/firstpage.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -

    {0} Documentation

    -

    {1}

    -

    © {2}

    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/graphics.tpl --- a/trunk/src/docgen/templates/default/html/graphics.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -

    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/listing.tpl --- a/trunk/src/docgen/templates/default/html/listing.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -

    {0} Contents

    -
    {1}
    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/listings.tpl --- a/trunk/src/docgen/templates/default/html/listings.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -

    {0} File List

    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/makefile.tpl --- a/trunk/src/docgen/templates/default/html/makefile.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,7 +0,0 @@ -#!/bin/sh - -for i in *.dot; do - F=`echo $i|sed 's/dot/{0}/'` - dot $i -T{0} -o$F -done - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/modules.tpl --- a/trunk/src/docgen/templates/default/html/modules.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -

    {0} Module List

    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/pagetemplate.tpl --- a/trunk/src/docgen/templates/default/html/pagetemplate.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,18 +0,0 @@ - - - - {1} - - - - - -
    - -
    diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/pagetemplate2.tpl --- a/trunk/src/docgen/templates/default/html/pagetemplate2.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,4 +0,0 @@ -
    -

    Generated by {0}

    - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/static/tab_b.gif Binary file trunk/src/docgen/templates/default/html/static/tab_b.gif has changed diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/static/tab_l.gif Binary file trunk/src/docgen/templates/default/html/static/tab_l.gif has changed diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/static/tab_r.gif Binary file trunk/src/docgen/templates/default/html/static/tab_r.gif has changed diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/static/tabs.css --- a/trunk/src/docgen/templates/default/html/static/tabs.css Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,102 +0,0 @@ -/* tabs styles, based on http://www.alistapart.com/articles/slidingdoors */ - -DIV.tabs -{ - float : left; - width : 100%; - background : url("tab_b.gif") repeat-x bottom; - margin-bottom : 4px; -} - -DIV.tabs UL -{ - margin : 0px; - padding-left : 10px; - list-style : none; -} - -DIV.tabs LI, DIV.tabs FORM -{ - display : inline; - margin : 0px; - padding : 0px; -} - -DIV.tabs FORM -{ - float : right; -} - -DIV.tabs A -{ - float : left; - background : url("tab_r.gif") no-repeat right top; - border-bottom : 1px solid #84B0C7; - font-size : x-small; - font-weight : bold; - text-decoration : none; -} - -DIV.tabs A:hover -{ - background-position: 100% -150px; -} - -DIV.tabs A:link, DIV.tabs A:visited, -DIV.tabs A:active, DIV.tabs A:hover -{ - color: #1A419D; -} - -DIV.tabs SPAN -{ - float : left; - display : block; - background : url("tab_l.gif") no-repeat left top; - padding : 5px 9px; - white-space : nowrap; -} - -DIV.tabs INPUT -{ - float : right; - display : inline; - font-size : 1em; -} - -DIV.tabs TD -{ - font-size : x-small; - font-weight : bold; - text-decoration : none; -} - - - -/* Commented Backslash Hack hides rule from IE5-Mac \*/ -DIV.tabs SPAN {float : none;} -/* End IE5-Mac hack */ - -DIV.tabs A:hover SPAN -{ - background-position: 0% -150px; -} - -DIV.tabs LI.current A -{ - background-position: 100% -150px; - border-width : 0px; -} - -DIV.tabs LI.current SPAN -{ - background-position: 0% -150px; - padding-bottom : 6px; -} - -DIV.nav -{ - background : none; - border : none; - border-bottom : 1px solid #84B0C7; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/stylesheet.tpl --- a/trunk/src/docgen/templates/default/html/stylesheet.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,33 +0,0 @@ -@import "tabs.css"; - -body {{ -background: #fff; -} - -h1, h2 {{ -text-align: center; -} - -h1 {{ -font-size: 1.5em; -} - -h2#version {{ -font-size: 0.8em; -} - -h2 {{ -font-size: 1.1em; -} - -hr {{ -height: 0; -border: 0; -border-bottom: 1px solid black; -} - -#generator {{ -text-align: right; -font-size: small; -font-style: italic; -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/html/toc.tpl --- a/trunk/src/docgen/templates/default/html/toc.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,15 +0,0 @@ - - - - {0} Reference Manual - - - - - - -

    Table of Contents

    -
    -{0} - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/classes.tpl --- a/trunk/src/docgen/templates/default/latex/classes.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -\chapter{{Class documentation} -\input{{classes} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/dependencies.tpl --- a/trunk/src/docgen/templates/default/latex/dependencies.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -\chapter{{Dependency diagram} -\input{{dependencies} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/firstpage.tpl --- a/trunk/src/docgen/templates/default/latex/firstpage.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,50 +0,0 @@ -\documentclass[{0}]{{book} -\usepackage{{a4wide} -\usepackage{{makeidx} -\usepackage{{fancyhdr} -\usepackage{{graphicx} -\usepackage{{hyperref} -\usepackage{{multicol} -\usepackage{{float} -\usepackage{{textcomp} -\usepackage{{alltt} -\usepackage[utf8]{{inputenc} -\usepackage{{listings} -\lstnewenvironment{{dcode} -{{ \lstset{{language=d} } -{{} -\lstset{{ - {5} literate= - {5} {{<=}{{{{$\leq$}}1 - {5} {{>=}{{{{$\geq$}}1 - {5} {{!=}{{{{$\neq$}}1 - {5} {{...}{{{{$\dots$}}1 - {5} {{~}{{{{$\sim$}}1, - stringstyle=\ttfamily, - inputencoding=utf8, - extendedchars=false, - columns=fixed, - basicstyle=\small -} -\hypersetup{{backref,colorlinks=true} -\makeindex -\setcounter{{tocdepth}{{1} -\newcommand{{\clearemptydoublepage}{{\newpage{{\pagestyle{{empty}\cleardoublepage}} -\def\thechapter{{\Roman{{chapter}} -\def\thesection{{\arabic{{chapter}.\arabic{{section}} -% \renewcommand{{\footrulewidth}{{0.4pt} - -\begin{{document} - -\begin{{titlepage} -\vspace*{{7cm} -\begin{{center} -{{\Large {1} Reference Manual\\[1ex]\large {2} }\\ -\vspace*{{1cm} -{{\large Generated by {3} }\\ -\vspace*{{0.5cm} -{{\small {4} }\\ -\end{{center} -\end{{titlepage} - -\clearemptydoublepage diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/graphics.tpl --- a/trunk/src/docgen/templates/default/latex/graphics.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -\includegraphics[width=1\textwidth,height=1\textheight,keepaspectratio]{{{0}} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/index.tpl --- a/trunk/src/docgen/templates/default/latex/index.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -\printindex diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/lastpage.tpl --- a/trunk/src/docgen/templates/default/latex/lastpage.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -\end{{document} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/listing.tpl --- a/trunk/src/docgen/templates/default/latex/listing.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -\section{{Module {0}} -\lstinputlisting[language=d]{{{1}} -\clearpage \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/listings.tpl --- a/trunk/src/docgen/templates/default/latex/listings.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -\chapter{{File listings} -\input{{files} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/makefile.tpl --- a/trunk/src/docgen/templates/default/latex/makefile.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,10 +0,0 @@ -#!/bin/sh - -for i in *.dot; do - F=`echo $i|sed 's/dot/{1}/'` - dot $i -T{1} -o$F -done - -pdflatex {0} -makeindex document -pdflatex {0} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/modules.tpl --- a/trunk/src/docgen/templates/default/latex/modules.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,2 +0,0 @@ -\chapter{{Module documentation} -\input{{modules} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/static/lstlang0.sty --- a/trunk/src/docgen/templates/default/latex/static/lstlang0.sty Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,22 +0,0 @@ -%% -%% D definition (c) 2007 Jari-Matti Mäkelä -%% -\lst@definelanguage{D}% - {morekeywords={abstract,alias,align,asm,assert,auto,body,bool,break,% - byte,case,cast,catch,cdouble,cent,cfloat,char,class,const,continue,% - creal,dchar,debug,default,delegate,delete,deprecated,do,double,% - else,enum,export,extern,false,final,finally,float,for,foreach,% - foreach_reverse,function,goto,idouble,if,ifloat,import,in,inout,% - int,interface,invariant,ireal,is,lazy,long,macro,mixin,module,new,% - null,out,override,package,pragma,private,protected,public,real,ref,% - return,scope,short,static,struct,super,switch,synchronized,template,% - this,throw,true,try,typedef,typeid,typeof,ubyte,ucent,uint,ulong,% - union,unittest,ushort,version,void,volatile,wchar,while,with},% - sensitive,% - - morecomment=[s]{/*}{*/},% - morecomment=[n]{/+}{+/},% - morecomment=[l]//,% - morestring=[b]",% - morestring=[b]`% - }[keywords,comments,strings]% diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/latex/toc.tpl --- a/trunk/src/docgen/templates/default/latex/toc.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6 +0,0 @@ -\tableofcontents -\thispagestyle{{empty} - -\clearemptydoublepage - -\setcounter{{page}{{1} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/classes.tpl --- a/trunk/src/docgen/templates/default/plaintext/classes.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -Class List ----------- - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/dependencies.tpl --- a/trunk/src/docgen/templates/default/plaintext/dependencies.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -Module Dependencies -------------------- - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/firstpage.tpl --- a/trunk/src/docgen/templates/default/plaintext/firstpage.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,7 +0,0 @@ -{0} - -Version {1} - -Generated by {2} - -{3} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/graphics.tpl --- a/trunk/src/docgen/templates/default/plaintext/graphics.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,1 +0,0 @@ -See {}. \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/listing.tpl --- a/trunk/src/docgen/templates/default/plaintext/listing.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -Module {0} -See {1}. - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/listings.tpl --- a/trunk/src/docgen/templates/default/plaintext/listings.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -File listings -------------- - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/makefile.tpl --- a/trunk/src/docgen/templates/default/plaintext/makefile.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,7 +0,0 @@ -#!/bin/sh - -for i in *.dot; do - F=`echo $i|sed 's/dot/{0}/'` - dot $i -T{0} -o$F -done - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/modules.tpl --- a/trunk/src/docgen/templates/default/plaintext/modules.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,3 +0,0 @@ -Module List ------------ - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/templates/default/plaintext/toc.tpl --- a/trunk/src/docgen/templates/default/plaintext/toc.tpl Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,10 +0,0 @@ -Table of Contents ------------------ - -1. Class documentation - -2. Module documentation - -3. File listings - -4. Dependency diagram diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/tests/common.d --- a/trunk/src/docgen/tests/common.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,25 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.tests.common; - -import docgen.misc.misc; -import docgen.misc.options; -import docgen.config.configurator; - -class TestDocGenerator : DocGenerator { - Configurator config; - - this() { - config = new DefaultConfigurator(); - } - - public void generate() { - - } - - public DocGeneratorOptions *options() { - return config.getConfiguration(); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/tests/doctemplate.d --- a/trunk/src/docgen/tests/doctemplate.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,30 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.tests.doctemplate; - -import docgen.tests.common; -import docgen.page.writers; -import tango.io.FileConduit; - -// doc template -//@unittest -void doctemplate1() { - auto gen = new TestDocGenerator; - auto fname = "doctemplate.tex"; - - auto gwf = new DefaultPageWriterFactory(gen); - auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); - auto writer = gwf.createPageWriter( [ file ], DocFormat.LaTeX ); - - writer.generateFirstPage(); - writer.generateTOC(null); - writer.generateModuleSection(null); - writer.generateListingSection(null); - writer.generateDepGraphSection(); - writer.generateIndexSection(); - writer.generateLastPage(); - - file.close(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/tests/graphs.d --- a/trunk/src/docgen/tests/graphs.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,160 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.tests.graphs; - -import docgen.tests.common; -import docgen.misc.parser; -import docgen.graphutils.writers; -import docgen.page.writers; -import tango.io.FileConduit; -import dil.semantic.Module; - -alias DepGraph.Edge Edge; -alias DepGraph.Vertex Vertex; - -void saveDefaultGraph(DepGraph depGraph, char[] fname) { - auto gen = new TestDocGenerator; - gen.options.graph.highlightCyclicVertices = true; - gen.options.graph.imageFormat = ImageFormat.SVG; - //gen.options.graph.graphFormat = GraphFormat.ModuleNames; - //gen.options.graph.graphFormat = GraphFormat.ModulePaths; - gen.options.graph.depth = 5; - auto ddf = new DefaultPageWriterFactory(gen); - auto gwf = new DefaultGraphWriterFactory(gen); - auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); - auto file2 = new FileConduit("docgen/teststuff/" ~ fname ~ "-2", FileConduit.WriteCreate); - - auto writer = gwf.createGraphWriter( - ddf.createPageWriter( [ file2 ], DocFormat.LaTeX), - GraphFormat.Dot - ); - - writer.generateDepGraph(depGraph, file); - - file.close(); - file2.close(); -} - -// no edges -//@unittest -void graph1() { - auto g = new DepGraph; - g.add(new Vertex("mod_a", "path.to.mod_a", 1)); - g.add(new Vertex("mod_b", "path.to.mod_b", 2)); - g.add(new Vertex("mod_c", "path.to.mod_c", 3)); - - saveDefaultGraph(g, "graph1.dot"); -} - - -// simple tree structure -//@unittest -void graph2() { - auto g = new DepGraph; - g.add(new Vertex("mod_a", "path.to.mod_a", 1)); - g.add(new Vertex("mod_b", "path.to.mod_b", 2)); - g.add(new Vertex("mod_c", "path.to.mod_c", 3)); - g.add(new Vertex("mod_d", "path.to.mod_d", 4)); - - g.connect(1, 0); - g.connect(2, 0); - g.connect(3, 2); - - saveDefaultGraph(g, "graph2.dot"); -} - -// circular imports -//@unittest -void graph3() { - auto g = new DepGraph; - g.add(new Vertex("mod_a", "path.to.mod_a", 1)); - g.add(new Vertex("mod_b", "path.to.mod_b", 2)); - g.add(new Vertex("mod_c", "path.to.mod_c", 3)); - g.add(new Vertex("mod_d", "path.to.mod_d", 4)); - - g.connect(1, 0); - g.connect(2, 1); - g.connect(0, 2); - - saveDefaultGraph(g, "graph3.dot"); -} - -// more complex graph -//@unittest -void graph4() { - auto g = new DepGraph; - g.add(new Vertex("mod_a", "path.to.mod_a", 1)); - g.add(new Vertex("mod_b", "path.to.mod_b", 2)); - g.add(new Vertex("mod_c", "path.to.mod_c", 3)); - g.add(new Vertex("mod_d", "path.to.mod_d", 4)); - g.add(new Vertex("mod_e", "path.to.mod_e", 5)); - g.add(new Vertex("mod_f", "path.to.mod_f", 6)); - g.add(new Vertex("mod_g", "path.to.mod_g", 7)); - - g.connect(1, 0); - g.connect(2, 1); - g.connect(0, 2); - g.connect(0, 3); - g.connect(0, 4); - g.connect(3, 1); - g.connect(4, 1); - g.connect(0, 6); - g.connect(5, 1); - g.connect(5, 6); - g.connect(6, 0); - - saveDefaultGraph(g, "graph4.dot"); -} - - -// parses the test modules and creates a dep graph -//@unittest -void graph5() { - auto gen = new TestDocGenerator; - gen.options.graph.highlightCyclicVertices = true; - gen.options.graph.imageFormat = ImageFormat.PDF; - gen.options.outputFormats = [ DocFormat.LaTeX ]; - auto fname = "dependencies.tex"; - auto imgFname = "depgraph.dot"; - - auto ddf = new DefaultPageWriterFactory(gen); - auto gwf = new DefaultGraphWriterFactory(gen); - auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); - auto imgFile = new FileConduit("docgen/teststuff/" ~ imgFname, FileConduit.WriteCreate); - - Module[] modules; - Edge[] edges; - Vertex[char[]] vertices; - int id = 1; - - Parser.loadModules( - [ "c" ], [ "docgen/teststuff/" ], - null, true, -1, - (char[] fqn, char[] path, Module m) { - vertices[m.moduleFQN] = new DepGraph.Vertex(m.moduleFQN, m.filePath, id++); - }, - (Module imported, Module importer, bool isPublic, bool isStatic) { - auto edge = vertices[imported.moduleFQN].addChild(vertices[importer.moduleFQN]); - edge.isPublic = isPublic; - edge.isStatic = isStatic; - edges ~= edge; - }, - modules - ); - - auto writer = gwf.createGraphWriter( - ddf.createPageWriter( [ file ], DocFormat.LaTeX ), - GraphFormat.Dot - ); - - auto graph = new DepGraph; - graph.edges = edges; - graph.vertices = vertices.values; - - writer.generateDepGraph(graph, imgFile); - - file.close(); - imgFile.close(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/tests/listing.d --- a/trunk/src/docgen/tests/listing.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,51 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.tests.listing; - -import docgen.misc.parser; -import docgen.tests.common; -import docgen.sourcelisting.writers; -import docgen.page.writers; -import tango.io.FileConduit; -import tango.text.Util; - -// doc template -//@unittest -void listing1() { - auto gen = new TestDocGenerator; - gen.options.outputFormats = [ DocFormat.LaTeX ]; - auto fname = "files.tex"; - - auto ddf = new DefaultPageWriterFactory(gen); - auto dlwf = new DefaultListingWriterFactory(gen); - auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); - - - Module[] modules; - - Parser.loadModules( - [ "c" ], [ "docgen/teststuff/" ], - null, true, -1, - (char[] fqn, char[] path, Module m) {}, - (Module imported, Module importer, bool isPublic, bool isStatic) {}, - modules - ); - - foreach(mod; modules) { - auto dstFname = replace(mod.moduleFQN.dup, '.', '_'); - - auto srcFile = new FileConduit(mod.filePath); - auto dstFile = new FileConduit("docgen/teststuff/_" ~ dstFname ~ ".d", FileConduit.WriteCreate); - auto writer = dlwf.createListingWriter( ddf.createPageWriter( [ file ], - DocFormat.LaTeX ), DocFormat.LaTeX ); - - writer.generateListing(srcFile, dstFile, mod.moduleFQN); - - srcFile.close(); - dstFile.close(); - } - - file.close(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/tests/parse.d --- a/trunk/src/docgen/tests/parse.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,65 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.tests.parse; - -import docgen.misc.parser; -import tango.io.FileConduit; -import tango.io.Print: Print; -import tango.text.convert.Layout : Layout; - -void saveToFile(char[] fname, void delegate(Print!(char) file) foo) { - auto file = new FileConduit("docgen/teststuff/" ~ fname, FileConduit.WriteCreate); - auto output = new Print!(char)(new Layout!(char), file); - - foo(output); - - file.close(); -} - -// load some test files -//@unittest -void parse1() { - saveToFile("parse1.txt", (Print!(char) file){ - Module[] modules; - - Parser.loadModules( - [ "c" ], [ "docgen/teststuff/" ], - null, true, -1, - (char[] fqn, char[] path, Module m) { - file.format("{0} = {1}\n", fqn, path); - }, - (Module imported, Module importer, bool isPublic, bool isStatic) { - file.format("{0} <- {1}\n", - imported ? imported.moduleFQN : "null"[], - importer ? importer.moduleFQN : "null"[] - ); - }, - modules - ); - }); -} - -// load the imports of dil -//@unittest -void parse2() { - saveToFile("parse2.txt", (Print!(char) file){ - Module[] modules; - - Parser.loadModules( - [ "docgen/testsuite" ], [".", "/home/jm/d/tango/"], - null, true, -1, - (char[] fqn, char[] path, Module m) { - file.format("{0} = {1}\n", fqn, path); - }, - (Module imported, Module importer, bool isPublic, bool isStatic) { - file.format("{0} <- {1}\n", - imported ? imported.moduleFQN : "null"[], - importer ? importer.moduleFQN : "null"[] - ); - }, - modules - ); - }); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/teststuff/a.d --- a/trunk/src/docgen/teststuff/a.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,4 +0,0 @@ -module a; - -void foo() {} -void bar() {} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/teststuff/b.d --- a/trunk/src/docgen/teststuff/b.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,4 +0,0 @@ -module b; - -import a; - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/teststuff/c.d --- a/trunk/src/docgen/teststuff/c.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,4 +0,0 @@ -module c; - -import a; -import b; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/teststuff/clean.sh --- a/trunk/src/docgen/teststuff/clean.sh Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,7 +0,0 @@ -rm _* -rm dep* -rm doc* -rm *.tex -rm graph* -rm parse* -touch modules.tex diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/teststuff/lstlang0.sty --- a/trunk/src/docgen/teststuff/lstlang0.sty Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,21 +0,0 @@ -%% -%% D definition (c) 2007 Jari-Matti Mäkelä -%% -\lst@definelanguage{D}% - {morekeywords={abstract,alias,align,asm,assert,auto,body,bool,break,% - byte,case,cast,catch,cdouble,cent,cfloat,char,class,const,continue,% - creal,dchar,debug,default,delegate,delete,deprecated,do,double,% - else,enum,export,extern,false,final,finally,float,for,foreach,% - foreach_reverse,function,goto,idouble,if,ifloat,import,in,inout,% - int,interface,invariant,ireal,is,lazy,long,macro,mixin,module,new,% - null,out,override,package,pragma,private,protected,public,real,ref,% - return,scope,short,static,struct,super,switch,synchronized,template,% - this,throw,true,try,typedef,typeid,typeof,ubyte,ucent,uint,ulong,% - union,unittest,ushort,version,void,volatile,wchar,while,with},% - sensitive,% - morecomment=[s]{/*}{*/},% - morecomment=[n]{/+}{+/},% - morecomment=[l]//,% - morestring=[b]",% - morestring=[b]`% - }[keywords,comments,strings]% \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/teststuff/modules.tex diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/docgen/testsuite.d --- a/trunk/src/docgen/testsuite.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,42 +0,0 @@ -/** - * Author: Jari-Matti Mäkelä - * License: GPL3 - */ -module docgen.testsuite; - -import docgen.tests.graphs; -import docgen.tests.parse; -import docgen.tests.doctemplate; -import docgen.tests.listing; -//import docgen.tests.sexp; -import tango.io.Stdout; - -/** - * A temporary test program for the docgen package. - * I'll replace this with proper unittests in the future. - * - */ -void main() { - Stdout("Running..\n")(); - - Stdout(" Test1\n")(); - graph1(); - Stdout(" Test2\n")(); - graph2(); - Stdout(" Test3\n")(); - graph3(); - Stdout(" Test4\n")(); - graph4(); - Stdout(" Test5\n")(); - graph5(); - Stdout(" Test6\n")(); - parse1(); - Stdout(" Test7\n")(); - parse2(); - Stdout(" Test8\n")(); - doctemplate1(); - Stdout(" Test9\n")(); - listing1(); -// loadConfig(); - Stdout("done.\n")(); -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/html.css --- a/trunk/src/html.css Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,70 +0,0 @@ -@charset "utf-8"; -.compilerinfo, .sourcecode, .linescolumn { - white-space: pre; - font-family: Monospace; - font-size: 0.8em; - margin:0; - padding:0; -} -.compilerinfo { - white-space: normal; - border: 1px solid #A22; - padding: 0.5em; - margin: 1em; -} -.compilerinfo .error { display: block; } -.linescolumn { - text-align: right; - padding-right: 0.3em; - border-right: 1px dotted gray; -} -.linescolumn a { - display: block; - color: #44B; - text-decoration: none; -} -/* Number */ -.n { color: teal; } -/* Keyword */ -.k { color: darkblue; font-weight: bold; } -/* Line, block and nested comments */ -.lc, .bc, .nc { color: green; } -/* Identifier */ -.i { color: black; } -/* String literal */ -.sl { color: red; } -/* Character literal */ -.cl { color: purple; } -/* All bracket types */ -.br { color: orange; } -/* Special tokens */ -.st { color: green; font-weight: bold; } -/* #line, hash line */ -.hl { color: green; } -/* filespec (e.g. #line number [filespec]) */ -.fs { color: purple;} -/* When the first line starts with #! it's a "shebang" */ -.shebang { color: gray; } -/* Operator */ -/*.op { color: royalblue; }*/ -/* Particular operators */ -/*.opaa { content: "and"; }*/ /*&& ∧*/ -/*.opoo { content: "or"; }*/ /*|| ∨*/ -/*.opn { content: "¬"; }*/ /*!*/ -/*.opne { content: "≠"; }*/ /*!=*/ -/*.ople { content: "≤"; }*/ /*<=*/ -/*.opge { content: "≥"; }*/ /*>=*/ -/*.oplg { content: "≶"; }*/ /*<>*/ -/* -d = Declaration -s = Statement -e = Expression -t = Type -o = Other -*/ -/* .d { background-color: #FFDDDD; } */ -/* .e { background-color: #DDDDFF; } */ -.d.Module .i, .d.Import .i { color: blue; } - /*.t .i,*/ .t.Identifier .i, .TemplateTypeParameter .i { color: #911; } -.t .br, .t .op { color: #911; } -.t .k { color: #911; font-weight: normal; } diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/html_map.d --- a/trunk/src/html_map.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,117 +0,0 @@ -/// A map of document elements and D tokens to format strings. -string[string] map = [ - "DocHead" : ``\n - ``\n - ``\n - ` `\n - ` `\n - ``\n - ``\n - ``\n, - "CompBegin" : `", - "LexerError" : `

    {0}({1},{2})L: {3}

    `\n, - "ParserError" : `

    {0}({1},{2})P: {3}

    `\n, - - "LineNumberBegin" : `\n", - - "DocEnd" : "\n
    `\n, - "CompEnd" : "
    \n
    `, - "LineNumberEnd" : "", - "LineNumber" : `{0}`, - - "SourceBegin" : `
    `\n,
-  "SourceEnd"   : "\n
    " - "\n" - "\n", - - // Node categories: - "Declaration" : "d", - "Statement" : "s", - "Expression" : "e", - "Type" : "t", - "Other" : "o", - - // {0} = node category. - // {1} = node class name: "Call", "If", "Class" etc. - // E.g.: ... - "NodeBegin" : ``, - "NodeEnd" : ``, - - "Identifier" : `{0}`, - "String" : `{0}`, - "Char" : `{0}`, - "Number" : `{0}`, - "Keyword" : `{0}`, - - "LineC" : `{0}`, - "BlockC" : `{0}`, - "NestedC" : `{0}`, - - "Shebang" : `{0}`, - "HLine" : `{0}`, // #line - "Filespec" : `{0}`, // #line N "filespec" - "Newline" : "{0}", // \n | \r | \r\n | LS | PS - "Illegal" : `{0}`, // A character not recognized by the lexer. - - "SpecialToken" : `{0}`, // __FILE__, __LINE__ etc. - - "(" : "(", - ")" : ")", - "[" : "[", - "]" : "]", - "{" : "{", - "}" : "}", - "." : ".", - ".." : "..", - "..." : "...", - "!<>=" : "!<>=", // Unordered - "!<>" : "!<>", // UorE - "!<=" : "!<=", // UorG - "!<" : "!<", // UorGorE - "!>=" : "!>=", // UorL - "!>" : "!>", // UorLorE - "<>=" : "<>=", // LorEorG - "<>" : "<>", // LorG - "=" : "=", - "==" : "==", - "!" : "!", - "!=" : "!=", - "<=" : "<=", - "<" : "<", - ">=" : ">=", - ">" : ">", - "<<=" : "<<=", - "<<" : "<<", - ">>=" : ">>=", - ">>" : ">>", - ">>>=" : ">>>=", - ">>>" : ">>>", - "|" : "|", - "||" : "||", - "|=" : "|=", - "&" : "&", - "&&" : "&&", - "&=" : "&=", - "+" : "+", - "++" : "++", - "+=" : "+=", - "-" : "-", - "--" : "--", - "-=" : "-=", - "/" : "/", - "/=" : "/=", - "*" : "*", - "*=" : "*=", - "%" : "%", - "%=" : "%=", - "^" : "^", - "^=" : "^=", - "~" : "~", - "~=" : "~=", - ":" : ":", - ";" : ";", - "?" : "?", - "," : ",", - "$" : "$", - "EOF" : "" -]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/lang_de.d --- a/trunk/src/lang_de.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,91 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ - -string lang_code = "de"; - -string[] messages = [ - // Lexer messages: - "illegales Zeichen gefunden: '{0}'", -// "ungültiges Unicodezeichen.", - "ungültige UTF-8-Sequenz: '{0}'", - // '' - "unterminiertes Zeichenliteral.", - "leeres Zeichenliteral.", - // #line - "erwartete 'line' nach '#'.", - "Ganzzahl nach #line erwartet.", -// `erwartete Dateispezifikation (z.B. "pfad\zur\datei".)`, - "unterminierte Dateispezifikation (filespec.)", - "ein Special Token muss mit einem Zeilenumbruch abgeschlossen werden.", - // "" - "unterminiertes Zeichenkettenliteral.", - // x"" - "Nicht-Hexzeichen '{0}' in Hexzeichenkette gefunden.", - "ungerade Anzahl von Hexziffern in Hexzeichenkette.", - "unterminierte Hexzeichenkette.", - // /* */ /+ +/ - "unterminierter Blockkommentar (/* */).", - "unterminierter verschachtelter Kommentar (/+ +/).", - // `` r"" - "unterminierte rohe Zeichenkette.", - "unterminierte Backquote-Zeichenkette.", - // \x \u \U - "undefinierte Escapesequenz '{0}' gefunden.", - "ungültige Unicode-Escapesequenz '{0}' gefunden.", - "unzureichende Anzahl von Hexziffern in Escapesequenz: '{0}'", - // \&[a-zA-Z][a-zA-Z0-9]+; - "undefinierte HTML-Entität '{0}'", - "unterminierte HTML-Entität '{0}'.", - "HTML-Entitäten müssen mit einem Buchstaben beginnen.", - // integer overflows - "Dezimalzahl überläuft im Vorzeichenbit.", - "Überlauf in Dezimalzahl.", - "Überlauf in Hexadezimalzahl.", - "Überlauf in Binärzahl.", - "Überlauf in Oktalzahl.", - "Überlauf in Fließkommazahl.", - "die Ziffern 8 und 9 sind in Oktalzahlen unzulässig.", - "ungültige Hexzahl; mindestens eine Hexziffer erforderlich.", - "ungültige Binärzahl; mindestens eine Binärziffer erforderlich.", - "der Exponent einer hexadezimalen Fließkommazahl ist erforderlich.", - "Hexadezimal-Exponenten müssen mit einer Dezimalziffer anfangen.", - "Exponenten müssen mit einer Dezimalziffer anfangen.", - - // Parser messages: - "erwartete '{0}', fand aber '{1}'.", - "'{0}' ist redundant.", - "Template-Tupel-Parameter dürfen nur am Ende auftreten.", - "der 'in'-Vertrag der Funktion wurde bereits geparsed.", - "der 'out'-Vertrag der Funktion wurde bereits geparsed.", - "es wurde kein Verbindungstyp angegeben.", - "unbekannter Verbindungstyp '{0}'; gültig sind C, C++, D, Windows, Pascal und System.", - "erwartete eine oder mehrere Basisklassen, nicht '{0}'.", - "Basisklassen sind in Vorwärtsdeklarationen nicht erlaubt.", - - // Help messages: - `dil v{0} -Copyright (c) 2007-2008, Aziz Köksal. Lizensiert unter der GPL3. - -Befehle: -{1} -Geben Sie 'dil help ' ein, um mehr Hilfe zu einem bestimmten Befehl zu -erhalten. - -Kompiliert mit {2} v{3} am {4}.`, - - `Generiere ein XML- oder HTML-Dokument aus einer D-Quelltextdatei. -Verwendung: - dil gen datei.d [Optionen] - -Optionen: - --syntax : generiere Elemente für den Syntaxbaum - --xml : verwende XML-Format (voreingestellt) - --html : verwende HTML-Format - -Beispiel: - dil gen Parser.d --html --syntax > Parser.html`, - - ``, -]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/lang_en.d --- a/trunk/src/lang_en.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,118 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ - -string lang_code = "en"; - -string[] messages = [ - // Lexer messages: - "illegal character found: '{0}'", -// "invalid Unicode character.", - "invalid UTF-8 sequence: '{0}'", - // '' - "unterminated character literal.", - "empty character literal.", - // #line - "expected 'line' after '#'.", - "integer expected after #line", -// `expected filespec string (e.g. "path\to\file".)`, - "unterminated filespec string.", - "expected a terminating newline after special token.", - // "" - "unterminated string literal.", - // x"" - "non-hex character '{0}' found in hex string.", - "odd number of hex digits in hex string.", - "unterminated hex string.", - // /* */ /+ +/ - "unterminated block comment (/* */).", - "unterminated nested comment (/+ +/).", - // `` r"" - "unterminated raw string.", - "unterminated back quote string.", - // \x \u \U - "found undefined escape sequence '{0}'.", - "found invalid Unicode escape sequence '{0}'.", - "insufficient number of hex digits in escape sequence: '{0}'", - // \&[a-zA-Z][a-zA-Z0-9]+; - "undefined HTML entity '{0}'", - "unterminated HTML entity '{0}'.", - "HTML entities must begin with a letter.", - // integer overflows - "decimal number overflows sign bit.", - "overflow in decimal number.", - "overflow in hexadecimal number.", - "overflow in binary number.", - "overflow in octal number.", - "overflow in float number.", - "digits 8 and 9 are not allowed in octal numbers.", - "invalid hex number; at least one hex digit expected.", - "invalid binary number; at least one binary digit expected.", - "the exponent of a hexadecimal float number is required.", - "hexadecimal float exponents must start with a digit.", - "exponents must start with a digit.", - - // Parser messages - "expected '{0}', but found '{1}'.", - "'{0}' is redundant.", - "template tuple parameters can only be last.", - "the functions 'in' contract was already parsed.", - "the functions 'out' contract was already parsed.", - "no linkage type was specified.", - "unrecognized linkage type '{0}'; valid types are C, C++, D, Windows, Pascal und System.", - "expected one or more base classes, not '{0}'.", - "base classes are not allowed in forward declarations.", - - // Help messages: - `dil v{0} -Copyright (c) 2007-2008 by Aziz Köksal. Licensed under the GPL3. - -Subcommands: -{1} -Type 'dil help ' for more help on a particular subcommand. - -Compiled with {2} v{3} on {4}.`, - - `Generate an XML or HTML document from a D source file. -Usage: - dil gen file.d [Options] - -Options: - --syntax : generate tags for the syntax tree - --xml : use XML format (default) - --html : use HTML format - -Example: - dil gen Parser.d --html --syntax > Parser.html`, - - `Parse a module and extract information from the resulting module dependency graph. -Usage: - dil igraph file.d Format [Options] - - The directory of file.d is implicitly added to the list of import paths. - -Format: - --dot : generate a dot document - Further options for --dot: - -gbp : Group modules by package names - -gbf : Group modules by full package name - -hle : highlight cyclic edges in the graph - -hlv : highlight modules in cyclic relationship - - --paths : print a list of paths to the modules imported by file.d - --list : print a list of the module names imported by file.d - Options common to --paths and --list: - -lN : print N levels. - -m : mark modules in cyclic relationships with a star. - -Options: - -Ipath : add 'path' to the list of import paths where modules are - looked for - -rREGEXP : exclude modules whose names match the regular expression - REGEXP - -i : include unlocatable modules - -Example: - dil igraph src/main.d`, -]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/lang_fi.d --- a/trunk/src/lang_fi.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,91 +0,0 @@ -/++ - Author: Jari-Matti Mäkelä - License: GPL3 -+/ - -string lang_code = "fi"; - -string[] messages = [ - // Lexer messages: - "virheellinen merkki: '{0}'", -// "virheellinen Unicode-merkki.", - "virheellinen UTF-8-merkkijono: '{0}'", - // '' - "päättämätön merkkiliteraali.", - "tyhjä merkkiliteraali.", - // #line - "odotettiin rivinumeroa '#':n jälkeen.", - "odotettiin kokonaislukua #line:n jälkeen", -// `odotettiin tiedostomäärittelyn merkkijonoa (esim. "polku\tiedostoon")`, - "päättämätön tiedostomäärittely.", - "odotettiin päättävää rivinvaihtoa erikoismerkin jälkeen.", - // "" - "päättämätön merkkijonoliteraali.", - // x"" - "ei-heksamerkki '{0}' heksajonossa.", - "pariton määrä heksanumeroita heksajonossa.", - "päättämätön heksajono.", - // /* */ /+ +/ - "päättämätön lohkokommentti (/* */).", - "päättämätön sisäkkäinen kommentti (/+ +/).", - // `` r"" - "päättämätön raakamerkkijono.", - "päättämätön gravisaksenttimerkkijono.", - // \x \u \U - "määrittelemätön escape-sekvenssi {0}.", - "virheellinen Unicode escape-merkki '{0}'.", - "riittämätön määrä heksanumeroita escape-sekvenssissä: '{0}'", - // \&[a-zA-Z][a-zA-Z0-9]+; - "määrittelemätön HTML-entiteetti '{0}'", - "päättämätön HTML-entiteetti {0}.", - "HTML-entiteettien tulee alkaa kirjaimella.", - // integer overflows - "desimaaliluku ylivuotaa etumerkin.", - "desimaaliluvun ylivuoto.", - "heksadesimaaliluvun ylivuoto.", - "binääriluvun ylivuoto.", - "oktaaliluvun ylivuoto.", - "liukuluvun ylivuoto.", - "numerot 8 ja 9 eivät ole sallittuja oktaaliluvuissa.", - "virheellinen heksaluku; odotettiin vähintään yhtä heksanumeroa.", - "virheellinen binääriluku; odotettiin vähintään yhtä binäärinumeroa.", - "heksadesimaalisen liukuluvun eksponentti vaaditaan.", - "heksadesimaalisen liukuluvun eksponentin tulee alkaa numerolla.", - "eksponenttien tulee alkaa numerolla.", - - // Parser messages - "odotettiin '{0}':a, mutta luettiin '{1}'.", - "'{0}' on redundantti.", - "tupla voi esiintyä ainoastaan mallin viimeisenä parametrina.", - "funktion alkuehto jäsennettiin jo.", - "funktion loppuehto jäsennettiin jo.", - "linkitystyyppiä ei määritelty.", - "tunnistamaton linkitystyyppi '{0}'; sallittuja tyyppejä ovat C, C++, D, Windows, Pascal ja System.", - "odotettiin yhtä tai useampaa luokkaa, ei '{0}':ta.", - "kantaluokat eivät ole sallittuja etukäteismäärittelyissä.", - - // Help messages: - `dil v{0} -Copyright (c) 2007-2008, Aziz Köksal. GPL3-lisensöity. - -Alikomennot: -{1} -Lisäohjeita tietystä alitoiminnosta saa kirjoittamalla 'dil help '. - -Käännetty {2}:n versiolla {3} {4}.`, - - `Luo XML- tai HTML-dokumentti D-lähdekoodista. - -Käyttö: - dil gen tiedosto.d [Valinnat] - -Valinnat: - --syntax : luo elementtejä syntaksipuun mukaisesti - --xml : käytä XML-muotoa (oletus) - --html : käytä HTML-muotoa - -Esimerkki: - dil gen Parser.d --html --syntax > Parser.html`, - - ``, -]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/lang_tr.d --- a/trunk/src/lang_tr.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,90 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ - -string lang_code = "tr"; - -string[] messages = [ - // Lexer messages: - "illegal karakter bulundu: '{0}'", -// "geçersiz Unikod karakteri.", - "geçersiz UTF-8 serisi: '{0}'", - // '' - "kapanmamış karakter sabiti.", - "boş karakter sabiti.", - // #line - "'#' karakter'den sonra 'line' beklendi.", - "'#line''den sonra rakam beklendi.", -// `filespec dizgisi beklendi (e.g. "yol\dosya".)`, - "kapanmamış filespec dizgisi.", - "özel belirtici'den (special token) sonra yeni bir satır beklendi.", - // "" - "kapanmamış çift tırnak dizgisi.", - // x"" - "heks sayı olmayan karakter '{0}' heks dizgisi içinde bulundu.", - "heks dizginin içindeki sayılar çifter çifter olmalıdır.", - "kapanmamış heks dizgisi.", - // /* */ /+ +/ - "kapanmamış blok açıklaması (/* */).", - "kapanmamış iç içe koyulabilen açıklaması (/+ +/).", - // `` r"" - "kapanmamış çiğ dizgisi.", - "kapanmamış ters tırnak dizgisi.", - // \x \u \U - "tanımlanmamış çıkış serisi '{0}' bulundu.", - "geçersiz Unikod çıkış serisi '{0}' bulundu.", - "heksadesimal çıkış serisi sayıları yeterli değil: '{0}'", - // \&[a-zA-Z][a-zA-Z0-9]+; - "tanımlanmamış HTML varlık '{0}'", - "kapanmamış HTML varlık '{0}'.", - "HTML varlık bir harf ile başlamalı.", - // integer overflows - "desimal rakamın bit işareti taşdı.", - "desimal rakam taşması.", - "heksadesimal rakam taşması.", - "binari rakam taşması.", - "oktal rakam taşması.", - "float rakam taşması.", - "8 ve 9 sayılar oktal rakamlar'da geçersizdir.", - "geçersiz heks rakam; minimum bir heks sayı gereklidir.", - "geçersiz binari rakam; minimum bir binari sayı gereklidir.", - "bir heksadesimal float rakamın üsü gereklidir.", - "heksadesimal float üsler desimal sayı ile başlamalı.", - "üsler desimal sayı ile başlamalı.", - - // Parser messages - "'{0}' beklendi, ama '{1}' bulundu.", - "'{0}' lüzumsuz.", - "şablon tuple parametre son sırada olmalı.", - "fonksiyonun 'in' kontratı daha önceden ayrıştırılmış.", - "fonksiyonun 'out' kontratı daha önceden ayrıştırılmış.", - "bağlantı tüp (linkage type) belirtilmedi.", - "bilinmeyen bağlantı tüpü (linkage type) '{0}'; geçerli olanlar C, C++, D, Windows, Pascal ve System.", - "expected one or more base classes, not '{0}'.", // TODO: translate - "base classes are not allowed in forward declarations.", // TODO: translate - - // Help messages: - `dil v{0} -Copyright (c) 2007-2008, Aziz Köksal. Lisans GPL3. - -Komutlar: -{1} -Belirli komut'a yardım edinmek için 'dil help ' yazınız. - -Bu yazılım {2} v{3} ile {4} tarihinde derletilmiş.`, - - `Bir D kaynak kodundan XML veya HTML dosyası oluştur. -Kullanım: - dil gen dosya.d [Seçenekler] - -Seçenekler: - --syntax : söz dizimi için etiketler yazdır - --xml : XML biçimi kullan (varsayılır) - --html : HTML biçimi kullan - -Örnek: - dil gen Parser.d --html --syntax > Parser.html`, - - ``, -]; diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/macros_dil.ddoc --- a/trunk/src/macros_dil.ddoc Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,22 +0,0 @@ -DDOC = - - - $(TITLE) - - -

    $(TITLE)

    -$(BODY) -
    -

    $(COPYRIGHT) Page generated by $(LINK2 http://code.google.com/p/dil, dil) on $(DATETIME).

    - - - -MODFQN = $(TITLE) -SRCFILE = ./htmlsrc/$(MODFQN).html -COPYRIGHT = Copyright © 2007-$(YEAR), Aziz Köksal. All rights reserved. -SYMBOL_ = $1 -SYMBOL = $1 - -PRE = 
    $0
    -DDD = --- -DMDBUG = $(LINK2 http://d.puremagic.com/issues/show_bug.cgi?id=$1, $2) diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/main.d --- a/trunk/src/main.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,576 +0,0 @@ -/++ - Author: Aziz Köksal - License: GPL3 -+/ -module main; - -import dil.parser.Parser; -import dil.lexer.Lexer, - dil.lexer.Token; -import dil.ast.Declarations, - dil.ast.Expressions, - dil.ast.Node, - dil.ast.Visitor; -import dil.semantic.Module; -import dil.semantic.Symbols; -import dil.semantic.Pass1, - dil.semantic.Pass2, - dil.semantic.Interpreter; -import dil.translator.German; -import dil.doc.Doc; -import dil.Messages; -import dil.CompilerInfo; -import dil.Information; -import dil.SourceText; -import dil.Compilation; - -import cmd.Generate; -import cmd.Statistics; -import cmd.ImportGraph; -import cmd.DDoc; - -import Settings; -import SettingsLoader; -// import TypeRules; -import common; - -import Integer = tango.text.convert.Integer; -import tango.stdc.stdio; -import tango.io.File; -import tango.text.Util; -import tango.time.StopWatch; -import tango.text.Ascii : icompare; - -/// Entry function of dil. -void main(char[][] args) -{ - auto infoMan = new InfoManager(); - SettingsLoader.SettingsLoader(infoMan).load(); - if (infoMan.hasInfo) - return printErrors(infoMan); - - if (args.length <= 1) - return Stdout(helpMain()).newline; - - string command = args[1]; - switch (command) - { - case "c", "compile": - if (args.length < 2) - return printHelp("compile"); - - string[] filePaths; - auto context = newCompilationContext(); - foreach (arg; args[2..$]) - { - if (parseDebugOrVersion(arg, context)) - {} - else - filePaths ~= arg; - } - infoMan = new InfoManager(); - foreach (filePath; filePaths) - { - auto mod = new Module(filePath, infoMan); - // Parse the file. - mod.parse(); - if (mod.hasErrors) - continue; - - // Start semantic analysis. - auto pass1 = new SemanticPass1(mod, context); - pass1.start(); - - void printSymbolTable(ScopeSymbol scopeSym, char[] indent) - { - foreach (member; scopeSym.members) - { - auto tokens = getDocTokens(member.node); - char[] docText; - foreach (token; tokens) - docText ~= token.srcText; - Stdout(indent).formatln("Id:{}, Symbol:{}, DocText:{}", member.name.str, member.classinfo.name, docText); - if (auto s = cast(ScopeSymbol)member) - printSymbolTable(s, indent ~ "→ "); - } - } - - printSymbolTable(mod, ""); - } - - infoMan.hasInfo && printErrors(infoMan); - break; - case "ddoc", "d": - if (args.length < 4) - return printHelp("ddoc"); - - auto destination = args[2]; - auto macroPaths = GlobalSettings.ddocFilePaths; - char[][] filePaths; - bool incUndoc; - bool writeXML; - bool verbose; - // Parse arguments. - auto context = newCompilationContext(); - foreach (arg; args[3..$]) - { - if (parseDebugOrVersion(arg, context)) - {} - else if (arg == "--xml") - writeXML = true; - else if (arg == "-i") - incUndoc = true; - else if (arg == "-v") - verbose = true; - else if (arg.length > 5 && icompare(arg[$-4..$], "ddoc") == 0) - macroPaths ~= arg; - else - filePaths ~= arg; - } - - infoMan = new InfoManager(); - // Execute command. - cmd.DDoc.execute(filePaths, destination, macroPaths, writeXML, - incUndoc, verbose, context, infoMan); - infoMan.hasInfo && printErrors(infoMan); - break; - case "gen", "generate": - char[] fileName; - GenOption options = GenOption.Tokens; - foreach (arg; args[2..$]) - { - switch (arg) - { - case "--syntax": - options |= GenOption.Syntax; break; - case "--xml": - options |= GenOption.XML; break; - case "--html": - options |= GenOption.HTML; break; - case "--lines": - options |= GenOption.PrintLines; break; - default: - fileName = arg; - } - } - if (!(options & (GenOption.XML | GenOption.HTML))) - options |= GenOption.XML; // Default to XML. - cmd.Generate.execute(fileName, options, infoMan); - infoMan.hasInfo && printErrors(infoMan); - break; - case "importgraph", "igraph": - string filePath; - string[] regexps; - string siStyle = "dashed"; // static import style - string piStyle = "bold"; // public import style - uint levels; - IGraphOption options; - auto context = newCompilationContext(); - foreach (arg; args[2..$]) - { - if (parseDebugOrVersion(arg, context)) - {} - else if (strbeg(arg, "-I")) - context.importPaths ~= arg[2..$]; - else if(strbeg(arg, "-r")) - regexps ~= arg[2..$]; - else if(strbeg(arg, "-l")) - levels = Integer.toInt(arg[2..$]); - else if(strbeg(arg, "-si")) - siStyle = arg[3..$]; - else if(strbeg(arg, "-pi")) - piStyle = arg[3..$]; - else - switch (arg) - { - case "--dot": - options |= IGraphOption.PrintDot; break; - case "--paths": - options |= IGraphOption.PrintPaths; break; - case "--list": - options |= IGraphOption.PrintList; break; - case "-i": - options |= IGraphOption.IncludeUnlocatableModules; break; - case "-hle": - options |= IGraphOption.HighlightCyclicEdges; break; - case "-hlv": - options |= IGraphOption.HighlightCyclicVertices; break; - case "-gbp": - options |= IGraphOption.GroupByPackageNames; break; - case "-gbf": - options |= IGraphOption.GroupByFullPackageName; break; - case "-m": - options |= IGraphOption.MarkCyclicModules; break; - default: - filePath = arg; - } - } - cmd.ImportGraph.execute(filePath, context, regexps, levels, siStyle, piStyle, options); - break; - case "stats", "statistics": - char[][] filePaths; - bool printTokensTable; - bool printNodesTable; - foreach (arg; args[2..$]) - if (arg == "--toktable") - printTokensTable = true; - else if (arg == "--asttable") - printNodesTable = true; - else - filePaths ~= arg; - cmd.Statistics.execute(filePaths, printTokensTable, printNodesTable); - break; - case "tok", "tokenize": - SourceText sourceText; - char[] filePath; - char[] separator; - bool ignoreWSToks; - bool printWS; - - foreach (arg; args[2..$]) - { - if (strbeg(arg, "-s")) - separator = arg[2..$]; - else if (arg == "-") - sourceText = new SourceText("stdin", readStdin()); - else if (arg == "-i") - ignoreWSToks = true; - else if (arg == "-ws") - printWS = true; - else - filePath = arg; - } - - separator || (separator = "\n"); - if (!sourceText) - sourceText = new SourceText(filePath, true); - - infoMan = new InfoManager(); - auto lx = new Lexer(sourceText, infoMan); - lx.scanAll(); - auto token = lx.firstToken(); - - for (; token.kind != TOK.EOF; token = token.next) - { - if (token.kind == TOK.Newline || ignoreWSToks && token.isWhitespace) - continue; - if (printWS && token.ws) - Stdout(token.wsChars); - Stdout(token.srcText)(separator); - } - - infoMan.hasInfo && printErrors(infoMan); - break; - case "trans", "translate": - if (args.length < 3) - return printHelp("trans"); - - if (args[2] != "German") - return Stdout.formatln("Error: unrecognized target language \"{}\"", args[2]); - - infoMan = new InfoManager(); - auto filePath = args[3]; - auto mod = new Module(filePath, infoMan); - // Parse the file. - mod.parse(); - if (!mod.hasErrors) - { // Translate - auto german = new GermanTranslator(Stdout, " "); - german.translate(mod.root); - } - printErrors(infoMan); - break; - case "profile": - if (args.length < 3) - break; - char[][] filePaths; - if (args[2] == "dstress") - { - auto text = cast(char[])(new File("dstress_files")).read(); - filePaths = split(text, "\0"); - } - else - filePaths = args[2..$]; - - StopWatch swatch; - swatch.start; - - foreach (filePath; filePaths) - (new Lexer(new SourceText(filePath, true))).scanAll(); - - Stdout.formatln("Scanned in {:f10}s.", swatch.stop); - break; - // case "parse": - // if (args.length == 3) - // parse(args[2]); - // break; - case "?", "help": - printHelp(args.length >= 3 ? args[2] : ""); - break; - // case "typerules": - // genHTMLTypeRulesTables(); - // break; - default: - } -} - -char[] readStdin() -{ - char[] text; - while (1) - { - auto c = getc(stdin); - if (c == EOF) - break; - text ~= c; - } - return text; -} - -/// Available commands. -const char[] COMMANDS = - " compile (c)\n" - " ddoc (d)\n" - " generate (gen)\n" - " help (?)\n" - " importgraph (igraph)\n" - " statistics (stats)\n" - " tokenize (tok)\n" - " translate (trans)\n"; - -bool strbeg(char[] str, char[] begin) -{ - if (str.length >= begin.length) - { - if (str[0 .. begin.length] == begin) - return true; - } - return false; -} - -/// Creates the global compilation context. -CompilationContext newCompilationContext() -{ - auto cc = new CompilationContext; - cc.importPaths = GlobalSettings.importPaths; - cc.addVersionId("dil"); - cc.addVersionId("all"); -version(D2) - cc.addVersionId("D_Version2"); - foreach (versionId; GlobalSettings.versionIds) - if (!Lexer.isReservedIdentifier(versionId)) - cc.versionIds[versionId] = true; - return cc; -} - -bool parseDebugOrVersion(string arg, CompilationContext context) -{ - if (strbeg(arg, "-debug")) - { - if (arg.length > 7) - { - auto val = arg[7..$]; - if (isdigit(val[0])) - context.debugLevel = Integer.toInt(val); - else if (!Lexer.isReservedIdentifier(val)) - context.addDebugId(val); - } - else - context.debugLevel = 1; - } - else if (arg.length > 9 && strbeg(arg, "-version=")) - { - auto val = arg[9..$]; - if (isdigit(val[0])) - context.versionLevel = Integer.toInt(val); - else if (!Lexer.isReservedIdentifier(val)) - context.addVersionId(val); - } - else - return false; - return true; -} - -/// Prints the errors collected in infoMan. -void printErrors(InfoManager infoMan) -{ - foreach (info; infoMan.info) - { - char[] errorFormat; - if (info.classinfo is LexerError.classinfo) - errorFormat = GlobalSettings.lexerErrorFormat; - else if (info.classinfo is ParserError.classinfo) - errorFormat = GlobalSettings.parserErrorFormat; - else if (info.classinfo is SemanticError.classinfo) - errorFormat = GlobalSettings.semanticErrorFormat; - else if (info.classinfo is Warning.classinfo) - errorFormat = "{0}: Warning: {3}"; - else - continue; - auto err = cast(Problem)info; - Stderr.formatln(errorFormat, err.filePath, err.loc, err.col, err.getMsg); - } -} - -/// Prints the compiler's main help message. -char[] helpMain() -{ - auto COMPILED_WITH = __VENDOR__; - auto COMPILED_VERSION = Format("{}.{,:d3}", __VERSION__/1000, __VERSION__%1000); - auto COMPILED_DATE = __TIMESTAMP__; - return FormatMsg(MID.HelpMain, VERSION, COMMANDS, COMPILED_WITH, - COMPILED_VERSION, COMPILED_DATE); -} - -/// Prints a help message for command. -void printHelp(char[] command) -{ - char[] msg; - switch (command) - { - case "c", "compile": - msg = `Compile D source files. -Usage: - dil compile file.d [file2.d, ...] [Options] - - This command only parses the source files and does little semantic analysis. - Errors are printed to standard error output. - -Options: - -debug : include debug code - -debug=level : include debug(l) code where l <= level - -debug=ident : include debug(ident) code - -version=level : include version(l) code where l >= level - -version=ident : include version(ident) code - -Example: - dil c src/main.d`; - break; - case "ddoc", "d": - msg = `Generate documentation from DDoc comments in D source files. -Usage: - dil ddoc Destination file.d [file2.d, ...] [Options] - - Destination is the folder where the documentation files are written to. - Files with the extension .ddoc are recognized as macro definition files. - -Options: - --xml : write XML instead of HTML documents - -i : include undocumented symbols - -v : verbose output - -Example: - dil d doc/ src/main.d src/macros_dil.ddoc -i`; - break; - case "gen", "generate": -// msg = GetMsg(MID.HelpGenerate); - msg = `Generate an XML or HTML document from a D source file. -Usage: - dil gen file.d [Options] - -Options: - --syntax : generate tags for the syntax tree - --xml : use XML format (default) - --html : use HTML format - --lines : print line numbers - -Example: - dil gen Parser.d --html --syntax > Parser.html`; - break; - case "importgraph", "igraph": -// msg = GetMsg(MID.HelpImportGraph); - msg = `Parse a module and build a module dependency graph based on its imports. -Usage: - dil igraph file.d Format [Options] - - The directory of file.d is implicitly added to the list of import paths. - -Format: - --dot : generate a dot document (default) - Options related to --dot: - -gbp : Group modules by package names - -gbf : Group modules by full package name - -hle : highlight cyclic edges in the graph - -hlv : highlight modules in cyclic relationships - -siSTYLE : the edge style to use for static imports - -piSTYLE : the edge style to use for public imports - STYLE can be: "dashed", "dotted", "solid", "invis" or "bold" - - --paths : print the file paths of the modules in the graph - - --list : print the names of the module in the graph - Options common to --paths and --list: - -lN : print N levels. - -m : use '*' to mark modules in cyclic relationships - -Options: - -Ipath : add 'path' to the list of import paths where modules are - looked for - -rREGEXP : exclude modules whose names match the regular expression - REGEXP - -i : include unlocatable modules - -Example: - dil igraph src/main.d --list - dil igraph src/main.d | dot -Tpng > main.png`; - break; - case "tok", "tokenize": - msg = `Print the tokens of a D source file. -Usage: - dil tok file.d [Options] - -Options: - - : reads text from the standard input. - -sSEPARATOR : print SEPARATOR instead of newline between tokens. - -i : ignore whitespace tokens (e.g. comments, shebang etc.) - -ws : print a token's preceding whitespace characters. - -Example: - echo "module foo; void func(){}" | dil tok - - dil tok main.d | grep ^[0-9]`; - break; - case "stats", "statistics": - msg = "Gather statistics about D source files. -Usage: - dil stat file.d [file2.d, ...] [Options] - -Options: - --toktable : print the count of all kinds of tokens in a table. - --asttable : print the count of all kinds of nodes in a table. - -Example: - dil stat src/dil/Parser.d src/dil/Lexer.d"; - break; - case "trans", "translate": - msg = `Translate a D source file to another language. -Usage: - dil translate Language file.d - - Languages that are supported: - *) German - -Example: - dil trans German src/main.d`; - break; - default: - msg = helpMain(); - } - Stdout(msg).newline; -} - -/+void parse(string fileName) -{ - auto mod = new Module(fileName); - mod.parse(); - - void print(Node[] decls, char[] indent) - { - foreach(decl; decls) - { - assert(decl !is null); - Stdout.formatln("{}{}: begin={} end={}", indent, decl.classinfo.name, decl.begin ? decl.begin.srcText : "\33[31mnull\33[0m", decl.end ? decl.end.srcText : "\33[31mnull\33[0m"); - print(decl.children, indent ~ " "); - } - } - print(mod.root.children, ""); -}+/ diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/predefined.ddoc --- a/trunk/src/predefined.ddoc Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,109 +0,0 @@ -DDOC = - - - $(TITLE) - - -

    $(TITLE)

    -$(BODY) -
    -

    $(COPYRIGHT) | Page generated by $(LINK2 http://code.google.com/p/dil, dil) on $(DATETIME).

    - - - -B = $0 -I = $0 -U = $0 -P =

    $0

    -DL =
    $0
    -DT =
    $0
    -DD =
    $0
    -TABLE = $0
    -TR = $0 -TH = $0 -TD = $0 -OL =
      $0
    -UL =
      $0
    -LI =
  • $0
    • -BIG = $0 -SMALL = $0 -BR =
    -LINK = $0 -LINK2 = $+ - -RED = $0 -BLUE = $0 -GREEN = $0 -YELLOW = $0 -BLACK = $0 -WHITE = $0 - -D_CODE = 
    $0
    -D_COMMENT = $(GREEN $0) -D_STRING = $(RED $0) -D_KEYWORD = $(BLUE $0) -D_PSYMBOL = $(U $0) -D_PARAM = $(I $0) - -DDOC_COMMENT = -DDOC_DECL = $(DT $(BIG $0)) -DDOC_DECL_DD = $(DD $0) -DDOC_DITTO = $(BR)$0 - -DDOC_SECTIONS = $0 -DDOC_SUMMARY = $0$(BR)$(BR) -DDOC_DESCRIPTION = $0$(BR)$(BR) -DDOC_AUTHORS = $(B Authors:)$(BR) -$0$(BR)$(BR) -DDOC_BUGS = $(RED BUGS:)$(BR) -$0$(BR)$(BR) -DDOC_COPYRIGHT = $(B Copyright:)$(BR) -$0$(BR)$(BR) -DDOC_DATE = $(B Date:)$(BR) -$0$(BR)$(BR) -DDOC_DEPRECATED = $(RED Deprecated:)$(BR) -$0$(BR)$(BR) -DDOC_EXAMPLES = $(B Examples:)$(BR) -$0$(BR)$(BR) -DDOC_HISTORY = $(B History:)$(BR) -$0$(BR)$(BR) -DDOC_LICENSE = $(B License:)$(BR) -$0$(BR)$(BR) -DDOC_RETURNS = $(B Returns:)$(BR) -$0$(BR)$(BR) -DDOC_SEE_ALSO = $(B See Also:)$(BR) -$0$(BR)$(BR) -DDOC_STANDARDS = $(B Standards:)$(BR) -$0$(BR)$(BR) -DDOC_THROWS = $(B Throws:)$(BR) -$0$(BR)$(BR) -DDOC_VERSION = $(B Version:)$(BR) -$0$(BR)$(BR) -DDOC_SECTION_H = $(B $0)$(BR) -DDOC_SECTION = $0$(BR)$(BR) - -DDOC_MEMBERS = $(DL $0) -DDOC_MODULE_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_CLASS_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_INTERFACE_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_STRUCT_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_UNION_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_TEMPLATE_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_ENUM_MEMBERS = $(DDOC_MEMBERS $0) - -DDOC_PARAMS = $(B Params:)$(BR) -$(TABLE $0)$(BR) -DDOC_PARAM_ROW = $(TR $0) -DDOC_PARAM_ID = $(TD $0) -DDOC_PARAM_DESC = $(TD $0) -DDOC_BLANKLINE = $(BR)$(BR) - -DDOC_PSYMBOL = $(U $0) -DDOC_KEYWORD = $(B $0) -DDOC_PARAM = $(I $0) - -ATTRIBUTES = [$0] -PROT = $0 -STC = $0 -LINKAGE = $0 -SYMBOL = $1 diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/predefined_xml.ddoc --- a/trunk/src/predefined_xml.ddoc Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,98 +0,0 @@ -DDOC = -$(TITLE) -$(BODY) -$(COPYRIGHT) -dil $(DDOC_VERSION) $(LINK http://code.google.com/p/dil) - $(DATETIME) - - - -B = $0 -I = $0 -U = $0 -P =

    $0

    -DL =
    $0
    -DT =
    $0
    -DD =
    $0
    -TABLE = $0
    -TR = $0 -TH = $0 -TD = $0 -OL =
      $0
    -UL =
      $0
    -LI =
  • $0
    • -BIG = $0 -SMALL = $0 -BR =
    -LINK = $0 -LINK2 = $+ - -RED = $0 -BLUE = $0 -GREEN = $0 -YELLOW = $0 -BLACK = $0 -WHITE = $0 - -D_CODE = $0 -D_COMMENT = $0 -D_STRING = $0 -D_COMMENT = $0 -D_PSYMBOL = $0 -D_PARAM = $0 - -DDOC_COMMENT = -DDOC_DECL = $2 -DDOC_DECL_DD = $0 -DDOC_DITTO = $0 - -DDOC_SECTIONS = $0 -DDOC_SECTION_T =
    $2
    -DDOC_SUMMARY = $(DDOC_SECTION_T summary, $0) -DDOC_DESCRIPTION = $(DDOC_SECTION_T description, $0) -DDOC_AUTHORS = $(DDOC_SECTION_T authors, $0) -DDOC_BUGS = $(DDOC_SECTION_T bugs, $0) -DDOC_COPYRIGHT = $(DDOC_SECTION_T copyright, $0) -DDOC_DATE = $(DDOC_SECTION_T date, $0) -DDOC_DEPRECATED = $(DDOC_SECTION_T deprecated, $0) -DDOC_EXAMPLES = $(DDOC_SECTION_T examples, $0) -DDOC_HISTORY = $(DDOC_SECTION_T history, $0) -DDOC_LICENSE = $(DDOC_SECTION_T license, $0) -DDOC_RETURNS = $(DDOC_SECTION_T returns, $0) -DDOC_SEE_ALSO = $(DDOC_SECTION_T seealso, $0) -DDOC_STANDARDS = $(DDOC_SECTION_T standards, $0) -DDOC_THROWS = $(DDOC_SECTION_T throws, $0) -DDOC_VERSION = $(DDOC_SECTION_T version, $0) -DDOC_SECTION_H = $(B $0)$(BR) -DDOC_SECTION = $0 - -DDOC_MEMBERS = $0 -DDOC_MODULE_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_CLASS_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_INTERFACE_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_STRUCT_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_UNION_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_TEMPLATE_MEMBERS = $(DDOC_MEMBERS $0) -DDOC_ENUM_MEMBERS = $(DDOC_MEMBERS $0) - -DDOC_PARAMS = $0 -DDOC_PARAM_ROW = $0 -DDOC_PARAM_ID = $0 -DDOC_PARAM_DESC = $0 -DDOC_BLANKLINE = - -DDOC_PSYMBOL = $(U $0) -DDOC_KEYWORD = $(B $0) -DDOC_PARAM = $0 - -ATTRIBUTES = $0 -ATTRIBUTE = $2 -PROT = $(ATTRIBUTE protection, $0) -STC = $(ATTRIBUTE storage, $0) -LINKAGE = $(ATTRIBUTE linkage, $0) -SYMBOL = $1 -PARENTS = $0 -TYPE = $0 -PARAMS = $0 -RETURNS = $0 -TEMPLATE_PARAMS = $0 diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/tests/forward01.d --- a/trunk/src/tests/forward01.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,7 +0,0 @@ -/++ - Author: Jari-Matti Mäkelä -+/ - -// Impossible circular composition. -struct A { B b; } -struct B { A a; } diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/tests/forward02.d --- a/trunk/src/tests/forward02.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,13 +0,0 @@ -/++ - Author: Jari-Matti Mäkelä -+/ - -// Valid circular composition because of pointer. -struct A { B* b; } -struct B { A a; } -// Equivalent to: -struct A { A* a; } - -// Valid circular composition because classes are reference types. -class C { D d; } -class D { C c; } diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/tests/forward03.d --- a/trunk/src/tests/forward03.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,18 +0,0 @@ -/++ - Author: Aziz Köksal -+/ - -// Impossible static circular reference. -const x = y; -const y = x; - -// Impossible static circular reference. -struct A -{ const int a = B.b; } -struct B -{ const int b = A.a; } - -struct C -{ - const x = C.x; -} \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/tests/forward04.d --- a/trunk/src/tests/forward04.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,6 +0,0 @@ -/++ - Author: Aziz Köksal -+/ - -class A { auto x = pi; } -mixin("const pi = 3.14;"); diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/tests/forward05.d --- a/trunk/src/tests/forward05.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,9 +0,0 @@ -/++ - Author: Aziz Köksal -+/ - -struct A -{ int a = B.x; } - -struct B -{ const int x = 1; } diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/about.ui --- a/trunk/src/translator/about.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,70 +0,0 @@ - - AboutDialog - - - Qt::WindowModal - - - - 0 - 0 - 414 - 157 - - - - About - - - - - - <html><head><meta name="qrichtext" content="1" /><style type="text/css"> -p, li { white-space: pre-wrap; } -</style></head><body style=" font-family:'Sans Serif'; font-size:9pt; font-weight:400; font-style:normal;"> -<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">A tool for managing message catalogues in different languages.</p> -<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">These catalogues can be used to make an application multilingual.</p> -<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"></p> -<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Copyright © 2007 by Aziz Köksal &lt;aziz.koeksal@gmail.com&gt;</p> -<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Licensed under the GPL2.</p></body></html> - - - Qt::RichText - - - Qt::AlignLeading|Qt::AlignLeft|Qt::AlignTop - - - true - - - - - - - QDialogButtonBox::Ok - - - - - - - - - buttonBox - accepted() - AboutDialog - close() - - - 346 - 274 - - - 317 - 256 - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/closing_project.ui --- a/trunk/src/translator/closing_project.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,125 +0,0 @@ - - ClosingProjectDialog - - - - 0 - 0 - 400 - 300 - - - - Closing Project - - - - - - The following documents have been modified: - - - - - - - - Title - - - - - Full Path - - - - - - - - Qt::Vertical - - - - 382 - 33 - - - - - - - - - - Qt::Horizontal - - - - 40 - 20 - - - - - - - - &Save Selected - - - - - - - &Discard All Changes - - - - - - - &Cancel - - - - - - - - - - - button_Save_Selected - clicked() - ClosingProjectDialog - accept() - - - 144 - 269 - - - 179 - 238 - - - - - button_Cancel - clicked() - ClosingProjectDialog - reject() - - - 337 - 268 - - - 328 - 248 - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/errors.py --- a/trunk/src/translator/errors.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,11 +0,0 @@ -# -*- coding: utf-8 -*- -# Author: Aziz Köksal -# License: GPL2 -import exceptions - -class LoadingError(exceptions.Exception): - def __init__(self, msg): - self.msg = msg - return - def __str__(self): - return self.msg diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/langfile.py --- a/trunk/src/translator/langfile.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,112 +0,0 @@ -# -*- coding: utf-8 -*- -# Author: Aziz Köksal -# License: GPL2 -import yaml -from errors import LoadingError - -# Avoid that all unicode strings are tagged with "!!python/unicode". -def unicode_representer(dumper, data): - return dumper.represent_scalar(u'tag:yaml.org,2002:str', data) -yaml.add_representer(unicode, unicode_representer) - -def newLangFile(langCode, authors, license): - return { - "LangCode":langCode, - "Authors":authors, - "License":license, - "Messages":[] - } - -class LangFile: - def __init__(self, filePath): - from os import path - self.filePath = path.abspath(filePath) - self.isSource = False - self.source = None - # Load language file and check data integrity. - try: - self.doc = yaml.load(open(filePath, "r")) - except yaml.YAMLError, e: - raise LoadingError(str(e)) - self.verify() - - def verify(self): - doc = self.doc - self.checkType(doc, dict) - try: - self.langCode = str(doc["LangCode"]) - self.authors = list(doc["Authors"]) - self.license = unicode(doc["License"]) - self.messages = list(doc["Messages"]) - except KeyError, e: - raise LoadingError("Missing member '%s' in '%s'" % (e.message, filePath)) - - authors = [] - for author in self.authors: - self.checkType(author, dict, "LangFile: author must be of type dict.") - try: - author["Name"] = unicode(author["Name"]) - author["EMail"] = str(author["EMail"]) - authors += [author] - except KeyError, e: - raise LoadingError("Author is missing '%s' in '%s'" % (e.message, filePath)) - self.authors = authors - - self.msgDict = {} # {ID : msg, ...} - for msg in self.messages: - self.checkType(msg, dict, "LangFile: messages must be of type dict.") - try: - msg["ID"] = int(msg["ID"]) - msg["Text"] = unicode(msg["Text"]) - msg["Annot"] = unicode(msg["Annot"]) - msg["LastEd"] = int(msg["LastEd"]) - except KeyError, e: - raise LoadingError("LangFile: a message is missing the '%s' key." % str(e)) - self.msgDict[msg["ID"]] = msg - - def checkType(self, var, type_, msg=""): - if not isinstance(var, type_): - raise LoadingError(msg) - - def setSource(self, sourceLangFile): - self.source = sourceLangFile - - def getMsg(self, ID): - for msg in self.messages: - if msg["ID"] == ID: - return msg - return None - - def createMissingMessages(self, IDs): - for ID in IDs: - if not self.msgDict.has_key(ID): - msg = self.createEmptyMsg(ID) - self.msgDict[ID] = msg - self.messages += [msg] - - def createEmptyMsg(self, ID): - return {"ID":ID,"Text":"","Annot":"","LastEd":""} - - def save(self): - self.doc["LangCode"] = self.langCode - self.doc["License"] = self.license - self.doc["Authors"] = self.authors - langFile = open(self.filePath, "w") - yaml.dump(self.doc, stream=langFile, allow_unicode=True) - langFile.close() - - def setLangCode(self, langCode): - self.langCode = langCode - - def setLicense(self, license): - self.license = license - - def setAuthors(self, authors): - self.authors = authors - - def getFileName(self): - from os import path - return path.basename(self.filePath) - - def getFilePath(self): - return self.filePath diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/langfile_properties.ui --- a/trunk/src/translator/langfile_properties.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,128 +0,0 @@ - - LangFilePropertiesDialog - - - - 0 - 0 - 400 - 228 - - - - Language File Properties - - - - - - - - Language code: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - langCodeField - - - - - - - - - - Creation date: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - creationDateField - - - - - - - - - - - - - Author(s): - - - Qt::AlignRight|Qt::AlignTop|Qt::AlignTrailing - - - - - - - - - Qt::Vertical - - - QSizePolicy::Fixed - - - - 20 - 10 - - - - - - - - Qt::Horizontal - - - QDialogButtonBox::Cancel|QDialogButtonBox::NoButton|QDialogButtonBox::Ok - - - - - - - - - buttonBox - accepted() - LangFilePropertiesDialog - accept() - - - 227 - 278 - - - 157 - 274 - - - - - buttonBox - rejected() - LangFilePropertiesDialog - reject() - - - 295 - 284 - - - 286 - 274 - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/make_ui.sh --- a/trunk/src/translator/make_ui.sh Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,5 +0,0 @@ -#!/bin/bash - -ui_file=$1 -echo $ui_file \> ui_`basename $ui_file .ui`.py -pyuic4 $ui_file > ui_`basename $ui_file .ui`.py diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/make_uis.sh --- a/trunk/src/translator/make_uis.sh Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,5 +0,0 @@ -#!/bin/bash - -for ui_file in *.ui; do - ./make_ui.sh $ui_file -done \ No newline at end of file diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/msg_form.ui --- a/trunk/src/translator/msg_form.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,107 +0,0 @@ - - MsgForm - - - - 0 - 0 - 456 - 512 - - - - Form - - - - - - Qt::Vertical - - - - - - - Messages: - - - - - - - - 1 - - - - - - - - - - - - - - Source string: - - - - - - - - - - - - - - Source annotation: - - - - - - - - - - - - - - Translation: - - - - - - - - - - - - - - Translator's annotation: - - - - - - - - - - - - - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/new_project.ui --- a/trunk/src/translator/new_project.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,127 +0,0 @@ - - NewProjectDialog - - - - 0 - 0 - 401 - 131 - - - - Create New Project - - - - - - - - - - Project name: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - projectName - - - - - - - - - - Project file: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - projectFilePath - - - - - - - - - - - - ... - - - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Horizontal - - - QDialogButtonBox::Cancel|QDialogButtonBox::NoButton|QDialogButtonBox::Ok - - - - - - - - - - - buttonBox - accepted() - NewProjectDialog - accept() - - - 228 - 108 - - - 157 - 130 - - - - - buttonBox - rejected() - NewProjectDialog - reject() - - - 296 - 114 - - - 286 - 130 - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/project.py --- a/trunk/src/translator/project.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,109 +0,0 @@ -# -*- coding: utf-8 -*- -# Author: Aziz Köksal -# License: GPL2 -import os -from errors import LoadingError -import langfile -import datetime -import yaml - -def newProjectData(projectName): - return { - "Name":projectName, - "LangFiles":[], - "SourceLangFile":'', - "MsgIDs":[], - "CreationDate":str(datetime.datetime.utcnow()), - "BuildScript":'' - } - -class Project: - # Members: - # name - # source - # langFilePaths - # langFile - # msgids - # creationDate - def __init__(self, projectPath): - self.projectPath = projectPath - # Load project file and check data integrity. - try: - self.doc = yaml.load(open(projectPath, "r")) - except yaml.YAMLError, e: - raise LoadingError(str(e)) - self.verify() - - def verify(self): - doc = self.doc - self.checkType(doc, dict) - try: - self.name = unicode(doc["Name"]) - self.srcLangFilePath = str(doc["SourceLangFile"]) - self.langFilePaths = list(doc["LangFiles"]) - self.msgIDs = list(doc["MsgIDs"]) - self.creationDate = str(doc["CreationDate"]) - self.buildScript = str(doc["BuildScript"]) - except KeyError, e: - raise LoadingError("Missing member '%s' in '%s'" % (e.message, projectPath)) - - for path in self.langFilePaths: - self.checkType(path, str) - - msgIDs = [] - for msg in self.msgIDs: - self.checkType(msg, dict) - try: - msg["ID"] = int(msg["ID"]) - msg["Name"] = unicode(msg["Name"]) - msg["Order"] = int(msg["Order"]) - except KeyError, e: - raise LoadingError("Project: a message is missing the '%s' key." % str(e)) - msgIDs += [msg] - self.msgIDs = msgIDs - - # Load language files. - self.langFiles = [] - IDList = [msg["ID"] for msgID in self.msgIDs] - for filePath in self.langFilePaths: - langFile = self.loadLangFile(filePath) - langFile.createMissingMessages(IDList) - self.langFiles += [langFile] - self.srcLangFile = self.loadLangFile(self.srcLangFilePath) - self.srcLangFile.createMissingMessages(IDList) - self.srcLangFile.isSource = True - self.langFiles += [self.srcLangFile] - - for langFile in self.langFiles: - langFile.setSource(self.srcLangFile) - - def checkType(self, var, type_): - if not isinstance(var, type_): - raise LoadingException("%s is not of type %s" % (str(var), str(type_))) - - def loadLangFile(self, filePath): - if not os.path.exists(filePath): - # Look in project directory. - projectDir = os.path.dirname(self.projectPath) - filePath2 = os.path.join(projectDir, filePath) - if not os.path.exists(filePath2): - raise LoadingError("Project: Language file '%s' doesn't exist"%filePath) - filePath = filePath2 - return langfile.LangFile(filePath) - - def setName(self, name): - self.name = name - - def setBuildScript(self, filePath): - self.buildScript = filePath - - def setCreationDate(self, date): - self.creationDate = date - - def save(self): - self.doc["Name"] = self.name - self.doc["BuildScript"] = self.buildScript - self.doc["CreationDate"] = self.creationDate - file_ = open(self.projectPath, "w") - yaml.dump(self.doc, stream=file_, allow_unicode=True) - file_.close() diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/project_properties.ui --- a/trunk/src/translator/project_properties.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,136 +0,0 @@ - - ProjectPropertiesDialog - - - - 0 - 0 - 400 - 152 - - - - Project Properties - - - - - - - - Project name: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - projectNameField - - - - - - - - - - Build script: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - buildScriptField - - - - - - - - - - - - ... - - - - - - - - - - - - Creation date: - - - Qt::AlignRight|Qt::AlignTrailing|Qt::AlignVCenter - - - - - - - - - Qt::Vertical - - - - 20 - 40 - - - - - - - - Qt::Horizontal - - - QDialogButtonBox::Cancel|QDialogButtonBox::NoButton|QDialogButtonBox::Ok - - - - - - - - - buttonBox - accepted() - ProjectPropertiesDialog - accept() - - - 227 - 278 - - - 157 - 274 - - - - - buttonBox - rejected() - ProjectPropertiesDialog - reject() - - - 295 - 284 - - - 286 - 274 - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/translator.py --- a/trunk/src/translator/translator.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,685 +0,0 @@ -#! /usr/bin/python -# -*- coding: utf-8 -*- -# Author: Aziz Köksal -# License: GPL2 -import sys, os -import yaml - -from PyQt4 import QtCore, QtGui -# User interface modules -from ui_translator import Ui_MainWindow -from ui_about import Ui_AboutDialog -from ui_new_project import Ui_NewProjectDialog -from ui_project_properties import Ui_ProjectPropertiesDialog -from ui_msg_form import Ui_MsgForm -from ui_closing_project import Ui_ClosingProjectDialog - -from project import Project, newProjectData - -g_scriptDir = sys.path[0] -g_CWD = os.getcwd() -g_projectExt = ".tproj" -g_catExt = ".cat" -g_settingsFile = os.path.join(g_scriptDir, "settings.yaml") -g_settings = {} - -Qt = QtCore.Qt -Qt.connect = QtCore.QObject.connect -Qt.disconnect = QtCore.QObject.disconnect -Qt.SIGNAL = QtCore.SIGNAL -Qt.SLOT = QtCore.SLOT - -def QTabWidgetCloseAll(self): - for i in range(self.count()-1,-1,-1): - widget = self.widget(i) - self.removeTab(i) - widget.close() -QtGui.QTabWidget.closeAll = QTabWidgetCloseAll - -class MainWindow(QtGui.QMainWindow, Ui_MainWindow): - def __init__(self): - QtGui.QMainWindow.__init__(self) - self.setupUi(self) - - self.project = None - # Modifications - self.pages = QtGui.QTabWidget() - self.setCentralWidget(self.pages) - self.disableMenuItems() - self.projectDock = QtGui.QDockWidget("Project", self) - self.projectTree = ProjectTree(self.projectDock, self) - self.projectDock.setWidget(self.projectTree) - self.addDockWidget(QtCore.Qt.LeftDockWidgetArea, self.projectDock) - # Custom connections - triggered = Qt.SIGNAL("triggered()") - Qt.connect(self.action_About, triggered, self.showAboutDialog) - Qt.connect(self.action_New_Project, triggered, self.createNewProject) - Qt.connect(self.action_Open_Project, triggered, self.openProjectAction) - Qt.connect(self.action_Close_Project, triggered, self.closeProjectAction) - Qt.connect(self.action_Save, triggered, self.saveForm) - Qt.connect(self.action_Save_All, triggered, self.saveAllForms) - Qt.connect(self.action_Close, triggered, self.closeForm) - Qt.connect(self.action_Close_All, triggered, self.closeAllForms) - Qt.connect(self.action_Properties, triggered, self.showProjectProperties) - Qt.connect(self.action_Add_Catalogue, triggered, self.addCatalogue) - Qt.connect(self.action_Add_New_Catalogue, triggered, self.addNewCatalogue) - Qt.connect(self.projectTree, Qt.SIGNAL("itemDoubleClicked(QTreeWidgetItem*,int)"), self.projectTreeItemDblClicked) - Qt.connect(self.projectTree, Qt.SIGNAL("onKeyEnter"), self.projectTreeItemActivated) - Qt.connect(self.projectTree, Qt.SIGNAL("onKeyDelete"), self.projectTreeItemDeleted) - - shortcut = QtGui.QShortcut(QtGui.QKeySequence(Qt.CTRL+Qt.Key_Tab), self) - Qt.connect(shortcut, Qt.SIGNAL("activated()"), self.nextDocument) - shortcut = QtGui.QShortcut(QtGui.QKeySequence(Qt.CTRL+Qt.SHIFT+Qt.Key_Tab), self) - Qt.connect(shortcut, Qt.SIGNAL("activated()"), self.prevDocument) - - self.readSettings() - - def nextDocument(self): - count = self.pages.count() - if count < 1: return - index = self.pages.currentIndex()+1 - if index == count: index = 0 - self.pages.setCurrentIndex(index) - - def prevDocument(self): - count = self.pages.count() - if count < 1: return - index = self.pages.currentIndex()-1 - if index == -1: index = count-1 - self.pages.setCurrentIndex(index) - - def showAboutDialog(self): - about = QtGui.QDialog() - Ui_AboutDialog().setupUi(about) - about.exec_() - - def showProjectProperties(self): - dialog = ProjectPropertiesDialog(self.project) - dialog.exec_() - self.projectTree.updateProjectName() - - def createNewProject(self): - if self.rejectClosingProject(): - return - dialog = NewProjectDialog() - code = dialog.exec_() - if code == QtGui.QDialog.Accepted: - self.closeProject() - self.openProject(str(dialog.projectFilePath.text())) - - def openProjectAction(self): - if self.rejectClosingProject(): - return - filePath = QtGui.QFileDialog.getOpenFileName(self, "Select Project File", g_CWD, "Translator Project (*%s)" % g_projectExt); - filePath = str(filePath) - if filePath: - self.closeProject() - self.openProject(filePath) - - def openProject(self, filePath): - from errors import LoadingError - try: - self.project = Project(filePath) - except LoadingError, e: - QtGui.QMessageBox.critical(self, "Error", u"Couldn't load project file:\n\n"+str(e)) - return - self.enableMenuItems() - self.projectTree.setProject(self.project) - - def closeProjectAction(self): - if not self.rejectClosingProject(): - self.closeProject() - - def addCatalogue(self): - filePath = QtGui.QFileDialog.getOpenFileName(self, "Select Project File", g_CWD, "Catalogue (*%s)" % g_catExt); - filePath = str(filePath) - # TODO: - #self.project.addLangFile(filePath) - - def addNewCatalogue(self): - pass - - def rejectClosingProject(self): - if self.project == None: - return False - - modifiedDocs = [] - # Check if any open document is modified. - for i in range(0, self.pages.count()): - if self.pages.widget(i).isModified: - modifiedDocs += [self.pages.widget(i)] - # Display dialog if so. - if len(modifiedDocs): - dialog = ClosingProjectDialog(modifiedDocs) - code = dialog.exec_() - if code == dialog.Accepted: - for doc in dialog.getSelectedDocs(): - self.saveDocument(doc) - elif code == dialog.Rejected: - return True - elif code == dialog.DiscardAll: - pass - - return False - - def closeProject(self): - if self.project == None: - return - self.project.save() - del self.project - self.project = None - self.disableMenuItems() - self.projectTree.clear() - self.pages.closeAll() - - def enableMenuItems(self): - #self.action_Close_Project.setEnabled(True) - for action in [ self.action_Save, - self.action_Save_All, - self.action_Close, - self.action_Close_All ]: - action.setEnabled(True) - self.menubar.insertMenu(self.menu_Help.menuAction(), self.menu_Project) - - def disableMenuItems(self): - #self.action_Close_Project.setEnabled(False) - for action in [ self.action_Save, - self.action_Save_All, - self.action_Close, - self.action_Close_All ]: - action.setEnabled(False) - self.menubar.removeAction(self.menu_Project.menuAction()) - - def projectTreeItemDblClicked(self, item, int): - self.projectTreeItemActivated(item) - - def projectTreeItemActivated(self, item): - if item == None: - return - - if isinstance(item, LangFileItem): - msgForm = None - if not item.isDocOpen(): - msgForm = item.openDoc() - msgForm.setModifiedCallback(self.formModified) - else: - msgForm = item.openDoc() - index = self.pages.indexOf(msgForm) - if index == -1: - index = self.pages.addTab(msgForm, msgForm.getDocumentTitle()) - self.pages.setCurrentIndex(index) - msgForm.updateData() - - def projectTreeItemDeleted(self, item): - pass - - def formModified(self, form): - # Append an asterisk to the tab label - index = self.pages.indexOf(form) - text = form.getDocumentTitle() + "*" - self.pages.setTabText(index, text) - - def saveForm(self): - self.saveDocument(self.pages.currentWidget()) - - def saveAllForms(self): - for i in range(0, self.pages.count()): - self.saveDocument(self.pages.widget(i)) - - def saveDocument(self, form): - if form.isModified: - # Reset tab text. - index = self.pages.indexOf(form) - text = form.getDocumentTitle() - self.pages.setTabText(index, text) - - form.save() - - def closeForm(self): - if self.pages.currentWidget(): - self.closeDocument(self.pages.currentWidget()) - - def closeAllForms(self): - for i in range(self.pages.count()-1, -1, -1): - self.closeDocument(self.pages.widget(i)) - - def closeDocument(self, form): - if form.isModified: - MB = QtGui.QMessageBox - button = MB.question(self, "Closing Document", "The document '%s' has been modified.\nDo you want to save the changes?" % form.getDocumentFullPath(), MB.Save | MB.Discard | MB.Cancel, MB.Cancel) - if button == MB.Cancel: - return False - if button == MB.Save: - self.saveDocument(form) - index = self.pages.indexOf(form) - self.pages.removeTab(index) - form.close() - return True - - def closeEvent(self, event): - if self.rejectClosingProject(): - event.ignore() - return - self.closeProject() - self.writeSettings() - # Closing application - - def moveToCenterOfDesktop(self): - rect = QtGui.QApplication.desktop().geometry() - self.move(rect.center() - self.rect().center()) - - def readSettings(self): - # Set default size - self.resize(QtCore.QSize(500, 400)) - doc = {} - try: - doc = yaml.load(open(g_settingsFile, "r")) - except: - self.moveToCenterOfDesktop() - return - - g_settings = doc - if not isinstance(doc, dict): - g_settings = {} - - try: - coord = doc["Window"] - size = QtCore.QSize(coord["Size"][0], coord["Size"][1]) - point = QtCore.QPoint(coord["Pos"][0], coord["Pos"][1]) - self.resize(size) - self.move(point) - except: - self.moveToCenterOfDesktop() - - def writeSettings(self): - # Save window coordinates - g_settings["Window"] = { - "Pos" : [self.pos().x(), self.pos().y()], - "Size" : [self.size().width(), self.size().height()] - } - yaml.dump(g_settings, open(g_settingsFile, "w")) #default_flow_style=False - - -class Document(QtGui.QWidget): - def __init__(self): - QtGui.QWidget.__init__(self) - self.isModified = False - self.modifiedCallback = None - self.documentTitle = "" - self.documentFullPath = "" - - def modified(self): - if not self.isModified: - self.isModified = True - self.modifiedCallback(self) - - def setModifiedCallback(self, func): - self.modifiedCallback = func - - def save(self): - self.isModified = False - - def close(self): - self.emit(Qt.SIGNAL("closed()")) - QtGui.QWidget.close(self) - - def getDocumentTitle(self): - return self.documentTitle - - def getDocumentFullPath(self): - return self.documentFullPath - - -class MessageItem(QtGui.QTreeWidgetItem): - def __init__(self, msg): - QtGui.QTreeWidgetItem.__init__(self, [str(msg["ID"]), msg["Text"], "Done"]) - self.msg = msg - - def getID(self): - return self.msg["ID"] - - def setMsgText(self, text): - self.msg["Text"] = text - - def setMsgAnnot(self, text): - self.msg["Annot"] = text - - -class MsgForm(Document, Ui_MsgForm): - def __init__(self, langFile): - Document.__init__(self) - self.documentTitle = langFile.getFileName() - self.documentFullPath = langFile.getFilePath() - self.setupUi(self) - self.vboxlayout.setMargin(0) - - self.langFile = langFile - self.currentItem = None - self.colID = 0 - self.colText = 1 - #self.colStat = 2 - #self.treeWidget.setColumnCount(3) - self.treeWidget.setColumnCount(2) - self.treeWidget.setHeaderLabels(["ID", "Text"]) #, "Status" - self.msgItemDict = {} # Maps msg IDs to msg items. - for msg in self.langFile.messages: - item = MessageItem(msg) - self.msgItemDict[msg["ID"]] = item - self.treeWidget.addTopLevelItem(item) - - Qt.connect(self.treeWidget, Qt.SIGNAL("currentItemChanged (QTreeWidgetItem *,QTreeWidgetItem *)"), self.treeItemChanged) - Qt.connect(self.translEdit, Qt.SIGNAL("textChanged()"), self.translEditTextChanged) - Qt.connect(self.translAnnotEdit, Qt.SIGNAL("textChanged()"), self.translAnnotEditTextChanged) - - #self.translEdit.focusOutEvent = self.translEditFocusOut - - def sourceMsgChanged(self, msg): - # TODO: - pass - - def treeItemChanged(self, current, previous): - if current == None: - self.setTranslMsg("") - self.setSourceMsg("") - return - ID = current.getID() - # Set the text controls. - # The slots receiving text changed signals do nothing if self.currentItem is None. - self.currentItem = None - self.setTranslMsg(self.langFile.getMsg(ID)) - self.setSourceMsg(self.langFile.source.getMsg(ID)) - self.currentItem = current - - def setTranslMsg(self, msg): - self.translEdit.setText(msg["Text"]) - self.translAnnotEdit.setText(msg["Annot"]) - - def setSourceMsg(self, msg): - self.sourceEdit.setText(msg["Text"]) - self.sourceAnnotEdit.setText(msg["Annot"]) - - #def translEditFocusOut(self, event): - #if self.currentItem: - #print self.currentItem.text(self.colText) - #if self.translEdit.document().isModified(): - #self.translEdit.document().setModified(False) - #print "translEdit was modified" - #QtGui.QTextEdit.focusOutEvent(self.translEdit, event) - - def translEditTextChanged(self): - if self.currentItem: - text = unicode(self.translEdit.toPlainText()) - self.currentItem.setText(self.colText, text) - self.currentItem.setMsgText(text) - self.modified() - - def translAnnotEditTextChanged(self): - if self.currentItem: - text = unicode(self.translAnnotEdit.toPlainText()) - self.currentItem.setMsgAnnot(text) - self.modified() - - def updateData(self): - if self.currentItem == None: - return - ID = self.currentItem.getID() - msg = self.langFile.source.getMsg(ID) - text = self.sourceEdit.toPlainText() - if text != msg["Text"]: - self.sourceEdit.setText(msg["Text"]) - text = self.sourceAnnotEdit.toPlainText() - if text != msg["Annot"]: - self.sourceAnnotEdit.setText(msg["Annot"]) - - def save(self): - Document.save(self) - self.langFile.save() - - -class MsgFormSource(MsgForm): - def __init__(self, langFile): - MsgForm.__init__(self, langFile) - - for x in [self.translEdit, - self.translAnnotEdit, - self.label_4, - self.label_5]: - x.close() - - Qt.connect(self.sourceEdit, Qt.SIGNAL("textChanged()"), self.sourceEditTextChanged) - Qt.connect(self.sourceAnnotEdit, Qt.SIGNAL("textChanged()"), self.sourceAnnotEditTextChanged) - - def treeItemChanged(self, current, previous): - if current == None: - self.setSourceMsg("") - return - ID = current.getID() - self.currentItem = None - self.setSourceMsg(self.langFile.getMsg(ID)) - self.currentItem = current - - def sourceEditTextChanged(self): - if self.currentItem: - text = unicode(self.sourceEdit.toPlainText()) - self.currentItem.setText(self.colText, text) - self.currentItem.setMsgText(text) - self.modified() - - def sourceAnnotEditTextChanged(self): - if self.currentItem: - text = unicode(self.sourceAnnotEdit.toPlainText()) - #self.currentItem.setText(self.colAnnot, text) - self.currentItem.setMsgAnnot(text) - self.modified() - - -class MsgIDItem(QtGui.QTreeWidgetItem): - def __init__(self, parent, text): - QtGui.QTreeWidgetItem.__init__(self, parent, [text]) - self.setFlags(self.flags()|Qt.ItemIsEditable); - -class LangFileItem(QtGui.QTreeWidgetItem): - def __init__(self, parent, langFile): - QtGui.QTreeWidgetItem.__init__(self, parent, [langFile.langCode]) - self.langFile = langFile - self.msgForm = None - - def isDocOpen(self): - return self.msgForm != None - - def openDoc(self): - if self.msgForm == None: - if self.langFile.isSource: - self.msgForm = MsgFormSource(self.langFile) - else: - self.msgForm = MsgForm(self.langFile) - Qt.connect(self.msgForm, Qt.SIGNAL("closed()"), self.docClosed) - return self.msgForm - - def docClosed(self): - self.msgForm = None - -class ProjectItem(QtGui.QTreeWidgetItem): - def __init__(self, text): - QtGui.QTreeWidgetItem.__init__(self, [text]) - self.setFlags(self.flags()|Qt.ItemIsEditable); - -class ProjectTree(QtGui.QTreeWidget): - def __init__(self, parent, mainWindow): - QtGui.QTreeWidget.__init__(self, parent) - self.mainWindow = mainWindow - self.project = None - self.topItem = None - self.msgIDsItem = None - self.headerItem().setHidden(True) - self.ignoreItemChanged = False - - def itemChanged(self, item, column): - if self.ignoreItemChanged: - return - text = unicode(item.text(0)) - #if hasattr(item, "textChanged"): - #item.textChanged(text) - if isinstance(item, ProjectItem): - self.project.setName(text) - print text - - def keyReleaseEvent(self, event): - Qt = QtCore.Qt - key = event.key() - if key in [Qt.Key_Enter, Qt.Key_Return]: - self.emit(Qt.SIGNAL("onKeyEnter"), self.currentItem()) - elif key == Qt.Key_Delete: - self.emit(Qt.SIGNAL("onKeyDelete"), self.currentItem()) - - def setProject(self, project): - self.project = project - - self.topItem = ProjectItem(self.project.name) - self.addTopLevelItem(self.topItem) - - for langFile in self.project.langFiles: - langFileItem = LangFileItem(self.topItem, langFile) - - self.msgIDsItem = QtGui.QTreeWidgetItem(self.topItem, ["Message IDs"]) - for msgID in self.project.msgIDs: - MsgIDItem(self.msgIDsItem, msgID["Name"]) - - for x in [self.topItem, self.msgIDsItem]: - x.setExpanded(True) - - Qt.connect(self, Qt.SIGNAL("itemChanged(QTreeWidgetItem*,int)"), self.itemChanged) - - def contextMenuEvent(self, event): - item = self.itemAt(event.pos()) - func_map = { - None : lambda item: None, - QtGui.QTreeWidgetItem : lambda item: None, - ProjectItem : self.showMenuProjectItem, - LangFileItem : self.showMenuLangFileItem, - MsgIDItem : self.showMenuMsgIDItem - } - func_map[type(item)](item) - - def showMenuProjectItem(self, item): - mousePos = QtGui.QCursor.pos() - menu = QtGui.QMenu() - actions = {} - actions[menu.addAction("Build")] = lambda: None - actions[menu.addAction("Properties")] = lambda: self.mainWindow.showProjectProperties() - actions[menu.exec_(mousePos)]() - - def showMenuLangFileItem(self, item): - print "LangFileItem" - - def showMenuMsgIDItem(self, item): - print "MsgIDItem" - - def updateProjectName(self): - self.ignoreItemChanged = True - self.topItem.setText(0, self.project.name) - self.ignoreItemChanged = False - - def clear(self): - self.project = None - self.topItem = None - self.msgIDsItem = None - Qt.disconnect(self, Qt.SIGNAL("itemChanged(QTreeWidgetItem*,int)"), self.itemChanged) - QtGui.QTreeWidget.clear(self) - - -class ClosingProjectDialog(QtGui.QDialog, Ui_ClosingProjectDialog): - DiscardAll = 2 - def __init__(self, docs): - QtGui.QDialog.__init__(self) - self.setupUi(self) - Qt.connect(self.button_Discard_All, Qt.SIGNAL("clicked()"), self.discardAll) - - self.items = [] - for doc in docs: - title = doc.getDocumentTitle() - path = doc.getDocumentFullPath() - item = QtGui.QTreeWidgetItem([title, path]) - item.doc = doc - item.setFlags(item.flags()|Qt.ItemIsUserCheckable); - item.setCheckState(0, Qt.Checked) - self.items += [item] - self.treeWidget.addTopLevelItems(self.items) - - self.button_Cancel.setFocus() - - def getSelectedDocs(self): - return [item.doc for item in self.items if item.checkState(0)] - - def discardAll(self): - self.done(self.DiscardAll) - - -class ProjectPropertiesDialog(QtGui.QDialog, Ui_ProjectPropertiesDialog): - def __init__(self, project): - QtGui.QDialog.__init__(self) - self.setupUi(self) - Qt.connect(self.pickFileButton, Qt.SIGNAL("clicked()"), self.pickFilePath) - - self.project = project - self.projectNameField.setText(self.project.name) - self.buildScriptField.setText(self.project.buildScript) - self.creationDateField.setText(self.project.creationDate) - - def pickFilePath(self): - filePath = QtGui.QFileDialog.getOpenFileName(self, "Select Build Script File", g_CWD, "Python Script (*.py)"); - if filePath: - self.buildScriptField.setText(str(filePath)) - - def accept(self): - self.project.setName(unicode(self.projectNameField.text())) - self.project.setBuildScript(unicode(self.buildScriptField.text())) - self.project.setCreationDate(str(self.creationDateField.text())) - QtGui.QDialog.accept(self) - - -class NewProjectDialog(QtGui.QDialog, Ui_NewProjectDialog): - def __init__(self): - QtGui.QDialog.__init__(self) - self.setupUi(self) - Qt.connect(self.pickFileButton, Qt.SIGNAL("clicked()"), self.pickFilePath) - - def pickFilePath(self): - filePath = QtGui.QFileDialog.getSaveFileName(self, "New Project File", g_CWD, "Translator Project (*%s)" % g_projectExt); - if filePath: - filePath = str(filePath) # Convert QString - if os.path.splitext(filePath)[1] != g_projectExt: - filePath += g_projectExt - self.projectFilePath.setText(filePath) - - def accept(self): - projectName = str(self.projectName.text()) - filePath = str(self.projectFilePath.text()) - - MB = QtGui.QMessageBox - if projectName == "": - MB.warning(self, "Warning", "Please, enter a name for the project.") - return - if filePath == "": - MB.warning(self, "Warning", "Please, choose or enter a path for the project file.") - return - - projectData = newProjectData(projectName) - - if os.path.splitext(filePath)[1] != g_projectExt: - filePath += g_projectExt - - try: - yaml.dump(projectData, open(filePath, "w"), default_flow_style=False) - except Exception, e: - MB.critical(self, "Error", str(e)) - return - - # Accept and close dialog. - QtGui.QDialog.accept(self) - -if __name__ == "__main__": - app = QtGui.QApplication(sys.argv) - main = MainWindow() - main.show() - sys.exit(app.exec_()) diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/translator.ui --- a/trunk/src/translator/translator.ui Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,174 +0,0 @@ - - MainWindow - - - - 0 - 0 - 608 - 464 - - - - Translator - - - - - - - - 0 - 0 - 608 - 29 - - - - - &File - - - - - - - - - - - - - - - &Help - - - - - - &Project - - - - - - - - - - - - - - - - - &Quit - - - Ctrl+Q - - - - - &About - - - - - &New Project... - - - Ctrl+N - - - - - &Add catalogue... - - - - - &Properties - - - - - &Build Project - - - - - &Open Project... - - - Ctrl+O - - - - - &Close - - - Ctrl+W - - - - - Add new catalogue... - - - - - &Close Project - - - Ctrl+F4 - - - - - &Save - - - Ctrl+S - - - - - Save &All - - - Ctrl+Shift+S - - - - - Clos&e All - - - Ctrl+Shift+W - - - - - - - action_Quit - triggered() - MainWindow - close() - - - -1 - -1 - - - 303 - 231 - - - - - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_about.py --- a/trunk/src/translator/ui_about.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,48 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'about.ui' -# -# Created: Sun Oct 21 17:33:29 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_AboutDialog(object): - def setupUi(self, AboutDialog): - AboutDialog.setObjectName("AboutDialog") - AboutDialog.setWindowModality(QtCore.Qt.WindowModal) - AboutDialog.resize(QtCore.QSize(QtCore.QRect(0,0,414,157).size()).expandedTo(AboutDialog.minimumSizeHint())) - - self.vboxlayout = QtGui.QVBoxLayout(AboutDialog) - self.vboxlayout.setObjectName("vboxlayout") - - self.label1 = QtGui.QLabel(AboutDialog) - self.label1.setTextFormat(QtCore.Qt.RichText) - self.label1.setAlignment(QtCore.Qt.AlignLeading|QtCore.Qt.AlignLeft|QtCore.Qt.AlignTop) - self.label1.setWordWrap(True) - self.label1.setObjectName("label1") - self.vboxlayout.addWidget(self.label1) - - self.buttonBox = QtGui.QDialogButtonBox(AboutDialog) - self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Ok) - self.buttonBox.setObjectName("buttonBox") - self.vboxlayout.addWidget(self.buttonBox) - - self.retranslateUi(AboutDialog) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),AboutDialog.close) - QtCore.QMetaObject.connectSlotsByName(AboutDialog) - - def retranslateUi(self, AboutDialog): - AboutDialog.setWindowTitle(QtGui.QApplication.translate("AboutDialog", "About", None, QtGui.QApplication.UnicodeUTF8)) - self.label1.setText(QtGui.QApplication.translate("AboutDialog", "\n" - "

    A tool for managing message catalogues in different languages.

    \n" - "

    These catalogues can be used to make an application multilingual.

    \n" - "

    \n" - "

    Copyright © 2007 by Aziz Köksal <aziz.koeksal@gmail.com>

    \n" - "

    Licensed under the GPL2.

    ", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_closing_project.py --- a/trunk/src/translator/ui_closing_project.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,65 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'closing_project.ui' -# -# Created: Sun Nov 11 23:01:28 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_ClosingProjectDialog(object): - def setupUi(self, ClosingProjectDialog): - ClosingProjectDialog.setObjectName("ClosingProjectDialog") - ClosingProjectDialog.resize(QtCore.QSize(QtCore.QRect(0,0,400,300).size()).expandedTo(ClosingProjectDialog.minimumSizeHint())) - - self.vboxlayout = QtGui.QVBoxLayout(ClosingProjectDialog) - self.vboxlayout.setObjectName("vboxlayout") - - self.label = QtGui.QLabel(ClosingProjectDialog) - self.label.setObjectName("label") - self.vboxlayout.addWidget(self.label) - - self.treeWidget = QtGui.QTreeWidget(ClosingProjectDialog) - self.treeWidget.setObjectName("treeWidget") - self.vboxlayout.addWidget(self.treeWidget) - - spacerItem = QtGui.QSpacerItem(382,33,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Expanding) - self.vboxlayout.addItem(spacerItem) - - self.hboxlayout = QtGui.QHBoxLayout() - self.hboxlayout.setObjectName("hboxlayout") - - spacerItem1 = QtGui.QSpacerItem(40,20,QtGui.QSizePolicy.Expanding,QtGui.QSizePolicy.Minimum) - self.hboxlayout.addItem(spacerItem1) - - self.button_Save_Selected = QtGui.QPushButton(ClosingProjectDialog) - self.button_Save_Selected.setObjectName("button_Save_Selected") - self.hboxlayout.addWidget(self.button_Save_Selected) - - self.button_Discard_All = QtGui.QPushButton(ClosingProjectDialog) - self.button_Discard_All.setObjectName("button_Discard_All") - self.hboxlayout.addWidget(self.button_Discard_All) - - self.button_Cancel = QtGui.QPushButton(ClosingProjectDialog) - self.button_Cancel.setDefault(True) - self.button_Cancel.setObjectName("button_Cancel") - self.hboxlayout.addWidget(self.button_Cancel) - self.vboxlayout.addLayout(self.hboxlayout) - - self.retranslateUi(ClosingProjectDialog) - QtCore.QObject.connect(self.button_Save_Selected,QtCore.SIGNAL("clicked()"),ClosingProjectDialog.accept) - QtCore.QObject.connect(self.button_Cancel,QtCore.SIGNAL("clicked()"),ClosingProjectDialog.reject) - QtCore.QMetaObject.connectSlotsByName(ClosingProjectDialog) - - def retranslateUi(self, ClosingProjectDialog): - ClosingProjectDialog.setWindowTitle(QtGui.QApplication.translate("ClosingProjectDialog", "Closing Project", None, QtGui.QApplication.UnicodeUTF8)) - self.label.setText(QtGui.QApplication.translate("ClosingProjectDialog", "The following documents have been modified:", None, QtGui.QApplication.UnicodeUTF8)) - self.treeWidget.headerItem().setText(0,QtGui.QApplication.translate("ClosingProjectDialog", "Title", None, QtGui.QApplication.UnicodeUTF8)) - self.treeWidget.headerItem().setText(1,QtGui.QApplication.translate("ClosingProjectDialog", "Full Path", None, QtGui.QApplication.UnicodeUTF8)) - self.button_Save_Selected.setText(QtGui.QApplication.translate("ClosingProjectDialog", "&Save Selected", None, QtGui.QApplication.UnicodeUTF8)) - self.button_Discard_All.setText(QtGui.QApplication.translate("ClosingProjectDialog", "&Discard All Changes", None, QtGui.QApplication.UnicodeUTF8)) - self.button_Cancel.setText(QtGui.QApplication.translate("ClosingProjectDialog", "&Cancel", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_langfile_properties.py --- a/trunk/src/translator/ui_langfile_properties.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,73 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'langfile_properties.ui' -# -# Created: Sat Nov 17 23:14:57 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_LangFilePropertiesDialog(object): - def setupUi(self, LangFilePropertiesDialog): - LangFilePropertiesDialog.setObjectName("LangFilePropertiesDialog") - LangFilePropertiesDialog.resize(QtCore.QSize(QtCore.QRect(0,0,400,228).size()).expandedTo(LangFilePropertiesDialog.minimumSizeHint())) - - self.vboxlayout = QtGui.QVBoxLayout(LangFilePropertiesDialog) - self.vboxlayout.setObjectName("vboxlayout") - - self.gridlayout = QtGui.QGridLayout() - self.gridlayout.setObjectName("gridlayout") - - self.label_2 = QtGui.QLabel(LangFilePropertiesDialog) - self.label_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label_2.setObjectName("label_2") - self.gridlayout.addWidget(self.label_2,1,0,1,1) - - self.langCodeField = QtGui.QLineEdit(LangFilePropertiesDialog) - self.langCodeField.setObjectName("langCodeField") - self.gridlayout.addWidget(self.langCodeField,1,1,1,1) - - self.label = QtGui.QLabel(LangFilePropertiesDialog) - self.label.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label.setObjectName("label") - self.gridlayout.addWidget(self.label,2,0,1,1) - - self.creationDateField = QtGui.QLineEdit(LangFilePropertiesDialog) - self.creationDateField.setObjectName("creationDateField") - self.gridlayout.addWidget(self.creationDateField,2,1,1,1) - - self.authorsField = QtGui.QTextEdit(LangFilePropertiesDialog) - self.authorsField.setObjectName("authorsField") - self.gridlayout.addWidget(self.authorsField,0,1,1,1) - - self.label_3 = QtGui.QLabel(LangFilePropertiesDialog) - self.label_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTop|QtCore.Qt.AlignTrailing) - self.label_3.setObjectName("label_3") - self.gridlayout.addWidget(self.label_3,0,0,1,1) - self.vboxlayout.addLayout(self.gridlayout) - - spacerItem = QtGui.QSpacerItem(20,10,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Fixed) - self.vboxlayout.addItem(spacerItem) - - self.buttonBox = QtGui.QDialogButtonBox(LangFilePropertiesDialog) - self.buttonBox.setOrientation(QtCore.Qt.Horizontal) - self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Cancel|QtGui.QDialogButtonBox.NoButton|QtGui.QDialogButtonBox.Ok) - self.buttonBox.setObjectName("buttonBox") - self.vboxlayout.addWidget(self.buttonBox) - self.label_2.setBuddy(self.langCodeField) - self.label.setBuddy(self.creationDateField) - - self.retranslateUi(LangFilePropertiesDialog) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),LangFilePropertiesDialog.accept) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("rejected()"),LangFilePropertiesDialog.reject) - QtCore.QMetaObject.connectSlotsByName(LangFilePropertiesDialog) - - def retranslateUi(self, LangFilePropertiesDialog): - LangFilePropertiesDialog.setWindowTitle(QtGui.QApplication.translate("LangFilePropertiesDialog", "Language File Properties", None, QtGui.QApplication.UnicodeUTF8)) - self.label_2.setText(QtGui.QApplication.translate("LangFilePropertiesDialog", "Language code:", None, QtGui.QApplication.UnicodeUTF8)) - self.label.setText(QtGui.QApplication.translate("LangFilePropertiesDialog", "Creation date:", None, QtGui.QApplication.UnicodeUTF8)) - self.label_3.setText(QtGui.QApplication.translate("LangFilePropertiesDialog", "Author(s):", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_msg_form.py --- a/trunk/src/translator/ui_msg_form.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,105 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'msg_form.ui' -# -# Created: Fri Nov 9 15:38:32 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_MsgForm(object): - def setupUi(self, MsgForm): - MsgForm.setObjectName("MsgForm") - MsgForm.resize(QtCore.QSize(QtCore.QRect(0,0,456,512).size()).expandedTo(MsgForm.minimumSizeHint())) - - self.vboxlayout = QtGui.QVBoxLayout(MsgForm) - self.vboxlayout.setObjectName("vboxlayout") - - self.splitter = QtGui.QSplitter(MsgForm) - self.splitter.setOrientation(QtCore.Qt.Vertical) - self.splitter.setObjectName("splitter") - - self.layoutWidget = QtGui.QWidget(self.splitter) - self.layoutWidget.setObjectName("layoutWidget") - - self.vboxlayout1 = QtGui.QVBoxLayout(self.layoutWidget) - self.vboxlayout1.setObjectName("vboxlayout1") - - self.label = QtGui.QLabel(self.layoutWidget) - self.label.setObjectName("label") - self.vboxlayout1.addWidget(self.label) - - self.treeWidget = QtGui.QTreeWidget(self.layoutWidget) - self.treeWidget.setObjectName("treeWidget") - self.vboxlayout1.addWidget(self.treeWidget) - - self.layoutWidget1 = QtGui.QWidget(self.splitter) - self.layoutWidget1.setObjectName("layoutWidget1") - - self.gridlayout = QtGui.QGridLayout(self.layoutWidget1) - self.gridlayout.setObjectName("gridlayout") - - self.vboxlayout2 = QtGui.QVBoxLayout() - self.vboxlayout2.setObjectName("vboxlayout2") - - self.label_2 = QtGui.QLabel(self.layoutWidget1) - self.label_2.setObjectName("label_2") - self.vboxlayout2.addWidget(self.label_2) - - self.sourceEdit = QtGui.QTextEdit(self.layoutWidget1) - self.sourceEdit.setObjectName("sourceEdit") - self.vboxlayout2.addWidget(self.sourceEdit) - self.gridlayout.addLayout(self.vboxlayout2,0,0,1,1) - - self.vboxlayout3 = QtGui.QVBoxLayout() - self.vboxlayout3.setObjectName("vboxlayout3") - - self.label_3 = QtGui.QLabel(self.layoutWidget1) - self.label_3.setObjectName("label_3") - self.vboxlayout3.addWidget(self.label_3) - - self.sourceAnnotEdit = QtGui.QTextEdit(self.layoutWidget1) - self.sourceAnnotEdit.setObjectName("sourceAnnotEdit") - self.vboxlayout3.addWidget(self.sourceAnnotEdit) - self.gridlayout.addLayout(self.vboxlayout3,0,1,1,1) - - self.vboxlayout4 = QtGui.QVBoxLayout() - self.vboxlayout4.setObjectName("vboxlayout4") - - self.label_4 = QtGui.QLabel(self.layoutWidget1) - self.label_4.setObjectName("label_4") - self.vboxlayout4.addWidget(self.label_4) - - self.translEdit = QtGui.QTextEdit(self.layoutWidget1) - self.translEdit.setObjectName("translEdit") - self.vboxlayout4.addWidget(self.translEdit) - self.gridlayout.addLayout(self.vboxlayout4,1,0,1,1) - - self.vboxlayout5 = QtGui.QVBoxLayout() - self.vboxlayout5.setObjectName("vboxlayout5") - - self.label_5 = QtGui.QLabel(self.layoutWidget1) - self.label_5.setObjectName("label_5") - self.vboxlayout5.addWidget(self.label_5) - - self.translAnnotEdit = QtGui.QTextEdit(self.layoutWidget1) - self.translAnnotEdit.setObjectName("translAnnotEdit") - self.vboxlayout5.addWidget(self.translAnnotEdit) - self.gridlayout.addLayout(self.vboxlayout5,1,1,1,1) - self.vboxlayout.addWidget(self.splitter) - - self.retranslateUi(MsgForm) - QtCore.QMetaObject.connectSlotsByName(MsgForm) - - def retranslateUi(self, MsgForm): - MsgForm.setWindowTitle(QtGui.QApplication.translate("MsgForm", "Form", None, QtGui.QApplication.UnicodeUTF8)) - self.label.setText(QtGui.QApplication.translate("MsgForm", "Messages:", None, QtGui.QApplication.UnicodeUTF8)) - self.treeWidget.headerItem().setText(0,QtGui.QApplication.translate("MsgForm", "1", None, QtGui.QApplication.UnicodeUTF8)) - self.label_2.setText(QtGui.QApplication.translate("MsgForm", "Source string:", None, QtGui.QApplication.UnicodeUTF8)) - self.label_3.setText(QtGui.QApplication.translate("MsgForm", "Source annotation:", None, QtGui.QApplication.UnicodeUTF8)) - self.label_4.setText(QtGui.QApplication.translate("MsgForm", "Translation:", None, QtGui.QApplication.UnicodeUTF8)) - self.label_5.setText(QtGui.QApplication.translate("MsgForm", "Translator\'s annotation:", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_new_project.py --- a/trunk/src/translator/ui_new_project.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,76 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'new_project.ui' -# -# Created: Sun Oct 21 17:33:29 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_NewProjectDialog(object): - def setupUi(self, NewProjectDialog): - NewProjectDialog.setObjectName("NewProjectDialog") - NewProjectDialog.resize(QtCore.QSize(QtCore.QRect(0,0,401,131).size()).expandedTo(NewProjectDialog.minimumSizeHint())) - - self.vboxlayout = QtGui.QVBoxLayout(NewProjectDialog) - self.vboxlayout.setObjectName("vboxlayout") - - self.gridlayout = QtGui.QGridLayout() - self.gridlayout.setObjectName("gridlayout") - - self.gridlayout1 = QtGui.QGridLayout() - self.gridlayout1.setObjectName("gridlayout1") - - self.label = QtGui.QLabel(NewProjectDialog) - self.label.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label.setObjectName("label") - self.gridlayout1.addWidget(self.label,0,0,1,1) - - self.projectName = QtGui.QLineEdit(NewProjectDialog) - self.projectName.setObjectName("projectName") - self.gridlayout1.addWidget(self.projectName,0,1,1,1) - - self.label_2 = QtGui.QLabel(NewProjectDialog) - self.label_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label_2.setObjectName("label_2") - self.gridlayout1.addWidget(self.label_2,1,0,1,1) - - self.hboxlayout = QtGui.QHBoxLayout() - self.hboxlayout.setObjectName("hboxlayout") - - self.projectFilePath = QtGui.QLineEdit(NewProjectDialog) - self.projectFilePath.setObjectName("projectFilePath") - self.hboxlayout.addWidget(self.projectFilePath) - - self.pickFileButton = QtGui.QToolButton(NewProjectDialog) - self.pickFileButton.setObjectName("pickFileButton") - self.hboxlayout.addWidget(self.pickFileButton) - self.gridlayout1.addLayout(self.hboxlayout,1,1,1,1) - self.gridlayout.addLayout(self.gridlayout1,0,0,1,1) - - spacerItem = QtGui.QSpacerItem(20,40,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Expanding) - self.gridlayout.addItem(spacerItem,1,0,1,1) - - self.buttonBox = QtGui.QDialogButtonBox(NewProjectDialog) - self.buttonBox.setOrientation(QtCore.Qt.Horizontal) - self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Cancel|QtGui.QDialogButtonBox.NoButton|QtGui.QDialogButtonBox.Ok) - self.buttonBox.setObjectName("buttonBox") - self.gridlayout.addWidget(self.buttonBox,2,0,1,1) - self.vboxlayout.addLayout(self.gridlayout) - self.label.setBuddy(self.projectName) - self.label_2.setBuddy(self.projectFilePath) - - self.retranslateUi(NewProjectDialog) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),NewProjectDialog.accept) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("rejected()"),NewProjectDialog.reject) - QtCore.QMetaObject.connectSlotsByName(NewProjectDialog) - - def retranslateUi(self, NewProjectDialog): - NewProjectDialog.setWindowTitle(QtGui.QApplication.translate("NewProjectDialog", "Create New Project", None, QtGui.QApplication.UnicodeUTF8)) - self.label.setText(QtGui.QApplication.translate("NewProjectDialog", "Project name:", None, QtGui.QApplication.UnicodeUTF8)) - self.label_2.setText(QtGui.QApplication.translate("NewProjectDialog", "Project file:", None, QtGui.QApplication.UnicodeUTF8)) - self.pickFileButton.setText(QtGui.QApplication.translate("NewProjectDialog", "...", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_project_properties.py --- a/trunk/src/translator/ui_project_properties.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,82 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'project_properties.ui' -# -# Created: Sat Nov 17 23:14:55 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_ProjectPropertiesDialog(object): - def setupUi(self, ProjectPropertiesDialog): - ProjectPropertiesDialog.setObjectName("ProjectPropertiesDialog") - ProjectPropertiesDialog.resize(QtCore.QSize(QtCore.QRect(0,0,400,152).size()).expandedTo(ProjectPropertiesDialog.minimumSizeHint())) - - self.vboxlayout = QtGui.QVBoxLayout(ProjectPropertiesDialog) - self.vboxlayout.setObjectName("vboxlayout") - - self.gridlayout = QtGui.QGridLayout() - self.gridlayout.setObjectName("gridlayout") - - self.label_2 = QtGui.QLabel(ProjectPropertiesDialog) - self.label_2.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label_2.setObjectName("label_2") - self.gridlayout.addWidget(self.label_2,0,0,1,1) - - self.projectNameField = QtGui.QLineEdit(ProjectPropertiesDialog) - self.projectNameField.setObjectName("projectNameField") - self.gridlayout.addWidget(self.projectNameField,0,1,1,1) - - self.label = QtGui.QLabel(ProjectPropertiesDialog) - self.label.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label.setObjectName("label") - self.gridlayout.addWidget(self.label,1,0,1,1) - - self.hboxlayout = QtGui.QHBoxLayout() - self.hboxlayout.setObjectName("hboxlayout") - - self.buildScriptField = QtGui.QLineEdit(ProjectPropertiesDialog) - self.buildScriptField.setObjectName("buildScriptField") - self.hboxlayout.addWidget(self.buildScriptField) - - self.pickFileButton = QtGui.QToolButton(ProjectPropertiesDialog) - self.pickFileButton.setObjectName("pickFileButton") - self.hboxlayout.addWidget(self.pickFileButton) - self.gridlayout.addLayout(self.hboxlayout,1,1,1,1) - - self.creationDateField = QtGui.QLineEdit(ProjectPropertiesDialog) - self.creationDateField.setObjectName("creationDateField") - self.gridlayout.addWidget(self.creationDateField,2,1,1,1) - - self.label_3 = QtGui.QLabel(ProjectPropertiesDialog) - self.label_3.setAlignment(QtCore.Qt.AlignRight|QtCore.Qt.AlignTrailing|QtCore.Qt.AlignVCenter) - self.label_3.setObjectName("label_3") - self.gridlayout.addWidget(self.label_3,2,0,1,1) - self.vboxlayout.addLayout(self.gridlayout) - - spacerItem = QtGui.QSpacerItem(20,40,QtGui.QSizePolicy.Minimum,QtGui.QSizePolicy.Expanding) - self.vboxlayout.addItem(spacerItem) - - self.buttonBox = QtGui.QDialogButtonBox(ProjectPropertiesDialog) - self.buttonBox.setOrientation(QtCore.Qt.Horizontal) - self.buttonBox.setStandardButtons(QtGui.QDialogButtonBox.Cancel|QtGui.QDialogButtonBox.NoButton|QtGui.QDialogButtonBox.Ok) - self.buttonBox.setObjectName("buttonBox") - self.vboxlayout.addWidget(self.buttonBox) - self.label_2.setBuddy(self.projectNameField) - self.label.setBuddy(self.buildScriptField) - - self.retranslateUi(ProjectPropertiesDialog) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("accepted()"),ProjectPropertiesDialog.accept) - QtCore.QObject.connect(self.buttonBox,QtCore.SIGNAL("rejected()"),ProjectPropertiesDialog.reject) - QtCore.QMetaObject.connectSlotsByName(ProjectPropertiesDialog) - - def retranslateUi(self, ProjectPropertiesDialog): - ProjectPropertiesDialog.setWindowTitle(QtGui.QApplication.translate("ProjectPropertiesDialog", "Project Properties", None, QtGui.QApplication.UnicodeUTF8)) - self.label_2.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "Project name:", None, QtGui.QApplication.UnicodeUTF8)) - self.label.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "Build script:", None, QtGui.QApplication.UnicodeUTF8)) - self.pickFileButton.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "...", None, QtGui.QApplication.UnicodeUTF8)) - self.label_3.setText(QtGui.QApplication.translate("ProjectPropertiesDialog", "Creation date:", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/translator/ui_translator.py --- a/trunk/src/translator/ui_translator.py Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,133 +0,0 @@ -# -*- coding: utf-8 -*- - -# Form implementation generated from reading ui file 'translator.ui' -# -# Created: Fri Nov 9 20:02:57 2007 -# by: PyQt4 UI code generator 4.1 -# -# WARNING! All changes made in this file will be lost! - -import sys -from PyQt4 import QtCore, QtGui - -class Ui_MainWindow(object): - def setupUi(self, MainWindow): - MainWindow.setObjectName("MainWindow") - MainWindow.resize(QtCore.QSize(QtCore.QRect(0,0,608,464).size()).expandedTo(MainWindow.minimumSizeHint())) - - self.centralwidget = QtGui.QWidget(MainWindow) - self.centralwidget.setObjectName("centralwidget") - - self.vboxlayout = QtGui.QVBoxLayout(self.centralwidget) - self.vboxlayout.setObjectName("vboxlayout") - MainWindow.setCentralWidget(self.centralwidget) - - self.menubar = QtGui.QMenuBar(MainWindow) - self.menubar.setGeometry(QtCore.QRect(0,0,608,29)) - self.menubar.setObjectName("menubar") - - self.menu_File = QtGui.QMenu(self.menubar) - self.menu_File.setObjectName("menu_File") - - self.menu_Help = QtGui.QMenu(self.menubar) - self.menu_Help.setObjectName("menu_Help") - - self.menu_Project = QtGui.QMenu(self.menubar) - self.menu_Project.setObjectName("menu_Project") - MainWindow.setMenuBar(self.menubar) - - self.statusbar = QtGui.QStatusBar(MainWindow) - self.statusbar.setObjectName("statusbar") - MainWindow.setStatusBar(self.statusbar) - - self.action_Quit = QtGui.QAction(MainWindow) - self.action_Quit.setObjectName("action_Quit") - - self.action_About = QtGui.QAction(MainWindow) - self.action_About.setObjectName("action_About") - - self.action_New_Project = QtGui.QAction(MainWindow) - self.action_New_Project.setObjectName("action_New_Project") - - self.action_Add_Catalogue = QtGui.QAction(MainWindow) - self.action_Add_Catalogue.setObjectName("action_Add_Catalogue") - - self.action_Properties = QtGui.QAction(MainWindow) - self.action_Properties.setObjectName("action_Properties") - - self.action_Build_Project = QtGui.QAction(MainWindow) - self.action_Build_Project.setObjectName("action_Build_Project") - - self.action_Open_Project = QtGui.QAction(MainWindow) - self.action_Open_Project.setObjectName("action_Open_Project") - - self.action_Close = QtGui.QAction(MainWindow) - self.action_Close.setObjectName("action_Close") - - self.action_Add_New_Catalogue = QtGui.QAction(MainWindow) - self.action_Add_New_Catalogue.setObjectName("action_Add_New_Catalogue") - - self.action_Close_Project = QtGui.QAction(MainWindow) - self.action_Close_Project.setObjectName("action_Close_Project") - - self.action_Save = QtGui.QAction(MainWindow) - self.action_Save.setObjectName("action_Save") - - self.action_Save_All = QtGui.QAction(MainWindow) - self.action_Save_All.setObjectName("action_Save_All") - - self.action_Close_All = QtGui.QAction(MainWindow) - self.action_Close_All.setObjectName("action_Close_All") - self.menu_File.addAction(self.action_Open_Project) - self.menu_File.addAction(self.action_New_Project) - self.menu_File.addSeparator() - self.menu_File.addAction(self.action_Save) - self.menu_File.addAction(self.action_Save_All) - self.menu_File.addSeparator() - self.menu_File.addAction(self.action_Close) - self.menu_File.addAction(self.action_Close_All) - self.menu_File.addSeparator() - self.menu_File.addAction(self.action_Quit) - self.menu_Help.addAction(self.action_About) - self.menu_Project.addAction(self.action_Add_Catalogue) - self.menu_Project.addAction(self.action_Add_New_Catalogue) - self.menu_Project.addAction(self.action_Build_Project) - self.menu_Project.addSeparator() - self.menu_Project.addAction(self.action_Properties) - self.menu_Project.addSeparator() - self.menu_Project.addAction(self.action_Close_Project) - self.menubar.addAction(self.menu_File.menuAction()) - self.menubar.addAction(self.menu_Project.menuAction()) - self.menubar.addAction(self.menu_Help.menuAction()) - - self.retranslateUi(MainWindow) - QtCore.QObject.connect(self.action_Quit,QtCore.SIGNAL("triggered()"),MainWindow.close) - QtCore.QMetaObject.connectSlotsByName(MainWindow) - - def retranslateUi(self, MainWindow): - MainWindow.setWindowTitle(QtGui.QApplication.translate("MainWindow", "Translator", None, QtGui.QApplication.UnicodeUTF8)) - self.menu_File.setTitle(QtGui.QApplication.translate("MainWindow", "&File", None, QtGui.QApplication.UnicodeUTF8)) - self.menu_Help.setTitle(QtGui.QApplication.translate("MainWindow", "&Help", None, QtGui.QApplication.UnicodeUTF8)) - self.menu_Project.setTitle(QtGui.QApplication.translate("MainWindow", "&Project", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Quit.setText(QtGui.QApplication.translate("MainWindow", "&Quit", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Quit.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+Q", None, QtGui.QApplication.UnicodeUTF8)) - self.action_About.setText(QtGui.QApplication.translate("MainWindow", "&About", None, QtGui.QApplication.UnicodeUTF8)) - self.action_New_Project.setText(QtGui.QApplication.translate("MainWindow", "&New Project...", None, QtGui.QApplication.UnicodeUTF8)) - self.action_New_Project.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+N", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Add_Catalogue.setText(QtGui.QApplication.translate("MainWindow", "&Add catalogue...", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Properties.setText(QtGui.QApplication.translate("MainWindow", "&Properties", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Build_Project.setText(QtGui.QApplication.translate("MainWindow", "&Build Project", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Open_Project.setText(QtGui.QApplication.translate("MainWindow", "&Open Project...", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Open_Project.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+O", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Close.setText(QtGui.QApplication.translate("MainWindow", "&Close", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Close.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+W", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Add_New_Catalogue.setText(QtGui.QApplication.translate("MainWindow", "Add new catalogue...", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Close_Project.setText(QtGui.QApplication.translate("MainWindow", "&Close Project", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Close_Project.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+F4", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Save.setText(QtGui.QApplication.translate("MainWindow", "&Save", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Save.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+S", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Save_All.setText(QtGui.QApplication.translate("MainWindow", "Save &All", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Save_All.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+Shift+S", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Close_All.setText(QtGui.QApplication.translate("MainWindow", "Clos&e All", None, QtGui.QApplication.UnicodeUTF8)) - self.action_Close_All.setShortcut(QtGui.QApplication.translate("MainWindow", "Ctrl+Shift+W", None, QtGui.QApplication.UnicodeUTF8)) - diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/util/uni.d --- a/trunk/src/util/uni.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,632 +0,0 @@ - -// Written in the D programming language. - -/* - * Placed into the Public Domain. - * Digital Mars, www.digitalmars.com - * Written by Walter Bright - */ - -/** - * Simple Unicode character classification functions. - * For ASCII classification, see $(LINK2 std_ctype.html, std.ctype). - * Macros: - * WIKI=Phobos/StdUni - * References: - * $(LINK2 http://www.digitalmars.com/d/ascii-table.html, ASCII Table), - * $(LINK2 http://en.wikipedia.org/wiki/Unicode, Wikipedia), - * $(LINK2 http://www.unicode.org, The Unicode Consortium) - * Trademarks: - * Unicode(tm) is a trademark of Unicode, Inc. - * Copyright: - * Public Domain. - */ - - -module util.uni; - -/** - * Returns !=0 if c is a Unicode lower case character. - */ -int isUniLower(dchar c) -{ - if (c <= 0x7F) - return (c >= 'a' && c <= 'z'); - - return isUniAlpha(c) && c == toUniLower(c); -} - -/** - * Returns !=0 if c is a Unicode upper case character. - */ -int isUniUpper(dchar c) -{ - if (c <= 0x7F) - return (c >= 'A' && c <= 'Z'); - - return isUniAlpha(c) && c == toUniUpper(c); -} - -/** - * If c is a Unicode upper case character, return the lower case - * equivalent, otherwise return c. - */ -dchar toUniLower(dchar c) -{ - if (c >= 'A' && c <= 'Z') - { - c += 32; - } - else if (c >= 0x00C0) - { - if ((c >= 0x00C0 && c <= 0x00D6) || (c >= 0x00D8 && c<=0x00DE)) - { - c += 32; - } - else if ((c >= 0x0100 && c < 0x0138) || (c > 0x0149 && c < 0x0178)) - { - if (c == 0x0130) - c = 0x0069; - else if ((c & 1) == 0) - c += 1; - } - else if (c == 0x0178) - { - c = 0x00FF; - } - else if ((c >= 0x0139 && c < 0x0149) || (c > 0x0178 && c < 0x017F)) - { - if (c & 1) - c += 1; - } - else if (c >= 0x0200 && c <= 0x0217) - { - if ((c & 1) == 0) - c += 1; - } - else if ((c >= 0x0401 && c <= 0x040C) || (c>= 0x040E && c <= 0x040F)) - { - c += 80; - } - else if (c >= 0x0410 && c <= 0x042F) - { - c += 32; - } - else if (c >= 0x0460 && c <= 0x047F) - { - if ((c & 1) == 0) - c += 1; - } - else if (c >= 0x0531 && c <= 0x0556) - { - c += 48; - } - else if (c >= 0x10A0 && c <= 0x10C5) - { - c += 48; - } - else if (c >= 0xFF21 && c <= 0xFF3A) - { - c += 32; - } - } - return c; -} - -/** - * If c is a Unicode lower case character, return the upper case - * equivalent, otherwise return c. - */ -dchar toUniUpper(dchar c) -{ - if (c >= 'a' && c <= 'z') - { - c -= 32; - } - else if (c >= 0x00E0) - { - if ((c >= 0x00E0 && c <= 0x00F6) || (c >= 0x00F8 && c <= 0x00FE)) - { - c -= 32; - } - else if (c == 0x00FF) - { - c = 0x0178; - } - else if ((c >= 0x0100 && c < 0x0138) || (c > 0x0149 && c < 0x0178)) - { - if (c == 0x0131) - c = 0x0049; - else if (c & 1) - c -= 1; - } - else if ((c >= 0x0139 && c < 0x0149) || (c > 0x0178 && c < 0x017F)) - { - if ((c & 1) == 0) - c = c-1; - } - else if (c == 0x017F) - { - c = 0x0053; - } - else if (c >= 0x0200 && c <= 0x0217) - { - if (c & 1) - c = c-1; - } - else if (c >= 0x0430 && c<= 0x044F) - { - c -= 32; - } - else if ((c >= 0x0451 && c <= 0x045C) || (c >=0x045E && c<= 0x045F)) - { - c -= 80; - } - else if (c >= 0x0460 && c <= 0x047F) - { - if (c & 1) - c -= 1; - } - else if (c >= 0x0561 && c < 0x0587) - { - c -= 48; - } - else if (c >= 0xFF41 && c <= 0xFF5A) - { - c -= 32; - } - } - return c; -} - - -/******************************* - * Return !=0 if u is a Unicode alpha character. - * (general Unicode category: Lu, Ll, Lt, Lm and Lo) - * - * Standards: Unicode 5.0.0 - */ - -int isUniAlpha(dchar u) -{ - static dchar table[][2] = - [ - [ 'A', 'Z' ], - [ 'a', 'z' ], - [ 0x00AA, 0x00AA ], - [ 0x00B5, 0x00B5 ], - [ 0x00BA, 0x00BA ], - [ 0x00C0, 0x00D6 ], - [ 0x00D8, 0x00F6 ], - [ 0x00F8, 0x02C1 ], - [ 0x02C6, 0x02D1 ], - [ 0x02E0, 0x02E4 ], - [ 0x02EE, 0x02EE ], - [ 0x037A, 0x037D ], - [ 0x0386, 0x0386 ], - [ 0x0388, 0x038A ], - [ 0x038C, 0x038C ], - [ 0x038E, 0x03A1 ], - [ 0x03A3, 0x03CE ], - [ 0x03D0, 0x03F5 ], - [ 0x03F7, 0x0481 ], - [ 0x048A, 0x0513 ], - [ 0x0531, 0x0556 ], - [ 0x0559, 0x0559 ], - [ 0x0561, 0x0587 ], - [ 0x05D0, 0x05EA ], - [ 0x05F0, 0x05F2 ], - [ 0x0621, 0x063A ], - [ 0x0640, 0x064A ], - [ 0x066E, 0x066F ], - [ 0x0671, 0x06D3 ], - [ 0x06D5, 0x06D5 ], - [ 0x06E5, 0x06E6 ], - [ 0x06EE, 0x06EF ], - [ 0x06FA, 0x06FC ], - [ 0x06FF, 0x06FF ], - [ 0x0710, 0x0710 ], - [ 0x0712, 0x072F ], - [ 0x074D, 0x076D ], - [ 0x0780, 0x07A5 ], - [ 0x07B1, 0x07B1 ], - [ 0x07CA, 0x07EA ], - [ 0x07F4, 0x07F5 ], - [ 0x07FA, 0x07FA ], - [ 0x0904, 0x0939 ], - [ 0x093D, 0x093D ], - [ 0x0950, 0x0950 ], - [ 0x0958, 0x0961 ], - [ 0x097B, 0x097F ], - [ 0x0985, 0x098C ], - [ 0x098F, 0x0990 ], - [ 0x0993, 0x09A8 ], - [ 0x09AA, 0x09B0 ], - [ 0x09B2, 0x09B2 ], - [ 0x09B6, 0x09B9 ], - [ 0x09BD, 0x09BD ], - [ 0x09CE, 0x09CE ], - [ 0x09DC, 0x09DD ], - [ 0x09DF, 0x09E1 ], - [ 0x09F0, 0x09F1 ], - [ 0x0A05, 0x0A0A ], - [ 0x0A0F, 0x0A10 ], - [ 0x0A13, 0x0A28 ], - [ 0x0A2A, 0x0A30 ], - [ 0x0A32, 0x0A33 ], - [ 0x0A35, 0x0A36 ], - [ 0x0A38, 0x0A39 ], - [ 0x0A59, 0x0A5C ], - [ 0x0A5E, 0x0A5E ], - [ 0x0A72, 0x0A74 ], - [ 0x0A85, 0x0A8D ], - [ 0x0A8F, 0x0A91 ], - [ 0x0A93, 0x0AA8 ], - [ 0x0AAA, 0x0AB0 ], - [ 0x0AB2, 0x0AB3 ], - [ 0x0AB5, 0x0AB9 ], - [ 0x0ABD, 0x0ABD ], - [ 0x0AD0, 0x0AD0 ], - [ 0x0AE0, 0x0AE1 ], - [ 0x0B05, 0x0B0C ], - [ 0x0B0F, 0x0B10 ], - [ 0x0B13, 0x0B28 ], - [ 0x0B2A, 0x0B30 ], - [ 0x0B32, 0x0B33 ], - [ 0x0B35, 0x0B39 ], - [ 0x0B3D, 0x0B3D ], - [ 0x0B5C, 0x0B5D ], - [ 0x0B5F, 0x0B61 ], - [ 0x0B71, 0x0B71 ], - [ 0x0B83, 0x0B83 ], - [ 0x0B85, 0x0B8A ], - [ 0x0B8E, 0x0B90 ], - [ 0x0B92, 0x0B95 ], - [ 0x0B99, 0x0B9A ], - [ 0x0B9C, 0x0B9C ], - [ 0x0B9E, 0x0B9F ], - [ 0x0BA3, 0x0BA4 ], - [ 0x0BA8, 0x0BAA ], - [ 0x0BAE, 0x0BB9 ], - [ 0x0C05, 0x0C0C ], - [ 0x0C0E, 0x0C10 ], - [ 0x0C12, 0x0C28 ], - [ 0x0C2A, 0x0C33 ], - [ 0x0C35, 0x0C39 ], - [ 0x0C60, 0x0C61 ], - [ 0x0C85, 0x0C8C ], - [ 0x0C8E, 0x0C90 ], - [ 0x0C92, 0x0CA8 ], - [ 0x0CAA, 0x0CB3 ], - [ 0x0CB5, 0x0CB9 ], - [ 0x0CBD, 0x0CBD ], - [ 0x0CDE, 0x0CDE ], - [ 0x0CE0, 0x0CE1 ], - [ 0x0D05, 0x0D0C ], - [ 0x0D0E, 0x0D10 ], - [ 0x0D12, 0x0D28 ], - [ 0x0D2A, 0x0D39 ], - [ 0x0D60, 0x0D61 ], - [ 0x0D85, 0x0D96 ], - [ 0x0D9A, 0x0DB1 ], - [ 0x0DB3, 0x0DBB ], - [ 0x0DBD, 0x0DBD ], - [ 0x0DC0, 0x0DC6 ], - [ 0x0E01, 0x0E30 ], - [ 0x0E32, 0x0E33 ], - [ 0x0E40, 0x0E46 ], - [ 0x0E81, 0x0E82 ], - [ 0x0E84, 0x0E84 ], - [ 0x0E87, 0x0E88 ], - [ 0x0E8A, 0x0E8A ], - [ 0x0E8D, 0x0E8D ], - [ 0x0E94, 0x0E97 ], - [ 0x0E99, 0x0E9F ], - [ 0x0EA1, 0x0EA3 ], - [ 0x0EA5, 0x0EA5 ], - [ 0x0EA7, 0x0EA7 ], - [ 0x0EAA, 0x0EAB ], - [ 0x0EAD, 0x0EB0 ], - [ 0x0EB2, 0x0EB3 ], - [ 0x0EBD, 0x0EBD ], - [ 0x0EC0, 0x0EC4 ], - [ 0x0EC6, 0x0EC6 ], - [ 0x0EDC, 0x0EDD ], - [ 0x0F00, 0x0F00 ], - [ 0x0F40, 0x0F47 ], - [ 0x0F49, 0x0F6A ], - [ 0x0F88, 0x0F8B ], - [ 0x1000, 0x1021 ], - [ 0x1023, 0x1027 ], - [ 0x1029, 0x102A ], - [ 0x1050, 0x1055 ], - [ 0x10A0, 0x10C5 ], - [ 0x10D0, 0x10FA ], - [ 0x10FC, 0x10FC ], - [ 0x1100, 0x1159 ], - [ 0x115F, 0x11A2 ], - [ 0x11A8, 0x11F9 ], - [ 0x1200, 0x1248 ], - [ 0x124A, 0x124D ], - [ 0x1250, 0x1256 ], - [ 0x1258, 0x1258 ], - [ 0x125A, 0x125D ], - [ 0x1260, 0x1288 ], - [ 0x128A, 0x128D ], - [ 0x1290, 0x12B0 ], - [ 0x12B2, 0x12B5 ], - [ 0x12B8, 0x12BE ], - [ 0x12C0, 0x12C0 ], - [ 0x12C2, 0x12C5 ], - [ 0x12C8, 0x12D6 ], - [ 0x12D8, 0x1310 ], - [ 0x1312, 0x1315 ], - [ 0x1318, 0x135A ], - [ 0x1380, 0x138F ], - [ 0x13A0, 0x13F4 ], - [ 0x1401, 0x166C ], - [ 0x166F, 0x1676 ], - [ 0x1681, 0x169A ], - [ 0x16A0, 0x16EA ], - [ 0x1700, 0x170C ], - [ 0x170E, 0x1711 ], - [ 0x1720, 0x1731 ], - [ 0x1740, 0x1751 ], - [ 0x1760, 0x176C ], - [ 0x176E, 0x1770 ], - [ 0x1780, 0x17B3 ], - [ 0x17D7, 0x17D7 ], - [ 0x17DC, 0x17DC ], - [ 0x1820, 0x1877 ], - [ 0x1880, 0x18A8 ], - [ 0x1900, 0x191C ], - [ 0x1950, 0x196D ], - [ 0x1970, 0x1974 ], - [ 0x1980, 0x19A9 ], - [ 0x19C1, 0x19C7 ], - [ 0x1A00, 0x1A16 ], - [ 0x1B05, 0x1B33 ], - [ 0x1B45, 0x1B4B ], - [ 0x1D00, 0x1DBF ], - [ 0x1E00, 0x1E9B ], - [ 0x1EA0, 0x1EF9 ], - [ 0x1F00, 0x1F15 ], - [ 0x1F18, 0x1F1D ], - [ 0x1F20, 0x1F45 ], - [ 0x1F48, 0x1F4D ], - [ 0x1F50, 0x1F57 ], - [ 0x1F59, 0x1F59 ], - [ 0x1F5B, 0x1F5B ], - [ 0x1F5D, 0x1F5D ], - [ 0x1F5F, 0x1F7D ], - [ 0x1F80, 0x1FB4 ], - [ 0x1FB6, 0x1FBC ], - [ 0x1FBE, 0x1FBE ], - [ 0x1FC2, 0x1FC4 ], - [ 0x1FC6, 0x1FCC ], - [ 0x1FD0, 0x1FD3 ], - [ 0x1FD6, 0x1FDB ], - [ 0x1FE0, 0x1FEC ], - [ 0x1FF2, 0x1FF4 ], - [ 0x1FF6, 0x1FFC ], - [ 0x2071, 0x2071 ], - [ 0x207F, 0x207F ], - [ 0x2090, 0x2094 ], - [ 0x2102, 0x2102 ], - [ 0x2107, 0x2107 ], - [ 0x210A, 0x2113 ], - [ 0x2115, 0x2115 ], - [ 0x2119, 0x211D ], - [ 0x2124, 0x2124 ], - [ 0x2126, 0x2126 ], - [ 0x2128, 0x2128 ], - [ 0x212A, 0x212D ], - [ 0x212F, 0x2139 ], - [ 0x213C, 0x213F ], - [ 0x2145, 0x2149 ], - [ 0x214E, 0x214E ], - [ 0x2183, 0x2184 ], - [ 0x2C00, 0x2C2E ], - [ 0x2C30, 0x2C5E ], - [ 0x2C60, 0x2C6C ], - [ 0x2C74, 0x2C77 ], - [ 0x2C80, 0x2CE4 ], - [ 0x2D00, 0x2D25 ], - [ 0x2D30, 0x2D65 ], - [ 0x2D6F, 0x2D6F ], - [ 0x2D80, 0x2D96 ], - [ 0x2DA0, 0x2DA6 ], - [ 0x2DA8, 0x2DAE ], - [ 0x2DB0, 0x2DB6 ], - [ 0x2DB8, 0x2DBE ], - [ 0x2DC0, 0x2DC6 ], - [ 0x2DC8, 0x2DCE ], - [ 0x2DD0, 0x2DD6 ], - [ 0x2DD8, 0x2DDE ], - [ 0x3005, 0x3006 ], - [ 0x3031, 0x3035 ], - [ 0x303B, 0x303C ], - [ 0x3041, 0x3096 ], - [ 0x309D, 0x309F ], - [ 0x30A1, 0x30FA ], - [ 0x30FC, 0x30FF ], - [ 0x3105, 0x312C ], - [ 0x3131, 0x318E ], - [ 0x31A0, 0x31B7 ], - [ 0x31F0, 0x31FF ], - [ 0x3400, 0x4DB5 ], - [ 0x4E00, 0x9FBB ], - [ 0xA000, 0xA48C ], - [ 0xA717, 0xA71A ], - [ 0xA800, 0xA801 ], - [ 0xA803, 0xA805 ], - [ 0xA807, 0xA80A ], - [ 0xA80C, 0xA822 ], - [ 0xA840, 0xA873 ], - [ 0xAC00, 0xD7A3 ], - [ 0xF900, 0xFA2D ], - [ 0xFA30, 0xFA6A ], - [ 0xFA70, 0xFAD9 ], - [ 0xFB00, 0xFB06 ], - [ 0xFB13, 0xFB17 ], - [ 0xFB1D, 0xFB1D ], - [ 0xFB1F, 0xFB28 ], - [ 0xFB2A, 0xFB36 ], - [ 0xFB38, 0xFB3C ], - [ 0xFB3E, 0xFB3E ], - [ 0xFB40, 0xFB41 ], - [ 0xFB43, 0xFB44 ], - [ 0xFB46, 0xFBB1 ], - [ 0xFBD3, 0xFD3D ], - [ 0xFD50, 0xFD8F ], - [ 0xFD92, 0xFDC7 ], - [ 0xFDF0, 0xFDFB ], - [ 0xFE70, 0xFE74 ], - [ 0xFE76, 0xFEFC ], - [ 0xFF21, 0xFF3A ], - [ 0xFF41, 0xFF5A ], - [ 0xFF66, 0xFFBE ], - [ 0xFFC2, 0xFFC7 ], - [ 0xFFCA, 0xFFCF ], - [ 0xFFD2, 0xFFD7 ], - [ 0xFFDA, 0xFFDC ], - [ 0x10000, 0x1000B ], - [ 0x1000D, 0x10026 ], - [ 0x10028, 0x1003A ], - [ 0x1003C, 0x1003D ], - [ 0x1003F, 0x1004D ], - [ 0x10050, 0x1005D ], - [ 0x10080, 0x100FA ], - [ 0x10300, 0x1031E ], - [ 0x10330, 0x10340 ], - [ 0x10342, 0x10349 ], - [ 0x10380, 0x1039D ], - [ 0x103A0, 0x103C3 ], - [ 0x103C8, 0x103CF ], - [ 0x10400, 0x1049D ], - [ 0x10800, 0x10805 ], - [ 0x10808, 0x10808 ], - [ 0x1080A, 0x10835 ], - [ 0x10837, 0x10838 ], - [ 0x1083C, 0x1083C ], - [ 0x1083F, 0x1083F ], - [ 0x10900, 0x10915 ], - [ 0x10A00, 0x10A00 ], - [ 0x10A10, 0x10A13 ], - [ 0x10A15, 0x10A17 ], - [ 0x10A19, 0x10A33 ], - [ 0x12000, 0x1236E ], - [ 0x1D400, 0x1D454 ], - [ 0x1D456, 0x1D49C ], - [ 0x1D49E, 0x1D49F ], - [ 0x1D4A2, 0x1D4A2 ], - [ 0x1D4A5, 0x1D4A6 ], - [ 0x1D4A9, 0x1D4AC ], - [ 0x1D4AE, 0x1D4B9 ], - [ 0x1D4BB, 0x1D4BB ], - [ 0x1D4BD, 0x1D4C3 ], - [ 0x1D4C5, 0x1D505 ], - [ 0x1D507, 0x1D50A ], - [ 0x1D50D, 0x1D514 ], - [ 0x1D516, 0x1D51C ], - [ 0x1D51E, 0x1D539 ], - [ 0x1D53B, 0x1D53E ], - [ 0x1D540, 0x1D544 ], - [ 0x1D546, 0x1D546 ], - [ 0x1D54A, 0x1D550 ], - [ 0x1D552, 0x1D6A5 ], - [ 0x1D6A8, 0x1D6C0 ], - [ 0x1D6C2, 0x1D6DA ], - [ 0x1D6DC, 0x1D6FA ], - [ 0x1D6FC, 0x1D714 ], - [ 0x1D716, 0x1D734 ], - [ 0x1D736, 0x1D74E ], - [ 0x1D750, 0x1D76E ], - [ 0x1D770, 0x1D788 ], - [ 0x1D78A, 0x1D7A8 ], - [ 0x1D7AA, 0x1D7C2 ], - [ 0x1D7C4, 0x1D7CB ], - [ 0x20000, 0x2A6D6 ], - [ 0x2F800, 0x2FA1D ], - ]; - - debug - { - for (int i = 0; i < table.length; i++) - { - assert(table[i][0] <= table[i][1]); - if (i < table.length - 1) - { -// if (table[i][1] >= table[i + 1][0]) -// printf("table[%d][1] = x%x, table[%d][0] = x%x\n", i, table[i][1], i + 1, table[i + 1][0]); - assert(table[i][1] < table[i + 1][0]); - } - } - } - - if (u < 0xAA) - { - if (u < 'A') - goto Lisnot; - if (u <= 'Z') - goto Lis; - if (u < 'a') - goto Lisnot; - if (u <= 'z') - goto Lis; - goto Lisnot; - } - - // Binary search - uint mid; - uint low; - uint high; - - low = 0; - high = table.length - 1; - while (cast(int)low <= cast(int)high) - { - mid = (low + high) >> 1; - if (u < table[mid][0]) - high = mid - 1; - else if (u > table[mid][1]) - low = mid + 1; - else - goto Lis; - } - -Lisnot: - debug - { - for (int i = 0; i < table.length; i++) - { - assert(u < table[i][0] || u > table[i][1]); - } - } - return 0; - -Lis: - debug - { - for (int i = 0; i < table.length; i++) - { - if (u >= table[i][0] && u <= table[i][1]) - return 1; - } - assert(0); // should have been in table - } - return 1; -} - -unittest -{ - for (uint i = 0; i < 0x80; i++) - { - if (i >= 'A' && i <= 'Z') - assert(isUniAlpha(i)); - else if (i >= 'a' && i <= 'z') - assert(isUniAlpha(i)); - else - assert(!isUniAlpha(i)); - } -} diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/xml.css --- a/trunk/src/xml.css Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,71 +0,0 @@ -@charset "utf-8"; -compilerinfo, sourcecode, linescolumn { - white-space: pre; - font-family: Monospace; - font-size: 0.8em; -} -compilerinfo, sourcecode { - display: block; -} -compilerinfo { - white-space: normal; - border: 1px solid #A22; - padding: 0.5em; - margin: 1em; -} -compilerinfo error { display: block; } -linescolumn { - display: block; - float: left; - text-align: right; - margin-right: 0.2em; - border-right: 1px solid gray; -} -linescolumn a { display: block; color: #555; } -/* Number */ -n { color: teal; } -/* Keyword */ -k { color: darkblue; font-weight: bold; } -/* Line, block and nested comments */ -lc, bc, nc { color: green; } -/* Identifier */ -i { color: black; } -/* String literal */ -sl { color: red; } -/* Character literal */ -cl { color: purple; } -/* All bracket types */ -br { color: orange; } -/* Special tokens */ -st { color: green; font-weight: bold; } -/* #line, hash line */ -hl { color: green; } -/* filespec (e.g. #line number [filespec]) */ -fs { color: purple;} -/* When the first line starts with #! it's a "shebang" */ -shebang { color: gray; } -/* Deprecated styles. */ -/* Operator */ -/*op { color: royalblue; }*/ -/* Particular operators */ -/*op[t=aa] { content: "and"; }*/ /*&& ∧*/ -/*op[t=oo] { content: "or"; }*/ /*|| ∨*/ -/*op[t=n] { content: "¬"; }*/ /*!*/ -/*op[t=ne] { content: "≠"; }*/ /*!=*/ -/*op[t=le] { content: "≤"; }*/ /*<=*/ -/*op[t=ge] { content: "≥"; }*/ /*>=*/ -/*op[t=lg] { content: "≶"; }*/ /*<>*/ -/* -d = Declaration -s = Statement -e = Expression -t = Type -o = Other -*/ -/* d { background-color: #FFDDDD; } */ -/* e { background-color: #DDDDFF; } */ -d[t=Illegal], s[t=Illegal] { background-color: #DD4422; } -d[t=Module] i, d[t=Import] i { color: blue; } -t > i { color: #911; } -t > br, t > op { color: #911; } -t[t=Integral] k { color: #911; font-weight: normal; } diff -r a3fab8b74a7d -r bcb74c9b895c trunk/src/xml_map.d --- a/trunk/src/xml_map.d Sat Mar 08 22:09:59 2008 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,107 +0,0 @@ -/// A map of document elements and D tokens to format strings. -string[string] map = [ - "DocHead" : ``\n - ``\n - "\n", - "DocEnd" : "\n", - "SourceBegin" : "", - "SourceEnd" : "\n", - "CompBegin" : "\n", - "CompEnd" : "\n", - "LexerError" : `{0}({1},{2})L: {3}`\n, - "ParserError" : `{0}({1},{2})P: {3}`\n, - "LineNumberBegin" : ``, - "LineNumberEnd" : ``, - "LineNumber" : `{0}`, - - // Node categories: - "Declaration" : "d", - "Statement" : "s", - "Expression" : "e", - "Type" : "t", - "Other" : "o", - - // {0} = node category. - // {1} = node class name: "Call", "If", "Class" etc. - // E.g.: ... - "NodeBegin" : `<{0} t="{1}">`, - "NodeEnd" : ``, - - "Identifier" : "{0}", - "String" : "{0}", - "Char" : "{0}", - "Number" : "{0}", - "Keyword" : "{0}", - - "LineC" : "{0}", - "BlockC" : "{0}", - "NestedC" : "{0}", - - "Shebang" : "{0}", - "HLine" : "{0}", // #line - "Filespec" : "{0}", // #line N "filespec" - "Newline" : "{0}", // \n | \r | \r\n | LS | PS - "Illegal" : "{0}", // A character not recognized by the lexer. - - "SpecialToken" : "{0}", // __FILE__, __LINE__ etc. - - "(" : "
    (
    ", - ")" : "
    )
    ", - "[" : "
    [
    ", - "]" : "
    ]
    ", - "{" : "
    {
    ", - "}" : "
    }
    ", - "." : ".", - ".." : "..", - "..." : "...", - "!<>=" : "!<>=", // Unordered - "!<>" : "!<>", // UorE - "!<=" : "!<=", // UorG - "!<" : "!<", // UorGorE - "!>=" : "!>=", // UorL - "!>" : "!>", // UorLorE - "<>=" : "<>=", // LorEorG - "<>" : "<>", // LorG - "=" : "=", - "==" : "==", - "!" : "!", - "!=" : "!=", - "<=" : "<=", - "<" : "<", - ">=" : ">=", - ">" : ">", - "<<=" : "<<=", - "<<" : "<<", - ">>=" : ">>=", - ">>" : ">>", - ">>>=" : ">>>=", - ">>>" : ">>>", - "|" : "|", - "||" : "||", - "|=" : "|=", - "&" : "&", - "&&" : "&&", - "&=" : "&=", - "+" : "+", - "++" : "++", - "+=" : "+=", - "-" : "-", - "--" : "--", - "-=" : "-=", - "/" : "/", - "/=" : "/=", - "*" : "*", - "*=" : "*=", - "%" : "%", - "%=" : "%=", - "^" : "^", - "^=" : "^=", - "~" : "~", - "~=" : "~=", - ":" : ":", - ";" : ";", - "?" : "?", - "," : ",", - "$" : "$", - "EOF" : "" -];