annotate base/src/java/nonstandard/SafeUtf.d @ 120:536e43f63c81

Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661 ===D2=== * added [Try]Immutable/Const/Shared templates to work with differenses in D1/D2 instead of version statements used these templates to work with strict type storage rules of dmd-2.053 * com.ibm.icu now also compilable with D2, but not tested yet * small fixes Snippet288 - shared data is in TLS ===Phobos=== * fixed critical bugs in Phobos implemention completely incorrect segfault prone fromStringz (Linux's port ruthless killer) terrible, incorrect StringBuffer realization (StyledText killer) * fixed small bugs as well Snippet72 - misprint in the snippet * implemented missed functionality for Phobos ByteArrayOutputStream implemented (image loading available) formatting correctly works for all DWT's cases As a result, folowing snippets now works with Phobos (Snippet### - what is fixed): Snippet24, 42, 111, 115, 130, 235, 276 - bad string formatting Snippet48, 282 - crash on image loading Snippet163, 189, 211, 213, 217, 218, 222 - crash on copy/cut in StyledText Snippet244 - hang-up ===Tango=== * few changes for the latest Tango trunc-r5661 * few small performance improvments ===General=== * implMissing-s for only one version changed to implMissingInTango/InPhobos * incorrect calls to Format in toString-s fixed * fixed loading \uXXXX characters in ResourceBundle * added good UTF-8 support for StyledText, TextLayout (Win32) and friends UTF functions revised and tested. It is now in java.nonstandard.*Utf modules StyledText and TextLayout (Win32) modules revised for UTF-8 support * removed small diferences in most identical files in *.swt.* folders *.swt.internal.image, *.swt.events and *.swt.custom are identical in Win32/Linux32 now 179 of 576 (~31%) files in *.swt.* folders are fully identical * Win32: snippets now have right subsystem, pretty icons and native system style controls * small fixes in snippets Snippet44 - it's not Snippet44 Snippet212 - functions work with different images and offsets arrays Win32: Snippet282 - crash on close if the button has an image Snippet293 - setGrayed is commented and others Win32: As a result, folowing snippets now works Snippet68 - color doesn't change Snippet163, 189, 211, 213, 217, 218, 222 - UTF-8 issues (see above) Snippet193 - no tabel headers
author Denis Shelomovskij <verylonglogin.reg@gmail.com>
date Sat, 09 Jul 2011 15:50:20 +0300
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
120
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
1 /**
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
2 * Stuff for working with narrow strings.
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
3 * Safe because of strong type checking.
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
4 *
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
5 * Authors: Denis Shelomovskij <verylonglogin.reg@gmail.com>
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
6 */
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
7 module java.nonstandard.SafeUtf;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
8
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
9 import java.nonstandard.UtfBase;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
10
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
11 private const bool UTFTypeCheck = true;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
12 mixin(UtfBaseText);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
13
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
14 unittest {
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
15 auto s = "abаб回家\U00064321\U00064321d"; assert(s.length == 1+1+2+2+3+3+4+4+1);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
16 auto ws = "abаб回家\U00064321\U00064321d"w; assert(ws.length == 1+1+1+1+1+1+2+2+1);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
17 auto dchars = "abаб回家\U00064321\U00064321d"d;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
18 auto starts = [1, 1, 1,0, 1,0, 1,0,0, 1,0,0, 1,0,0,0, 1,0,0,0, 1];
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
19 auto wstarts = [1, 1, 1, 1, 1, 1, 1,0, 1,0 , 1];
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
20 assert(s.length == starts.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
21 assert(ws.length == wstarts.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
22
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
23 auto strides = [1, 1, 2, 2, 3, 3, 4, 4, 1];
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
24 auto wstrides = [1, 1, 1, 1, 1, 1, 2, 2, 1];
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
25 auto shifts0 = [0, 1, 1+1, 1+1+2, 1+1+2+2, 1+1+2+2+3, 1+1+2+2+3+3, 1+1+2+2+3+3+4, 1+1+2+2+3+3+4+4];
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
26 assert(strides.length == dchars.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
27 assert(wstrides.length == dchars.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
28 assert(shifts0.length == dchars.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
29
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
30 UTF8index prevStart = 0;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
31 UCSindex n = 0;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
32 foreach(UTF8index i, char ch; s) {
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
33 assert(s.isUTF8sequenceStart(i) == starts[i]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
34 if(starts[i]) {
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
35 s.validateUTF8index(i);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
36 assert(s.UTF8strideAt(i) == strides[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
37 assert(s.toUTF8shift(0, n) == shifts0[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
38 assert(s.toUTF8shift(shifts0[n], -n) == -shifts0[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
39 if(i) assert(s.offsetBefore(i) == prevStart);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
40 assert(s[0 .. val(i)].UCScount == n);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
41 assert(s[val(i) .. $].UCScount == strides.length - n);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
42
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
43 UTF8shift di;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
44 assert(s.dcharAt(i, di) == dchars[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
45 assert(di == strides[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
46 if(i) assert(s.dcharBefore(i) == s.dcharAt(prevStart));
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
47 if(i) assert(s.dcharAfter(prevStart) == s.dcharAt(i));
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
48 auto dcharStr = s[val(i) .. val(i) + strides[n]];
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
49 assert(s.dcharAsStringAt(i, di) == dcharStr && di == dcharStr.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
50 assert(dcharToString(s.dcharAt(i)) == dcharStr);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
51 prevStart = i;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
52 ++n;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
53 }
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
54 UTF8index t = i;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
55 s.adjustUTF8index(t);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
56 assert(t == prevStart);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
57 }
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
58
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
59 n = 0;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
60 foreach(UTF16index i, wchar ch; ws)
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
61 if(wstarts[i]) {
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
62 //s.validateUTF16index(i);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
63 UTF16shift di;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
64 assert(ws.dcharAt(i, di) == dchars[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
65 assert(di == wstrides[n]);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
66 ++n;
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
67 }
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
68
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
69 s.validateUTF8index(s.length);
536e43f63c81 Comprehensive update for Win32/Linux32 dmd-2.053/dmd-1.068+Tango-r5661
Denis Shelomovskij <verylonglogin.reg@gmail.com>
parents:
diff changeset
70 }