Commit graph

40 commits

Author SHA1 Message Date
Markus Scherer
51e21af692 ICU-22707 Unicode 16 aug16 2024-08-19 17:15:00 -07:00
Markus Scherer
4312a4e5cb ICU-22707 Unicode 16 beta jun04 2024-07-18 23:56:34 +00:00
Markus Scherer
560e4bbf41 ICU-22707 preparseucd.py 2024-04-29 17:00:55 -07:00
Markus Scherer
d8659b476d ICU-22404 new properties IDS_Unary_Operator, ID_Compat_Math_*, NFKC_SCF 2023-09-16 14:41:51 -07:00
Markus Scherer
81a6edb287 ICU-22404 Unicode 15.1 data 20230811 plus UTS46 fix 2023-08-16 14:25:22 -07:00
Elango Cheran
2e45e6ec0e ICU-22404 Unicode 15.1 beta data files & API constants
See #2492

Co-authored-by: Andy Heninger <andy.heninger@gmail.com>
Co-authored-by: Robin Leroy <egg.robin.leroy@gmail.com>
2023-07-13 19:26:14 -07:00
Markus Scherer
e1be738ccb ICU-21980 Unicode 15 pre-beta data files, new prop values 2022-05-25 18:23:11 +00:00
gnrunge
f37a5e0090 ICU-21796 Rename bazel build files from BUILD to BUILD.bazel. This can
prevent conflicts when ICU users have their own BUILD files already.
2021-12-16 06:55:09 -08:00
Markus Scherer
41aa7159ea ICU-21635 Unicode 14 data files 20210820, line break LB30b.2
See #1807
2021-08-23 22:11:49 +00:00
Markus Scherer
d4c92ebcfc ICU-21635 Unicode 14 beta 2021-06-21 22:26:15 +00:00
Elango Cheran
227c729b0e ICU-21117 Use Bazel to automate generation of Unicode data files 2021-03-24 10:39:38 -07:00
Markus Scherer
a7e378d587 ICU-20893 Unicode 13 beta
See PR #915, see changes.txt
- Unicode 13 beta data as of 2019-nov-21
- uprops.icu format version 7.7 with more bits for Script/Script_Extensions
- more bits in spoof checker ScriptSet
- root line break rules adjusted for UAX 14 changes, from Andy
- line break tailorings not yet in sync with root
2019-11-21 17:35:53 -08:00
Markus Scherer
0565894534 ICU-20497 Unicode 12.1 2019-04-04 10:23:24 -07:00
Markus Scherer
98589d9cc7 ICU-20203 Unicode 12 final data (only trivial changes) 2019-03-13 08:57:05 -07:00
Markus Scherer
ea7c030961 ICU-20203 update ICU to Unicode 12 beta
- data as of 2018-nov-26
- API constants for new blocks & scripts
- sync RBBIMonkeyTest.java test data with C++
2018-11-28 23:13:07 +01:00
Markus Scherer
fe3eb3ed5c
ICU-13530 add UCPTrie/CodePointTrie, switch normalization to use it (#48)
* ICU-13530 copy C/C++ files UTrie2 -> UTrie3

X-SVN-Rev: 40754

* ICU-13530 UTrie3 new files copied from UTrie2: rename types/functions/macros

X-SVN-Rev: 40755

* ICU-13530 debug-print building each UTrie2

X-SVN-Rev: 40756

* ICU-13530 remove two-byte-UTF-8 errorValue block; move highValue from end of data array into header; add errorValue to header

X-SVN-Rev: 40762

* ICU-13530 UTrie3 U16_NEXT/PREV: errorValue for unpaired surrogates

X-SVN-Rev: 40763

* ICU-13530 no more separate values for lead surrogate code units

X-SVN-Rev: 40764

* ICU-13530 change from 11:5 trie bits to 10:6 for simpler UTF-8 code

X-SVN-Rev: 40766

* ICU-13530 UTrie2 build UTrie3 as well, print sizes

X-SVN-Rev: 40767

* ICU-13530 debug-print countSame, sumOverlaps, countInitial

X-SVN-Rev: 40768

* ICU-13530 debug-print whether trie is for CanonIterData

X-SVN-Rev: 40769

* ICU-13530 no index-shift for BMP data, no separate index-2 for 2-byte UTF-8; builder changes incomplete

X-SVN-Rev: 40777

* ICU-13530 remove errorValue and highStart from UNewTrie3

X-SVN-Rev: 40778

* ICU-13530 rewrite UTrie3 builder code

X-SVN-Rev: 40783

* ICU-13530 UTrie3 bug fixes

X-SVN-Rev: 40788

* ICU-13530 fully re-inline _UTRIE3_U8_NEXT()

X-SVN-Rev: 40790

* ICU-13530 find most common all-same data block for dataNullBlock and initialValue

X-SVN-Rev: 40792

* ICU-13530 UTrie3 iterator functions take start and return the end of a range, rather than callback call for each range

X-SVN-Rev: 40800

* ICU-13530 mask off unused data value bits before building a UTrie3 with values less than 32 bits wide

X-SVN-Rev: 40803

* ICU-13530 split utrie3builder.h out of utrie3.h

X-SVN-Rev: 40804

* ICU-13530 separate types UTrie3 vs. UTrie3Builder, implement builder as wrapper over C++ class Trie3Builder in .cpp

X-SVN-Rev: 40809

* ICU-13530 function to make a UTrie3Builder from a UTrie3

X-SVN-Rev: 40810

* ICU-13530 debug-print some data; some cleanup

X-SVN-Rev: 40865

* ICU-13530 BMP 10:6 but supplementary 10:6:4

X-SVN-Rev: 40984

* ICU-13530 move errorValue & highValue to the end of the data table, minimal padding to 4 bytes

X-SVN-Rev: 41011

* ICU-13530 index-1 table gap of index-2 null blocks

X-SVN-Rev: 41018

* ICU-13530 test with more than 128k compacted data

X-SVN-Rev: 41034

* ICU-13530 supplementary bits 11:5:4 saves a little space

X-SVN-Rev: 41039

* ICU-13530 supplementary bits 6:5:5:4 instead of gap: about same size but simpler

X-SVN-Rev: 41050

* ICU-13530 remove unnecessary utrie3_clone(built trie)

X-SVN-Rev: 41058

* ICU-13530 remove unnecessary UTrie3StringIterator

X-SVN-Rev: 41059

* ICU-13530 back to UTRIE3_GET...() macros *returning* data values

X-SVN-Rev: 41060

* ICU-13530 fast vs. small

X-SVN-Rev: 41066

* ICU-13530 always load NFC data, add simple normalization performance test

X-SVN-Rev: 41110

* ICU-13530 change normalization main trie to UTrie3 with special values for lead surrogates; forbid non-inert surrogate code *points* because unable to store values different from code *units*; runtime code work around that for code point lookup and iteration; adjust UTS 46 for normalization no longer mapping unpaired surrogates to U+FFFD

X-SVN-Rev: 41122

* ICU-13530 simplenormperf bug fix and NFC base line

X-SVN-Rev: 41126

* ICU-13530 move normalization getRange skipping lead surrogates to API getRangeSkipLead()

X-SVN-Rev: 41182

* ICU-13530 switch CanonIterData and gennorm2 Norms to UTrie3

X-SVN-Rev: 41183

* ICU-13530 remove unused overwrite parameter from setRange()

X-SVN-Rev: 41184

* ICU-13530 getRange skip lead -> fixed surrogates

X-SVN-Rev: 41219

* ICU-13530 minor cleanup

X-SVN-Rev: 41221

* ICU-13530 UTS 46 code map unpaired surrogates to U+FFFD before normalization

X-SVN-Rev: 41224

* ICU-13530 minor internal-docs cleanup

X-SVN-Rev: 41225

* ICU-13530 rename UTrie3 to UCPTrie, and other name changes

X-SVN-Rev: 41226

* ICU-13530 add 8-bit data option; add type-any & valueBits-any for fromBinary(); macros consistently source type then data width

X-SVN-Rev: 41234

* ICU-13530 scrub the API docs for the proposal

X-SVN-Rev: 41319

* ICU-13530 tag internal definitions as such, or move them to an internal header

X-SVN-Rev: 41320

* ICU-13530 Java API skeleton

X-SVN-Rev: 41326

* ICU-13530 API feedback: ValueWidth, MutableCodePointTrie, base CodePointMap, ...

X-SVN-Rev: 41382

* ICU-13530 add UCPTrie valueWidth field and padding, and combine data pointers into a union

X-SVN-Rev: 41408

* ICU-13530 switch some macros to using dataAccess parameter: separate index vs. data lookups, no macro variant for each value width

X-SVN-Rev: 41409

* ICU-13530 StringIterator is no longer a java.util.Iterator (bad fit)

X-SVN-Rev: 41455

* ICU-13530 CodePointTrie.java code complete

X-SVN-Rev: 41518

* ICU-13530 finish Java port incl test; keep C++ parallel

* ICU-13530 adjust API for feedback: rename HandleValue to FilterValue, change getRange+getRangeFixedSurr(bool allSurr) to enum RangeOption+getRange(enum option); change remaining C macros to use dataAccess for 16/32/8-bit value widths; fix/clarify some API docs

* ICU-13530 add javadoc

* ICU-13530 document UCPTrie binary data format

* ICU-13530 update .nrm formatVersion 3->4, document change in surrogate handling with new trie

* ICU-13530 re-hardcode NFC data

* move trie swapper code into new file; add new files to Windows project files; turn off trie debugging

* ICU-13530 minor cleanup

* ICU-13530 test more range starts; fix a C test leak

* ICU-13530 regenerate Java data from scratch

* ICU-13530 review feedback changes: API docs typos, more @internal, C++11 field initializers, fix potential leak in MutableCodePointTrie::fromUCPTrie()

* ICU-13530 rename interface FilterValue to ValueFilter
2018-09-27 14:27:38 -07:00
Markus Scherer
03303a6cb6 ICU-13630 Unicode 11 beta data apr02 (security apr03), fix ICU4C tests except RBBI
X-SVN-Rev: 41191
2018-04-03 23:09:49 +00:00
Markus Scherer
1752b5c8c9 ICU-13630 Unicode 11 beta data mar06, API constants for new property values
X-SVN-Rev: 41092
2018-03-09 23:53:02 +00:00
Markus Scherer
edce2be62c ICU-12985 Unicode 10 data 20170418, new property values, adjust tools & tests
X-SVN-Rev: 40079
2017-04-26 21:17:13 +00:00
Markus Scherer
466a569c58 ICU-12900 mostly still Unicode 9.0 but Unicode 10 beta (20170322) segmentation & bidi data and draft emoji 5.0 (also 20170322)
X-SVN-Rev: 39915
2017-03-23 02:14:00 +00:00
Markus Scherer
962c595814 ICU-12669 draft Emoji 4.0 properties data
X-SVN-Rev: 39067
2016-08-18 21:08:23 +00:00
Michael Ow
61607c2773 ICU-12564 Update copyright notice in trunk
X-SVN-Rev: 38848
2016-06-15 18:58:17 +00:00
Markus Scherer
5e69db5c2f ICU-12526 integrate Unicode 9 beta
X-SVN-Rev: 38753
2016-05-19 22:48:18 +00:00
Markus Scherer
c10fdc6e29 ICU-11802 add 4 Emoji properties from emoji-data.txt 2.0
X-SVN-Rev: 38183
2016-01-21 04:39:21 +00:00
Markus Scherer
7c3234e250 ICU-11574 Unicode 8.0 final data
X-SVN-Rev: 37548
2015-06-12 11:10:10 +00:00
Markus Scherer
fc290db4c1 ICU-11574 near-final Unicode 8 data with new block API constants, code adjustments & fixes; svn merge --reintegrate branches/markus/uni80
X-SVN-Rev: 37473
2015-05-28 23:16:00 +00:00
Markus Scherer
29fc864adf ICU-11002 Unicode 7.0 final IdnaMappingTable
X-SVN-Rev: 36020
2014-07-10 19:05:06 +00:00
Markus Scherer
567aebade1 ICU-10821 Unicode 7.0 in trunk, from branches/markus/uni70@35631
X-SVN-Rev: 35632
2014-04-14 22:30:09 +00:00
Markus Scherer
889c8b2429 ICU-10678 cherry-pick Unicode 7.0 U+20BD RUBLE SIGN for ICU 53
X-SVN-Rev: 35241
2014-02-26 21:34:14 +00:00
Markus Scherer
2982958b06 ICU-10128 update ICU to Unicode 6.3 beta (merge from branches/markus/uni63 at r33661)
X-SVN-Rev: 33662
2013-05-15 21:51:04 +00:00
Markus Scherer
6876123c0e ICU-9437 update to Unicode 6.2; merge in branches/markus/uni62 r32274
X-SVN-Rev: 32282
2012-08-28 23:37:10 +00:00
Markus Scherer
2b7a619808 ICU-8995 update to 2011-12-15 version of IdnaMappingTable.txt
X-SVN-Rev: 31230
2012-01-19 18:51:47 +00:00
Markus Scherer
b19326fa13 ICU-9023 reduce norm2/nfkc.txt to a delta over nfc.txt
X-SVN-Rev: 31201
2012-01-12 01:03:31 +00:00
Markus Scherer
d3fce873c6 ICU-8972 generate norm2/nfkc_cf.txt from preparseucd.py
X-SVN-Rev: 31198
2012-01-10 23:00:05 +00:00
Markus Scherer
6d038075aa ICU-8972 gennorm2 parse * Unicode version lines from .txt input files
X-SVN-Rev: 31193
2012-01-10 21:04:58 +00:00
Markus Scherer
2f5489adb1 ICU-8972 norm2/nf[k]c.txt files generated by preparseucd.py
X-SVN-Rev: 31175
2011-12-28 19:24:15 +00:00
Markus Scherer
43bb4f2cf4 ICU-8827 Unicode 6.1 update: svn merge branches/markus/uni61 at r31062 into the trunk
X-SVN-Rev: 31071
2011-12-08 22:37:24 +00:00
Markus Scherer
b5e1330176 ICU-7264 merge Unicode 6.0 into trunk from branches/markus/uni60 -r 28339:28657
X-SVN-Rev: 28661
2010-09-21 00:12:49 +00:00
Markus Scherer
3360fe4708 ICU-7144 merge UTS #46 (IDNA2008) C/C++ feature into trunk, merging in -r 27794:28172 icu/branches/markus/uts46
X-SVN-Rev: 28195
2010-06-15 15:54:53 +00:00
Markus Scherer
8ddbd1394c ICU-7273 merge in Normalizer2 API & code, and ICU-5785 UnicodeSet::span(UnicodeString) and ICU-7296 tempSubString()/retainBetween(); merge -r 26971:27150 branches/markus/norm2
X-SVN-Rev: 27155
2010-01-06 23:50:03 +00:00