mirror of
https://github.com/unicode-org/icu.git
synced 2025-04-08 23:10:40 +00:00
Update javadoc; fix compile error
X-SVN-Rev: 1166
This commit is contained in:
Alan Liu
parent
2947282e42
commit
71b21e2eeb
2 changed files with 556 additions and 418 deletions
|
|
@ -5,8 +5,8 @@
|
|||
*******************************************************************************
|
||||
*
|
||||
* $Source: /xsrl/Nsvn/icu/icu4j/src/com/ibm/icu/text/RuleBasedTransliterator.java,v $
|
||||
* $Date: 2000/04/19 16:34:18 $
|
||||
* $Revision: 1.19 $
|
||||
* $Date: 2000/04/19 17:35:23 $
|
||||
* $Revision: 1.20 $
|
||||
*
|
||||
*****************************************************************************************
|
||||
*/
|
||||
|
|
@ -18,247 +18,263 @@ import java.text.ParsePosition;
|
|||
import com.ibm.util.Utility;
|
||||
|
||||
/**
|
||||
* A transliterator that reads a set of rules in order to determine how to perform
|
||||
* translations. Rules are stored in resource bundles indexed by name. Rules are separated by
|
||||
* semicolons (';'). To include a literal semicolon, prefix it with a backslash ('\;').
|
||||
* Whitespace, as defined by <code>Character.isWhitespace()</code>, is ignored. If the first
|
||||
* non-blank character on a line is '#', the entire line is ignored as a comment. </p>
|
||||
* A transliterator that reads a set of rules in order to
|
||||
* determine how to perform translations. Rules are stored in
|
||||
* resource bundles indexed by name. Rules are separated by
|
||||
* semicolons (';'). To include a literal semicolon, prefix it with
|
||||
* a backslash ('\'). Whitespace, as defined by <code>Character.isWhitespace()</code>,
|
||||
* is ignored. If the first non-blank character on a line is '#',
|
||||
* the entire line is ignored as a comment. </p>
|
||||
*
|
||||
* <p>Each set of rules consists of two groups, one forward, and one reverse. This is a
|
||||
* convention that is not enforced; rules for one direction may be omitted, with the result
|
||||
* that translations in that direction will not modify the source text. </p>
|
||||
* <p>Each set of rules consists of two groups, one forward, and one
|
||||
* reverse. This is a convention that is not enforced; rules for one
|
||||
* direction may be omitted, with the result that translations in
|
||||
* that direction will not modify the source text. Alternatively,
|
||||
* bidirectional forward-reverse rules may be specified for
|
||||
* symmetrical transformations.</p>
|
||||
*
|
||||
* <p><b>Rule syntax</b> </p>
|
||||
*
|
||||
* <p>Rule statements take one of the following forms:
|
||||
* <p>Rule statements take one of the following forms: </p>
|
||||
*
|
||||
* <dl>
|
||||
* <dt><code>alefmadda=\u0622</code></dt>
|
||||
* <dd><strong>Variable definition.</strong> The name on the left is assigned the character or
|
||||
* expression on the right. Names may not contain any special characters (see list below).
|
||||
* Duplicate names (including duplicates of simple variables or category names) cause an
|
||||
* exception to be thrown. If the right hand side consists of one character, then the
|
||||
* variable stands for that character. In this example, after this statement, instances of
|
||||
* the left hand name surrounded by braces, "<code>{alefmadda}</code>", will be
|
||||
* replaced by the Unicode character U+0622. If the right hand side is longer than one
|
||||
* character, then it is interpreted as a character category expression; see below for
|
||||
* details.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>softvowel=[eiyEIY]</code></dt>
|
||||
* <dd><strong>Category definition.</strong> The name on the left is assigned to stand for a
|
||||
* set of characters. The same rules for names of simple variables apply. After this
|
||||
* statement, the left hand variable will be interpreted as indicating a set of characters in
|
||||
* appropriate contexts. The pattern syntax defining sets of characters is defined by {@link
|
||||
* UnicodeSet}. Examples of valid patterns are:<table>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[abc]</code></td>
|
||||
* <td>The set containing the characters 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[^abc]</code></td>
|
||||
* <td>The set of all characters <em>except</em> 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[A-Z]</code></td>
|
||||
* <td>The set of all characters from 'A' to 'Z' in Unicode order.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[:Lu:]</code></td>
|
||||
* <td>The set of Unicode uppercase letters. See <a href="http://www.unicode.org">www.unicode.org</a>
|
||||
* for a complete list of categories and their two-letter codes.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[^a-z[:Lu:][:Ll:]]</code></td>
|
||||
* <td>The set of all characters <em>except</em> 'a' through 'z' and uppercase or lowercase
|
||||
* letters.</td>
|
||||
* </tr>
|
||||
* </table>
|
||||
* <p>See {@link UnicodeSet} for more documentation and examples. </p>
|
||||
* </dd>
|
||||
* <dt><code>ai>{alefmadda}</code></dt>
|
||||
* <dd><strong>Forward translation rule.</strong> This rule states that the string on the left
|
||||
* will be changed to the string on the right when performing forward transliteration.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>ai<{alefmadda}</code></dt>
|
||||
* <dd><strong>Reverse translation rule.</strong> This rule states that the string on the right
|
||||
* will be changed to the string on the left when performing reverse transliteration.</dd>
|
||||
* <dt><code>alefmadda=\u0622</code></dt>
|
||||
* <dd><strong>Variable definition.</strong> The name on the
|
||||
* left is assigned the character or expression on the
|
||||
* right. Names may not contain any special characters (see
|
||||
* list below). Duplicate names (including duplicates of
|
||||
* simple variables or category names) cause an exception to
|
||||
* be thrown. If the right hand side consists of one
|
||||
* character, then the variable stands for that character.
|
||||
* In this example, after this statement, instances of the
|
||||
* left hand name surrounded by braces, "<code>{alefmadda}</code>",
|
||||
* will be replaced by the Unicode character U+0622. If the
|
||||
* right hand side is longer than one character, then it is
|
||||
* interpreted as a character category expression; see below
|
||||
* for details.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>softvowel=[eiyEIY]</code></dt>
|
||||
* <dd><strong>Category definition.</strong> The name on the
|
||||
* left is assigned to stand for a set of characters. The
|
||||
* same rules for names of simple variables apply. After
|
||||
* this statement, the left hand variable will be
|
||||
* interpreted as indicating a set of characters in
|
||||
* appropriate contexts. The pattern syntax defining sets of
|
||||
* characters is defined by {@link UnicodeSet}. Examples of
|
||||
* valid patterns are:<table border="0">
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[abc]</code></td>
|
||||
* <td valign="top">The set containing the
|
||||
* characters 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[^abc]</code></td>
|
||||
* <td valign="top">The set of all characters <em>except</em>
|
||||
* 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[A-Z]</code></td>
|
||||
* <td valign="top">The set of all characters from
|
||||
* 'A' to 'Z' in Unicode order.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[:Lu:]</code></td>
|
||||
* <td valign="top">The set of Unicode uppercase
|
||||
* letters. See <a href="http://www.unicode.org">www.unicode.org</a>
|
||||
* for a complete list of categories and their
|
||||
* two-letter codes.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[^a-z[:Lu:][:Ll:]]</code></td>
|
||||
* <td valign="top">The set of all characters <em>except</em>
|
||||
* 'a' through 'z' and uppercase or lowercase
|
||||
* letters.</td>
|
||||
* </tr>
|
||||
* </table>
|
||||
* <p>See {@link UnicodeSet} for more documentation and
|
||||
* examples. </p>
|
||||
* </dd>
|
||||
* <dt><code>ai>{alefmadda}</code></dt>
|
||||
* <dd><strong>Forward translation rule.</strong> This rule
|
||||
* states that the string on the left will be changed to the
|
||||
* string on the right when performing forward
|
||||
* transliteration.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>ai<{alefmadda}</code></dt>
|
||||
* <dd><strong>Reverse translation rule.</strong> This rule
|
||||
* states that the string on the right will be changed to
|
||||
* the string on the left when performing reverse
|
||||
* transliteration.</dd>
|
||||
* </dl>
|
||||
*
|
||||
* <dl>
|
||||
* <dt><code>ai<>{alefmadda}</code></dt>
|
||||
* <dd><strong>Bidirectional translation rule.</strong> This rule states that the string on the
|
||||
* right will be changed to the string on the left when performing forward transliteration,
|
||||
* and vice versa when performing reverse transliteration.</dd>
|
||||
* <dt><code>ai<>{alefmadda}</code></dt>
|
||||
* <dd><strong>Bidirectional translation rule.</strong> This
|
||||
* rule states that the string on the right will be changed
|
||||
* to the string on the left when performing forward
|
||||
* transliteration, and vice versa when performing reverse
|
||||
* transliteration.</dd>
|
||||
* </dl>
|
||||
*
|
||||
* <p>Forward and reverse translation rules consist of a <em>match pattern</em> and an <em>output
|
||||
* string</em>. The match pattern consists of literal characters, optionally preceded by
|
||||
* context, and optionally followed by context. Context characters, like literal pattern
|
||||
* characters, must be matched in the text being transliterated. However, unlike literal
|
||||
* pattern characters, they are not replaced by the output text. For example, the pattern
|
||||
* "<code>(abc)def</code>" indicates the characters "<code>def</code>"
|
||||
* must be preceded by "<code>abc</code>" for a successful match. If there is a
|
||||
* successful match, "<code>def</code>" will be replaced, but not "<code>abc</code>".
|
||||
* The initial '<code>(</code>' is optional, so "<code>abc)def</code>" is
|
||||
* equivalent to "<code>(abc)def</code>". Another example is "<code>123(456)</code>"
|
||||
* (or "<code>123(456</code>") in which the literal pattern "<code>123</code>"
|
||||
* must be followed by "<code>456</code>". </p>
|
||||
* <p>Translation rules consist of a <em>match pattern</em> and an <em>output
|
||||
* string</em>. The match pattern consists of literal characters,
|
||||
* optionally preceded by context, and optionally followed by
|
||||
* context. Context characters, like literal pattern characters,
|
||||
* must be matched in the text being transliterated. However, unlike
|
||||
* literal pattern characters, they are not replaced by the output
|
||||
* text. For example, the pattern "<code>(abc)def</code>"
|
||||
* indicates the characters "<code>def</code>" must be
|
||||
* preceded by "<code>abc</code>" for a successful match.
|
||||
* If there is a successful match, "<code>def</code>" will
|
||||
* be replaced, but not "<code>abc</code>". The initial '<code>(</code>'
|
||||
* is optional, so "<code>abc)def</code>" is equivalent to
|
||||
* "<code>(abc)def</code>". Another example is "<code>123(456)</code>"
|
||||
* (or "<code>123(456</code>") in which the literal
|
||||
* pattern "<code>123</code>" must be followed by "<code>456</code>".
|
||||
* </p>
|
||||
*
|
||||
* <p>The output string of a forward or reverse rule consists of characters to replace the
|
||||
* literal pattern characters. If the output string contains the character '<code>|</code>',
|
||||
* this is taken to indicate the location of the <em>cursor</em> after replacement. The
|
||||
* cursor is the point in the text at which the next replacement, if any, will be applied. </p>
|
||||
* <p>The output string of a forward or reverse rule consists of
|
||||
* characters to replace the literal pattern characters. If the
|
||||
* output string contains the character '<code>|</code>', this is
|
||||
* taken to indicate the location of the <em>cursor</em> after
|
||||
* replacement. The cursor is the point in the text at which the
|
||||
* next replacement, if any, will be applied. </p>
|
||||
*
|
||||
* <p>In addition to being defined in variables, <code>UnicodeSet</code> patterns may be
|
||||
* embedded directly into rule strings. Thus, the following two rules are equivalent:</p>
|
||||
* <p>In addition to being defined in variables, <code>UnicodeSet</code>
|
||||
* patterns may be embedded directly into rule strings. Thus, the
|
||||
* following two rules are equivalent:</p>
|
||||
*
|
||||
* <blockquote>
|
||||
* <p><code>vowel=[aeiou]; {vowel}>*; # One way to do this<br>
|
||||
* [aeiou]>*;
|
||||
* #
|
||||
* Another way</code></p>
|
||||
* <p><code>vowel=[aeiou]; {vowel}>*; # One way to do this<br>
|
||||
* [aeiou]>*;
|
||||
* #
|
||||
* Another way</code></p>
|
||||
* </blockquote>
|
||||
*
|
||||
* <p><b>Segments</b></p>
|
||||
*
|
||||
* <p>Segments of the input string can be matched and copied to the
|
||||
* output string. This makes certain sets of rules simpler and more
|
||||
* general, and makes reordering possible. For example:</p>
|
||||
*
|
||||
* <blockquote>
|
||||
* <p><code>$([a-z]$) > $1 $1;
|
||||
* #
|
||||
* double lowercase letters<br>
|
||||
* $([:Lu:]$) $([:Ll:]$) > $2 $1; # reverse order of Lu-Ll
|
||||
* pairs</code></p>
|
||||
* </blockquote>
|
||||
*
|
||||
* <p>The segment of the input string to be copied is delimited by
|
||||
* "<code>$(</code>" and "<code>)$</code>". Up
|
||||
* to nine segments may be defined. Segments may not overlap. In the
|
||||
* output string, "<code>$1</code>" through "<code>$9</code>"
|
||||
* represent the input string segments, in left-to-right order of
|
||||
* definition.</p>
|
||||
*
|
||||
* <p><b>Example</b> </p>
|
||||
*
|
||||
* <p>The following example rules illustrate many of the features of the rule language. </p>
|
||||
* <p>The following example rules illustrate many of the features of
|
||||
* the rule language. </p>
|
||||
*
|
||||
* <table cellpadding="4">
|
||||
* <tr valign="top">
|
||||
* <td>Rule 1.</td>
|
||||
* <td nowrap><code>(abc)def>x|y</code></td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td>Rule 2.</td>
|
||||
* <td nowrap><code>xyz>r</code></td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td>Rule 3.</td>
|
||||
* <td nowrap><code>yz>q</code></td>
|
||||
* </tr>
|
||||
* <table border="0" cellpadding="4">
|
||||
* <tr>
|
||||
* <td valign="top">Rule 1.</td>
|
||||
* <td valign="top" nowrap><code>(abc)def>x|y</code></td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top">Rule 2.</td>
|
||||
* <td valign="top" nowrap><code>xyz>r</code></td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top">Rule 3.</td>
|
||||
* <td valign="top" nowrap><code>yz>q</code></td>
|
||||
* </tr>
|
||||
* </table>
|
||||
*
|
||||
* <p>Applying these rules to the string "<code>adefabcdefz</code>" yields the
|
||||
* following results: </p>
|
||||
* <p>Applying these rules to the string "<code>adefabcdefz</code>"
|
||||
* yields the following results: </p>
|
||||
*
|
||||
* <table cellpadding="4">
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>|adefabcdefz</code></td>
|
||||
* <td>Initial state, no rules match. Advance cursor.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>a|defabcdefz</code></td>
|
||||
* <td>Still no match. Rule 1 does not match because the preceding context is not present.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>ad|efabcdefz</code></td>
|
||||
* <td>Still no match. Keep advancing until there is a match...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>ade|fabcdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adef|abcdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefa|bcdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefab|cdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefabc|defz</code></td>
|
||||
* <td>Rule 1 matches; replace "<code>def</code>" with "<code>xy</code>"
|
||||
* and back up the cursor to before the '<code>y</code>'.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefabcx|yz</code></td>
|
||||
* <td>Although "<code>xyz</code>" is present, rule 2 does not match because the
|
||||
* cursor is before the '<code>y</code>', not before the '<code>x</code>'. Rule 3 does match.
|
||||
* Replace "<code>yz</code>" with "<code>q</code>".</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefabcxq|</code></td>
|
||||
* <td>The cursor is at the end; transliteration is complete.</td>
|
||||
* </tr>
|
||||
* <table border="0" cellpadding="4">
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>|adefabcdefz</code></td>
|
||||
* <td valign="top">Initial state, no rules match. Advance
|
||||
* cursor.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>a|defabcdefz</code></td>
|
||||
* <td valign="top">Still no match. Rule 1 does not match
|
||||
* because the preceding context is not present.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>ad|efabcdefz</code></td>
|
||||
* <td valign="top">Still no match. Keep advancing until
|
||||
* there is a match...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>ade|fabcdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adef|abcdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefa|bcdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefab|cdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefabc|defz</code></td>
|
||||
* <td valign="top">Rule 1 matches; replace "<code>def</code>"
|
||||
* with "<code>xy</code>" and back up the cursor
|
||||
* to before the '<code>y</code>'.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefabcx|yz</code></td>
|
||||
* <td valign="top">Although "<code>xyz</code>" is
|
||||
* present, rule 2 does not match because the cursor is
|
||||
* before the '<code>y</code>', not before the '<code>x</code>'.
|
||||
* Rule 3 does match. Replace "<code>yz</code>"
|
||||
* with "<code>q</code>".</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefabcxq|</code></td>
|
||||
* <td valign="top">The cursor is at the end;
|
||||
* transliteration is complete.</td>
|
||||
* </tr>
|
||||
* </table>
|
||||
*
|
||||
* <p>The order of rules is significant. If multiple rules may match at some point, the first
|
||||
* matching rule is applied. </p>
|
||||
* <p>The order of rules is significant. If multiple rules may match
|
||||
* at some point, the first matching rule is applied. </p>
|
||||
*
|
||||
* <p>Forward and reverse rules may have an empty output string. Otherwise, an empty left or
|
||||
* right hand side of any statement is a syntax error. </p>
|
||||
* <p>Forward and reverse rules may have an empty output string.
|
||||
* Otherwise, an empty left or right hand side of any statement is a
|
||||
* syntax error. </p>
|
||||
*
|
||||
* <p>Single quotes are used to quote the special characters <code>=><{}[]()|</code>.
|
||||
* To specify a single quote itself, inside or outside of quotes, use two single quotes in a
|
||||
* row. For example, the rule "<code>'>'>o''clock</code>" changes the string
|
||||
* "<code>></code>" to the string "<code>o'clock</code>". </p>
|
||||
* To specify a single quote itself, inside or outside of quotes,
|
||||
* use two single quotes in a row. For example, the rule "<code>'>'>o''clock</code>"
|
||||
* changes the string "<code>></code>" to the string
|
||||
* "<code>o'clock</code>". </p>
|
||||
*
|
||||
* <p><b>Notes</b> </p>
|
||||
*
|
||||
* <p>While a RuleBasedTransliterator is being built, it checks that the rules are added in
|
||||
* proper order. For example, if the rule "a>x" is followed by the rule
|
||||
* "ab>y", then the second rule will throw an exception. The reason is that the
|
||||
* second rule can never be triggered, since the first rule always matches anything it
|
||||
* matches. In other words, the first rule <em>masks</em> the second rule. </p>
|
||||
* <p>While a RuleBasedTransliterator is being built, it checks that
|
||||
* the rules are added in proper order. For example, if the rule
|
||||
* "a>x" is followed by the rule "ab>y",
|
||||
* then the second rule will throw an exception. The reason is that
|
||||
* the second rule can never be triggered, since the first rule
|
||||
* always matches anything it matches. In other words, the first
|
||||
* rule <em>masks</em> the second rule. </p>
|
||||
*
|
||||
* <p>Copyright (c) IBM Corporation 1999-2000. All rights reserved.</p>
|
||||
*
|
||||
* @author Alan Liu
|
||||
* @version $RCSfile: RuleBasedTransliterator.java,v $ $Revision: 1.19 $ $Date: 2000/04/19 16:34:18 $
|
||||
*
|
||||
* $Log: RuleBasedTransliterator.java,v $
|
||||
* Revision 1.19 2000/04/19 16:34:18 alan
|
||||
* Add segment support.
|
||||
*
|
||||
* Revision 1.18 2000/04/12 20:17:45 alan
|
||||
* Delegate replace operation to rule object
|
||||
*
|
||||
* Revision 1.17 2000/03/10 04:07:23 johnf
|
||||
* Copyright update
|
||||
*
|
||||
* Revision 1.16 2000/02/24 20:46:49 liu
|
||||
* Add infinite loop check
|
||||
*
|
||||
* Revision 1.15 2000/02/10 07:36:25 johnf
|
||||
* fixed imports for com.ibm.util.Utility
|
||||
*
|
||||
* Revision 1.14 2000/02/03 18:18:42 Alan
|
||||
* Use array rather than hashtable for char-to-set map
|
||||
*
|
||||
* Revision 1.13 2000/01/27 18:59:19 Alan
|
||||
* Use Position rather than int[] and move all subclass overrides to one method (handleTransliterate)
|
||||
*
|
||||
* Revision 1.12 2000/01/18 17:51:09 Alan
|
||||
* Remove "keyboard" from method names. Make maximum context a field of Transliterator, and have subclasses set it.
|
||||
*
|
||||
* Revision 1.11 2000/01/18 02:30:49 Alan
|
||||
* Add Jamo-Hangul, Hangul-Jamo, fix rules, add compound ID support
|
||||
*
|
||||
* Revision 1.10 2000/01/13 23:53:23 Alan
|
||||
* Fix bugs found during ICU port
|
||||
*
|
||||
* Revision 1.9 2000/01/11 04:12:06 Alan
|
||||
* Cleanup, embellish comments
|
||||
*
|
||||
* Revision 1.8 2000/01/11 02:25:03 Alan
|
||||
* Rewrite UnicodeSet and RBT parsers for better performance and new syntax
|
||||
*
|
||||
* Revision 1.7 2000/01/06 01:36:36 Alan
|
||||
* Allow string arrays in rule resource bundles
|
||||
*
|
||||
* Revision 1.6 2000/01/04 21:43:57 Alan
|
||||
* Add rule indexing, and move masking check to TransliterationRuleSet.
|
||||
*
|
||||
* Revision 1.5 1999/12/22 01:40:54 Alan
|
||||
* Consolidate rule pattern anteContext, key, and postContext into one string.
|
||||
*
|
||||
* Revision 1.4 1999/12/22 01:05:54 Alan
|
||||
* Improve masking checking; turn it off by default, for better performance
|
||||
* @version $RCSfile: RuleBasedTransliterator.java,v $ $Revision: 1.20 $ $Date: 2000/04/19 17:35:23 $
|
||||
*/
|
||||
public class RuleBasedTransliterator extends Transliterator {
|
||||
|
||||
|
|
@ -981,7 +997,7 @@ public class RuleBasedTransliterator extends Transliterator {
|
|||
data.ruleSet.addRule(new TransliterationRule(
|
||||
left.text, left.ante, left.post,
|
||||
right.text, right.cursor,
|
||||
left.getSegments(), data));
|
||||
left.getSegments()));
|
||||
|
||||
return pos;
|
||||
}
|
||||
|
|
@ -1193,3 +1209,56 @@ public class RuleBasedTransliterator extends Transliterator {
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* $Log: RuleBasedTransliterator.java,v $
|
||||
* Revision 1.20 2000/04/19 17:35:23 alan
|
||||
* Update javadoc; fix compile error
|
||||
*
|
||||
* Revision 1.19 2000/04/19 16:34:18 alan
|
||||
* Add segment support.
|
||||
*
|
||||
* Revision 1.18 2000/04/12 20:17:45 alan
|
||||
* Delegate replace operation to rule object
|
||||
*
|
||||
* Revision 1.17 2000/03/10 04:07:23 johnf
|
||||
* Copyright update
|
||||
*
|
||||
* Revision 1.16 2000/02/24 20:46:49 liu
|
||||
* Add infinite loop check
|
||||
*
|
||||
* Revision 1.15 2000/02/10 07:36:25 johnf
|
||||
* fixed imports for com.ibm.util.Utility
|
||||
*
|
||||
* Revision 1.14 2000/02/03 18:18:42 Alan
|
||||
* Use array rather than hashtable for char-to-set map
|
||||
*
|
||||
* Revision 1.13 2000/01/27 18:59:19 Alan
|
||||
* Use Position rather than int[] and move all subclass overrides to one method (handleTransliterate)
|
||||
*
|
||||
* Revision 1.12 2000/01/18 17:51:09 Alan
|
||||
* Remove "keyboard" from method names. Make maximum context a field of Transliterator, and have subclasses set it.
|
||||
*
|
||||
* Revision 1.11 2000/01/18 02:30:49 Alan
|
||||
* Add Jamo-Hangul, Hangul-Jamo, fix rules, add compound ID support
|
||||
*
|
||||
* Revision 1.10 2000/01/13 23:53:23 Alan
|
||||
* Fix bugs found during ICU port
|
||||
*
|
||||
* Revision 1.9 2000/01/11 04:12:06 Alan
|
||||
* Cleanup, embellish comments
|
||||
*
|
||||
* Revision 1.8 2000/01/11 02:25:03 Alan
|
||||
* Rewrite UnicodeSet and RBT parsers for better performance and new syntax
|
||||
*
|
||||
* Revision 1.7 2000/01/06 01:36:36 Alan
|
||||
* Allow string arrays in rule resource bundles
|
||||
*
|
||||
* Revision 1.6 2000/01/04 21:43:57 Alan
|
||||
* Add rule indexing, and move masking check to TransliterationRuleSet.
|
||||
*
|
||||
* Revision 1.5 1999/12/22 01:40:54 Alan
|
||||
* Consolidate rule pattern anteContext, key, and postContext into one string.
|
||||
*
|
||||
* Revision 1.4 1999/12/22 01:05:54 Alan
|
||||
* Improve masking checking; turn it off by default, for better performance
|
||||
*/
|
||||
|
|
|
|||
|
|
@ -5,8 +5,8 @@
|
|||
*******************************************************************************
|
||||
*
|
||||
* $Source: /xsrl/Nsvn/icu/icu4j/src/com/ibm/text/Attic/RuleBasedTransliterator.java,v $
|
||||
* $Date: 2000/04/19 16:34:18 $
|
||||
* $Revision: 1.19 $
|
||||
* $Date: 2000/04/19 17:35:23 $
|
||||
* $Revision: 1.20 $
|
||||
*
|
||||
*****************************************************************************************
|
||||
*/
|
||||
|
|
@ -18,247 +18,263 @@ import java.text.ParsePosition;
|
|||
import com.ibm.util.Utility;
|
||||
|
||||
/**
|
||||
* A transliterator that reads a set of rules in order to determine how to perform
|
||||
* translations. Rules are stored in resource bundles indexed by name. Rules are separated by
|
||||
* semicolons (';'). To include a literal semicolon, prefix it with a backslash ('\;').
|
||||
* Whitespace, as defined by <code>Character.isWhitespace()</code>, is ignored. If the first
|
||||
* non-blank character on a line is '#', the entire line is ignored as a comment. </p>
|
||||
* A transliterator that reads a set of rules in order to
|
||||
* determine how to perform translations. Rules are stored in
|
||||
* resource bundles indexed by name. Rules are separated by
|
||||
* semicolons (';'). To include a literal semicolon, prefix it with
|
||||
* a backslash ('\'). Whitespace, as defined by <code>Character.isWhitespace()</code>,
|
||||
* is ignored. If the first non-blank character on a line is '#',
|
||||
* the entire line is ignored as a comment. </p>
|
||||
*
|
||||
* <p>Each set of rules consists of two groups, one forward, and one reverse. This is a
|
||||
* convention that is not enforced; rules for one direction may be omitted, with the result
|
||||
* that translations in that direction will not modify the source text. </p>
|
||||
* <p>Each set of rules consists of two groups, one forward, and one
|
||||
* reverse. This is a convention that is not enforced; rules for one
|
||||
* direction may be omitted, with the result that translations in
|
||||
* that direction will not modify the source text. Alternatively,
|
||||
* bidirectional forward-reverse rules may be specified for
|
||||
* symmetrical transformations.</p>
|
||||
*
|
||||
* <p><b>Rule syntax</b> </p>
|
||||
*
|
||||
* <p>Rule statements take one of the following forms:
|
||||
* <p>Rule statements take one of the following forms: </p>
|
||||
*
|
||||
* <dl>
|
||||
* <dt><code>alefmadda=\u0622</code></dt>
|
||||
* <dd><strong>Variable definition.</strong> The name on the left is assigned the character or
|
||||
* expression on the right. Names may not contain any special characters (see list below).
|
||||
* Duplicate names (including duplicates of simple variables or category names) cause an
|
||||
* exception to be thrown. If the right hand side consists of one character, then the
|
||||
* variable stands for that character. In this example, after this statement, instances of
|
||||
* the left hand name surrounded by braces, "<code>{alefmadda}</code>", will be
|
||||
* replaced by the Unicode character U+0622. If the right hand side is longer than one
|
||||
* character, then it is interpreted as a character category expression; see below for
|
||||
* details.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>softvowel=[eiyEIY]</code></dt>
|
||||
* <dd><strong>Category definition.</strong> The name on the left is assigned to stand for a
|
||||
* set of characters. The same rules for names of simple variables apply. After this
|
||||
* statement, the left hand variable will be interpreted as indicating a set of characters in
|
||||
* appropriate contexts. The pattern syntax defining sets of characters is defined by {@link
|
||||
* UnicodeSet}. Examples of valid patterns are:<table>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[abc]</code></td>
|
||||
* <td>The set containing the characters 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[^abc]</code></td>
|
||||
* <td>The set of all characters <em>except</em> 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[A-Z]</code></td>
|
||||
* <td>The set of all characters from 'A' to 'Z' in Unicode order.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[:Lu:]</code></td>
|
||||
* <td>The set of Unicode uppercase letters. See <a href="http://www.unicode.org">www.unicode.org</a>
|
||||
* for a complete list of categories and their two-letter codes.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>[^a-z[:Lu:][:Ll:]]</code></td>
|
||||
* <td>The set of all characters <em>except</em> 'a' through 'z' and uppercase or lowercase
|
||||
* letters.</td>
|
||||
* </tr>
|
||||
* </table>
|
||||
* <p>See {@link UnicodeSet} for more documentation and examples. </p>
|
||||
* </dd>
|
||||
* <dt><code>ai>{alefmadda}</code></dt>
|
||||
* <dd><strong>Forward translation rule.</strong> This rule states that the string on the left
|
||||
* will be changed to the string on the right when performing forward transliteration.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>ai<{alefmadda}</code></dt>
|
||||
* <dd><strong>Reverse translation rule.</strong> This rule states that the string on the right
|
||||
* will be changed to the string on the left when performing reverse transliteration.</dd>
|
||||
* <dt><code>alefmadda=\u0622</code></dt>
|
||||
* <dd><strong>Variable definition.</strong> The name on the
|
||||
* left is assigned the character or expression on the
|
||||
* right. Names may not contain any special characters (see
|
||||
* list below). Duplicate names (including duplicates of
|
||||
* simple variables or category names) cause an exception to
|
||||
* be thrown. If the right hand side consists of one
|
||||
* character, then the variable stands for that character.
|
||||
* In this example, after this statement, instances of the
|
||||
* left hand name surrounded by braces, "<code>{alefmadda}</code>",
|
||||
* will be replaced by the Unicode character U+0622. If the
|
||||
* right hand side is longer than one character, then it is
|
||||
* interpreted as a character category expression; see below
|
||||
* for details.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>softvowel=[eiyEIY]</code></dt>
|
||||
* <dd><strong>Category definition.</strong> The name on the
|
||||
* left is assigned to stand for a set of characters. The
|
||||
* same rules for names of simple variables apply. After
|
||||
* this statement, the left hand variable will be
|
||||
* interpreted as indicating a set of characters in
|
||||
* appropriate contexts. The pattern syntax defining sets of
|
||||
* characters is defined by {@link UnicodeSet}. Examples of
|
||||
* valid patterns are:<table border="0">
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[abc]</code></td>
|
||||
* <td valign="top">The set containing the
|
||||
* characters 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[^abc]</code></td>
|
||||
* <td valign="top">The set of all characters <em>except</em>
|
||||
* 'a', 'b', and 'c'.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[A-Z]</code></td>
|
||||
* <td valign="top">The set of all characters from
|
||||
* 'A' to 'Z' in Unicode order.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[:Lu:]</code></td>
|
||||
* <td valign="top">The set of Unicode uppercase
|
||||
* letters. See <a href="http://www.unicode.org">www.unicode.org</a>
|
||||
* for a complete list of categories and their
|
||||
* two-letter codes.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>[^a-z[:Lu:][:Ll:]]</code></td>
|
||||
* <td valign="top">The set of all characters <em>except</em>
|
||||
* 'a' through 'z' and uppercase or lowercase
|
||||
* letters.</td>
|
||||
* </tr>
|
||||
* </table>
|
||||
* <p>See {@link UnicodeSet} for more documentation and
|
||||
* examples. </p>
|
||||
* </dd>
|
||||
* <dt><code>ai>{alefmadda}</code></dt>
|
||||
* <dd><strong>Forward translation rule.</strong> This rule
|
||||
* states that the string on the left will be changed to the
|
||||
* string on the right when performing forward
|
||||
* transliteration.</dd>
|
||||
* <dt> </dt>
|
||||
* <dt><code>ai<{alefmadda}</code></dt>
|
||||
* <dd><strong>Reverse translation rule.</strong> This rule
|
||||
* states that the string on the right will be changed to
|
||||
* the string on the left when performing reverse
|
||||
* transliteration.</dd>
|
||||
* </dl>
|
||||
*
|
||||
* <dl>
|
||||
* <dt><code>ai<>{alefmadda}</code></dt>
|
||||
* <dd><strong>Bidirectional translation rule.</strong> This rule states that the string on the
|
||||
* right will be changed to the string on the left when performing forward transliteration,
|
||||
* and vice versa when performing reverse transliteration.</dd>
|
||||
* <dt><code>ai<>{alefmadda}</code></dt>
|
||||
* <dd><strong>Bidirectional translation rule.</strong> This
|
||||
* rule states that the string on the right will be changed
|
||||
* to the string on the left when performing forward
|
||||
* transliteration, and vice versa when performing reverse
|
||||
* transliteration.</dd>
|
||||
* </dl>
|
||||
*
|
||||
* <p>Forward and reverse translation rules consist of a <em>match pattern</em> and an <em>output
|
||||
* string</em>. The match pattern consists of literal characters, optionally preceded by
|
||||
* context, and optionally followed by context. Context characters, like literal pattern
|
||||
* characters, must be matched in the text being transliterated. However, unlike literal
|
||||
* pattern characters, they are not replaced by the output text. For example, the pattern
|
||||
* "<code>(abc)def</code>" indicates the characters "<code>def</code>"
|
||||
* must be preceded by "<code>abc</code>" for a successful match. If there is a
|
||||
* successful match, "<code>def</code>" will be replaced, but not "<code>abc</code>".
|
||||
* The initial '<code>(</code>' is optional, so "<code>abc)def</code>" is
|
||||
* equivalent to "<code>(abc)def</code>". Another example is "<code>123(456)</code>"
|
||||
* (or "<code>123(456</code>") in which the literal pattern "<code>123</code>"
|
||||
* must be followed by "<code>456</code>". </p>
|
||||
* <p>Translation rules consist of a <em>match pattern</em> and an <em>output
|
||||
* string</em>. The match pattern consists of literal characters,
|
||||
* optionally preceded by context, and optionally followed by
|
||||
* context. Context characters, like literal pattern characters,
|
||||
* must be matched in the text being transliterated. However, unlike
|
||||
* literal pattern characters, they are not replaced by the output
|
||||
* text. For example, the pattern "<code>(abc)def</code>"
|
||||
* indicates the characters "<code>def</code>" must be
|
||||
* preceded by "<code>abc</code>" for a successful match.
|
||||
* If there is a successful match, "<code>def</code>" will
|
||||
* be replaced, but not "<code>abc</code>". The initial '<code>(</code>'
|
||||
* is optional, so "<code>abc)def</code>" is equivalent to
|
||||
* "<code>(abc)def</code>". Another example is "<code>123(456)</code>"
|
||||
* (or "<code>123(456</code>") in which the literal
|
||||
* pattern "<code>123</code>" must be followed by "<code>456</code>".
|
||||
* </p>
|
||||
*
|
||||
* <p>The output string of a forward or reverse rule consists of characters to replace the
|
||||
* literal pattern characters. If the output string contains the character '<code>|</code>',
|
||||
* this is taken to indicate the location of the <em>cursor</em> after replacement. The
|
||||
* cursor is the point in the text at which the next replacement, if any, will be applied. </p>
|
||||
* <p>The output string of a forward or reverse rule consists of
|
||||
* characters to replace the literal pattern characters. If the
|
||||
* output string contains the character '<code>|</code>', this is
|
||||
* taken to indicate the location of the <em>cursor</em> after
|
||||
* replacement. The cursor is the point in the text at which the
|
||||
* next replacement, if any, will be applied. </p>
|
||||
*
|
||||
* <p>In addition to being defined in variables, <code>UnicodeSet</code> patterns may be
|
||||
* embedded directly into rule strings. Thus, the following two rules are equivalent:</p>
|
||||
* <p>In addition to being defined in variables, <code>UnicodeSet</code>
|
||||
* patterns may be embedded directly into rule strings. Thus, the
|
||||
* following two rules are equivalent:</p>
|
||||
*
|
||||
* <blockquote>
|
||||
* <p><code>vowel=[aeiou]; {vowel}>*; # One way to do this<br>
|
||||
* [aeiou]>*;
|
||||
* #
|
||||
* Another way</code></p>
|
||||
* <p><code>vowel=[aeiou]; {vowel}>*; # One way to do this<br>
|
||||
* [aeiou]>*;
|
||||
* #
|
||||
* Another way</code></p>
|
||||
* </blockquote>
|
||||
*
|
||||
* <p><b>Segments</b></p>
|
||||
*
|
||||
* <p>Segments of the input string can be matched and copied to the
|
||||
* output string. This makes certain sets of rules simpler and more
|
||||
* general, and makes reordering possible. For example:</p>
|
||||
*
|
||||
* <blockquote>
|
||||
* <p><code>$([a-z]$) > $1 $1;
|
||||
* #
|
||||
* double lowercase letters<br>
|
||||
* $([:Lu:]$) $([:Ll:]$) > $2 $1; # reverse order of Lu-Ll
|
||||
* pairs</code></p>
|
||||
* </blockquote>
|
||||
*
|
||||
* <p>The segment of the input string to be copied is delimited by
|
||||
* "<code>$(</code>" and "<code>)$</code>". Up
|
||||
* to nine segments may be defined. Segments may not overlap. In the
|
||||
* output string, "<code>$1</code>" through "<code>$9</code>"
|
||||
* represent the input string segments, in left-to-right order of
|
||||
* definition.</p>
|
||||
*
|
||||
* <p><b>Example</b> </p>
|
||||
*
|
||||
* <p>The following example rules illustrate many of the features of the rule language. </p>
|
||||
* <p>The following example rules illustrate many of the features of
|
||||
* the rule language. </p>
|
||||
*
|
||||
* <table cellpadding="4">
|
||||
* <tr valign="top">
|
||||
* <td>Rule 1.</td>
|
||||
* <td nowrap><code>(abc)def>x|y</code></td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td>Rule 2.</td>
|
||||
* <td nowrap><code>xyz>r</code></td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td>Rule 3.</td>
|
||||
* <td nowrap><code>yz>q</code></td>
|
||||
* </tr>
|
||||
* <table border="0" cellpadding="4">
|
||||
* <tr>
|
||||
* <td valign="top">Rule 1.</td>
|
||||
* <td valign="top" nowrap><code>(abc)def>x|y</code></td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top">Rule 2.</td>
|
||||
* <td valign="top" nowrap><code>xyz>r</code></td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top">Rule 3.</td>
|
||||
* <td valign="top" nowrap><code>yz>q</code></td>
|
||||
* </tr>
|
||||
* </table>
|
||||
*
|
||||
* <p>Applying these rules to the string "<code>adefabcdefz</code>" yields the
|
||||
* following results: </p>
|
||||
* <p>Applying these rules to the string "<code>adefabcdefz</code>"
|
||||
* yields the following results: </p>
|
||||
*
|
||||
* <table cellpadding="4">
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>|adefabcdefz</code></td>
|
||||
* <td>Initial state, no rules match. Advance cursor.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>a|defabcdefz</code></td>
|
||||
* <td>Still no match. Rule 1 does not match because the preceding context is not present.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>ad|efabcdefz</code></td>
|
||||
* <td>Still no match. Keep advancing until there is a match...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>ade|fabcdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adef|abcdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefa|bcdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefab|cdefz</code></td>
|
||||
* <td>...</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefabc|defz</code></td>
|
||||
* <td>Rule 1 matches; replace "<code>def</code>" with "<code>xy</code>"
|
||||
* and back up the cursor to before the '<code>y</code>'.</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefabcx|yz</code></td>
|
||||
* <td>Although "<code>xyz</code>" is present, rule 2 does not match because the
|
||||
* cursor is before the '<code>y</code>', not before the '<code>x</code>'. Rule 3 does match.
|
||||
* Replace "<code>yz</code>" with "<code>q</code>".</td>
|
||||
* </tr>
|
||||
* <tr valign="top">
|
||||
* <td nowrap><code>adefabcxq|</code></td>
|
||||
* <td>The cursor is at the end; transliteration is complete.</td>
|
||||
* </tr>
|
||||
* <table border="0" cellpadding="4">
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>|adefabcdefz</code></td>
|
||||
* <td valign="top">Initial state, no rules match. Advance
|
||||
* cursor.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>a|defabcdefz</code></td>
|
||||
* <td valign="top">Still no match. Rule 1 does not match
|
||||
* because the preceding context is not present.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>ad|efabcdefz</code></td>
|
||||
* <td valign="top">Still no match. Keep advancing until
|
||||
* there is a match...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>ade|fabcdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adef|abcdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefa|bcdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefab|cdefz</code></td>
|
||||
* <td valign="top">...</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefabc|defz</code></td>
|
||||
* <td valign="top">Rule 1 matches; replace "<code>def</code>"
|
||||
* with "<code>xy</code>" and back up the cursor
|
||||
* to before the '<code>y</code>'.</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefabcx|yz</code></td>
|
||||
* <td valign="top">Although "<code>xyz</code>" is
|
||||
* present, rule 2 does not match because the cursor is
|
||||
* before the '<code>y</code>', not before the '<code>x</code>'.
|
||||
* Rule 3 does match. Replace "<code>yz</code>"
|
||||
* with "<code>q</code>".</td>
|
||||
* </tr>
|
||||
* <tr>
|
||||
* <td valign="top" nowrap><code>adefabcxq|</code></td>
|
||||
* <td valign="top">The cursor is at the end;
|
||||
* transliteration is complete.</td>
|
||||
* </tr>
|
||||
* </table>
|
||||
*
|
||||
* <p>The order of rules is significant. If multiple rules may match at some point, the first
|
||||
* matching rule is applied. </p>
|
||||
* <p>The order of rules is significant. If multiple rules may match
|
||||
* at some point, the first matching rule is applied. </p>
|
||||
*
|
||||
* <p>Forward and reverse rules may have an empty output string. Otherwise, an empty left or
|
||||
* right hand side of any statement is a syntax error. </p>
|
||||
* <p>Forward and reverse rules may have an empty output string.
|
||||
* Otherwise, an empty left or right hand side of any statement is a
|
||||
* syntax error. </p>
|
||||
*
|
||||
* <p>Single quotes are used to quote the special characters <code>=><{}[]()|</code>.
|
||||
* To specify a single quote itself, inside or outside of quotes, use two single quotes in a
|
||||
* row. For example, the rule "<code>'>'>o''clock</code>" changes the string
|
||||
* "<code>></code>" to the string "<code>o'clock</code>". </p>
|
||||
* To specify a single quote itself, inside or outside of quotes,
|
||||
* use two single quotes in a row. For example, the rule "<code>'>'>o''clock</code>"
|
||||
* changes the string "<code>></code>" to the string
|
||||
* "<code>o'clock</code>". </p>
|
||||
*
|
||||
* <p><b>Notes</b> </p>
|
||||
*
|
||||
* <p>While a RuleBasedTransliterator is being built, it checks that the rules are added in
|
||||
* proper order. For example, if the rule "a>x" is followed by the rule
|
||||
* "ab>y", then the second rule will throw an exception. The reason is that the
|
||||
* second rule can never be triggered, since the first rule always matches anything it
|
||||
* matches. In other words, the first rule <em>masks</em> the second rule. </p>
|
||||
* <p>While a RuleBasedTransliterator is being built, it checks that
|
||||
* the rules are added in proper order. For example, if the rule
|
||||
* "a>x" is followed by the rule "ab>y",
|
||||
* then the second rule will throw an exception. The reason is that
|
||||
* the second rule can never be triggered, since the first rule
|
||||
* always matches anything it matches. In other words, the first
|
||||
* rule <em>masks</em> the second rule. </p>
|
||||
*
|
||||
* <p>Copyright (c) IBM Corporation 1999-2000. All rights reserved.</p>
|
||||
*
|
||||
* @author Alan Liu
|
||||
* @version $RCSfile: RuleBasedTransliterator.java,v $ $Revision: 1.19 $ $Date: 2000/04/19 16:34:18 $
|
||||
*
|
||||
* $Log: RuleBasedTransliterator.java,v $
|
||||
* Revision 1.19 2000/04/19 16:34:18 alan
|
||||
* Add segment support.
|
||||
*
|
||||
* Revision 1.18 2000/04/12 20:17:45 alan
|
||||
* Delegate replace operation to rule object
|
||||
*
|
||||
* Revision 1.17 2000/03/10 04:07:23 johnf
|
||||
* Copyright update
|
||||
*
|
||||
* Revision 1.16 2000/02/24 20:46:49 liu
|
||||
* Add infinite loop check
|
||||
*
|
||||
* Revision 1.15 2000/02/10 07:36:25 johnf
|
||||
* fixed imports for com.ibm.util.Utility
|
||||
*
|
||||
* Revision 1.14 2000/02/03 18:18:42 Alan
|
||||
* Use array rather than hashtable for char-to-set map
|
||||
*
|
||||
* Revision 1.13 2000/01/27 18:59:19 Alan
|
||||
* Use Position rather than int[] and move all subclass overrides to one method (handleTransliterate)
|
||||
*
|
||||
* Revision 1.12 2000/01/18 17:51:09 Alan
|
||||
* Remove "keyboard" from method names. Make maximum context a field of Transliterator, and have subclasses set it.
|
||||
*
|
||||
* Revision 1.11 2000/01/18 02:30:49 Alan
|
||||
* Add Jamo-Hangul, Hangul-Jamo, fix rules, add compound ID support
|
||||
*
|
||||
* Revision 1.10 2000/01/13 23:53:23 Alan
|
||||
* Fix bugs found during ICU port
|
||||
*
|
||||
* Revision 1.9 2000/01/11 04:12:06 Alan
|
||||
* Cleanup, embellish comments
|
||||
*
|
||||
* Revision 1.8 2000/01/11 02:25:03 Alan
|
||||
* Rewrite UnicodeSet and RBT parsers for better performance and new syntax
|
||||
*
|
||||
* Revision 1.7 2000/01/06 01:36:36 Alan
|
||||
* Allow string arrays in rule resource bundles
|
||||
*
|
||||
* Revision 1.6 2000/01/04 21:43:57 Alan
|
||||
* Add rule indexing, and move masking check to TransliterationRuleSet.
|
||||
*
|
||||
* Revision 1.5 1999/12/22 01:40:54 Alan
|
||||
* Consolidate rule pattern anteContext, key, and postContext into one string.
|
||||
*
|
||||
* Revision 1.4 1999/12/22 01:05:54 Alan
|
||||
* Improve masking checking; turn it off by default, for better performance
|
||||
* @version $RCSfile: RuleBasedTransliterator.java,v $ $Revision: 1.20 $ $Date: 2000/04/19 17:35:23 $
|
||||
*/
|
||||
public class RuleBasedTransliterator extends Transliterator {
|
||||
|
||||
|
|
@ -981,7 +997,7 @@ public class RuleBasedTransliterator extends Transliterator {
|
|||
data.ruleSet.addRule(new TransliterationRule(
|
||||
left.text, left.ante, left.post,
|
||||
right.text, right.cursor,
|
||||
left.getSegments(), data));
|
||||
left.getSegments()));
|
||||
|
||||
return pos;
|
||||
}
|
||||
|
|
@ -1193,3 +1209,56 @@ public class RuleBasedTransliterator extends Transliterator {
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* $Log: RuleBasedTransliterator.java,v $
|
||||
* Revision 1.20 2000/04/19 17:35:23 alan
|
||||
* Update javadoc; fix compile error
|
||||
*
|
||||
* Revision 1.19 2000/04/19 16:34:18 alan
|
||||
* Add segment support.
|
||||
*
|
||||
* Revision 1.18 2000/04/12 20:17:45 alan
|
||||
* Delegate replace operation to rule object
|
||||
*
|
||||
* Revision 1.17 2000/03/10 04:07:23 johnf
|
||||
* Copyright update
|
||||
*
|
||||
* Revision 1.16 2000/02/24 20:46:49 liu
|
||||
* Add infinite loop check
|
||||
*
|
||||
* Revision 1.15 2000/02/10 07:36:25 johnf
|
||||
* fixed imports for com.ibm.util.Utility
|
||||
*
|
||||
* Revision 1.14 2000/02/03 18:18:42 Alan
|
||||
* Use array rather than hashtable for char-to-set map
|
||||
*
|
||||
* Revision 1.13 2000/01/27 18:59:19 Alan
|
||||
* Use Position rather than int[] and move all subclass overrides to one method (handleTransliterate)
|
||||
*
|
||||
* Revision 1.12 2000/01/18 17:51:09 Alan
|
||||
* Remove "keyboard" from method names. Make maximum context a field of Transliterator, and have subclasses set it.
|
||||
*
|
||||
* Revision 1.11 2000/01/18 02:30:49 Alan
|
||||
* Add Jamo-Hangul, Hangul-Jamo, fix rules, add compound ID support
|
||||
*
|
||||
* Revision 1.10 2000/01/13 23:53:23 Alan
|
||||
* Fix bugs found during ICU port
|
||||
*
|
||||
* Revision 1.9 2000/01/11 04:12:06 Alan
|
||||
* Cleanup, embellish comments
|
||||
*
|
||||
* Revision 1.8 2000/01/11 02:25:03 Alan
|
||||
* Rewrite UnicodeSet and RBT parsers for better performance and new syntax
|
||||
*
|
||||
* Revision 1.7 2000/01/06 01:36:36 Alan
|
||||
* Allow string arrays in rule resource bundles
|
||||
*
|
||||
* Revision 1.6 2000/01/04 21:43:57 Alan
|
||||
* Add rule indexing, and move masking check to TransliterationRuleSet.
|
||||
*
|
||||
* Revision 1.5 1999/12/22 01:40:54 Alan
|
||||
* Consolidate rule pattern anteContext, key, and postContext into one string.
|
||||
*
|
||||
* Revision 1.4 1999/12/22 01:05:54 Alan
|
||||
* Improve masking checking; turn it off by default, for better performance
|
||||
*/
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue