Compare commits

..

No commits in common. "organicmaps" and "v1.0.3" have entirely different histories.

41 changed files with 220 additions and 778 deletions

View file

@ -1,37 +0,0 @@
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake
# For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby
name: Test
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
ruby-version: ['2.6', '2.7', '3.0', '3.1']
steps:
- uses: actions/checkout@v3
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
- name: Install dependencies
run: bundle install
- name: Run tests
run: bundle exec rake test

View file

@ -1,34 +1,3 @@
# 1.1.2 (2022-11-15)
- Bugfix: Fixed a runtime error caused by a missing rexml dependency in Ruby 3 (#312)
# 1.1.1 (2021-01-28)
- Bugfix: Properly parse multiline comments in Android XML files (#300)
# 1.1 (2020-07-09)
- Feature: Add --escape-all-tags option to force escaping of Android styling tags (#281)
- Improvement: Twine now requires Ruby 2.4 or greater and rubyzip 2.0 or greater (#297)
- Bugfix: Fix issues with the Django formatter (#289)
# 1.0.6 (2019-05-28)
- Improvement: Support more Android styling tags (#278)
- Improvement: Update Android output path for default language (#276)
# 1.0.5 (2019-02-24)
- Bugfix: Incorrect language detection when reading localization files (#251)
- Bugfix: Double quotes in Android files could be converted to single quotes (#254)
- Bugfix: Properly escape quotes when writing gettext files (#268)
# 1.0.4 (2018-05-30)
- Feature: Add a --quiet option (#245)
- Bugfix: Consume child HTML tags in Android formatter (#247)
- Bugfix: Let consume-localization-archive return a non-zero status (#246)
# 1.0.3 (2018-01-26)
- Bugfix: Workaround a possible crash in safe_yaml (#237)

View file

@ -1,6 +1,8 @@
# Twine
Twine is a command line tool for managing your strings and their translations. These are all stored in a single text file and then Twine uses this file to import and export localization files in a variety of types, including iOS and Mac OS X `.strings` files, Android `.xml` files, gettext `.po` files, and [jquery-localize][jquerylocalize] `.json` files. This allows individuals and companies to easily share translations across multiple projects, as well as export localization files in any format the user wants.
[![Continuous Integration by CircleCI](https://circleci.com/gh/teespring/twine.svg?style=shield)](https://circleci.com/gh/teespring/twine)
Twine is a command line tool for managing your strings and their translations. These are all stored in a master text file and then Twine uses this file to import and export localization files in a variety of types, including iOS and Mac OS X `.strings` files, Android `.xml` files, gettext `.po` files, and [jquery-localize][jquerylocalize] `.json` files. This allows individuals and companies to easily share translations across multiple projects, as well as export localization files in any format the user wants.
## Install
@ -22,7 +24,7 @@ Twine supports [`printf` style placeholders][printf] with one peculiarity: `@` i
Tags are used by Twine as a way to only work with a subset of your definitions at any given point in time. Each definition can be assigned zero or more tags which are separated by commas. Tags are optional, though highly recommended. You can get a list of all definitions currently missing tags by executing the [`validate-twine-file`](#validate-twine-file) command with the `--pedantic` option.
When generating a localization file, you can specify which definitions should be included using the `--tags` option. Provide a comma separated list of tags to match all definitions that contain any of the tags (`--tags tag1,tag2` matches all definitions tagged with `tag1` _or_ `tag2`). Provide multiple `--tags` options to match definitions containing all specified tags (`--tags tag1 --tags tag2` matches all definitions tagged with `tag1` _and_ `tag2`). You can match definitions _not_ containing a tag by prefixing the tag with a tilde (`--tags ~tag1` matches all definitions _not_ tagged with `tag1`). All three options are combinable.
When generating a localization file, you can specify which definitions should be included using the `--tags` option. Provide a comma separated list of tags to match all definitions that contain any of the tags (`--tags tag1,tag2` matches all definitions tagged with `tag1` _or_ `tag2`). Provide multiple `--tags` options to match defintions containing all specified tags (`--tags tag1 --tags tag2` matches all definitions tagged with `tag1` _and_ `tag2`). You can match definitions _not_ containing a tag by prefixing the tag with a tilde (`--tags ~tag1` matches all definitions _not_ tagged with `tag1`). All three options are combinable.
### Whitespace
@ -78,8 +80,8 @@ Twine currently supports the following output formats:
* [Android String Resources][androidstrings] (format: android)
* HTML tags will be escaped by replacing `<` with `&lt`
* Tags inside `<![CDATA[` won't be escaped.
* Supports [basic styling][androidstyling] according to [Android documentation](https://developer.android.com/guide/topics/resources/string-resource.html#StylingWithHTML). All of the documented tags are supported, in addition to `<a>` links.
* These tags will *not* be escaped if the string doesn't contain placeholders. You can reference them directly in your layouts or by using [`getText()`](https://developer.android.com/reference/android/content/res/Resources.html#getText(int)) to read them programmatically.
* Supports [basic styling][androidstyling] with `<b>`, `<i>`, `<u>` and `<a>` links.
* These tags will *not* be escaped if the string doesn't contain placeholders. You can reference them directly in your layouts or by using [`getText()`](https://developer.android.com/reference/android/content/res/Resources.html#getText(int)) to read them programatically.
* These tags *will* be escaped if the string contains placeholders. You can use [`getString()`](https://developer.android.com/reference/android/content/res/Resources.html#getString(int,%20java.lang.Object...)) combined with [`fromHtml`](https://developer.android.com/reference/android/text/Html.html#fromHtml(java.lang.String)) as shown in the [documentation][androidstyling] to display them.
* See [\#212](https://github.com/scelis/twine/issues/212) for details.
* [Gettext PO Files][gettextpo] (format: gettext)
@ -148,7 +150,7 @@ This command validates that the Twine data file can be parsed, contains no dupli
The easiest way to create your first Twine data file is to run the [`consume-all-localization-files`](#consume-all-localization-files) command. The one caveat is to first create a blank file to use as your starting point. Then, just point the `consume-all-localization-files` command at a directory in your project containing all of your localization files.
$ touch twine.txt
$ twine consume-all-localization-files twine.txt Resources/Locales --developer-language en --consume-all --consume-comments --format apple/android/gettext/jquery/django/tizen/flash
$ twine consume-all-localization-files twine.txt Resources/Locales --developer-language en --consume-all --consume-comments
## Twine and Your Build Process
@ -172,10 +174,7 @@ Now, whenever you build your application, Xcode will automatically invoke Twine
### Android Studio/Gradle
#### Standard
Add the following code to `app/build.gradle`:
Add the following task at the top level in app/build.gradle:
```
task generateLocalizations {
String script = 'if hash twine 2>/dev/null; then twine generate-localization-file twine.txt ./src/main/res/values/generated_strings.xml; fi'
@ -184,46 +183,10 @@ task generateLocalizations {
args '-c', script
}
}
preBuild {
dependsOn generateLocalizations
}
```
#### Using [jruby](http://jruby.org)
Now every time you build your app the localization files are generated from the Twine file.
With this approach, developers do not need to manually install ruby, gem, or twine.
Add the following code to `app/build.gradle`:
```
buildscript {
repositories { jcenter() }
dependencies {
/* NOTE: Set your preferred version of jruby here. */
classpath "com.github.jruby-gradle:jruby-gradle-plugin:1.5.0"
}
}
apply plugin: 'com.github.jruby-gradle.base'
dependencies {
/* NOTE: Set your preferred version of twine here. */
jrubyExec 'rubygems:twine:1.1'
}
task generateLocalizations (type: JRubyExec) {
dependsOn jrubyPrepare
jrubyArgs '-S'
script "twine"
scriptArgs 'generate-localization-file', 'twine.txt', './src/main/res/values/generated_strings.xml'
}
preBuild {
dependsOn generateLocalizations
}
```
## User Interface
@ -261,4 +224,4 @@ Many thanks to all of the contributors to the Twine project, including:
[djangopo]: https://docs.djangoproject.com/en/dev/topics/i18n/translation/
[tizen]: https://developer.tizen.org/documentation/articles/localization
[flash]: http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/resources/IResourceManager.html#getString()
[printf]: https://en.wikipedia.org/wiki/Printf_format_string
[printf]: https://en.wikipedia.org/wiki/Printf_format_string

28
circle.yml Normal file
View file

@ -0,0 +1,28 @@
machine:
environment:
BUNDLE_INSTALL_PATH: "./vendor/bundle" # circle caches this by default
TEST_RUBIES: "system 2.1 2.2 2.3"
xcode:
version: "8.2"
dependencies:
override:
- >
for v in $TEST_RUBIES; do
echo
echo "****************************************"
echo "Installing gems on Ruby version: $v"
echo "****************************************"
chruby-exec $v -- bundle install --path $BUNDLE_INSTALL_PATH
done
test:
override:
- >
for v in $TEST_RUBIES; do
echo
echo "*******************************"
echo "Testing on Ruby version: $v"
echo "********************************"
chruby-exec $v -- bundle exec rake test TESTOPTS="--ci-dir=$CIRCLE_TEST_REPORTS/reports"
done

View file

@ -172,7 +172,7 @@
\f0\fs36 \cf0 /**\
* Apple Strings File\
* Generated by Twine\
* Generated by Twine 0.8.1\
* Language: en\
*/\
\

View file

@ -30,6 +30,12 @@ If we map the _input_ for each method to the example file, it looks like this
As stated at the beginning, the output produced by a formatter depends on the formatter. The output of the Apple formatter would for example be
```
/**
* Apple Strings File
* Generated by Twine 0.8.1
* Language: en
*/
/********** General **********/
"yes" = "Yes";
@ -55,7 +61,7 @@ Formatters inherit from [`Abstract`](/lib/twine/formatters/abstract.rb) and need
The `Abstract` formatter also specifies two utility methods to be used when read a localization file, `set_translation_for_key` and `set_comment_for_key`, however the actual parsing is too formatter specific and must be implemented in the `read` method of a formatter.
Which methods to overwrite to produce the desired output depends pretty much on the format. Again, looking at the [bundled formatters](/lib/twine/formatters) will provide some insight.
Which methods to overwrite to produce the desired output depends pretty much on the format. Again, looking at the [bundeled formatters](/lib/twine/formatters) will provide some insight.
Finally, to make a formatter available, it needs to be added to the list of formatters
@ -77,13 +83,8 @@ Plugins are specified as values for the `gems` key. The following is an example
gems: wicked_twine
```
Multiple gems can also be specified in the yaml file.
Multiple gems can also be specfied in the yaml file.
```
gems: [wicked_twine, some_other_plugin]
```
## Sample Plugins
* [appium-twine](https://github.com/appium/appium_twine)
* [twine-flutter](https://github.com/tiknil/twine-flutter)

View file

@ -31,7 +31,6 @@ module Twine
require 'twine/formatters/abstract'
require 'twine/formatters/android'
require 'twine/formatters/apple'
require 'twine/formatters/apple_plural'
require 'twine/formatters/django'
require 'twine/formatters/flash'
require 'twine/formatters/gettext'

View file

@ -45,14 +45,6 @@ module Twine
files are UTF-16 without BOM, you need to specify if it's UTF-16LE or UTF16-BE.
DESC
},
escape_all_tags: {
switch: ['--[no-]escape-all-tags'],
description: <<-DESC,
Always escape all HTML tags. By default the Android formatter will ONLY escape styling tags, if a
string also contains placeholders. This flag enforces that styling tags are escaped regardless of
placeholders.
DESC
},
file_name: {
switch: ['-n', '--file-name FILE_NAME'],
description: 'This flag may be used to overwrite the default file name of the format.'
@ -86,10 +78,6 @@ module Twine
switch: ['-p', '--[no-]pedantic'],
description: 'When validating a Twine file, perform additional checks that go beyond pure validity (like presence of tags).'
},
quiet: {
switch: ['-q', '--[no-]quiet'],
description: 'Suppress all console output except error messages.'
},
tags: {
switch: ['-t', '--tags TAG1,TAG2,TAG3', Array],
description: <<-DESC,
@ -119,11 +107,9 @@ module Twine
optional_options: [
:developer_language,
:encoding,
:escape_all_tags,
:format,
:include,
:languages,
:quiet,
:tags,
:untagged,
:validate
@ -142,11 +128,9 @@ module Twine
:create_folders,
:developer_language,
:encoding,
:escape_all_tags,
:file_name,
:format,
:include,
:quiet,
:tags,
:untagged,
:validate
@ -162,9 +146,7 @@ module Twine
optional_options: [
:developer_language,
:encoding,
:escape_all_tags,
:include,
:quiet,
:tags,
:untagged,
:validate
@ -182,7 +164,6 @@ module Twine
:format,
:languages,
:output_path,
:quiet,
:tags
],
option_validation: Proc.new { |options|
@ -202,7 +183,6 @@ module Twine
:encoding,
:format,
:output_path,
:quiet,
:tags
],
example: 'twine consume-all-localization-files twine.txt Resources/Locales/ --developer-language en --tags DefaultTag1,DefaultTag2'
@ -217,7 +197,6 @@ module Twine
:encoding,
:format,
:output_path,
:quiet,
:tags
],
example: 'twine consume-localization-archive twine.txt LocDrop5.zip'
@ -227,8 +206,7 @@ module Twine
arguments: [:twine_file],
optional_options: [
:developer_language,
:pedantic,
:quiet
:pedantic
],
example: 'twine validate-twine-file twine.txt'
}
@ -249,7 +227,7 @@ module Twine
mapped_command = DEPRECATED_COMMAND_MAPPINGS[command]
if mapped_command
Twine::stdout.puts "WARNING: Twine commands names have changed. `#{command}` is now `#{mapped_command}`. The old command is deprecated and will soon stop working. For more information please check the documentation at https://github.com/mobiata/twine"
Twine::stderr.puts "WARNING: Twine commands names have changed. `#{command}` is now `#{mapped_command}`. The old command is deprecated and will soon stop working. For more information please check the documentation at https://github.com/mobiata/twine"
command = mapped_command
end

View file

@ -3,9 +3,6 @@ require 'fileutils'
module Twine
module Formatters
class Abstract
SUPPORTS_PLURAL = false
LANGUAGE_CODE_WITH_OPTIONAL_REGION_CODE = "[a-z]{2}(?:-[A-Za-z]{2})?"
attr_accessor :twine_file
attr_accessor :options
@ -41,7 +38,7 @@ module Twine
definition.translations[lang] = value
end
elsif @options[:consume_all]
Twine::stdout.puts "Adding new definition '#{key}' to twine file."
Twine::stderr.puts "Adding new definition '#{key}' to twine file."
current_section = @twine_file.sections.find { |s| s.name == 'Uncategorized' }
unless current_section
current_section = TwineSection.new('Uncategorized')
@ -57,7 +54,7 @@ module Twine
@twine_file.definitions_by_key[key] = current_definition
@twine_file.definitions_by_key[key].translations[lang] = value
else
Twine::stdout.puts "WARNING: '#{key}' not found in twine file."
Twine::stderr.puts "Warning: '#{key}' not found in twine file."
end
if !@twine_file.language_codes.include?(lang)
@twine_file.add_language_code(lang)
@ -79,12 +76,7 @@ module Twine
end
def determine_language_given_path(path)
only_language_and_region = /^#{LANGUAGE_CODE_WITH_OPTIONAL_REGION_CODE}$/i
basename = File.basename(path, File.extname(path))
return basename if basename =~ only_language_and_region
return basename if @twine_file.language_codes.include? basename
path.split(File::SEPARATOR).reverse.find { |segment| segment =~ only_language_and_region }
raise NotImplementedError.new("You must implement determine_language_given_path in your formatter class.")
end
def output_path_for_language(lang)
@ -140,13 +132,7 @@ module Twine
end
def format_definition(definition, lang)
formatted_definition = [format_comment(definition, lang)]
if self.class::SUPPORTS_PLURAL && definition.is_plural?
formatted_definition << format_plural(definition, lang)
else
formatted_definition << format_key_value(definition, lang)
end
formatted_definition.compact.join
[format_comment(definition, lang), format_key_value(definition, lang)].compact.join
end
def format_comment(definition, lang)
@ -157,21 +143,10 @@ module Twine
key_value_pattern % { key: format_key(definition.key.dup), value: format_value(value.dup) }
end
def format_plural(definition, lang)
plural_hash = definition.plural_translation_for_lang(lang)
if plural_hash
format_plural_keys(definition.key.dup, plural_hash)
end
end
def key_value_pattern
raise NotImplementedError.new("You must implement key_value_pattern in your formatter class.")
end
def format_plural_keys(key, plural_hash)
raise NotImplementedError.new("You must implement format_plural_keys in your formatter class.")
end
def format_key(key)
key
end

View file

@ -7,17 +7,6 @@ module Twine
class Android < Abstract
include Twine::Placeholders
SUPPORTS_PLURAL = true
LANG_CODES = Hash[
'zh' => 'zh-Hans',
'zh-CN' => 'zh-Hans',
'zh-HK' => 'zh-Hant',
# See https://developer.android.com/reference/java/util/Locale#legacy-language-codes
'iw' => 'he',
'in' => 'id',
'ji' => 'yi'
]
def format_name
'android'
end
@ -44,22 +33,15 @@ module Twine
# see http://developer.android.com/guide/topics/resources/providing-resources.html#AlternativeResources
match = /^values-([a-z]{2}(-r[a-z]{2})?)$/i.match(segment)
if match
lang = match[1].sub('-r', '-')
return LANG_CODES.fetch(lang, lang)
end
return match[1].sub('-r', '-') if match
end
end
return super
return
end
def output_path_for_language(lang)
if lang == @twine_file.language_codes[0]
"values"
else
"values-#{lang}".gsub(/-(\p{Lu})/, '-r\1')
end
"values-#{lang}".gsub(/-(\p{Lu})/, '-r\1')
end
def set_translation_for_key(key, lang, value)
@ -74,20 +56,18 @@ module Twine
def read(io, lang)
document = REXML::Document.new io, :compress_whitespace => %w{ string }
document.context[:attribute_quote] = :quote
comment = nil
document.root.children.each do |child|
if child.is_a? REXML::Comment
content = child.string.strip
content.gsub!(/[\s]+/, ' ')
comment = content if content.length > 0 and not content.start_with?("SECTION:")
elsif child.is_a? REXML::Element
next unless child.name == 'string'
key = child.attributes['name']
content = child.children.map(&:to_s).join
set_translation_for_key(key, lang, content)
set_translation_for_key(key, lang, child.text)
set_comment_for_key(key, comment) if comment
comment = nil
@ -96,7 +76,7 @@ module Twine
end
def format_header(lang)
"<?xml version=\"1.0\" encoding=\"utf-8\"?>"
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Android Strings File -->\n<!-- Generated by Twine #{Twine::VERSION} -->\n<!-- Language: #{lang} -->"
end
def format_sections(twine_file, lang)
@ -108,25 +88,15 @@ module Twine
end
def format_section_header(section)
"#{space(4)}<!-- SECTION: #{section.name} -->"
"\t<!-- SECTION: #{section.name} -->"
end
def format_comment(definition, lang)
"#{space(4)}<!-- #{definition.comment.gsub('--', '—')} -->\n" if definition.comment
"\t<!-- #{definition.comment.gsub('--', '—')} -->\n" if definition.comment
end
def key_value_pattern
"#{space(4)}<string name=\"%{key}\">%{value}</string>"
end
def format_plural_keys(key, plural_hash)
result = "#{space(4)}<plurals name=\"#{key}\">\n"
result += plural_hash.map{|quantity,value| "#{space(8)}<item quantity=\"#{quantity}\">#{escape_value(value)}</item>"}.join("\n")
result += "\n#{space(4)}</plurals>"
end
def space(level)
' ' * level
"\t<string name=\"%{key}\">%{value}</string>"
end
def gsub_unless(text, pattern, replacement)
@ -138,29 +108,24 @@ module Twine
# http://developer.android.com/guide/topics/resources/string-resource.html#FormattingAndStyling
def escape_value(value)
inside_cdata = /<\!\[CDATA\[((?!\]\]>).)*$/ # opening CDATA tag ('<![CDATA[') not followed by a closing tag (']]>')
inside_opening_tag = /<(a|font|span|p)\s?((?!>).)*$/ # tag start ('<a ', '<font ', '<span ' or '<p ') not followed by a '>'
inside_cdata = /<\!\[CDATA\[((?!\]\]>).)*$/ # opening CDATA tag ('<![CDATA[') not followed by a closing tag (']]>')
inside_opening_anchor_tag = /<a\s?((?!>).)*$/ # anchor tag start ('<a ') not followed by a '>'
# escape double and single quotes and & signs
value = gsub_unless(value, '"', '\\"') { |substring| substring =~ inside_cdata || substring =~ inside_opening_tag }
value = gsub_unless(value, '"', '\\"') { |substring| substring =~ inside_cdata || substring =~ inside_opening_anchor_tag }
value = gsub_unless(value, "'", "\\'") { |substring| substring =~ inside_cdata }
value = gsub_unless(value, /&/, '&amp;') { |substring| substring =~ inside_cdata || substring =~ inside_opening_tag }
value = gsub_unless(value, /&/, '&amp;') { |substring| substring =~ inside_cdata || substring =~ inside_opening_anchor_tag }
# if `value` contains a placeholder, escape all angle brackets
# if not, escape opening angle brackes unless it's a supported styling tag
# https://github.com/scelis/twine/issues/212
# https://stackoverflow.com/questions/3235131/#18199543
if number_of_twine_placeholders(value) > 0 or @options[:escape_all_tags]
# matches all `<` but <![CDATA
angle_bracket = /<(?!(\/?(\!\[CDATA)))/
else
# matches all '<' but <b>, <em>, <i>, <cite>, <dfn>, <big>, <small>, <font>, <tt>, <s>,
# <strike>, <del>, <u>, <super>, <sub>, <ul>, <li>, <br>, <div>, <span>, <p>, <a>
# and <![CDATA
angle_bracket = /<(?!(\/?(b|em|i|cite|dfn|big|small|font|tt|s|strike|del|u|super|sub|ul|li|br|div|span|p|a|\!\[CDATA)))/
if number_of_twine_placeholders(value) > 0
angle_bracket = /<(?!(\/?(\!\[CDATA)))/ # matches all `<` but <![CDATA
else
angle_bracket = /<(?!(\/?(b|u|i|a|\!\[CDATA)))/ # matches all `<` but <b>, <u>, <i>, <a> and <![CDATA
end
value = gsub_unless(value, angle_bracket, '&lt;') { |substring| substring =~ inside_cdata }
value = gsub_unless(value, '\n', "\n\\n") { |substring| substring =~ inside_cdata }
# escape non resource identifier @ signs (http://developer.android.com/guide/topics/resources/accessing-resources.html#ResourcesFromXml)
resource_identifier_regex = /@(?!([a-z\.]+:)?[a-z+]+\/[a-zA-Z_]+)/ # @[<package_name>:]<resource_type>/<resource_name>

View file

@ -1,8 +1,6 @@
module Twine
module Formatters
class Apple < Abstract
include Twine::Placeholders
def format_name
'apple'
end
@ -32,7 +30,7 @@ module Twine
end
end
return super
return
end
def output_path_for_language(lang)
@ -65,21 +63,20 @@ module Twine
end
end
def format_file(lang)
result = super
result += "\n"
def format_header(lang)
"/**\n * Apple Strings File\n * Generated by Twine #{Twine::VERSION}\n * Language: #{lang}\n */"
end
def format_section_header(section)
"\n/********** #{section.name} **********/\n"
"/********** #{section.name} **********/\n"
end
def key_value_pattern
"\"%{key}\" = \"%{value}\";"
"\"%{key}\" = \"%{value}\";\n"
end
def format_comment(definition, lang)
"\n/* #{definition.comment.gsub('*/', '* /')} */\n" if definition.comment
"/* #{definition.comment.gsub('*/', '* /')} */\n" if definition.comment
end
def format_key(key)
@ -87,14 +84,8 @@ module Twine
end
def format_value(value)
# Replace Android's %s with iOS %@
value = convert_placeholders_from_android_to_twine(value)
escape_quotes(value)
end
def should_include_definition(definition, lang)
return !definition.is_plural? && super
end
end
end
end

View file

@ -1,72 +0,0 @@
module Twine
module Formatters
class ApplePlural < Apple
include Twine::Placeholders
SUPPORTS_PLURAL = true
def format_name
'apple-plural'
end
def extension
'.stringsdict'
end
def default_file_name
'Localizable.stringsdict'
end
def format_footer(lang)
footer = "</dict>\n</plist>"
end
def format_file(lang)
result = super
result += format_footer(lang)
end
def format_header(lang)
header = "<\?xml version=\"1.0\" encoding=\"UTF-8\"\?>\n"
header += "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
header += "<plist version=\"1.0\">\n<dict>"
end
def format_section_header(section)
"<!-- ********** #{section.name} **********/ -->\n"
end
def format_plural_keys(key, plural_hash)
result = "\t<key>#{key}</key>\n"
result += "\t<dict>\n"
result += "\t\t<key>NSStringLocalizedFormatKey</key>\n"
result += "\t\t<string>\%\#@value@</string>\n"
result += "\t\t<key>value</key>\n"
result += "\t\t<dict>\n"
result += "\t\t\t<key>NSStringFormatSpecTypeKey</key>\n"
result += "\t\t\t<string>NSStringPluralRuleType</string>\n"
result += "\t\t\t<key>NSStringFormatValueTypeKey</key>\n"
result += "\t\t\t<string>d</string>\n"
# Replace Android's %s with iOS %@
result += plural_hash.map{|quantity,value| "\t\t\t<key>#{quantity}</key>\n\t\t\t<string>#{convert_placeholders_from_android_to_twine(value)}</string>"}.join("\n")
result += "\n"
result += "\t\t</dict>\n"
result += "\t</dict>\n"
end
def format_comment(definition, lang)
"<!-- #{definition.comment.gsub('--', '—')} -->\n" if definition.comment
end
def read(io, lang)
raise NotImplementedError.new("Reading \".stringdict\" files not implemented yet")
end
def should_include_definition(definition, lang)
return definition.is_plural? && definition.plural_translation_for_lang(lang)
end
end
end
end
Twine::Formatters.formatters << Twine::Formatters::ApplePlural.new

View file

@ -1,6 +1,5 @@
module Twine
module Formatters
# For a description of the .po file format, see https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html
class Django < Abstract
def format_name
'django'
@ -14,11 +13,22 @@ module Twine
'strings.po'
end
def read(io, lang)
comment_regex = /^\s*#\. *"?(.*)"?$/
key_regex = /^msgid *"(.*)"$/
value_regex = /^msgstr *"(.*)"$/m
def determine_language_given_path(path)
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
match = /([a-z]{2}(-[A-Za-z]{2})?)\.po$/.match(segment)
return match[1] if match
end
return
end
def read(io, lang)
comment_regex = /#\. *"?(.*)"?$/
key_regex = /msgid *"(.*)"$/
value_regex = /msgstr *"(.*)"$/m
last_comment = nil
while line = io.gets
comment_match = comment_regex.match(line)
if comment_match
@ -54,12 +64,11 @@ module Twine
end
def format_header(lang)
# see https://www.gnu.org/software/trans-coord/manual/gnun/html_node/PO-Header.html for details
"# Django Strings File\n# Generated by Twine\n# Language: #{lang}\nmsgid \"\"\nmsgstr \"\"\n\"Content-Type: text/plain; charset=UTF-8\\n\""
"##\n # Django Strings File\n # Generated by Twine #{Twine::VERSION}\n # Language: #{lang}\nmsgid \"\"\nmsgstr \"\"\n\"Content-Type: text/plain; charset=UTF-8\\n\""
end
def format_section_header(section)
"# --------- #{section.name} --------- #\n"
"#--------- #{section.name} ---------#\n"
end
def format_definition(definition, lang)

View file

@ -15,6 +15,11 @@ module Twine
'resources.properties'
end
def determine_language_given_path(path)
# match two-letter language code, optionally followed by a two letter region code
path.split(File::SEPARATOR).reverse.find { |segment| segment =~ /^([a-z]{2}(-[a-z]{2})?)$/i }
end
def set_translation_for_key(key, lang, value)
value = convert_placeholders_from_flash_to_twine(value)
super(key, lang, value)
@ -42,7 +47,7 @@ module Twine
end
def format_header(lang)
"## Flash Strings File\n## Generated by Twine\n## Language: #{lang}"
"## Flash Strings File\n## Generated by Twine #{Twine::VERSION}\n## Language: #{lang}"
end
def format_section_header(section)

View file

@ -15,6 +15,16 @@ module Twine
'strings.po'
end
def determine_language_given_path(path)
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
match = /([a-z]{2}(-[A-Za-z]{2})?)\.po$/.match(segment)
return match[1] if match
end
return
end
def read(io, lang)
comment_regex = /#.? *"(.*)"$/
key_regex = /msgctxt *"(.*)"$/
@ -55,7 +65,7 @@ module Twine
end
def format_header(lang)
"msgid \"\"\nmsgstr \"\"\n\"Language: #{lang}\"\n\"X-Generator: Twine #{Twine::VERSION}\"\n"
"msgid \"\"\nmsgstr \"\"\n\"Language: #{lang}\\n\"\n\"X-Generator: Twine #{Twine::VERSION}\\n\"\n"
end
def format_section_header(section)
@ -76,15 +86,15 @@ module Twine
end
def format_key(key)
"msgctxt \"#{escape_quotes(key)}\"\n"
"msgctxt \"#{key}\"\n"
end
def format_base_translation(definition)
"msgid \"#{escape_quotes(definition.translations[@default_lang])}\"\n"
"msgid \"#{definition.translations[@default_lang]}\"\n"
end
def format_value(value)
"msgstr \"#{escape_quotes(value)}\"\n"
"msgstr \"#{value}\"\n"
end
end
end

View file

@ -14,17 +14,22 @@ module Twine
end
def determine_language_given_path(path)
match = /^.+([a-z]{2}-[A-Z]{2})\.json$/.match File.basename(path)
return match[1] if match
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
match = /^((.+)-)?([^-]+)\.json$/.match(segment)
if match
return match[3]
end
end
return super
return
end
def read(io, lang)
begin
require "json"
rescue LoadError
raise Twine::Error.new "You must run `gem install json` in order to read or write jquery-localize files."
raise Twine::Error.new "You must run 'gem install json' in order to read or write jquery-localize files."
end
json = JSON.load(io)
@ -41,7 +46,7 @@ module Twine
def format_sections(twine_file, lang)
sections = twine_file.sections.map { |section| format_section(section, lang) }
sections.delete_if(&:empty?)
sections.delete_if &:empty?
sections.join(",\n\n")
end

View file

@ -91,7 +91,7 @@ module Twine
end
def format_header(lang)
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Tizen Strings File -->\n<!-- Generated by Twine -->\n<!-- Language: #{lang} -->"
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Tizen Strings File -->\n<!-- Generated by Twine #{Twine::VERSION} -->\n<!-- Language: #{lang} -->"
end
def format_sections(twine_file, lang)

View file

@ -13,16 +13,10 @@ module Twine
def fallback_languages(language)
fallback_mapping = {
'zh-CN' => 'zh-Hans', # if we don't have a zh-CN translation, try zh-Hans before en
'zh-TW' => 'zh-Hant' # if we don't have a zh-TW translation, try zh-Hant before en
}
# Regional dialect fallbacks to generic language (for example: 'es-MX' to 'es' instead of default 'en').
if language.match(/([a-zA-Z]{2})-[a-zA-Z]+/)
generic_language = language.gsub(/([a-zA-Z])-[a-zA-Z]+/, '\1')
end
[fallback_mapping[language], generic_language, default_language].flatten.compact
[fallback_mapping[language], default_language].flatten.compact
end
def process(language)
@ -48,14 +42,6 @@ module Twine
new_definition = definition.dup
new_definition.translations[language] = value
if definition.is_plural?
# If definition is plural, but no translation found -> create
# Then check 'other' key
if !(new_definition.plural_translations[language] ||= {}).key? 'other'
new_definition.plural_translations[language]['other'] = value
end
end
new_section.definitions << new_definition
result.definitions_by_key[new_definition.key] = new_definition
end

View file

@ -72,11 +72,5 @@ module Twine
def convert_placeholders_from_flash_to_twine(input)
input.gsub /\{\d+\}/, '%@'
end
# Python supports placeholders in the form of `%(amount)03d`
# see https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting
def contains_python_specific_placeholder(input)
/%\([a-zA-Z0-9_-]+\)#{PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH}#{PLACEHOLDER_TYPES}/.match(input) != nil
end
end
end

View file

@ -57,7 +57,7 @@ module Twine
end
def join_path *paths
File.expand_path File.join(*paths)
File.expand_path File.join *paths
end
end
end

View file

@ -5,14 +5,6 @@ Twine::Plugin.new # Initialize plugins first in Runner.
module Twine
class Runner
class NullOutput
def puts(message)
end
def string
""
end
end
def self.run(args)
options = CLI.parse(args)
@ -43,9 +35,6 @@ module Twine
def initialize(options = {}, twine_file = TwineFile.new)
@options = options
@twine_file = twine_file
if @options[:quite]
Twine::stdout = NullOutput.new
end
end
def write_twine_data(path)
@ -87,7 +76,7 @@ module Twine
end
unless formatter
raise Twine::Error.new "Could not determine format given the contents of #{@options[:output_path]}. Try using `--format`."
raise Twine::Error.new "Could not determine format given the contents of #{@options[:output_path]}"
end
file_name = @options[:file_name] || formatter.default_file_name
@ -101,7 +90,7 @@ module Twine
output = formatter.format_file(lang)
unless output
Twine::stdout.puts "Skipping file at path #{file_path} since it would not contain any translations."
Twine::stderr.puts "Skipping file at path #{file_path} since it would not contain any translations."
next
end
@ -123,7 +112,7 @@ module Twine
file_path = File.join(output_path, file_name)
output = formatter.format_file(lang)
unless output
Twine::stdout.puts "Skipping file at path #{file_path} since it would not contain any translations."
Twine::stderr.puts "Skipping file at path #{file_path} since it would not contain any translations."
next
end
@ -159,7 +148,7 @@ module Twine
output = formatter.format_file(lang)
unless output
Twine::stdout.puts "Skipping file #{file_name} since it would not contain any translations."
Twine::stderr.puts "Skipping file #{file_name} since it would not contain any translations."
next
end
@ -208,7 +197,6 @@ module Twine
raise Twine::Error.new("File does not exist: #{@options[:input_path]}")
end
error_encountered = false
Dir.mktmpdir do |temp_dir|
Zip::File.open(@options[:input_path]) do |zipfile|
zipfile.each do |entry|
@ -221,7 +209,6 @@ module Twine
read_localization_file(real_path)
rescue Twine::Error => e
Twine::stderr.puts "#{e.message}"
error_encountered = true
end
end
end
@ -229,10 +216,6 @@ module Twine
output_path = @options[:output_path] || @options[:twine_file]
write_twine_data(output_path)
if error_encountered
raise Twine::Error.new("At least one file could not be consumed")
end
end
def validate_twine_file
@ -241,7 +224,6 @@ module Twine
duplicate_keys = Set.new
keys_without_tags = Set.new
invalid_keys = Set.new
keys_with_python_only_placeholders = Set.new
valid_key_regex = /^[A-Za-z0-9_]+$/
@twine_file.sections.each do |section|
@ -254,8 +236,6 @@ module Twine
keys_without_tags.add(definition.key) if definition.tags == nil or definition.tags.length == 0
invalid_keys << definition.key unless definition.key =~ valid_key_regex
keys_with_python_only_placeholders << definition.key if definition.translations.values.any? { |v| Placeholders.contains_python_specific_placeholder(v) }
end
end
@ -278,10 +258,6 @@ module Twine
errors << "Found key(s) with invalid characters:\n#{join_keys.call(invalid_keys)}"
end
unless keys_with_python_only_placeholders.empty?
errors << "Found key(s) with placeholders that are only supported by Python:\n#{join_keys.call(keys_with_python_only_placeholders)}"
end
raise Twine::Error.new errors.join("\n\n") unless errors.empty?
Twine::stdout.puts "#{@options[:twine_file]} is valid."
@ -301,16 +277,21 @@ module Twine
end
end
def determine_language_given_path(path)
code = File.basename(path, File.extname(path))
return code if @twine_file.language_codes.include? code
end
def formatter_for_format(format)
find_formatter { |f| f.format_name == format }
end
def find_formatter(&block)
formatters = Formatters.formatters.select(&block)
formatters = Formatters.formatters.select &block
if formatters.empty?
return nil
elsif formatters.size > 1
raise Twine::Error.new("Unable to determine format. Candidates are: #{formatters.map(&:format_name).join(', ')}. Please specify the format you want using `--format`")
raise Twine::Error.new("Unable to determine format. Candidates are: #{formatters.map(&:format_name).join(', ')}. Please specify the format you want using '--format'")
end
formatter = formatters.first
formatter.twine_file = @twine_file
@ -341,12 +322,12 @@ module Twine
end
unless formatter
raise Twine::Error.new "Unable to determine format of #{path}. Try using `--format`."
raise Twine::Error.new "Unable to determine format of #{path}"
end
lang = lang || formatter.determine_language_given_path(path)
lang = lang || determine_language_given_path(path) || formatter.determine_language_given_path(path)
unless lang
raise Twine::Error.new "Unable to determine language for #{path}. Try using `--lang`."
raise Twine::Error.new "Unable to determine language for #{path}"
end
@twine_file.language_codes << lang unless @twine_file.language_codes.include? lang

View file

@ -1,13 +1,9 @@
module Twine
class TwineDefinition
PLURAL_KEYS = %w(zero one two few many other)
attr_reader :key
attr_accessor :comment
attr_accessor :tags
attr_reader :translations
attr_reader :plural_translations
attr_reader :is_plural
attr_accessor :reference
attr_accessor :reference_key
@ -16,7 +12,6 @@ module Twine
@comment = nil
@tags = nil
@translations = {}
@plural_translations = {}
end
def comment
@ -49,22 +44,12 @@ module Twine
end
def translation_for_lang(lang)
translation = [lang].flatten.map { |l| @translations[l] }.compact.first
translation = [lang].flatten.map { |l| @translations[l] }.first
translation = reference.translation_for_lang(lang) if translation.nil? && reference
return translation
end
def plural_translation_for_lang(lang)
if @plural_translations.has_key? lang
@plural_translations[lang].dup.sort_by { |key,_| TwineDefinition::PLURAL_KEYS.index(key) }.to_h
end
end
def is_plural?
!@plural_translations.empty?
end
end
class TwineSection
@ -152,12 +137,11 @@ module Twine
parsed = true
end
else
match = /^([^:=]+)(?::([^=]+))?=(.*)$/.match(line)
match = /^([^=]+)=(.*)$/.match(line)
if match
key = match[1].strip
plural_key = match[2].to_s.strip
value = match[3].strip
value = match[2].strip
value = value[1..-2] if value[0] == '`' && value[-1] == '`'
case key
@ -171,18 +155,7 @@ module Twine
if !@language_codes.include? key
add_language_code(key)
end
# Providing backward compatibility
# for formatters without plural support
if plural_key.empty? || plural_key == 'other'
current_definition.translations[key] = value
end
if !plural_key.empty?
if !TwineDefinition::PLURAL_KEYS.include? plural_key
warn("Unknown plural key #{plural_key}")
next
end
(current_definition.plural_translations[key] ||= {})[plural_key] = value
end
current_definition.translations[key] = value
end
parsed = true
end
@ -217,7 +190,7 @@ module Twine
value = write_value(definition, dev_lang, f)
if !value && !definition.reference_key
Twine::stdout.puts "WARNING: #{definition.key} does not exist in developer language '#{dev_lang}'"
puts "Warning: #{definition.key} does not exist in developer language '#{dev_lang}'"
end
if definition.reference_key

View file

@ -1,3 +1,3 @@
module Twine
VERSION = '1.1.2-om'
VERSION = '1.0.3'
end

Binary file not shown.

View file

@ -1,12 +1,15 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Android Strings File -->
<!-- Generated by Twine <%= Twine::VERSION %> -->
<!-- Language: en -->
<resources>
<!-- SECTION: Section 1 -->
<!-- comment key1 -->
<string name="key1">value1-english</string>
<string name="key2">value2-english</string>
<!-- SECTION: Section 1 -->
<!-- comment key1 -->
<string name="key1">value1-english</string>
<string name="key2">value2-english</string>
<!-- SECTION: Section 2 -->
<string name="key3">value3-english</string>
<!-- comment key4 -->
<string name="key4">value4-english</string>
<!-- SECTION: Section 2 -->
<string name="key3">value3-english</string>
<!-- comment key4 -->
<string name="key4">value4-english</string>
</resources>

View file

@ -1,3 +1,9 @@
/**
* Apple Strings File
* Generated by Twine <%= Twine::VERSION %>
* Language: en
*/
/********** Section 1 **********/
/* comment key1 */

View file

@ -1,11 +1,12 @@
# Django Strings File
# Generated by Twine
# Language: en
##
# Django Strings File
# Generated by Twine <%= Twine::VERSION %>
# Language: en
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
# --------- Section 1 --------- #
#--------- Section 1 ---------#
#. comment key1
# base translation: "value1-english"
@ -17,7 +18,7 @@ msgid "key2"
msgstr "value2-english"
# --------- Section 2 --------- #
#--------- Section 2 ---------#
# base translation: "value3-english"
msgid "key3"

View file

@ -1,5 +1,5 @@
## Flash Strings File
## Generated by Twine
## Generated by Twine <%= Twine::VERSION %>
## Language: en
## Section 1 ##

View file

@ -1,7 +1,7 @@
msgid ""
msgstr ""
"Language: en"
"X-Generator: Twine <%= Twine::VERSION %>"
"Language: en\n"
"X-Generator: Twine <%= Twine::VERSION %>\n"
# SECTION: Section 1

View file

@ -1,10 +0,0 @@
msgid ""
msgstr ""
"Language: en"
"X-Generator: Twine <%= Twine::VERSION %>"
# SECTION: Section
msgctxt "key"
msgid "foo \"bar\" baz"
msgstr "foo \"bar\" baz"

View file

@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Tizen Strings File -->
<!-- Generated by Twine -->
<!-- Generated by Twine <%= Twine::VERSION %> -->
<!-- Language: en -->
<string_table Bversion="2.0.0.201311071819" Dversion="20120315">
<!-- SECTION: Section 1 -->

View file

@ -48,13 +48,6 @@ class CLITest < TwineTest
assert_equal 'UTF16', @options[:encoding]
end
def assert_option_escape_all_tags
parse_with "--escape-all-tags"
assert @options[:escape_all_tags]
parse_with "--no-escape-all-tags"
refute @options[:escape_all_tags]
end
def assert_option_format
random_format = Twine::Formatters.formatters.sample.format_name.downcase
parse_with "--format #{random_format}"
@ -89,13 +82,6 @@ class CLITest < TwineTest
assert_equal @output_path, @options[:output_path]
end
def assert_option_quiet
parse_with '--quiet'
assert @options[:quiet]
parse_with '--no-quiet'
refute @options[:quiet]
end
def assert_option_tags
# single tag
random_tag = "tag#{rand(100)}"
@ -170,7 +156,7 @@ class TestGenerateLocalizationFileCLI < CLITest
def test_missing_argument
assert_raises Twine::Error do
parse "generate-localization-file #{@twine_file_path}"
parse "generate-localization-file #{@twine_file}"
end
end
@ -184,12 +170,10 @@ class TestGenerateLocalizationFileCLI < CLITest
assert_help
assert_option_developer_language
assert_option_encoding
assert_option_escape_all_tags
assert_option_format
assert_option_include
assert_option_single_language
assert_raises(Twine::Error) { assert_option_multiple_languages }
assert_option_quiet
assert_option_tags
assert_option_untagged
assert_option_validate
@ -225,10 +209,8 @@ class TestGenerateAllLocalizationFilesCLI < CLITest
assert_help
assert_option_developer_language
assert_option_encoding
assert_option_escape_all_tags
assert_option_format
assert_option_include
assert_option_quiet
assert_option_tags
assert_option_untagged
assert_option_validate
@ -277,9 +259,7 @@ class TestGenerateLocalizationArchiveCLI < CLITest
assert_help
assert_option_developer_language
assert_option_encoding
assert_option_escape_all_tags
assert_option_include
assert_option_quiet
assert_option_tags
assert_option_untagged
assert_option_validate
@ -298,7 +278,7 @@ class TestGenerateLocalizationArchiveCLI < CLITest
def test_deprecated_command_prints_warning
parse "generate-loc-drop #{@twine_file_path} #{@output_path} --format apple"
assert_match "WARNING: Twine commands names have changed.", Twine::stdout.string
assert_match "WARNING: Twine commands names have changed.", Twine::stderr.string
end
end
@ -337,7 +317,6 @@ class TestConsumeLocalizationFileCLI < CLITest
assert_option_single_language
assert_raises(Twine::Error) { assert_option_multiple_languages }
assert_option_output_path
assert_option_quiet
assert_option_tags
end
end
@ -375,7 +354,6 @@ class TestConsumeAllLocalizationFilesCLI < CLITest
assert_option_encoding
assert_option_format
assert_option_output_path
assert_option_quiet
assert_option_tags
end
end
@ -413,7 +391,6 @@ class TestConsumeLocalizationArchiveCLI < CLITest
assert_option_encoding
assert_option_format
assert_option_output_path
assert_option_quiet
assert_option_tags
end
@ -424,7 +401,7 @@ class TestConsumeLocalizationArchiveCLI < CLITest
def test_deprecated_command_prints_warning
parse "consume-loc-drop #{@twine_file_path} #{@input_path}"
assert_match "WARNING: Twine commands names have changed.", Twine::stdout.string
assert_match "WARNING: Twine commands names have changed.", Twine::stderr.string
end
end
@ -455,7 +432,6 @@ class TestValidateTwineFileCLI < CLITest
def test_options
assert_help
assert_option_developer_language
assert_option_quiet
end
def test_option_pedantic

View file

@ -4,30 +4,24 @@ class TestConsumeLocalizationArchive < CommandTest
def setup
super
options = {}
options[:input_path] = fixture_path 'consume_localization_archive.zip'
options[:output_path] = @output_path
options[:format] = 'apple'
@twine_file = build_twine_file 'en', 'es' do
add_section 'Section' do
add_definition key1: 'value1'
end
end
end
def new_runner(options = {})
options[:input_path] = fixture_path 'consume_localization_archive.zip'
options[:output_path] = @output_path
Twine::Runner.new(options, @twine_file)
@runner = Twine::Runner.new(options, @twine_file)
end
def test_consumes_zip_file
new_runner(format: 'android').consume_localization_archive
@runner.consume_localization_archive
assert @twine_file.definitions_by_key['key1'].translations['en'], 'value1-english'
assert @twine_file.definitions_by_key['key1'].translations['es'], 'value1-spanish'
end
def test_raises_error_if_format_ambiguous
assert_raises Twine::Error do
new_runner.consume_localization_archive
end
end
end

View file

@ -47,87 +47,26 @@ class TestAndroidFormatter < FormatterTest
'a "good" way' => 'a \"good\" way',
'<b>bold</b>' => '<b>bold</b>',
'<em>bold</em>' => '<em>bold</em>',
'<i>italic</i>' => '<i>italic</i>',
'<cite>italic</cite>' => '<cite>italic</cite>',
'<dfn>italic</dfn>' => '<dfn>italic</dfn>',
'<big>larger</big>' => '<big>larger</big>',
'<small>smaller</small>' => '<small>smaller</small>',
'<font color="#45C1D0">F</font>' => '<font color="#45C1D0">F</font>',
'<tt>monospaced</tt>' => '<tt>monospaced</tt>',
'<s>strike</s>' => '<s>strike</s>',
'<strike>strike</strike>' => '<strike>strike</strike>',
'<del>strike</del>' => '<del>strike</del>',
'<u>underline</u>' => '<u>underline</u>',
'<super>superscript</super>'=> '<super>superscript</super>',
'<sub>subscript</sub>' => '<sub>subscript</sub>',
'<ul>bullet point</ul>' => '<ul>bullet point</ul>',
'<li>bullet point</li>' => '<li>bullet point</li>',
'<br>line break' => '<br>line break',
'<div>division</div>' => '<div>division</div>',
'<span style="color:#45C1D0">inline</span>' => '<span style="color:#45C1D0">inline</span>',
'<p>para</p>' => '<p>para</p>',
'<p dir="ltr">para</p>' => '<p dir="ltr">para</p>',
'<b>%@</b>' => '&lt;b>%s&lt;/b>',
'<em>%@</em>' => '&lt;em>%s&lt;/em>',
'<i>%@</i>' => '&lt;i>%s&lt;/i>',
'<cite>%@</cite>' => '&lt;cite>%s&lt;/cite>',
'<dfn>%@</dfn>' => '&lt;dfn>%s&lt;/dfn>',
'<big>%@</big>' => '&lt;big>%s&lt;/big>',
'<small>%@</small>' => '&lt;small>%s&lt;/small>',
'<font color="#45C1D0>%@</font>' => '&lt;font color="#45C1D0>%s&lt;/font>',
'<tt>%@</tt>' => '&lt;tt>%s&lt;/tt>',
'<s>%@</s>' => '&lt;s>%s&lt;/s>',
'<strike>%@</strike>' => '&lt;strike>%s&lt;/strike>',
'<del>%@</del>' => '&lt;del>%s&lt;/del>',
'<u>%@</u>' => '&lt;u>%s&lt;/u>',
'<super>%@</super>' => '&lt;super>%s&lt;/super>',
'<sub>%@</sub>' => '&lt;sub>%s&lt;/sub>',
'<ul>%@</ul>' => '&lt;ul>%s&lt;/ul>',
'<li>%@</li>' => '&lt;li>%s&lt;/li>',
'<br>%@' => '&lt;br>%s',
'<div>%@</div>' => '&lt;div>%s&lt;/div>',
'<span style="color:#45C1D0">%@</span>' => '&lt;span style="color:#45C1D0">%s&lt;/span>',
'<p>%@</p>' => '&lt;p>%s&lt;/p>',
'<p dir="ltr">%@</p>' => '&lt;p dir="ltr">%s&lt;/p>',
'<span>inline</span>' => '&lt;span>inline&lt;/span>',
'<p>paragraph</p>' => '&lt;p>paragraph&lt;/p>',
'<a href="target">link</a>' => '<a href="target">link</a>',
'<a href="target">"link"</a>' => '<a href="target">\"link\"</a>',
'<a href="target"></a>"out"' => '<a href="target"></a>\"out\"',
'<a href="http://url.com?param=1&param2=3&param3=%20">link</a>' => '<a href="http://url.com?param=1&param2=3&param3=%20">link</a>',
'<q>escaped</q><![CDATA[]]>' => '&lt;q>escaped&lt;/q><![CDATA[]]>',
'<![CDATA[]]><q>escaped</q>' => '<![CDATA[]]>&lt;q>escaped&lt;/q>',
'<![CDATA[<q>unescaped</q>]]>' => '<![CDATA[<q>unescaped</q>]]>',
'<![CDATA[<q>unescaped with %@</q>]]>' => '<![CDATA[<q>unescaped with %s</q>]]>',
'<![CDATA[]]><![CDATA[<q>unescaped</q>]]>' => '<![CDATA[]]><![CDATA[<q>unescaped</q>]]>',
'<p>escaped</p><![CDATA[]]>' => '&lt;p>escaped&lt;/p><![CDATA[]]>',
'<![CDATA[]]><p>escaped</p>' => '<![CDATA[]]>&lt;p>escaped&lt;/p>',
'<![CDATA[<p>unescaped</p>]]>' => '<![CDATA[<p>unescaped</p>]]>',
'<![CDATA[<p>unescaped with %@</p>]]>' => '<![CDATA[<p>unescaped with %s</p>]]>',
'<![CDATA[]]><![CDATA[<p>unescaped</p>]]>' => '<![CDATA[]]><![CDATA[<p>unescaped</p>]]>',
'<![CDATA[&]]>' => '<![CDATA[&]]>',
'<![CDATA[\']]>' => '<![CDATA[\']]>',
@ -138,11 +77,6 @@ class TestAndroidFormatter < FormatterTest
'<xliff:g id="42">untouched</xliff:g>' => '<xliff:g id="42">untouched</xliff:g>',
'<xliff:g id="1">first</xliff:g> inbetween <xliff:g id="2">second</xliff:g>' => '<xliff:g id="1">first</xliff:g> inbetween <xliff:g id="2">second</xliff:g>'
}
@escape_all_test_values = {
'<b>bold</b>' => '&lt;b>bold&lt;/b>',
'<i>italic</i>' => '&lt;i>italic&lt;/i>',
'<u>underline</u>' => '&lt;u>underline&lt;/u>'
}
end
def test_read_format
@ -167,54 +101,6 @@ class TestAndroidFormatter < FormatterTest
assert_equal 'This is\n a string', @empty_twine_file.definitions_by_key["foo"].translations['en']
end
def test_read_multiline_comment
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<!-- multiline
comment -->
<string name="foo">This is
a string</string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'multiline comment', @empty_twine_file.definitions_by_key["foo"].comment
end
def test_read_html_tags
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="foo">Hello, <b>BOLD</b></string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'Hello, <b>BOLD</b>', @empty_twine_file.definitions_by_key["foo"].translations['en']
end
def test_double_quotes_are_not_modified
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="foo">Hello, <a href="http://www.foo.com">BOLD</a></string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'Hello, <a href="http://www.foo.com">BOLD</a>', @empty_twine_file.definitions_by_key["foo"].translations['en']
end
def test_set_translation_converts_leading_spaces
@formatter.set_translation_for_key 'key1', 'en', "\u0020value"
assert_equal ' value', @empty_twine_file.definitions_by_key['key1'].translations['en']
@ -240,11 +126,6 @@ class TestAndroidFormatter < FormatterTest
@formatter.set_translation_for_key 'key1', 'en', input
assert_equal expected, @empty_twine_file.definitions_by_key['key1'].translations['en']
end
@escape_all_test_values.each do |expected, input|
@formatter.set_translation_for_key 'key1', 'en', input
assert_equal expected, @empty_twine_file.definitions_by_key['key1'].translations['en']
end
end
def test_format_file
@ -273,11 +154,6 @@ class TestAndroidFormatter < FormatterTest
@escape_test_values.each do |input, expected|
assert_equal expected, @formatter.format_value(input)
end
@formatter.options.merge!({ escape_all_tags: true })
@escape_all_test_values.each do |input, expected|
assert_equal expected, @formatter.format_value(input)
end
end
def test_format_value_escapes_non_resource_identifier_at_signs
@ -289,24 +165,8 @@ class TestAndroidFormatter < FormatterTest
assert_equal identifier, @formatter.format_value(identifier)
end
def test_deducts_language_from_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("#{language}.xml")
end
def test_recognize_every_twine_language_from_filename
twine_file = build_twine_file "not-a-lang-code" do
add_section "Section" do
add_definition key: "value"
end
end
@formatter.twine_file = twine_file
assert_equal "not-a-lang-code", @formatter.determine_language_given_path("not-a-lang-code.xml")
end
def test_deducts_language_from_resource_folder
language = KNOWN_LANGUAGES.sample
language = %w(en de fr).sample
assert_equal language, @formatter.determine_language_given_path("res/values-#{language}")
end
@ -325,13 +185,6 @@ class TestAndroidFormatter < FormatterTest
def test_output_path_with_region
assert_equal 'values-en-rGB', @formatter.output_path_for_language('en-GB')
end
def test_output_path_respects_default_lang
@formatter.twine_file.language_codes.concat KNOWN_LANGUAGES
non_default_language = KNOWN_LANGUAGES[1..-1].sample
assert_equal 'values', @formatter.output_path_for_language(KNOWN_LANGUAGES[0])
assert_equal "values-#{non_default_language}", @formatter.output_path_for_language(non_default_language)
end
end
class TestAppleFormatter < FormatterTest
@ -345,22 +198,6 @@ class TestAppleFormatter < FormatterTest
assert_file_contents_read_correctly
end
def test_deducts_language_from_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("#{language}.strings")
end
def test_recognize_every_twine_language_from_filename
twine_file = build_twine_file "not-a-lang-code" do
add_section "Section" do
add_definition key: "value"
end
end
@formatter.twine_file = twine_file
assert_equal "not-a-lang-code", @formatter.determine_language_given_path("not-a-lang-code.strings")
end
def test_deducts_language_from_resource_folder
language = %w(en de fr).sample
assert_equal language, @formatter.determine_language_given_path("#{language}.lproj/Localizable.strings")
@ -465,21 +302,6 @@ class TestJQueryFormatter < FormatterTest
def test_format_value_with_newline
assert_equal "value\nwith\nline\nbreaks", @formatter.format_value("value\nwith\nline\nbreaks")
end
def test_deducts_language_from_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("#{language}.json")
end
def test_deducts_language_from_extended_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("something-#{language}.json")
end
def test_deducts_language_from_path
language = %w(en-GB de fr).sample
assert_equal language, @formatter.determine_language_given_path("/output/#{language}/#{@formatter.default_file_name}")
end
end
class TestGettextFormatter < FormatterTest
@ -509,21 +331,6 @@ class TestGettextFormatter < FormatterTest
language = "en-GB"
assert_equal language, @formatter.determine_language_given_path("#{language}.po")
end
def test_deducts_language_from_path
language = %w(en-GB de fr).sample
assert_equal language, @formatter.determine_language_given_path("/output/#{language}/#{@formatter.default_file_name}")
end
def test_quoted_strings
formatter = Twine::Formatters::Gettext.new
formatter.twine_file = build_twine_file "not-a-lang-code" do
add_section "Section" do
add_definition key: "foo \"bar\" baz"
end
end
assert_equal content('formatter_gettext_quotes.po'), formatter.format_file('en')
end
end
class TestTizenFormatter < FormatterTest
@ -544,6 +351,7 @@ class TestTizenFormatter < FormatterTest
formatter.twine_file = @twine_file
assert_equal content('formatter_tizen.xml'), formatter.format_file('en')
end
end
class TestDjangoFormatter < FormatterTest
@ -567,24 +375,6 @@ class TestDjangoFormatter < FormatterTest
language = "en-GB"
assert_equal language, @formatter.determine_language_given_path("#{language}.po")
end
def test_deducts_language_from_path
language = %w(en-GB de fr).sample
assert_equal language, @formatter.determine_language_given_path("/output/#{language}/#{@formatter.default_file_name}")
end
def test_ignores_commented_out_strings
content = <<-EOCONTENT
#~ msgid "foo"
#~ msgstr "This should be ignored"
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_nil @empty_twine_file.definitions_by_key["foo"]
end
end
class TestFlashFormatter < FormatterTest
@ -615,10 +405,10 @@ class TestFlashFormatter < FormatterTest
def test_deducts_language_from_resource_folder
language = %w(en de fr).sample
assert_equal language, @formatter.determine_language_given_path("locale/#{language}/#{@formatter.default_file_name}")
assert_equal language, @formatter.determine_language_given_path("locale/#{language}")
end
def test_deducts_language_and_region_from_resource_folder
assert_equal 'de-AT', @formatter.determine_language_given_path("locale/de-AT/#{@formatter.default_file_name}")
assert_equal 'de-AT', @formatter.determine_language_given_path("locale/de-AT")
end
end

View file

@ -24,6 +24,10 @@ class TestGenerateAllLocalizationFiles < CommandTest
def setup
super
Dir.mkdir File.join @output_dir, 'values-en'
# both Android and Tizen can handle folders containing `values-en`
android_formatter = prepare_mock_formatter(Twine::Formatters::Android)
tizen_formatter = prepare_mock_formatter(Twine::Formatters::Tizen, false)
end
def new_runner(options = {})
@ -43,8 +47,8 @@ class TestGenerateAllLocalizationFiles < CommandTest
end
class TestDoNotCreateFolders < TestGenerateAllLocalizationFiles
def new_runner(twine_file = nil, options = {})
super(false, twine_file, options)
def new_runner(twine_file = nil)
super(false, twine_file)
end
def test_fails_if_output_folder_does_not_exist
@ -56,53 +60,39 @@ class TestGenerateAllLocalizationFiles < CommandTest
def test_does_not_create_language_folders
Dir.mkdir File.join @output_dir, 'en.lproj'
new_runner.generate_all_localization_files
refute File.exist?(File.join(@output_dir, 'es.lproj')), "language folder should not be created"
refute File.exists?(File.join(@output_dir, 'es.lproj')), "language folder should not be created"
end
def test_prints_empty_file_warnings
Dir.mkdir File.join @output_dir, 'en.lproj'
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file).generate_all_localization_files
assert_match "Skipping file at path", Twine::stdout.string
end
def test_does_not_print_empty_file_warnings_if_quite
Dir.mkdir File.join @output_dir, 'en.lproj'
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file, quite: true).generate_all_localization_files
refute_match "Skipping file at path", Twine::stdout.string
assert_match "Skipping file at path", Twine::stderr.string
end
end
class TestCreateFolders < TestGenerateAllLocalizationFiles
def new_runner(twine_file = nil, options = {})
super(true, twine_file, options)
def new_runner(twine_file = nil)
super(true, twine_file)
end
def test_creates_output_folder
FileUtils.remove_entry_secure @output_dir
new_runner.generate_all_localization_files
assert File.exist? @output_dir
assert File.exists? @output_dir
end
def test_creates_language_folders
new_runner.generate_all_localization_files
assert File.exist?(File.join(@output_dir, 'en.lproj')), "language folder 'en.lproj' should be created"
assert File.exist?(File.join(@output_dir, 'es.lproj')), "language folder 'es.lproj' should be created"
assert File.exists?(File.join(@output_dir, 'en.lproj')), "language folder 'en.lproj' should be created"
assert File.exists?(File.join(@output_dir, 'es.lproj')), "language folder 'es.lproj' should be created"
end
def test_prints_empty_file_warnings
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file).generate_all_localization_files
assert_match "Skipping file at path", Twine::stdout.string
end
def test_does_not_print_empty_file_warnings_if_quite
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file, quite: true).generate_all_localization_files
refute_match "Skipping file at path", Twine::stdout.string
assert_match "Skipping file at path", Twine::stderr.string
end
end

View file

@ -1,7 +1,8 @@
require 'command_test'
class TestGenerateLocalizationArchive < CommandTest
def new_runner(twine_file = nil, options = {})
def new_runner(twine_file = nil)
options = {}
options[:output_path] = @output_path
options[:format] = 'apple'
@ -19,7 +20,7 @@ class TestGenerateLocalizationArchive < CommandTest
def test_generates_zip_file
new_runner.generate_localization_archive
assert File.exist?(@output_path), "zip file should exist"
assert File.exists?(@output_path), "zip file should exist"
end
def test_zip_file_structure
@ -44,13 +45,7 @@ class TestGenerateLocalizationArchive < CommandTest
def test_prints_empty_file_warnings
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file).generate_localization_archive
assert_match "Skipping file", Twine::stdout.string
end
def test_does_not_print_empty_file_warnings_if_quite
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file, quite: true).generate_localization_archive
refute_match "Skipping file", Twine::stdout.string
assert_match "Skipping file", Twine::stderr.string
end
class TestValidate < CommandTest

View file

@ -122,21 +122,4 @@ class PlaceholderTest < TwineTest
assert_equal "some %@ more %@ text %@", from_flash("some {0} more {1} text {2}")
end
end
class PythonPlaceholder < PlaceholderTest
def test_negative_for_regular_placeholders
assert_equal false, Twine::Placeholders.contains_python_specific_placeholder(placeholder)
end
def test_positive_for_named_placeholders
inputs = [
"%(language)s has",
"For %(number)03d quotes",
"bought on %(app_name)s"
]
inputs.each do |input|
assert_equal true, Twine::Placeholders.contains_python_specific_placeholder(input)
end
end
end
end

View file

@ -58,12 +58,4 @@ class TestValidateTwineFile < CommandTest
Twine::Runner.new(@options.merge(pedantic: true), @twine_file).validate_twine_file
end
end
def test_reports_python_specific_placeholders
random_definition.translations["en"] = "%(python_only)s"
assert_raises Twine::Error do
Twine::Runner.new(@options, @twine_file).validate_twine_file
end
end
end

View file

@ -1,6 +1,6 @@
require 'erb'
require 'minitest/autorun'
require "mocha/minitest"
require "mocha/mini_test"
require 'securerandom'
require 'stringio'
require 'twine'
@ -23,7 +23,7 @@ class TwineTest < Minitest::Test
end
def teardown
FileUtils.remove_entry_secure @output_dir if File.exist? @output_dir
FileUtils.remove_entry_secure @output_dir if File.exists? @output_dir
Twine::Formatters.formatters.clear
Twine::Formatters.formatters.concat @formatters
super

2
twine
View file

@ -1,3 +1,3 @@
#!/bin/sh
BASEDIR=$(dirname $0)
ruby -rrubygems -I $BASEDIR/lib $BASEDIR/bin/twine "$@"
ruby -rubygems -I $BASEDIR/lib $BASEDIR/bin/twine "$@"

View file

@ -6,7 +6,7 @@ Gem::Specification.new do |s|
s.version = Twine::VERSION
s.date = Time.now.strftime('%Y-%m-%d')
s.summary = "Manage strings and their translations for your iOS, Android and other projects."
s.homepage = "https://github.com/scelis/twine"
s.homepage = "https://github.com/mobiata/twine"
s.email = "twine@mobiata.com"
s.authors = [ "Sebastian Celis" ]
s.has_rdoc = false
@ -18,11 +18,10 @@ Gem::Specification.new do |s|
s.files += Dir.glob("test/**/*")
s.test_files = Dir.glob("test/test_*")
s.required_ruby_version = ">= 2.6"
s.add_runtime_dependency('rexml', "~> 3.2")
s.add_runtime_dependency('rubyzip', "~> 2.0")
s.required_ruby_version = ">= 2.0"
s.add_runtime_dependency('rubyzip', "~> 1.1")
s.add_runtime_dependency('safe_yaml', "~> 1.0")
s.add_development_dependency('rake', "~> 13.0")
s.add_development_dependency('rake', "~> 10.4")
s.add_development_dependency('minitest', "~> 5.5")
s.add_development_dependency('minitest-ci', "~> 3.0")
s.add_development_dependency('mocha', "~> 1.1")