Compare commits

..

No commits in common. "organicmaps" and "v0.8.1" have entirely different histories.

71 changed files with 1912 additions and 7475 deletions

View file

@ -1,37 +0,0 @@
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# This workflow will download a prebuilt Ruby version, install dependencies and run tests with Rake
# For more information see: https://github.com/marketplace/actions/setup-ruby-jruby-and-truffleruby
name: Test
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
permissions:
contents: read
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
ruby-version: ['2.6', '2.7', '3.0', '3.1']
steps:
- uses: actions/checkout@v3
- name: Set up Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: ${{ matrix.ruby-version }}
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
- name: Install dependencies
run: bundle install
- name: Run tests
run: bundle exec rake test

2
.gitignore vendored
View file

@ -1,5 +1,3 @@
*.gem
.idea/
*.lock
.ruby-version
.DS_Store

View file

@ -1,59 +0,0 @@
# 1.1.2 (2022-11-15)
- Bugfix: Fixed a runtime error caused by a missing rexml dependency in Ruby 3 (#312)
# 1.1.1 (2021-01-28)
- Bugfix: Properly parse multiline comments in Android XML files (#300)
# 1.1 (2020-07-09)
- Feature: Add --escape-all-tags option to force escaping of Android styling tags (#281)
- Improvement: Twine now requires Ruby 2.4 or greater and rubyzip 2.0 or greater (#297)
- Bugfix: Fix issues with the Django formatter (#289)
# 1.0.6 (2019-05-28)
- Improvement: Support more Android styling tags (#278)
- Improvement: Update Android output path for default language (#276)
# 1.0.5 (2019-02-24)
- Bugfix: Incorrect language detection when reading localization files (#251)
- Bugfix: Double quotes in Android files could be converted to single quotes (#254)
- Bugfix: Properly escape quotes when writing gettext files (#268)
# 1.0.4 (2018-05-30)
- Feature: Add a --quiet option (#245)
- Bugfix: Consume child HTML tags in Android formatter (#247)
- Bugfix: Let consume-localization-archive return a non-zero status (#246)
# 1.0.3 (2018-01-26)
- Bugfix: Workaround a possible crash in safe_yaml (#237)
- Bugfix: Fix an error caused by combining %@ with other placeholders (#235)
# 1.0.2 (2018-01-20)
- Improvement: Better support for placeholders in HTML styled Android strings (#212)
# 1.0.1 (2017-10-17)
- Bugfix: Always prefer the passed-in formatter (#221)
# 1.0 (2017-10-16)
- Feature: Fail twine commands if there's more than one formatter candidate (#201)
- Feature: In the Apple formatter, use developer language for the base localization (#219)
- Bugfix: Preserve basic HTML styling for Android strings (#212)
- Bugfix: `twine --version` reports "unknown" (#213)
- Bugfix: Support spaces in command line arguments (#206)
- Bugfix: 'r' missing before region code in Android output folder (#203)
- Bugfix: .po formatter language detection (#199)
- Bugfix: Add 'q' placeholder for android strings (#194)
- Bugfix: Boolean command line parameters are always true (#191)
# 0.10.1 (2017-01-19)
- Bugfix: Xcode integration (#184)

View file

@ -1,6 +0,0 @@
If you're reporting a bug, please do the following:
- Mention the Twine version you're using
- Provide a minimal but complete input file causing the error (inline with triple ` is fine)
- Quote the exact twine command that's being run
- Provide the complete _expected_ output (again, inline is fine)

261
README.md
View file

@ -1,28 +1,39 @@
# Twine
Twine is a command line tool for managing your strings and their translations. These are all stored in a single text file and then Twine uses this file to import and export localization files in a variety of types, including iOS and Mac OS X `.strings` files, Android `.xml` files, gettext `.po` files, and [jquery-localize][jquerylocalize] `.json` files. This allows individuals and companies to easily share translations across multiple projects, as well as export localization files in any format the user wants.
Twine is a command line tool for managing your strings and their translations. These strings are all stored in a master text file and then Twine uses this file to import and export strings in a variety of file types, including iOS and Mac OS X `.strings` files, Android `.xml` files, gettext `.po` files, and [jquery-localize][jquerylocalize] `.json` files. This allows individuals and companies to easily share strings across multiple projects, as well as export strings in any format the user wants.
## Install
### As a Gem
Twine is most easily installed as a Gem.
$ gem install twine
## Twine File Format
### From Source
Twine stores everything in a single file, the Twine data file. The format of this file is a slight variant of the [Git][git] config file format, which itself is based on the old [Windows INI file][INI] format. The entire file is broken up into sections, which are created by placing the section name between two pairs of square brackets. Sections are optional, but they are the recommended way of grouping your definitions into smaller, more manageable chunks.
You can also run Twine directly from source. However, it requires [rubyzip][rubyzip] in order to create and read standard zip files.
Each grouping section contains N definitions. These definitions start with the key placed within a single pair of square brackets. It then contains a number of key-value pairs, including a comment, a comma-separated list of tags and all of the translations.
$ gem install rubyzip
$ git clone git://github.com/mobiata/twine.git
$ cd twine
$ ./twine --help
Make sure you run the `twine` executable at the root of the project as it properly sets up your Ruby library path. The `bin/twine` executable does not.
## String File Format
Twine stores all of its strings in a single file. The format of this file is a slight variant of the [Git][git] config file format, which itself is based on the old [Windows INI file][INI] format. The entire file is broken up into sections, which are created by placing the section name between two pairs of square brackets. Sections are optional, but they are a recommended way of breaking your strings into smaller, more manageable chunks.
Each grouping section contains N string definitions. These string definitions start with the string key placed within a single pair of square brackets. This string definition then contains a number of key-value pairs, including a comment, a comma-separated list of tags (which are used by Twine to select a subset of strings), and all of the translations.
### Placeholders
Twine supports [`printf` style placeholders][printf] with one peculiarity: `@` is used for strings instead of `s`. This is because Twine started out as a tool for iOS and OS X projects.
Twine supports [`printf` style placeholders](https://en.wikipedia.org/wiki/Printf_format_string) with one peculiarity: `@` is used for strings instead of `s`. This is because Twine started out as a tool for iOS and OS X projects.
### Tags
Tags are used by Twine as a way to only work with a subset of your definitions at any given point in time. Each definition can be assigned zero or more tags which are separated by commas. Tags are optional, though highly recommended. You can get a list of all definitions currently missing tags by executing the [`validate-twine-file`](#validate-twine-file) command with the `--pedantic` option.
When generating a localization file, you can specify which definitions should be included using the `--tags` option. Provide a comma separated list of tags to match all definitions that contain any of the tags (`--tags tag1,tag2` matches all definitions tagged with `tag1` _or_ `tag2`). Provide multiple `--tags` options to match definitions containing all specified tags (`--tags tag1 --tags tag2` matches all definitions tagged with `tag1` _and_ `tag2`). You can match definitions _not_ containing a tag by prefixing the tag with a tilde (`--tags ~tag1` matches all definitions _not_ tagged with `tag1`). All three options are combinable.
Tags are used by Twine as a way to only work with a subset of your strings at any given point in time. Each string can be assigned zero or more tags which are separated by commas. Tags are optional, though highly recommended. You can get a list of all strings currently missing tags by executing the `validate-strings-file` command.
### Whitespace
@ -30,125 +41,116 @@ Whitepace in this file is mostly ignored. If you absolutely need to put spaces a
### References
If you want a definition to inherit the values of another definition, you can use a reference. Any property not specified for a definition will be taken from the reference.
If you want a key to inherit the values of another key, you can use a reference. Any property not specified for a key will be taken from the reference.
### Example
```ini
[[General]]
[yes]
en = Yes
es = Sí
fr = Oui
ja = はい
[no]
en = No
fr = Non
ja = いいえ
[[General]]
[yes]
en = Yes
es = Sí
fr = Oui
ja = はい
[no]
en = No
fr = Non
ja = いいえ
[[Errors]]
[path_not_found_error]
en = The file '%@' could not be found.
tags = app1,app6
comment = An error describing when a path on the filesystem could not be found.
[network_unavailable_error]
en = The network is currently unavailable.
tags = app1
comment = An error describing when the device can not connect to the internet.
[dismiss_error]
ref = yes
en = Dismiss
[[Errors]]
[path_not_found_error]
en = The file '%@' could not be found.
tags = app1,app6
comment = An error describing when a path on the filesystem could not be found.
[network_unavailable_error]
en = The network is currently unavailable.
tags = app1
comment = An error describing when the device can not connect to the internet.
[dismiss_error]
ref = yes
en = Dismiss
[[Escaping Example]]
[list_item_separator]
en = `, `
tags = mytag
comment = A string that should be placed between multiple items in a list. For example: Red, Green, Blue
[grave_accent_quoted_string]
en = ``%@``
tags = myothertag
comment = This string will evaluate to `%@`.
```
[[Escaping Example]]
[list_item_separator]
en = `, `
tags = mytag
comment = A string that should be placed between multiple items in a list. For example: Red, Green, Blue
[grave_accent_quoted_string]
en = ``%@``
tags = myothertag
comment = This string will evaluate to `%@`.
## Supported Output Formats
Twine currently supports the following output formats:
Twine currently supports the following formats for outputting strings:
* [iOS and OS X String Resources][applestrings] (format: apple)
* [Android String Resources][androidstrings] (format: android)
* HTML tags will be escaped by replacing `<` with `&lt`
* Tags inside `<![CDATA[` won't be escaped.
* Supports [basic styling][androidstyling] according to [Android documentation](https://developer.android.com/guide/topics/resources/string-resource.html#StylingWithHTML). All of the documented tags are supported, in addition to `<a>` links.
* These tags will *not* be escaped if the string doesn't contain placeholders. You can reference them directly in your layouts or by using [`getText()`](https://developer.android.com/reference/android/content/res/Resources.html#getText(int)) to read them programmatically.
* These tags *will* be escaped if the string contains placeholders. You can use [`getString()`](https://developer.android.com/reference/android/content/res/Resources.html#getString(int,%20java.lang.Object...)) combined with [`fromHtml`](https://developer.android.com/reference/android/text/Html.html#fromHtml(java.lang.String)) as shown in the [documentation][androidstyling] to display them.
* See [\#212](https://github.com/scelis/twine/issues/212) for details.
* [Gettext PO Files][gettextpo] (format: gettext)
* [jquery-localize Language Files][jquerylocalize] (format: jquery)
* [Django PO Files][djangopo] (format: django)
* [Tizen String Resources][tizen] (format: tizen)
* [Flash/Flex Properties][flash] (format: flash)
If you would like to enable Twine to create localization files in another format, read the wiki page on how to create an appropriate formatter.
If you would like to enable twine to create language files in another format, create an appropriate formatter in `lib/twine/formatters`.
## Usage
Usage: twine COMMAND TWINE_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]
Usage: twine COMMAND STRINGS_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]
### Commands
#### `generate-localization-file`
#### `generate-string-file`
This command creates a localization file from the Twine data file. If the output file would not contain any translations, Twine will exit with an error.
This command creates an Apple or Android strings file from the master strings data file.
$ twine generate-localization-file /path/to/twine.txt values-ja.xml --tags common,app1
$ twine generate-localization-file /path/to/twine.txt Localizable.strings --lang ja --tags mytag
$ twine generate-localization-file /path/to/twine.txt all-english.strings --lang en
$ twine generate-string-file /path/to/strings.txt values-ja.xml --tags common,app1
$ twine generate-string-file /path/to/strings.txt Localizable.strings --lang ja --tags mytag
$ twine generate-string-file /path/to/strings.txt all-english.strings --lang en
#### `generate-all-localization-files`
#### `generate-all-string-files`
This command is a convenient way to call [`generate-localization-file`](#generate-localization-file) multiple times. It uses standard conventions to figure out exactly which files to create given a parent directory. For example, if you point it to a parent directory containing `en.lproj`, `fr.lproj`, and `ja.lproj` subdirectories, Twine will create a `Localizable.strings` file of the appropriate language in each of them. However, files that would not contain any translations will not be created; instead warnings will be logged to `stderr`. This is often the command you will want to execute during the build phase of your project.
This command is a convenient way to call `generate-string-file` multiple times. It uses standard Mac OS X, iOS, and Android conventions to figure out exactly which files to create given a parent directory. For example, if you point it to a parent directory containing `en.lproj`, `fr.lproj`, and `ja.lproj` subdirectories, Twine will create a `Localizable.strings` file of the appropriate language in each of them. This is often the command you will want to execute during the build phase of your project.
$ twine generate-all-localization-files /path/to/twine.txt /path/to/project/locales/directory --tags common,app1
$ twine generate-all-string-files /path/to/strings.txt /path/to/project/locales/directory --tags common,app1
#### `consume-localization-file`
#### `consume-string-file`
This command slurps all of the translations from a localization file and incorporates the translated strings into the Twine data file. This is a simple way to incorporate any changes made to a single file by one of your translators. It will only identify definitions that already exist in the data file.
This command slurps all of the strings from a `.strings` or `.xml` file and incorporates the translated text into the master strings data file. This is a simple way to incorporate any changes made to a single file by one of your translators. It will only identify strings that already exist in the master data file.
$ twine consume-localization-file /path/to/twine.txt fr.strings
$ twine consume-localization-file /path/to/twine.txt Localizable.strings --lang ja
$ twine consume-localization-file /path/to/twine.txt es.xml
$ twine consume-string-file /path/to/strings.txt fr.strings
$ twine consume-string-file /path/to/strings.txt Localizable.strings --lang ja
$ twine consume-string-file /path/to/strings.txt es.xml
#### `consume-all-localization-files`
#### `consume-all-string-files`
This command reads in a folder containing many localization files. These files should be in a standard folder hierarchy so that Twine knows the language of each file. When combined with the `--developer-language`, `--consume-comments`, and `--consume-all` flags, this command is a great way to create your initial Twine data file from an existing project. Just make sure that you create a blank Twine data file first!
This command reads in a folder containing many `.strings` or `.xml` files. These files should be in a standard folder hierarchy so that twine knows the language of each file. When combined with the `--developer-language`, `--consume-comments`, and `--consume-all` flags, this command is a great way to create your initial strings data file from an existing iOS or Android project. Just make sure that you create a blank strings.txt file, first!
$ twine consume-all-localization-files twine.txt Resources/Locales --developer-language en --consume-all --consume-comments
$ twine consume-all-string-files strings.txt Resources/Locales --developer-language en --consume-all --consume-comments
#### `generate-localization-archive`
#### `generate-loc-drop`
This command is a convenient way to generate a zip file containing files created by the [`generate-localization-file`](#generate-localization-file) command. If a file would not contain any translated strings, it is skipped and a warning is logged to `stderr`. This command can be used to create a single zip containing a large number of translations in all languages which you can then hand off to your translation team.
This command is a convenient way to generate a zip file containing files created by the `generate-string-file` command. It is often used for creating a single zip containing a large number of strings in all languages which you can then hand off to your translation team.
$ twine generate-localization-archive /path/to/twine.txt LocDrop1.zip
$ twine generate-localization-archive /path/to/twine.txt LocDrop2.zip --lang en,fr,ja,ko --tags common,app1
$ twine generate-loc-drop /path/to/strings.txt LocDrop1.zip
$ twine generate-loc-drop /path/to/strings.txt LocDrop2.zip --lang en,fr,ja,ko --tags common,app1
#### `consume-localization-archive`
#### `consume-loc-drop`
This command is a convenient way of taking a zip file and executing the [`consume-localization-file`](#consume-localization-file) command on each file within the archive. It is most often used to incorporate all of the changes made by the translation team after they have completed work on a localization archive.
This command is a convenient way of taking a zip file and executing the `consume-string-file` command on each file within the archive. It is most often used to incorporate all of the changes made by the translation team after they have completed work on a localization drop.
$ twine consume-localization-archive /path/to/twine.txt LocDrop2.zip
$ twine consume-loc-drop /path/to/strings.txt LocDrop2.zip
#### `validate-twine-file`
#### `validate-strings-file`
This command validates that the Twine data file can be parsed, contains no duplicate keys, and that no key contains invalid characters. It will exit with a non-zero status code if any of those criteria are not met.
This command validates that the strings file can be parsed, contains no duplicate keys, and that all strings have at least one tag. It will exit with a non-zero status code if any of those criteria are not met.
$ twine validate-twine-file /path/to/twine.txt
$ twine validate-strings-file /path/to/strings.txt
## Creating Your First Twine Data File
## Creating Your First strings.txt File
The easiest way to create your first Twine data file is to run the [`consume-all-localization-files`](#consume-all-localization-files) command. The one caveat is to first create a blank file to use as your starting point. Then, just point the `consume-all-localization-files` command at a directory in your project containing all of your localization files.
The easiest way to create your first strings.txt file is to run the `consume-all-string-files` command. The one caveat is to first create a blank strings.txt file to use as your starting point. Then, just point the `consume-all-string-files` command at a directory in your project containing all of your iOS, OS X, or Android strings files.
$ touch twine.txt
$ twine consume-all-localization-files twine.txt Resources/Locales --developer-language en --consume-all --consume-comments --format apple/android/gettext/jquery/django/tizen/flash
$ touch strings.txt
$ twine consume-all-string-files strings.txt Resources/Locales --developer-language en --consume-all --consume-comments
## Twine and Your Build Process
@ -157,12 +159,12 @@ The easiest way to create your first Twine data file is to run the [`consume-all
It is easy to incorporate Twine right into your iOS and OS X app build processes.
1. In your project folder, create all of the `.lproj` directories that you need. It does not really matter where they are. We tend to put them in `Resources/Locales/`.
2. Run the [`generate-all-localization-files`](#generate-all-localization-files) command to create all of the `.strings` files you need in these directories. For example,
2. Run the `generate-all-string-files` command to create all of the string files you need in these directories. For example,
$ twine generate-all-localization-files twine.txt Resources/Locales/ --tags tag1,tag2
$ twine generate-all-string-files strings.txt Resources/Locales/ --tags tag1,tag2
Make sure you point Twine at your data file, the directory that contains all of your `.lproj` directories, and the tags that describe the definitions you want to use for this project.
3. Drag the `Resources/Locales/` directory to the Xcode project navigator so that Xcode knows to include all of these `.strings` files in your build.
Make sure you point Twine at your strings data file, the directory that contains all of your `.lproj` directories, and the tags that describe the strings you want to use for this project.
3. Drag the `Resources/Locales/` directory to the Xcode project navigator so that Xcode knows to include all of these strings files in your build.
4. In Xcode, navigate to the "Build Phases" tab of your target.
5. Click on the "Add Build Phase" button and select "Add Run Script".
6. Drag the new "Run Script" build phase up so that it runs earlier in the build process. It doesn't really matter where, as long as it happens before the resources are copied to your bundle.
@ -172,66 +174,46 @@ Now, whenever you build your application, Xcode will automatically invoke Twine
### Android Studio/Gradle
#### Standard
Add the following code to `app/build.gradle`:
Add the following task at the top level in app/build.gradle:
```
task generateLocalizations {
String script = 'if hash twine 2>/dev/null; then twine generate-localization-file twine.txt ./src/main/res/values/generated_strings.xml; fi'
exec {
executable "sh"
args '-c', script
}
}
preBuild {
dependsOn generateLocalizations
task generateStrings {
String script = 'if hash twine 2>/dev/null; then twine generate-string-file strings.txt ./src/main/res/values/generated_strings.xml; fi'
exec {
executable "sh"
args '-c', script
}
}
```
#### Using [jruby](http://jruby.org)
Now every time you build your app the strings are generated from the twine file.
With this approach, developers do not need to manually install ruby, gem, or twine.
Add the following code to `app/build.gradle`:
```
buildscript {
repositories { jcenter() }
dependencies {
/* NOTE: Set your preferred version of jruby here. */
classpath "com.github.jruby-gradle:jruby-gradle-plugin:1.5.0"
}
}
apply plugin: 'com.github.jruby-gradle.base'
dependencies {
/* NOTE: Set your preferred version of twine here. */
jrubyExec 'rubygems:twine:1.1'
}
task generateLocalizations (type: JRubyExec) {
dependsOn jrubyPrepare
jrubyArgs '-S'
script "twine"
scriptArgs 'generate-localization-file', 'twine.txt', './src/main/res/values/generated_strings.xml'
}
preBuild {
dependsOn generateLocalizations
}
```
## User Interface
* [Twine TextMate 2 Bundle](https://github.com/mobiata/twine.tmbundle) — This [TextMate 2](https://github.com/textmate/textmate) bundle will make it easier for you to work with Twine files. In particular, it lets you use code folding to easily collapse and expand both definitions and sections.
* [Twine TextMate 2 Bundle](https://github.com/mobiata/twine.tmbundle) — This [TextMate 2](https://github.com/textmate/textmate) bundle will make it easier for you to work with Twine strings files. In particular, it lets you use code folding to easily collapse and expand both strings and sections.
* [twine_ui](https://github.com/Daij-Djan/twine_ui) — A user interface for Twine written by [Dominik Pich](https://github.com/Daij-Djan/). Consider using this if you would prefer to use Twine without dropping to a command line.
## Extending Twine
## Plugin Support
If there's a format Twine does not yet support and you're keen to change that, check out the [documentation](documentation/formatters.md).
Twine supports a basic plugin infrastructure, allowing third-party code to provide support for additional formatters. Twine will read a yaml config file specifying which plugins to load from three locations.
0. `./twine.yml` The current working directory
0. `~/.twine` The home directory
0. `/etc/twine.yml` The etc directory
Plugins are specified as values for the `gems` key. The following is an example config:
```
gems: appium_twine
```
Multiple gems can also be specfied in the yaml file.
```
gems: [appium_twine, some_other_plugin]
```
[appium_twine](https://github.com/appium/appium_twine) is a sample plugin used to provide a C# formatter.
## Contributors
@ -245,7 +227,7 @@ Many thanks to all of the contributors to the Twine project, including:
* [Kevin Wood](https://github.com/kwood)
* [Mohammad Hejazi](https://github.com/MohammadHejazi)
* [Robert Guo](http://www.robertguo.me/)
* [Sebastian Ludwig](https://github.com/sebastianludwig)
* [sebastianludwig](https://github.com/sebastianludwig)
* [Sergey Pisarchik](https://github.com/SergeyPisarchik)
* [Shai Shamir](https://github.com/pichirichi)
@ -253,12 +235,9 @@ Many thanks to all of the contributors to the Twine project, including:
[rubyzip]: http://rubygems.org/gems/rubyzip
[git]: http://git-scm.org/
[INI]: http://en.wikipedia.org/wiki/INI_file
[applestrings]: https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/LoadingResources/Strings/Strings.html
[applestrings]: http://developer.apple.com/documentation/Cocoa/Conceptual/LoadingResources/Strings/Strings.html
[androidstrings]: http://developer.android.com/guide/topics/resources/string-resource.html
[androidstyling]: http://developer.android.com/guide/topics/resources/string-resource.html#FormattingAndStyling
[gettextpo]: http://www.gnu.org/savannah-checkouts/gnu/gettext/manual/html_node/PO-Files.html
[jquerylocalize]: https://github.com/coderifous/jquery-localize
[djangopo]: https://docs.djangoproject.com/en/dev/topics/i18n/translation/
[tizen]: https://developer.tizen.org/documentation/articles/localization
[flash]: http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/resources/IResourceManager.html#getString()
[printf]: https://en.wikipedia.org/wiki/Printf_format_string

File diff suppressed because it is too large Load diff

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 68 KiB

View file

@ -1,89 +0,0 @@
# How Formatters Work
It's a formatters job to transform a Twine file into a localization file of a specific format. That task is solved using a hierarchical approach. This document describes the process in detail using the following Twine file as an example.
```
[[General]]
[yes]
en = Yes
[no]
en = No
[[Messages]]
[success]
en = All good
comment = Everything worked
```
A Twine file consists of multiple _sections_, each _section_ contains _keys_ with translated _values_, _comments_ and so on. If we highlight the components of the example file, it looks like this:
![Highlighted Twine file components](assets/formatter_1.png)
A formatter takes each of these components (and a few more) and formats them, working it's way from the outside inwards. First the method `format_file` is called, which doesn't do much more than calling `format_sections` which in turn calls `format_section` for each section and so on. To get an overview which method calls which, look at the next picture, where each method is represented by a block that is called by its surrounding block - read it from top to bottom and from the outside in.
![formatting method calling structure](assets/formatter_2.png)
If we map the _input_ for each method to the example file, it looks like this
![highlighted formatter method input](assets/formatter_3.png)
As stated at the beginning, the output produced by a formatter depends on the formatter. The output of the Apple formatter would for example be
```
/********** General **********/
"yes" = "Yes";
"no" = "No";
/********** Messages **********/
/* Everything worked */
"success" = "All good";
```
Or, highlighted by the method that produces each piece of the output
![highlighted formatter method output](assets/formatter_4.png)
The process described above is implemented by the structure giving formatter base class [`Abstract`](/lib/twine/formatters/abstract.rb). To generate a desired output format, a formatter overwrites just enough methods to gain as much control as it needs - it's basically the [Template Method Pattern](https://en.wikipedia.org/wiki/Template_method_pattern) applied again and again.
# Write a Formatter
Formatters inherit from [`Abstract`](/lib/twine/formatters/abstract.rb) and need to specify some information about the format they are supporting like the format name, the default file name, if they can handle a directory of localization files and so on - just take a look at the class and the [other formatters](/lib/twine/formatters) to get an idea.
The `Abstract` formatter also specifies two utility methods to be used when read a localization file, `set_translation_for_key` and `set_comment_for_key`, however the actual parsing is too formatter specific and must be implemented in the `read` method of a formatter.
Which methods to overwrite to produce the desired output depends pretty much on the format. Again, looking at the [bundled formatters](/lib/twine/formatters) will provide some insight.
Finally, to make a formatter available, it needs to be added to the list of formatters
```
Twine::Formatters.formatters << MyWickedFormatter.new
```
# Plugins
Once a formatter has been developed, it can be distributed as a gem and loaded as a plugin. Twine will read a yaml config file specifying which plugins to load from three locations.
0. `./twine.yml` The current working directory
0. `~/.twine` The home directory
0. `/etc/twine.yml` The etc directory
Plugins are specified as values for the `gems` key. The following is an example config:
```
gems: wicked_twine
```
Multiple gems can also be specified in the yaml file.
```
gems: [wicked_twine, some_other_plugin]
```
## Sample Plugins
* [appium-twine](https://github.com/appium/appium_twine)
* [twine-flutter](https://github.com/tiknil/twine-flutter)

View file

@ -21,22 +21,13 @@ module Twine
class Error < StandardError
end
require 'twine/version'
require 'twine/plugin'
require 'twine/twine_file'
require 'twine/cli'
require 'twine/stringsfile'
require 'twine/encoding'
require 'twine/output_processor'
require 'twine/placeholders'
require 'twine/formatters'
require 'twine/formatters/abstract'
require 'twine/formatters/android'
require 'twine/formatters/apple'
require 'twine/formatters/apple_plural'
require 'twine/formatters/django'
require 'twine/formatters/flash'
require 'twine/formatters/gettext'
require 'twine/formatters/jquery'
require 'twine/formatters/tizen'
require 'twine/runner'
require 'twine/cli'
require 'twine/version'
end

View file

@ -1,433 +1,169 @@
require 'optparse'
require 'io/console'
module Twine
module CLI
ALL_FORMATS = Formatters.formatters.map(&:format_name).map(&:downcase)
OPTIONS = {
consume_all: {
switch: ['-a', '--[no-]consume-all'],
description: 'Normally Twine will ignore any translation keys that do not exist in your Twine file.',
boolean: true
},
consume_comments: {
switch: ['-c', '--[no-]consume-comments'],
description: <<-DESC,
Normally Twine will ignore all comments in the file. With this flag set, any
comments encountered will be read and parsed into the Twine data file. This is especially useful
when creating your first Twine data file from an existing project.
DESC
boolean: true
},
create_folders: {
switch: ['-r', '--[no-]create-folders'],
description: <<-DESC,
This flag may be used to create output folders for all languages, if they don't exist yet.
As a result all languages will be exported, not only the ones where an output folder already exists.
DESC
boolean: true
},
developer_language: {
switch: ['-d', '--developer-language LANG'],
description: <<-DESC,
When writing the Twine data file, set the specified language as the "developer language". In
practice, this just means that this language will appear first in the Twine data file. When
generating files this language will be used as default language and its translations will be
used if a definition is not localized for the output language.
DESC
},
encoding: {
switch: ['-e', '--encoding ENCODING'],
description: <<-DESC,
Twine defaults to encoding all output files in UTF-8. This flag will tell Twine to use an alternate
encoding for these files. For example, you could use this to write Apple .strings files in UTF-16.
When reading files, Twine does its best to determine the encoding automatically. However, if the
files are UTF-16 without BOM, you need to specify if it's UTF-16LE or UTF16-BE.
DESC
},
escape_all_tags: {
switch: ['--[no-]escape-all-tags'],
description: <<-DESC,
Always escape all HTML tags. By default the Android formatter will ONLY escape styling tags, if a
string also contains placeholders. This flag enforces that styling tags are escaped regardless of
placeholders.
DESC
},
file_name: {
switch: ['-n', '--file-name FILE_NAME'],
description: 'This flag may be used to overwrite the default file name of the format.'
},
format: {
switch: ['-f', '--format FORMAT', ALL_FORMATS],
description: <<-DESC,
The file format to read or write: (#{ALL_FORMATS.join(', ')}). Additional formatters can be placed in the formats/ directory.
DESC
},
:include => {
switch: ['-i', '--include SET', [:all, :translated, :untranslated]],
description: <<-DESC,
This flag will determine which definitions are included. It's possible values are:
all: All definitions both translated and untranslated for the specified language are included.
This is the default value.
translated: Only definitions with translation for the specified language are included.
untranslated: Only definitions without translation for the specified language are included.
DESC
default: :all
},
languages: {
switch: ['-l', '--lang LANGUAGES', Array],
description: 'Comma separated list of language codes to use for the specified action.'
},
output_path: {
switch: ['-o', '--output-file OUTPUT_FILE'],
description: 'Write a new Twine file at this location instead of replacing the original file.'
},
pedantic: {
switch: ['-p', '--[no-]pedantic'],
description: 'When validating a Twine file, perform additional checks that go beyond pure validity (like presence of tags).'
},
quiet: {
switch: ['-q', '--[no-]quiet'],
description: 'Suppress all console output except error messages.'
},
tags: {
switch: ['-t', '--tags TAG1,TAG2,TAG3', Array],
description: <<-DESC,
Only definitions with ANY of the specified tags will be processed. Specify this option multiple
times to only include definitions with ALL of the specified tags. Prefix a tag with ~ to include
definitions NOT containing that tag. Omit this option to match all definitions in the Twine data file.
DESC
repeated: true
},
untagged: {
switch: ['-u', '--[no-]untagged'],
description: <<-DESC,
If you have specified tags using the --tags flag, then only those tags will be selected. If you also
want to select all definitions that are untagged, then you can specify this option to do so.
DESC
},
validate: {
switch: ['--[no-]validate'],
description: 'Validate the Twine file before formatting it.'
}
}
COMMANDS = {
'generate-localization-file' => {
description: 'Generates a localization file in a certain LANGUAGE given a particular FORMAT. This script will attempt to guess both the language and the format given the filename and extension. For example, "ko.xml" will generate a Korean language file for Android.',
arguments: [:twine_file, :output_path],
optional_options: [
:developer_language,
:encoding,
:escape_all_tags,
:format,
:include,
:languages,
:quiet,
:tags,
:untagged,
:validate
],
option_validation: Proc.new { |options|
if options[:languages] and options[:languages].length > 1
raise Twine::Error.new 'specify only a single language for the `generate-localization-file` command.'
end
},
example: 'twine generate-localization-file twine.txt ko.xml --tags FT'
},
'generate-all-localization-files' => {
description: 'Generates all the localization files necessary for a given project. The parent directory to all of the locale-specific directories in your project should be specified as the INPUT_OR_OUTPUT_PATH. This command will most often be executed by your build script so that each build always contains the most recent translations.',
arguments: [:twine_file, :output_path],
optional_options: [
:create_folders,
:developer_language,
:encoding,
:escape_all_tags,
:file_name,
:format,
:include,
:quiet,
:tags,
:untagged,
:validate
],
example: 'twine generate-all-localization-files twine.txt Resources/Locales/ --tags FT,FB'
},
'generate-localization-archive' => {
description: 'Generates a zip archive of localization files in a given format. The purpose of this command is to create a very simple archive that can be handed off to a translation team. The translation team can unzip the archive, translate all of the strings in the archived files, zip everything back up, and then hand that final archive back to be consumed by the consume-localization-archive command.',
arguments: [:twine_file, :output_path],
required_options: [
:format
],
optional_options: [
:developer_language,
:encoding,
:escape_all_tags,
:include,
:quiet,
:tags,
:untagged,
:validate
],
example: 'twine generate-localization-archive twine.txt LocDrop5.zip --tags FT,FB --format android --lang de,en,en-GB,ja,ko'
},
'consume-localization-file' => {
description: 'Slurps all of the translations from a localization file into the specified TWINE_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.',
arguments: [:twine_file, :input_path],
optional_options: [
:consume_all,
:consume_comments,
:developer_language,
:encoding,
:format,
:languages,
:output_path,
:quiet,
:tags
],
option_validation: Proc.new { |options|
if options[:languages] and options[:languages].length > 1
raise Twine::Error.new 'specify only a single language for the `consume-localization-file` command.'
end
},
example: 'twine consume-localization-file twine.txt ja.strings'
},
'consume-all-localization-files' => {
description: 'Slurps all of the translations from a directory into the specified TWINE_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.',
arguments: [:twine_file, :input_path],
optional_options: [
:consume_all,
:consume_comments,
:developer_language,
:encoding,
:format,
:output_path,
:quiet,
:tags
],
example: 'twine consume-all-localization-files twine.txt Resources/Locales/ --developer-language en --tags DefaultTag1,DefaultTag2'
},
'consume-localization-archive' => {
description: 'Consumes an archive of translated files. This archive should be in the same format as the one created by the generate-localization-archive command.',
arguments: [:twine_file, :input_path],
optional_options: [
:consume_all,
:consume_comments,
:developer_language,
:encoding,
:format,
:output_path,
:quiet,
:tags
],
example: 'twine consume-localization-archive twine.txt LocDrop5.zip'
},
'validate-twine-file' => {
description: 'Validates that the given Twine file is parseable, contains no duplicates, and that no key contains invalid characters. Exits with a non-zero exit code if those criteria are not met.',
arguments: [:twine_file],
optional_options: [
:developer_language,
:pedantic,
:quiet
],
example: 'twine validate-twine-file twine.txt'
}
}
DEPRECATED_COMMAND_MAPPINGS = {
'generate-loc-drop' => 'generate-localization-archive', # added on 17.01.2017 - version 0.10
'consume-loc-drop' => 'consume-localization-archive' # added on 17.01.2017 - version 0.10
NEEDED_COMMAND_ARGUMENTS = {
'generate-string-file' => 3,
'generate-all-string-files' => 3,
'consume-string-file' => 3,
'consume-all-string-files' => 3,
'generate-loc-drop' => 3,
'consume-loc-drop' => 3,
'validate-strings-file' => 2
}
def self.parse(args)
command = args.select { |a| a[0] != '-' }[0]
args = args.reject { |a| a == command }
if args.any? { |a| a == '--version' }
Twine::stdout.puts "Twine version #{Twine::VERSION}"
return false
end
mapped_command = DEPRECATED_COMMAND_MAPPINGS[command]
if mapped_command
Twine::stdout.puts "WARNING: Twine commands names have changed. `#{command}` is now `#{mapped_command}`. The old command is deprecated and will soon stop working. For more information please check the documentation at https://github.com/mobiata/twine"
command = mapped_command
end
if command.nil?
print_help(args)
return false
elsif not COMMANDS.keys.include? command
raise Twine::Error.new "Invalid command: #{command}"
end
parse_command_options(command, args)
end
private
def self.print_help(args)
verbose = false
help_parser = OptionParser.new
help_parser.banner = 'Usage: twine [command] [options]'
help_parser.define('-h', '--help', 'Show this message.')
help_parser.define('--verbose', 'More detailed help.') { verbose = true }
help_parser.parse!(args)
Twine::stdout.puts help_parser.help
Twine::stdout.puts ''
Twine::stdout.puts 'Commands:'
COMMANDS.each do |name, properties|
if verbose
Twine::stdout.puts ''
Twine::stdout.puts ''
Twine::stdout.puts "# #{name}"
Twine::stdout.puts ''
Twine::stdout.puts properties[:description]
else
Twine::stdout.puts "- #{name}"
options = {}
parser = OptionParser.new do |opts|
opts.banner = 'Usage: twine COMMAND STRINGS_FILE [INPUT_OR_OUTPUT_PATH] [--lang LANG1,LANG2...] [--tags TAG1,TAG2,TAG3...] [--format FORMAT]'
opts.separator ''
opts.separator 'The purpose of this script is to convert back and forth between multiple data formats, allowing us to treat our strings (and translations) as data stored in a text file. We can then use the data file to create drops for the localization team, consume similar drops returned by the localization team, and create formatted string files to ship with your products. Twine currently supports iOS, OS X, Android, gettext, and jquery-localize string files.'
opts.separator ''
opts.separator 'Commands:'
opts.separator ''
opts.separator 'generate-string-file -- Generates a string file in a certain LANGUAGE given a particular FORMAT. This script will attempt to guess both the language and the format given the filename and extension. For example, "ko.xml" will generate a Korean language file for Android.'
opts.separator ''
opts.separator 'generate-all-string-files -- Generates all the string files necessary for a given project. The parent directory to all of the locale-specific directories in your project should be specified as the INPUT_OR_OUTPUT_PATH. This command will most often be executed by your build script so that each build always contains the most recent strings.'
opts.separator ''
opts.separator 'consume-string-file -- Slurps all of the strings from a translated strings file into the specified STRINGS_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.'
opts.separator ''
opts.separator 'consume-all-string-files -- Slurps all of the strings from a directory into the specified STRINGS_FILE. If you have some files returned to you by your translators you can use this command to incorporate all of their changes. This script will attempt to guess both the language and the format given the filename and extension. For example, "ja.strings" will assume that the file is a Japanese iOS strings file.'
opts.separator ''
opts.separator 'generate-loc-drop -- Generates a zip archive of strings files in any format. The purpose of this command is to create a very simple archive that can be handed off to a translation team. The translation team can unzip the archive, translate all of the strings in the archived files, zip everything back up, and then hand that final archive back to be consumed by the consume-loc-drop command.'
opts.separator ''
opts.separator 'consume-loc-drop -- Consumes an archive of translated files. This archive should be in the same format as the one created by the generate-loc-drop command.'
opts.separator ''
opts.separator 'validate-strings-file -- Validates that the given strings file is parseable, contains no duplicates, and that every string has a tag. Exits with a non-zero exit code if those criteria are not met.'
opts.separator ''
opts.separator 'General Options:'
opts.separator ''
opts.on('-l', '--lang LANGUAGES', Array, 'The language code(s) to use for the specified action.') do |langs|
options[:languages] = langs
end
end
Twine::stdout.puts ''
Twine::stdout.puts 'type `twine [command] --help` for further information about a command.'
end
# source: https://www.safaribooksonline.com/library/view/ruby-cookbook/0596523696/ch01s15.html
def self.word_wrap(s, width)
s.gsub(/(.{1,#{width}})(\s+|\Z)/, "\\1\n").rstrip
end
def self.indent(string, first_line, following_lines)
lines = string.split("\n")
indentation = ' ' * following_lines
lines.map! { |line| indentation + line }
result = lines.join("\n").strip
' ' * first_line + result
end
# ensure the description forms a neat block on the right
def self.prepare_description!(options, summary_width)
lines = options[:description].split "\n"
# remove leadinge HEREDOC spaces
space_match = lines[0].match(/^\s+/)
if space_match
leading_spaces = space_match[0].length
lines.map! { |l| l[leading_spaces..-1] }
end
merged_lines = []
lines.each do |line|
# if the line is a continuation of the previous one
if not merged_lines.empty? and (line[0] != ' ' or line[0, 4] == ' ')
merged_lines[-1] += ' ' + line.strip
else
merged_lines << line.rstrip
opts.on('-t', '--tags TAGS', Array, 'The tag(s) to use for the specified action. Only strings with that tag will be processed. Do not specify any tags to match all strings in the strings data file.') do |tags|
options[:tags] = tags
end
end
if IO.console
console_width = IO.console.winsize[1]
else
console_width = 100
end
summary_width += 7 # account for description padding
max_description_width = console_width - summary_width
merged_lines.map! do |line|
if line[0] == ' '
line = word_wrap(line.strip, max_description_width - 2)
line = indent(line, 2, 4)
else
line = word_wrap(line, max_description_width)
opts.on('-u', '--untagged', 'If you have specified tags using the --tags flag, then only those tags will be selected. If you also want to select all strings that are untagged, then you can specify this option to do so.') do |u|
options[:untagged] = true
end
line
end
options[:switch] << indent(merged_lines.join("\n"), 0, summary_width)
end
def self.parse_command_options(command_name, args)
command = COMMANDS[command_name]
result = {
command: command_name
}
parser = OptionParser.new
parser.banner = "Usage: twine #{command_name} #{command[:arguments].map { |c| "[#{c}]" }.join(' ')} [options]"
[:required_options, :optional_options].each do |option_type|
options = command[option_type]
if options and options.size > 0
parser.separator ''
parser.separator option_type.to_s.gsub('_', ' ').capitalize + ":"
options.each do |option_name|
option = OPTIONS[option_name]
result[option_name] = option[:default] if option[:default]
prepare_description!(option, parser.summary_width)
parser.define(*option[:switch]) do |value|
if option[:repeated]
result[option_name] = (result[option_name] || []) << value
else
result[option_name] = value
end
end
formats = Formatters.formatters.map(&:format_name)
opts.on('-f', '--format FORMAT', "The file format to read or write (#{formats.join(', ')}). Additional formatters can be placed in the formats/ directory.") do |format|
unless formats.include?(format.downcase)
raise Twine::Error.new "Invalid format: #{format}"
end
options[:format] = format.downcase
end
end
parser.define('-h', '--help', 'Show this message.') do
Twine::stdout.puts parser.help
return false
end
parser.separator ''
parser.separator 'Examples:'
parser.separator ''
parser.separator "> #{command[:example]}"
begin
parser.parse! args
rescue OptionParser::ParseError => e
raise Twine::Error.new e.message
end
arguments = args.reject { |a| a[0] == '-' }
number_of_missing_arguments = command[:arguments].size - arguments.size
if number_of_missing_arguments > 0
missing_arguments = command[:arguments][-number_of_missing_arguments, number_of_missing_arguments]
raise Twine::Error.new "#{number_of_missing_arguments} missing argument#{number_of_missing_arguments > 1 ? "s" : ""}: #{missing_arguments.join(', ')}. Check `twine #{command_name} -h`"
end
if args.length > command[:arguments].size
raise Twine::Error.new "Unknown argument: #{args[command[:arguments].size]}"
end
if command[:required_options]
command[:required_options].each do |option_name|
if result[option_name] == nil
raise Twine::Error.new "missing option: #{OPTIONS[option_name][:switch][0]}"
opts.on('-a', '--consume-all', 'Normally, when consuming a string file, Twine will ignore any string keys that do not exist in your master file.') do |a|
options[:consume_all] = true
end
opts.on('-i', '--include SET', "This flag will determine which strings are included when generating strings files. It's possible values:",
" all: All strings both translated and untranslated for the specified language are included. This is the default value.",
" translated: Only translated strings are included.",
" untranslated: Only untranslated strings are included.") do |set|
unless ['all', 'translated', 'untranslated'].include?(set.downcase)
raise Twine::Error.new "Invalid include flag: #{set}"
end
options[:include] = set.downcase
end
unless options[:include]
options[:include] = 'all'
end
opts.on('-o', '--output-file OUTPUT_FILE', 'Write the new strings database to this file instead of replacing the original file. This flag is only useful when running the consume-string-file or consume-loc-drop commands.') do |o|
options[:output_path] = o
end
opts.on('-n', '--file-name FILE_NAME', 'When running the generate-all-string-files command, this flag may be used to overwrite the default file name of the format.') do |n|
options[:file_name] = n
end
opts.on('-r', '--create-folders', "When running the generate-all-string-files command, this flag may be used to create output folders for all languages, if they don't exist yet. As a result all languages will be exported, not only the ones where an output folder already exists.") do |r|
options[:create_folders] = true
end
opts.on('-d', '--developer-language LANG', 'When writing the strings data file, set the specified language as the "developer language". In practice, this just means that this language will appear first in the strings data file. When generating files this language will be used as default language and its translations will be used if a key is not localized for the output language.') do |d|
options[:developer_language] = d
end
opts.on('-c', '--consume-comments', 'Normally, when consuming a string file, Twine will ignore all comments in the file. With this flag set, any comments encountered will be read and parsed into the strings data file. This is especially useful when creating your first strings data file from an existing project.') do |c|
options[:consume_comments] = true
end
opts.on('-e', '--encoding ENCODING', 'Twine defaults to encoding all output files in UTF-8. This flag will tell Twine to use an alternate encoding for these files. For example, you could use this to write Apple .strings files in UTF-16. This flag is currently only supported in Ruby 1.9.3 or greater.') do |e|
unless "".respond_to? :encode
raise Twine::Error.new "The --encoding flag is only supported on Ruby 1.9.3 or greater."
end
options[:output_encoding] = e
end
opts.on('--validate', 'Validate the strings file before formatting it') do
options[:validate] = true
end
opts.on('-p', '--pedantic', 'When validating a strings file, perform additional checks that go beyond pure validity (like presence of tags)') do
options[:pedantic] = true
end
opts.on('-h', '--help', 'Show this message.') do |h|
puts opts.help
exit
end
opts.on('--version', 'Print the version number and exit.') do |x|
puts "Twine version #{Twine::VERSION}"
exit
end
opts.separator ''
opts.separator 'Examples:'
opts.separator ''
opts.separator '> twine generate-string-file strings.txt ko.xml --tags FT'
opts.separator '> twine generate-all-string-files strings.txt Resources/Locales/ --tags FT,FB'
opts.separator '> twine consume-string-file strings.txt ja.strings'
opts.separator '> twine consume-all-string-files strings.txt Resources/Locales/ --developer-language en --tags DefaultTag1,DefaultTag2'
opts.separator '> twine generate-loc-drop strings.txt LocDrop5.zip --tags FT,FB --format android --lang de,en,en-GB,ja,ko'
opts.separator '> twine consume-loc-drop strings.txt LocDrop5.zip'
opts.separator '> twine validate-strings-file strings.txt'
end
parser.parse! args
if args.length == 0
puts parser.help
exit
end
command[:option_validation].call(result) if command[:option_validation]
number_of_needed_arguments = NEEDED_COMMAND_ARGUMENTS[args[0]]
unless number_of_needed_arguments
raise Twine::Error.new "Invalid command: #{args[0]}"
end
options[:command] = args[0]
command[:arguments].each do |argument_name|
result[argument_name] = args.shift
if args.length < 2
raise Twine::Error.new 'You must specify your strings file.'
end
options[:strings_file] = args[1]
if args.length < number_of_needed_arguments
raise Twine::Error.new 'Not enough arguments.'
elsif args.length > number_of_needed_arguments
raise Twine::Error.new "Unknown argument: #{args[number_of_needed_arguments]}"
end
result
case options[:command]
when 'generate-string-file'
options[:output_path] = args[2]
if options[:languages] and options[:languages].length > 1
raise Twine::Error.new 'Please only specify a single language for the generate-string-file command.'
end
when 'generate-all-string-files'
options[:output_path] = args[2]
when 'consume-string-file'
options[:input_path] = args[2]
if options[:languages] and options[:languages].length > 1
raise Twine::Error.new 'Please only specify a single language for the consume-string-file command.'
end
when 'consume-all-string-files'
options[:input_path] = args[2]
when 'generate-loc-drop'
options[:output_path] = args[2]
if !options[:format]
raise Twine::Error.new 'You must specify a format.'
end
when 'consume-loc-drop'
options[:input_path] = args[2]
when 'validate-strings-file'
end
return options
end
end
end

View file

@ -1,22 +1,20 @@
module Twine
module Encoding
def self.encoding_for_path path
File.open(path, 'rb') do |f|
begin
a = f.readbyte
b = f.readbyte
if (a == 0xfe && b == 0xff)
return 'UTF-16BE'
elsif (a == 0xff && b == 0xfe)
return 'UTF-16LE'
end
rescue EOFError
end
end
def self.bom(path)
first_bytes = IO.binread(path, 2)
return nil unless first_bytes
first_bytes = first_bytes.codepoints.map.to_a
return 'UTF-16BE' if first_bytes == [0xFE, 0xFF]
return 'UTF-16LE' if first_bytes == [0xFF, 0xFE]
rescue EOFError
return nil
end
def self.has_bom?(path)
!bom(path).nil?
end
def self.encoding_for_path(path)
bom(path) || 'UTF-8'
'UTF-8'
end
end
end

View file

@ -18,3 +18,7 @@ module Twine
end
end
end
Dir[File.join(File.dirname(__FILE__), 'formatters', '*.rb')].each do |file|
require file
end

View file

@ -3,14 +3,11 @@ require 'fileutils'
module Twine
module Formatters
class Abstract
SUPPORTS_PLURAL = false
LANGUAGE_CODE_WITH_OPTIONAL_REGION_CODE = "[a-z]{2}(?:-[A-Za-z]{2})?"
attr_accessor :twine_file
attr_accessor :strings
attr_accessor :options
def initialize
@twine_file = TwineFile.new
@strings = StringsFile.new
@options = {}
end
@ -23,7 +20,7 @@ module Twine
end
def can_handle_directory?(path)
Dir.entries(path).any? { |item| /^.+#{Regexp.escape(extension)}$/.match(item) }
raise NotImplementedError.new("You must implement can_handle_directory? in your formatter class.")
end
def default_file_name
@ -33,98 +30,88 @@ module Twine
def set_translation_for_key(key, lang, value)
value = value.gsub("\n", "\\n")
if @twine_file.definitions_by_key.include?(key)
definition = @twine_file.definitions_by_key[key]
reference = @twine_file.definitions_by_key[definition.reference_key] if definition.reference_key
if @strings.strings_map.include?(key)
row = @strings.strings_map[key]
reference = @strings.strings_map[row.reference_key] if row.reference_key
if !reference or value != reference.translations[lang]
definition.translations[lang] = value
row.translations[lang] = value
end
elsif @options[:consume_all]
Twine::stdout.puts "Adding new definition '#{key}' to twine file."
current_section = @twine_file.sections.find { |s| s.name == 'Uncategorized' }
Twine::stderr.puts "Adding new string '#{key}' to strings data file."
current_section = @strings.sections.find { |s| s.name == 'Uncategorized' }
unless current_section
current_section = TwineSection.new('Uncategorized')
@twine_file.sections.insert(0, current_section)
current_section = StringsSection.new('Uncategorized')
@strings.sections.insert(0, current_section)
end
current_definition = TwineDefinition.new(key)
current_section.definitions << current_definition
current_row = StringsRow.new(key)
current_section.rows << current_row
if @options[:tags] && @options[:tags].length > 0
current_definition.tags = @options[:tags]
current_row.tags = @options[:tags]
end
@twine_file.definitions_by_key[key] = current_definition
@twine_file.definitions_by_key[key].translations[lang] = value
@strings.strings_map[key] = current_row
@strings.strings_map[key].translations[lang] = value
else
Twine::stdout.puts "WARNING: '#{key}' not found in twine file."
Twine::stderr.puts "Warning: '#{key}' not found in strings data file."
end
if !@twine_file.language_codes.include?(lang)
@twine_file.add_language_code(lang)
if !@strings.language_codes.include?(lang)
@strings.add_language_code(lang)
end
end
def set_comment_for_key(key, comment)
return unless @options[:consume_comments]
if @twine_file.definitions_by_key.include?(key)
definition = @twine_file.definitions_by_key[key]
if @strings.strings_map.include?(key)
row = @strings.strings_map[key]
reference = @twine_file.definitions_by_key[definition.reference_key] if definition.reference_key
reference = @strings.strings_map[row.reference_key] if row.reference_key
if !reference or comment != reference.raw_comment
definition.comment = comment
row.comment = comment
end
end
end
def determine_language_given_path(path)
only_language_and_region = /^#{LANGUAGE_CODE_WITH_OPTIONAL_REGION_CODE}$/i
basename = File.basename(path, File.extname(path))
return basename if basename =~ only_language_and_region
return basename if @twine_file.language_codes.include? basename
path.split(File::SEPARATOR).reverse.find { |segment| segment =~ only_language_and_region }
raise NotImplementedError.new("You must implement determine_language_given_path in your formatter class.")
end
def output_path_for_language(lang)
lang
end
def read(io, lang)
raise NotImplementedError.new("You must implement read in your formatter class.")
def read_file(path, lang)
raise NotImplementedError.new("You must implement read_file in your formatter class.")
end
def format_file(lang)
output_processor = Processors::OutputProcessor.new(@twine_file, @options)
processed_twine_file = output_processor.process(lang)
return nil if processed_twine_file.definitions_by_key.empty?
def format_file(strings, lang)
header = format_header(lang)
result = ""
result += header + "\n" if header
result += format_sections(processed_twine_file, lang)
result += format_sections(strings, lang)
end
def format_header(lang)
end
def format_sections(twine_file, lang)
sections = twine_file.sections.map { |section| format_section(section, lang) }
def format_sections(strings, lang)
sections = strings.sections.map { |section| format_section(section, lang) }
sections.compact.join("\n")
end
def format_section_header(section)
end
def should_include_definition(definition, lang)
return !definition.translation_for_lang(lang).nil?
def should_include_row(row, lang)
row.translated_string_for_lang(lang)
end
def format_section(section, lang)
definitions = section.definitions.select { |definition| should_include_definition(definition, lang) }
return if definitions.empty?
rows = section.rows.select { |row| should_include_row(row, lang) }
return if rows.empty?
result = ""
@ -133,45 +120,28 @@ module Twine
result += "\n#{section_header}" if section_header
end
definitions.map! { |definition| format_definition(definition, lang) }
definitions.compact! # remove nil definitions
definitions.map! { |definition| "\n#{definition}" } # prepend newline
result += definitions.join
rows.map! { |row| format_row(row, lang) }
rows.compact! # remove nil entries
rows.map! { |row| "\n#{row}" } # prepend newline
result += rows.join
end
def format_definition(definition, lang)
formatted_definition = [format_comment(definition, lang)]
if self.class::SUPPORTS_PLURAL && definition.is_plural?
formatted_definition << format_plural(definition, lang)
else
formatted_definition << format_key_value(definition, lang)
end
formatted_definition.compact.join
def format_row(row, lang)
[format_comment(row, lang), format_key_value(row, lang)].compact.join
end
def format_comment(definition, lang)
def format_comment(row, lang)
end
def format_key_value(definition, lang)
value = definition.translation_for_lang(lang)
key_value_pattern % { key: format_key(definition.key.dup), value: format_value(value.dup) }
end
def format_plural(definition, lang)
plural_hash = definition.plural_translation_for_lang(lang)
if plural_hash
format_plural_keys(definition.key.dup, plural_hash)
end
def format_key_value(row, lang)
value = row.translated_string_for_lang(lang)
key_value_pattern % { key: format_key(row.key.dup), value: format_value(value.dup) }
end
def key_value_pattern
raise NotImplementedError.new("You must implement key_value_pattern in your formatter class.")
end
def format_plural_keys(key, plural_hash)
raise NotImplementedError.new("You must implement format_plural_keys in your formatter class.")
end
def format_key(key)
key
end
@ -183,6 +153,50 @@ module Twine
def escape_quotes(text)
text.gsub('"', '\\\\"')
end
def write_file(path, lang)
output_processor = Processors::OutputProcessor.new(@strings, @options)
processed_strings = output_processor.process(lang)
encoding = @options[:output_encoding] || 'UTF-8'
File.open(path, "w:#{encoding}") do |f|
f.puts format_file(processed_strings, lang)
end
end
def write_all_files(path)
file_name = @options[:file_name] || default_file_name
if @options[:create_folders]
@strings.language_codes.each do |lang|
output_path = File.join(path, output_path_for_language(lang))
FileUtils.mkdir_p(output_path)
file_path = File.join(output_path, file_name)
write_file(file_path, lang)
end
else
language_written = false
Dir.foreach(path) do |item|
next if item == "." or item == ".."
item = File.join(path, item)
next unless File.directory?(item)
lang = determine_language_given_path(item)
next unless lang
file_path = File.join(item, file_name)
write_file(file_path, lang)
language_written = true
end
if !language_written
raise Twine::Error.new("Failed to generate any files: No languages found at #{path}")
end
end
end
end
end
end

View file

@ -7,15 +7,14 @@ module Twine
class Android < Abstract
include Twine::Placeholders
SUPPORTS_PLURAL = true
LANG_CODES = Hash[
'zh' => 'zh-Hans',
'zh-CN' => 'zh-Hans',
'zh-HK' => 'zh-Hant',
# See https://developer.android.com/reference/java/util/Locale#legacy-language-codes
'iw' => 'he',
'zh-rCN' => 'zh-Hans',
'zh-rHK' => 'zh-Hant',
'en-rGB' => 'en-UK',
'in' => 'id',
'ji' => 'yi'
'nb' => 'no'
# TODO: spanish
]
def format_name
@ -31,35 +30,26 @@ module Twine
end
def default_file_name
'strings.xml'
return 'strings.xml'
end
def determine_language_given_path(path)
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
if segment == 'values'
return @twine_file.language_codes[0]
return @strings.language_codes[0]
else
# The language is defined by a two-letter ISO 639-1 language code, optionally followed by a two letter ISO 3166-1-alpha-2 region code (preceded by lowercase "r").
# see http://developer.android.com/guide/topics/resources/providing-resources.html#AlternativeResources
match = /^values-([a-z]{2}(-r[a-z]{2})?)$/i.match(segment)
match = /^values-(.*)$/.match(segment)
if match
lang = match[1].sub('-r', '-')
return LANG_CODES.fetch(lang, lang)
lang = match[1]
lang = LANG_CODES.fetch(lang, lang)
lang.sub!('-r', '-')
return lang
end
end
end
return super
end
def output_path_for_language(lang)
if lang == @twine_file.language_codes[0]
"values"
else
"values-#{lang}".gsub(/-(\p{Lu})/, '-r\1')
end
return
end
def set_translation_for_key(key, lang, value)
@ -72,127 +62,78 @@ module Twine
super(key, lang, value)
end
def read(io, lang)
document = REXML::Document.new io, :compress_whitespace => %w{ string }
document.context[:attribute_quote] = :quote
def read_file(path, lang)
resources_regex = /<resources(?:[^>]*)>(.*)<\/resources>/m
key_regex = /<string name="(\w+)">/
comment_regex = /<!-- (.*) -->/
value_regex = /<string name="\w+">(.*)<\/string>/
key = nil
value = nil
comment = nil
document.root.children.each do |child|
if child.is_a? REXML::Comment
content = child.string.strip
content.gsub!(/[\s]+/, ' ')
comment = content if content.length > 0 and not content.start_with?("SECTION:")
elsif child.is_a? REXML::Element
next unless child.name == 'string'
key = child.attributes['name']
File.open(path, 'r:UTF-8') do |f|
content_match = resources_regex.match(f.read)
if content_match
for line in content_match[1].split(/\r?\n/)
key_match = key_regex.match(line)
if key_match
key = key_match[1]
value_match = value_regex.match(line)
value = value_match ? value_match[1] : ""
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("SECTION:")
set_comment_for_key(key, comment)
end
comment = nil
end
content = child.children.map(&:to_s).join
set_translation_for_key(key, lang, content)
set_comment_for_key(key, comment) if comment
comment = nil
end
comment_match = comment_regex.match(line)
if comment_match
comment = comment_match[1]
end
end
end
end
end
def format_header(lang)
"<?xml version=\"1.0\" encoding=\"utf-8\"?>"
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Android Strings File -->\n<!-- Generated by Twine #{Twine::VERSION} -->\n<!-- Language: #{lang} -->"
end
def format_sections(twine_file, lang)
def format_sections(strings, lang)
result = '<resources>'
result += super + "\n"
result += "</resources>\n"
result += '</resources>'
end
def format_section_header(section)
"#{space(4)}<!-- SECTION: #{section.name} -->"
"\t<!-- SECTION: #{section.name} -->"
end
def format_comment(definition, lang)
"#{space(4)}<!-- #{definition.comment.gsub('--', '—')} -->\n" if definition.comment
def format_comment(row, lang)
"\t<!-- #{row.comment.gsub('--', '—')} -->\n" if row.comment
end
def key_value_pattern
"#{space(4)}<string name=\"%{key}\">%{value}</string>"
"\t<string name=\"%{key}\">%{value}</string>"
end
def format_plural_keys(key, plural_hash)
result = "#{space(4)}<plurals name=\"#{key}\">\n"
result += plural_hash.map{|quantity,value| "#{space(8)}<item quantity=\"#{quantity}\">#{escape_value(value)}</item>"}.join("\n")
result += "\n#{space(4)}</plurals>"
end
def space(level)
' ' * level
end
def gsub_unless(text, pattern, replacement)
text.gsub(pattern) do |match|
match_start_position = Regexp.last_match.offset(0)[0]
yield(text[0, match_start_position]) ? match : replacement
end
end
# http://developer.android.com/guide/topics/resources/string-resource.html#FormattingAndStyling
def escape_value(value)
inside_cdata = /<\!\[CDATA\[((?!\]\]>).)*$/ # opening CDATA tag ('<![CDATA[') not followed by a closing tag (']]>')
inside_opening_tag = /<(a|font|span|p)\s?((?!>).)*$/ # tag start ('<a ', '<font ', '<span ' or '<p ') not followed by a '>'
# escape double and single quotes and & signs
value = gsub_unless(value, '"', '\\"') { |substring| substring =~ inside_cdata || substring =~ inside_opening_tag }
value = gsub_unless(value, "'", "\\'") { |substring| substring =~ inside_cdata }
value = gsub_unless(value, /&/, '&amp;') { |substring| substring =~ inside_cdata || substring =~ inside_opening_tag }
# if `value` contains a placeholder, escape all angle brackets
# if not, escape opening angle brackes unless it's a supported styling tag
# https://github.com/scelis/twine/issues/212
# https://stackoverflow.com/questions/3235131/#18199543
if number_of_twine_placeholders(value) > 0 or @options[:escape_all_tags]
# matches all `<` but <![CDATA
angle_bracket = /<(?!(\/?(\!\[CDATA)))/
else
# matches all '<' but <b>, <em>, <i>, <cite>, <dfn>, <big>, <small>, <font>, <tt>, <s>,
# <strike>, <del>, <u>, <super>, <sub>, <ul>, <li>, <br>, <div>, <span>, <p>, <a>
# and <![CDATA
angle_bracket = /<(?!(\/?(b|em|i|cite|dfn|big|small|font|tt|s|strike|del|u|super|sub|ul|li|br|div|span|p|a|\!\[CDATA)))/
end
value = gsub_unless(value, angle_bracket, '&lt;') { |substring| substring =~ inside_cdata }
value = gsub_unless(value, '\n', "\n\\n") { |substring| substring =~ inside_cdata }
# escape non resource identifier @ signs (http://developer.android.com/guide/topics/resources/accessing-resources.html#ResourcesFromXml)
resource_identifier_regex = /@(?!([a-z\.]+:)?[a-z+]+\/[a-zA-Z_]+)/ # @[<package_name>:]<resource_type>/<resource_name>
value.gsub(resource_identifier_regex, '\@')
end
# see http://developer.android.com/guide/topics/resources/string-resource.html#FormattingAndStyling
# however unescaped HTML markup like in "Welcome to <b>Android</b>!" is stripped when retrieved with getString() (http://stackoverflow.com/questions/9891996/)
def format_value(value)
value = value.dup
# convert placeholders (e.g. %@ -> %s)
# Android enforces the following rules on the values
# 1) apostrophes and quotes must be escaped with a backslash
value = escape_quotes(value)
value.gsub!("'", "\\\\'")
# 2) HTML escape the string
value = CGI.escapeHTML(value)
# 3) convert placeholders (e.g. %@ -> %s)
value = convert_placeholders_from_twine_to_android(value)
# capture xliff tags and replace them with a placeholder
xliff_tags = []
value.gsub! /<xliff:g.+?<\/xliff:g>/ do
xliff_tags << $&
'TWINE_XLIFF_TAG_PLACEHOLDER'
end
# escape everything outside xliff tags
value = escape_value(value)
# put xliff tags back into place
xliff_tags.each do |xliff_tag|
# escape content of xliff tags
xliff_tag.gsub! /(<xliff:g.*?>)(.*)(<\/xliff:g>)/ do "#{$1}#{escape_value($2)}#{$3}" end
value.sub! 'TWINE_XLIFF_TAG_PLACEHOLDER', xliff_tag
end
# replace beginning and end spaces with \u0020. Otherwise Android strips them.
# 4) escape non resource identifier @ signs (http://developer.android.com/guide/topics/resources/accessing-resources.html#ResourcesFromXml)
resource_identifier_regex = /@(?!([a-z\.]+:)?[a-z+]+\/[a-zA-Z_]+)/ # @[<package_name>:]<resource_type>/<resource_name>
value.gsub!(resource_identifier_regex, '\@')
# 5) replace beginning and end spaces with \0020. Otherwise Android strips them.
value.gsub(/\A *| *\z/) { |spaces| '\u0020' * spaces.length }
end

View file

@ -1,8 +1,6 @@
module Twine
module Formatters
class Apple < Abstract
include Twine::Placeholders
def format_name
'apple'
end
@ -16,7 +14,7 @@ module Twine
end
def default_file_name
'Localizable.strings'
return 'Localizable.strings'
end
def determine_language_given_path(path)
@ -24,62 +22,87 @@ module Twine
path_arr.each do |segment|
match = /^(.+)\.lproj$/.match(segment)
if match
if match[1] == "Base"
return @options[:developer_language]
else
if match[1] != "Base"
return match[1]
end
end
end
return super
return
end
def output_path_for_language(lang)
"#{lang}.lproj"
end
def read(io, lang)
last_comment = nil
while line = io.gets
# matches a `key = "value"` line, where key may be quoted or unquoted. The former may also contain escaped characters
match = /^\s*((?:"(?:[^"\\]|\\.)+")|(?:[^"\s=]+))\s*=\s*"((?:[^"\\]|\\.)*)"/.match(line)
if match
key = match[1]
key = key[1..-2] if key[0] == '"' and key[-1] == '"'
key.gsub!('\\"', '"')
value = match[2]
value.gsub!('\\"', '"')
set_translation_for_key(key, lang, value)
if last_comment
set_comment_for_key(key, last_comment)
end
end
match = /\/\* (.*) \*\//.match(line)
if match
last_comment = match[1]
def read_file(path, lang)
encoding = Twine::Encoding.encoding_for_path(path)
sep = nil
if !encoding.respond_to?(:encode)
# This code is not necessary in 1.9.3 and does not work as it did in 1.8.7.
if encoding.end_with? 'LE'
sep = "\x0a\x00"
elsif encoding.end_with? 'BE'
sep = "\x00\x0a"
else
last_comment = nil
sep = "\n"
end
end
if encoding.index('UTF-16')
mode = "rb:#{encoding}"
else
mode = "r:#{encoding}"
end
File.open(path, mode) do |f|
last_comment = nil
while line = (sep) ? f.gets(sep) : f.gets
if encoding.index('UTF-16')
if line.respond_to? :encode!
line.encode!('UTF-8')
else
require 'iconv'
line = Iconv.iconv('UTF-8', encoding, line).join
end
end
match = /"((?:[^"\\]|\\.)+)"\s*=\s*"((?:[^"\\]|\\.)*)"/.match(line)
if match
key = match[1]
key.gsub!('\\"', '"')
value = match[2]
value.gsub!('\\"', '"')
set_translation_for_key(key, lang, value)
if last_comment
set_comment_for_key(key, last_comment)
end
end
match = /\/\* (.*) \*\//.match(line)
if match
last_comment = match[1]
else
last_comment = nil
end
end
end
end
def format_file(lang)
result = super
result += "\n"
def format_header(lang)
"/**\n * Apple Strings File\n * Generated by Twine #{Twine::VERSION}\n * Language: #{lang}\n */"
end
def format_section_header(section)
"\n/********** #{section.name} **********/\n"
"/********** #{section.name} **********/\n"
end
def key_value_pattern
"\"%{key}\" = \"%{value}\";"
"\"%{key}\" = \"%{value}\";\n"
end
def format_comment(definition, lang)
"\n/* #{definition.comment.gsub('*/', '* /')} */\n" if definition.comment
def format_comment(row, lang)
"/* #{row.comment.gsub('*/', '* /')} */\n" if row.comment
end
def format_key(key)
@ -87,14 +110,8 @@ module Twine
end
def format_value(value)
# Replace Android's %s with iOS %@
value = convert_placeholders_from_android_to_twine(value)
escape_quotes(value)
end
def should_include_definition(definition, lang)
return !definition.is_plural? && super
end
end
end
end

View file

@ -1,72 +0,0 @@
module Twine
module Formatters
class ApplePlural < Apple
include Twine::Placeholders
SUPPORTS_PLURAL = true
def format_name
'apple-plural'
end
def extension
'.stringsdict'
end
def default_file_name
'Localizable.stringsdict'
end
def format_footer(lang)
footer = "</dict>\n</plist>"
end
def format_file(lang)
result = super
result += format_footer(lang)
end
def format_header(lang)
header = "<\?xml version=\"1.0\" encoding=\"UTF-8\"\?>\n"
header += "<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
header += "<plist version=\"1.0\">\n<dict>"
end
def format_section_header(section)
"<!-- ********** #{section.name} **********/ -->\n"
end
def format_plural_keys(key, plural_hash)
result = "\t<key>#{key}</key>\n"
result += "\t<dict>\n"
result += "\t\t<key>NSStringLocalizedFormatKey</key>\n"
result += "\t\t<string>\%\#@value@</string>\n"
result += "\t\t<key>value</key>\n"
result += "\t\t<dict>\n"
result += "\t\t\t<key>NSStringFormatSpecTypeKey</key>\n"
result += "\t\t\t<string>NSStringPluralRuleType</string>\n"
result += "\t\t\t<key>NSStringFormatValueTypeKey</key>\n"
result += "\t\t\t<string>d</string>\n"
# Replace Android's %s with iOS %@
result += plural_hash.map{|quantity,value| "\t\t\t<key>#{quantity}</key>\n\t\t\t<string>#{convert_placeholders_from_android_to_twine(value)}</string>"}.join("\n")
result += "\n"
result += "\t\t</dict>\n"
result += "\t</dict>\n"
end
def format_comment(definition, lang)
"<!-- #{definition.comment.gsub('--', '—')} -->\n" if definition.comment
end
def read(io, lang)
raise NotImplementedError.new("Reading \".stringdict\" files not implemented yet")
end
def should_include_definition(definition, lang)
return definition.is_plural? && definition.plural_translation_for_lang(lang)
end
end
end
end
Twine::Formatters.formatters << Twine::Formatters::ApplePlural.new

View file

@ -1,6 +1,5 @@
module Twine
module Formatters
# For a description of the .po file format, see https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html
class Django < Abstract
def format_name
'django'
@ -10,64 +9,112 @@ module Twine
'.po'
end
def default_file_name
'strings.po'
def can_handle_directory?(path)
Dir.entries(path).any? { |item| /^.+\.po$/.match(item) }
end
def read(io, lang)
comment_regex = /^\s*#\. *"?(.*)"?$/
key_regex = /^msgid *"(.*)"$/
value_regex = /^msgstr *"(.*)"$/m
def default_file_name
return 'strings.po'
end
while line = io.gets
comment_match = comment_regex.match(line)
if comment_match
comment = comment_match[1]
def determine_language_given_path(path)
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
match = /(..)\.po$/.match(segment)
if match
return match[1]
end
end
return
end
key_match = key_regex.match(line)
if key_match
key = key_match[1].gsub('\\"', '"')
end
value_match = value_regex.match(line)
if value_match
value = value_match[1].gsub(/"\n"/, '').gsub('\\"', '"')
end
def read_file(path, lang)
comment_regex = /#\. *"?(.*)"?$/
key_regex = /msgid *"(.*)"$/
value_regex = /msgstr *"(.*)"$/m
if key and key.length > 0 and value and value.length > 0
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("--------- ")
set_comment_for_key(key, comment)
encoding = Twine::Encoding.encoding_for_path(path)
sep = nil
if !encoding.respond_to?(:encode)
# This code is not necessary in 1.9.3 and does not work as it did in 1.8.7.
if encoding.end_with? 'LE'
sep = "\x0a\x00"
elsif encoding.end_with? 'BE'
sep = "\x00\x0a"
else
sep = "\n"
end
end
if encoding.index('UTF-16')
mode = "rb:#{encoding}"
else
mode = "r:#{encoding}"
end
File.open(path, mode) do |f|
last_comment = nil
while line = (sep) ? f.gets(sep) : f.gets
if encoding.index('UTF-16')
if line.respond_to? :encode!
line.encode!('UTF-8')
else
require 'iconv'
line = Iconv.iconv('UTF-8', encoding, line).join
end
end
key = nil
value = nil
comment = nil
comment_match = comment_regex.match(line)
if comment_match
comment = comment_match[1]
end
key_match = key_regex.match(line)
if key_match
key = key_match[1].gsub('\\"', '"')
end
value_match = value_regex.match(line)
if value_match
value = value_match[1].gsub(/"\n"/, '').gsub('\\"', '"')
end
if key and key.length > 0 and value and value.length > 0
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("--------- ")
set_comment_for_key(key, comment)
end
key = nil
value = nil
comment = nil
end
end
end
end
def format_file(lang)
@default_lang = @twine_file.language_codes[0]
def format_file(strings, lang)
@default_lang = strings.language_codes[0]
result = super
@default_lang = nil
result
end
def format_header(lang)
# see https://www.gnu.org/software/trans-coord/manual/gnun/html_node/PO-Header.html for details
"# Django Strings File\n# Generated by Twine\n# Language: #{lang}\nmsgid \"\"\nmsgstr \"\"\n\"Content-Type: text/plain; charset=UTF-8\\n\""
"##\n # Django Strings File\n # Generated by Twine #{Twine::VERSION}\n # Language: #{lang}\n"
end
def format_section_header(section)
"# --------- #{section.name} --------- #\n"
"#--------- #{section.name} ---------#\n"
end
def format_definition(definition, lang)
[format_comment(definition, lang), format_base_translation(definition), format_key_value(definition, lang)].compact.join
def format_row(row, lang)
[format_comment(row, lang), format_base_translation(row), format_key_value(row, lang)].compact.join
end
def format_base_translation(definition)
base_translation = definition.translations[@default_lang]
def format_base_translation(row)
base_translation = row.translations[@default_lang]
"# base translation: \"#{base_translation}\"\n" if base_translation
end
@ -76,8 +123,8 @@ module Twine
"msgstr \"%{value}\"\n"
end
def format_comment(definition, lang)
"#. #{escape_quotes(definition.comment)}\n" if definition.comment
def format_comment(row, lang)
"#. #{escape_quotes(row.comment)}\n" if row.comment
end
def format_key(key)

View file

@ -1,8 +1,6 @@
module Twine
module Formatters
class Flash < Abstract
include Twine::Placeholders
def format_name
'flash'
end
@ -11,46 +9,81 @@ module Twine
'.properties'
end
def can_handle_directory?(path)
return false
end
def default_file_name
'resources.properties'
return 'resources.properties'
end
def set_translation_for_key(key, lang, value)
value = convert_placeholders_from_flash_to_twine(value)
super(key, lang, value)
def determine_language_given_path(path)
return
end
def read(io, lang)
last_comment = nil
while line = io.gets
match = /((?:[^"\\]|\\.)+)\s*=\s*((?:[^"\\]|\\.)*)/.match(line)
if match
key = match[1]
value = match[2].strip
set_translation_for_key(key, lang, value)
set_comment_for_key(key, last_comment) if last_comment
def read_file(path, lang)
encoding = Twine::Encoding.encoding_for_path(path)
sep = nil
if !encoding.respond_to?(:encode)
# This code is not necessary in 1.9.3 and does not work as it did in 1.8.7.
if encoding.end_with? 'LE'
sep = "\x0a\x00"
elsif encoding.end_with? 'BE'
sep = "\x00\x0a"
else
sep = "\n"
end
end
if encoding.index('UTF-16')
mode = "rb:#{encoding}"
else
mode = "r:#{encoding}"
end
File.open(path, mode) do |f|
last_comment = nil
while line = (sep) ? f.gets(sep) : f.gets
if encoding.index('UTF-16')
if line.respond_to? :encode!
line.encode!('UTF-8')
else
require 'iconv'
line = Iconv.iconv('UTF-8', encoding, line).join
end
end
match = /((?:[^"\\]|\\.)+)\s*=\s*((?:[^"\\]|\\.)*)/.match(line)
if match
key = match[1]
value = match[2].strip
value.gsub!(/\{[0-9]\}/, '%@')
set_translation_for_key(key, lang, value)
if last_comment
set_comment_for_key(key, last_comment)
end
end
match = /# *(.*)/.match(line)
if match
last_comment = match[1]
else
last_comment = nil
end
end
match = /# *(.*)/.match(line)
last_comment = match ? match[1] : nil
end
end
def format_sections(twine_file, lang)
super + "\n"
end
def format_header(lang)
"## Flash Strings File\n## Generated by Twine\n## Language: #{lang}"
"## Flash Strings File\n## Generated by Twine #{Twine::VERSION}\n## Language: #{lang}"
end
def format_section_header(section)
"## #{section.name} ##\n"
end
def format_comment(definition, lang)
"# #{definition.comment}\n" if definition.comment
def format_comment(row, lang)
"# #{row.comment}\n" if row.comment
end
def key_value_pattern
@ -58,7 +91,8 @@ module Twine
end
def format_value(value)
convert_placeholders_from_twine_to_flash(value)
placeHolderNumber = -1
value.gsub(/%[d@]/) { placeHolderNumber += 1; '{%d}' % placeHolderNumber }
end
end
end

View file

@ -11,80 +11,97 @@ module Twine
'.po'
end
def default_file_name
'strings.po'
def can_handle_directory?(path)
Dir.entries(path).any? { |item| /^.+\.po$/.match(item) }
end
def read(io, lang)
def default_file_name
return 'strings.po'
end
def determine_language_given_path(path)
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
match = /(..)\.po$/.match(segment)
if match
return match[1]
end
end
return
end
def read_file(path, lang)
comment_regex = /#.? *"(.*)"$/
key_regex = /msgctxt *"(.*)"$/
value_regex = /msgstr *"(.*)"$/m
while item = io.gets("\n\n")
key = nil
value = nil
comment = nil
comment_match = comment_regex.match(item)
if comment_match
comment = comment_match[1]
end
key_match = key_regex.match(item)
if key_match
key = key_match[1].gsub('\\"', '"')
end
value_match = value_regex.match(item)
if value_match
value = value_match[1].gsub(/"\n"/, '').gsub('\\"', '"')
end
if key and key.length > 0 and value and value.length > 0
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("SECTION:")
set_comment_for_key(key, comment)
end
File.open(path, 'r:UTF-8') do |f|
while item = f.gets("\n\n")
key = nil
value = nil
comment = nil
comment_match = comment_regex.match(item)
if comment_match
comment = comment_match[1]
end
key_match = key_regex.match(item)
if key_match
key = key_match[1].gsub('\\"', '"')
end
value_match = value_regex.match(item)
if value_match
value = value_match[1].gsub(/"\n"/, '').gsub('\\"', '"')
end
if key and key.length > 0 and value and value.length > 0
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("SECTION:")
set_comment_for_key(key, comment)
end
comment = nil
end
end
end
end
def format_file(lang)
@default_lang = twine_file.language_codes[0]
def format_file(strings, lang)
@default_lang = strings.language_codes[0]
result = super
@default_lang = nil
result
end
def format_header(lang)
"msgid \"\"\nmsgstr \"\"\n\"Language: #{lang}\"\n\"X-Generator: Twine #{Twine::VERSION}\"\n"
"msgid \"\"\nmsgstr \"\"\n\"Language: #{lang}\\n\"\n\"X-Generator: Twine #{Twine::VERSION}\\n\"\n"
end
def format_section_header(section)
"# SECTION: #{section.name}"
end
def should_include_definition(definition, lang)
super and !definition.translation_for_lang(@default_lang).nil?
def should_include_row(row, lang)
super and row.translated_string_for_lang(@default_lang)
end
def format_comment(definition, lang)
"#. \"#{escape_quotes(definition.comment)}\"\n" if definition.comment
def format_comment(row, lang)
"#. \"#{escape_quotes(row.comment)}\"\n" if row.comment
end
def format_key_value(definition, lang)
value = definition.translation_for_lang(lang)
[format_key(definition.key.dup), format_base_translation(definition), format_value(value.dup)].compact.join
def format_key_value(row, lang)
value = row.translated_string_for_lang(lang)
[format_key(row.key.dup), format_base_translation(row), format_value(value.dup)].compact.join
end
def format_key(key)
"msgctxt \"#{escape_quotes(key)}\"\n"
"msgctxt \"#{key}\"\n"
end
def format_base_translation(definition)
"msgid \"#{escape_quotes(definition.translations[@default_lang])}\"\n"
def format_base_translation(row)
"msgid \"#{row.translations[@default_lang]}\"\n"
end
def format_value(value)
"msgstr \"#{escape_quotes(value)}\"\n"
"msgstr \"#{value}\"\n"
end
end
end

View file

@ -9,39 +9,47 @@ module Twine
'.json'
end
def can_handle_directory?(path)
Dir.entries(path).any? { |item| /^.+\.json$/.match(item) }
end
def default_file_name
'localize.json'
return 'localize.json'
end
def determine_language_given_path(path)
match = /^.+([a-z]{2}-[A-Z]{2})\.json$/.match File.basename(path)
return match[1] if match
path_arr = path.split(File::SEPARATOR)
path_arr.each do |segment|
match = /^((.+)-)?([^-]+)\.json$/.match(segment)
if match
return match[3]
end
end
return super
return
end
def read(io, lang)
def read_file(path, lang)
begin
require "json"
rescue LoadError
raise Twine::Error.new "You must run `gem install json` in order to read or write jquery-localize files."
raise Twine::Error.new "You must run 'gem install json' in order to read or write jquery-localize files."
end
json = JSON.load(io)
json.each do |key, value|
set_translation_for_key(key, lang, value)
open(path) do |io|
json = JSON.load(io)
json.each do |key, value|
set_translation_for_key(key, lang, value)
end
end
end
def format_file(lang)
result = super
return result unless result
"{\n#{super}\n}\n"
def format_file(strings, lang)
"{\n#{super}\n}"
end
def format_sections(twine_file, lang)
sections = twine_file.sections.map { |section| format_section(section, lang) }
sections.delete_if(&:empty?)
def format_sections(strings, lang)
sections = strings.sections.map { |section| format_section(section, lang) }
sections.join(",\n\n")
end
@ -49,11 +57,11 @@ module Twine
end
def format_section(section, lang)
definitions = section.definitions.dup
rows = section.rows.dup
definitions.map! { |definition| format_definition(definition, lang) }
definitions.compact! # remove nil definitions
definitions.join(",\n")
rows.map! { |row| format_row(row, lang) }
rows.compact! # remove nil entries
rows.join(",\n")
end
def key_value_pattern

View file

@ -33,7 +33,7 @@ module Twine
end
def default_file_name
'strings.xml'
return 'strings.xml'
end
def determine_language_given_path(path)
@ -49,7 +49,7 @@ module Twine
return
end
def read(io, lang)
def read_file(path, lang)
resources_regex = /<resources(?:[^>]*)>(.*)<\/resources>/m
key_regex = /<string name="(\w+)">/
comment_regex = /<!-- (.*) -->/
@ -58,56 +58,58 @@ module Twine
value = nil
comment = nil
content_match = resources_regex.match(io.read)
if content_match
for line in content_match[1].split(/\r?\n/)
key_match = key_regex.match(line)
if key_match
key = key_match[1]
value_match = value_regex.match(line)
if value_match
value = value_match[1]
value = CGI.unescapeHTML(value)
value.gsub!('\\\'', '\'')
value.gsub!('\\"', '"')
value = convert_placeholders_from_android_to_twine(value)
value.gsub!(/(\\u0020)*|(\\u0020)*\z/) { |spaces| ' ' * (spaces.length / 6) }
else
value = ""
File.open(path, 'r:UTF-8') do |f|
content_match = resources_regex.match(f.read)
if content_match
for line in content_match[1].split(/\r?\n/)
key_match = key_regex.match(line)
if key_match
key = key_match[1]
value_match = value_regex.match(line)
if value_match
value = value_match[1]
value = CGI.unescapeHTML(value)
value.gsub!('\\\'', '\'')
value.gsub!('\\"', '"')
value = convert_placeholders_from_android_to_twine(value)
value.gsub!(/(\\u0020)*|(\\u0020)*\z/) { |spaces| ' ' * (spaces.length / 6) }
else
value = ""
end
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("SECTION:")
set_comment_for_key(key, comment)
end
comment = nil
end
set_translation_for_key(key, lang, value)
if comment and comment.length > 0 and !comment.start_with?("SECTION:")
set_comment_for_key(key, comment)
end
comment = nil
end
comment_match = comment_regex.match(line)
if comment_match
comment = comment_match[1]
comment_match = comment_regex.match(line)
if comment_match
comment = comment_match[1]
end
end
end
end
end
def format_header(lang)
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Tizen Strings File -->\n<!-- Generated by Twine -->\n<!-- Language: #{lang} -->"
"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<!-- Tizen Strings File -->\n<!-- Generated by Twine #{Twine::VERSION} -->\n<!-- Language: #{lang} -->"
end
def format_sections(twine_file, lang)
def format_sections(strings, lang)
result = '<string_table Bversion="2.0.0.201311071819" Dversion="20120315">'
result += super + "\n"
result += "</string_table>\n"
result += '</string_table>'
end
def format_section_header(section)
"\t<!-- SECTION: #{section.name} -->"
end
def format_comment(definition, lang)
"\t<!-- #{definition.comment.gsub('--', '—')} -->\n" if definition.comment
def format_comment(row, lang)
"\t<!-- #{row.comment.gsub('--', '—')} -->\n" if row.comment
end
def key_value_pattern

View file

@ -2,62 +2,48 @@ module Twine
module Processors
class OutputProcessor
def initialize(twine_file, options)
@twine_file = twine_file
def initialize(strings, options)
@strings = strings
@options = options
end
def default_language
@options[:developer_language] || @twine_file.language_codes[0]
@options[:developer_language] || @strings.language_codes[0]
end
def fallback_languages(language)
fallback_mapping = {
'zh-CN' => 'zh-Hans', # if we don't have a zh-CN translation, try zh-Hans before en
'zh-TW' => 'zh-Hant' # if we don't have a zh-TW translation, try zh-Hant before en
}
# Regional dialect fallbacks to generic language (for example: 'es-MX' to 'es' instead of default 'en').
if language.match(/([a-zA-Z]{2})-[a-zA-Z]+/)
generic_language = language.gsub(/([a-zA-Z])-[a-zA-Z]+/, '\1')
end
[fallback_mapping[language], generic_language, default_language].flatten.compact
[fallback_mapping[language], default_language].flatten.compact
end
def process(language)
result = TwineFile.new
result = StringsFile.new
result.language_codes.concat @twine_file.language_codes
@twine_file.sections.each do |section|
new_section = TwineSection.new section.name
result.language_codes.concat @strings.language_codes
@strings.sections.each do |section|
new_section = StringsSection.new section.name
section.definitions.each do |definition|
next unless definition.matches_tags?(@options[:tags], @options[:untagged])
section.rows.each do |row|
next unless row.matches_tags?(@options[:tags], @options[:untagged])
value = definition.translation_for_lang(language)
value = row.translated_string_for_lang(language)
next if value && @options[:include] == :untranslated
next if value && @options[:include] == 'untranslated'
if value.nil? && @options[:include] != :translated
value = definition.translation_for_lang(fallback_languages(language))
if value.nil? && @options[:include] != 'translated'
value = row.translated_string_for_lang(fallback_languages(language))
end
next unless value
new_definition = definition.dup
new_definition.translations[language] = value
new_row = row.dup
new_row.translations[language] = value
if definition.is_plural?
# If definition is plural, but no translation found -> create
# Then check 'other' key
if !(new_definition.plural_translations[language] ||= {}).key? 'other'
new_definition.plural_translations[language]['other'] = value
end
end
new_section.definitions << new_definition
result.definitions_by_key[new_definition.key] = new_definition
new_section.rows << new_row
result.strings_map[new_row.key] = new_row
end
result.sections << new_section

View file

@ -2,42 +2,34 @@ module Twine
module Placeholders
extend self
# Note: the ` ` (single space) flag is NOT supported
PLACEHOLDER_FLAGS_WIDTH_PRECISION_LENGTH = '([-+0#])?(\d+|\*)?(\.(\d+|\*))?(hh?|ll?|L|z|j|t|q)?'
PLACEHOLDER_FLAGS_WIDTH_PRECISION_LENGTH = '([-+ 0#])?(\d+|\*)?(\.(\d+|\*))?(hh?|ll?|L|z|j|t)?'
PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH = '(\d+\$)?' + PLACEHOLDER_FLAGS_WIDTH_PRECISION_LENGTH
PLACEHOLDER_TYPES = '[diufFeEgGxXoscpaA]'
PLACEHOLDER_REGEX = /%#{PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH}#{PLACEHOLDER_TYPES}/
def number_of_twine_placeholders(input)
input.scan(PLACEHOLDER_REGEX).size
end
def convert_twine_string_placeholder(input)
# %@ -> %s
input.gsub(/(%#{PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH})@/, '\1s')
end
# http://developer.android.com/guide/topics/resources/string-resource.html#FormattingAndStyling
# http://stackoverflow.com/questions/4414389/android-xml-percent-symbol
# https://github.com/mobiata/twine/pull/106
def convert_placeholders_from_twine_to_android(input)
# %@ -> %s
value = convert_twine_string_placeholder(input)
placeholder_types = '[diufFeEgGxXoscpaA]'
number_of_placeholders = number_of_twine_placeholders(value)
# %@ -> %s
value = input.gsub(/(%#{PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH})@/, '\1s')
placeholder_syntax = PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH + placeholder_types
placeholder_regex = /%#{placeholder_syntax}/
number_of_placeholders = value.scan(placeholder_regex).size
return value if number_of_placeholders == 0
# got placeholders -> need to double single percent signs
# % -> %% (but %% -> %%, %d -> %d)
placeholder_syntax = PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH + PLACEHOLDER_TYPES
single_percent_regex = /([^%])(%)(?!(%|#{placeholder_syntax}))/
value.gsub! single_percent_regex, '\1%%'
return value if number_of_placeholders < 2
# number placeholders
non_numbered_placeholder_regex = /%(#{PLACEHOLDER_FLAGS_WIDTH_PRECISION_LENGTH}#{PLACEHOLDER_TYPES})/
non_numbered_placeholder_regex = /%(#{PLACEHOLDER_FLAGS_WIDTH_PRECISION_LENGTH}#{placeholder_types})/
number_of_non_numbered_placeholders = value.scan(non_numbered_placeholder_regex).size
@ -58,25 +50,5 @@ module Twine
# %s -> %@
input.gsub(placeholder_regex, '\1@')
end
# http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/resources/IResourceManager.html#getString()
# http://soenkerohde.com/2008/07/flex-localization/comment-page-1/
def convert_placeholders_from_twine_to_flash(input)
value = convert_twine_string_placeholder(input)
value.gsub(PLACEHOLDER_REGEX).each_with_index do |match, index|
"{#{index}}"
end
end
def convert_placeholders_from_flash_to_twine(input)
input.gsub /\{\d+\}/, '%@'
end
# Python supports placeholders in the form of `%(amount)03d`
# see https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting
def contains_python_specific_placeholder(input)
/%\([a-zA-Z0-9_-]+\)#{PLACEHOLDER_PARAMETER_FLAGS_WIDTH_PRECISION_LENGTH}#{PLACEHOLDER_TYPES}/.match(input) != nil
end
end
end

View file

@ -1,4 +1,3 @@
require 'date'
require 'safe_yaml/load'
SafeYAML::OPTIONS[:suppress_warnings] = true
@ -57,7 +56,7 @@ module Twine
end
def join_path *paths
File.expand_path File.join(*paths)
File.expand_path File.join *paths
end
end
end

View file

@ -5,72 +5,54 @@ Twine::Plugin.new # Initialize plugins first in Runner.
module Twine
class Runner
class NullOutput
def puts(message)
end
def string
""
end
end
def self.run(args)
options = CLI.parse(args)
return unless options
twine_file = TwineFile.new
twine_file.read options[:twine_file]
runner = new(options, twine_file)
strings = StringsFile.new
strings.read options[:strings_file]
runner = new(options, strings)
case options[:command]
when 'generate-localization-file'
runner.generate_localization_file
when 'generate-all-localization-files'
runner.generate_all_localization_files
when 'consume-localization-file'
runner.consume_localization_file
when 'consume-all-localization-files'
runner.consume_all_localization_files
when 'generate-localization-archive'
runner.generate_localization_archive
when 'consume-localization-archive'
runner.consume_localization_archive
when 'validate-twine-file'
runner.validate_twine_file
when 'generate-string-file'
runner.generate_string_file
when 'generate-all-string-files'
runner.generate_all_string_files
when 'consume-string-file'
runner.consume_string_file
when 'consume-all-string-files'
runner.consume_all_string_files
when 'generate-loc-drop'
runner.generate_loc_drop
when 'consume-loc-drop'
runner.consume_loc_drop
when 'validate-strings-file'
runner.validate_strings_file
end
end
def initialize(options = {}, twine_file = TwineFile.new)
def initialize(options = {}, strings = StringsFile.new)
@options = options
@twine_file = twine_file
if @options[:quite]
Twine::stdout = NullOutput.new
end
@strings = strings
end
def write_twine_data(path)
def write_strings_data(path)
if @options[:developer_language]
@twine_file.set_developer_language_code(@options[:developer_language])
@strings.set_developer_language_code(@options[:developer_language])
end
@twine_file.write(path)
@strings.write(path)
end
def generate_localization_file
validate_twine_file if @options[:validate]
def generate_string_file
validate_strings_file if @options[:validate]
lang = nil
lang = @options[:languages][0] if @options[:languages]
formatter, lang = prepare_read_write(@options[:output_path], lang)
output = formatter.format_file(lang)
raise Twine::Error.new "Nothing to generate! The resulting file would not contain any translations." unless output
IO.write(@options[:output_path], output, encoding: output_encoding)
write_string_file(@options[:output_path], lang)
end
def generate_all_localization_files
validate_twine_file if @options[:validate]
def generate_all_string_files
validate_strings_file if @options[:validate]
if !File.directory?(@options[:output_path])
if @options[:create_folders]
@ -80,65 +62,48 @@ module Twine
end
end
if @options[:format]
formatter = formatter_for_format(@options[:format])
else
formatter = find_formatter { |f| f.can_handle_directory?(@options[:output_path]) }
end
formatter_for_directory = find_formatter { |f| f.can_handle_directory?(@options[:output_path]) }
formatter = formatter_for_format(@options[:format]) || formatter_for_directory
unless formatter
raise Twine::Error.new "Could not determine format given the contents of #{@options[:output_path]}. Try using `--format`."
end
file_name = @options[:file_name] || formatter.default_file_name
if @options[:create_folders]
@twine_file.language_codes.each do |lang|
output_path = File.join(@options[:output_path], formatter.output_path_for_language(lang))
FileUtils.mkdir_p(output_path)
file_path = File.join(output_path, file_name)
output = formatter.format_file(lang)
unless output
Twine::stdout.puts "Skipping file at path #{file_path} since it would not contain any translations."
next
end
IO.write(file_path, output, encoding: output_encoding)
end
else
language_found = false
Dir.foreach(@options[:output_path]) do |item|
next if item == "." or item == ".."
output_path = File.join(@options[:output_path], item)
next unless File.directory?(output_path)
lang = formatter.determine_language_given_path(output_path)
next unless lang
language_found = true
file_path = File.join(output_path, file_name)
output = formatter.format_file(lang)
unless output
Twine::stdout.puts "Skipping file at path #{file_path} since it would not contain any translations."
next
end
IO.write(file_path, output, encoding: output_encoding)
end
unless language_found
raise Twine::Error.new("Failed to generate any files: No languages found at #{@options[:output_path]}")
end
raise Twine::Error.new "Could not determine format given the contents of #{@options[:output_path]}"
end
formatter.write_all_files(@options[:output_path])
end
def generate_localization_archive
validate_twine_file if @options[:validate]
def consume_string_file
lang = nil
if @options[:languages]
lang = @options[:languages][0]
end
read_string_file(@options[:input_path], lang)
output_path = @options[:output_path] || @options[:strings_file]
write_strings_data(output_path)
end
def consume_all_string_files
if !File.directory?(@options[:input_path])
raise Twine::Error.new("Directory does not exist: #{@options[:output_path]}")
end
Dir.glob(File.join(@options[:input_path], "**/*")) do |item|
if File.file?(item)
begin
read_string_file(item)
rescue Twine::Error => e
Twine::stderr.puts "#{e.message}"
end
end
end
output_path = @options[:output_path] || @options[:strings_file]
write_strings_data(output_path)
end
def generate_loc_drop
validate_strings_file if @options[:validate]
require_rubyzip
@ -146,116 +111,70 @@ module Twine
File.delete(@options[:output_path])
end
Dir.mktmpdir do |temp_dir|
Dir.mktmpdir do |dir|
Zip::File.open(@options[:output_path], Zip::File::CREATE) do |zipfile|
zipfile.mkdir('Locales')
formatter = formatter_for_format(@options[:format])
@twine_file.language_codes.each do |lang|
@strings.language_codes.each do |lang|
if @options[:languages] == nil || @options[:languages].length == 0 || @options[:languages].include?(lang)
file_name = lang + formatter.extension
temp_path = File.join(temp_dir, file_name)
real_path = File.join(dir, file_name)
zip_path = File.join('Locales', file_name)
output = formatter.format_file(lang)
unless output
Twine::stdout.puts "Skipping file #{file_name} since it would not contain any translations."
next
end
IO.write(temp_path, output, encoding: output_encoding)
zipfile.add(zip_path, temp_path)
formatter.write_file(real_path, lang)
zipfile.add(zip_path, real_path)
end
end
end
end
end
def consume_localization_file
lang = nil
if @options[:languages]
lang = @options[:languages][0]
end
read_localization_file(@options[:input_path], lang)
output_path = @options[:output_path] || @options[:twine_file]
write_twine_data(output_path)
end
def consume_all_localization_files
if !File.directory?(@options[:input_path])
raise Twine::Error.new("Directory does not exist: #{@options[:input_path]}")
end
Dir.glob(File.join(@options[:input_path], "**/*")) do |item|
if File.file?(item)
begin
read_localization_file(item)
rescue Twine::Error => e
Twine::stderr.puts "#{e.message}"
end
end
end
output_path = @options[:output_path] || @options[:twine_file]
write_twine_data(output_path)
end
def consume_localization_archive
def consume_loc_drop
require_rubyzip
if !File.file?(@options[:input_path])
raise Twine::Error.new("File does not exist: #{@options[:input_path]}")
end
error_encountered = false
Dir.mktmpdir do |temp_dir|
Dir.mktmpdir do |dir|
Zip::File.open(@options[:input_path]) do |zipfile|
zipfile.each do |entry|
next if entry.name.end_with? '/' or File.basename(entry.name).start_with? '.'
real_path = File.join(temp_dir, entry.name)
FileUtils.mkdir_p(File.dirname(real_path))
zipfile.extract(entry.name, real_path)
begin
read_localization_file(real_path)
rescue Twine::Error => e
Twine::stderr.puts "#{e.message}"
error_encountered = true
if !entry.name.end_with?'/' and !File.basename(entry.name).start_with?'.'
real_path = File.join(dir, entry.name)
FileUtils.mkdir_p(File.dirname(real_path))
zipfile.extract(entry.name, real_path)
begin
read_string_file(real_path)
rescue Twine::Error => e
Twine::stderr.puts "#{e.message}"
end
end
end
end
end
output_path = @options[:output_path] || @options[:twine_file]
write_twine_data(output_path)
if error_encountered
raise Twine::Error.new("At least one file could not be consumed")
end
output_path = @options[:output_path] || @options[:strings_file]
write_strings_data(output_path)
end
def validate_twine_file
total_definitions = 0
def validate_strings_file
total_strings = 0
all_keys = Set.new
duplicate_keys = Set.new
keys_without_tags = Set.new
invalid_keys = Set.new
keys_with_python_only_placeholders = Set.new
valid_key_regex = /^[A-Za-z0-9_]+$/
@twine_file.sections.each do |section|
section.definitions.each do |definition|
total_definitions += 1
@strings.sections.each do |section|
section.rows.each do |row|
total_strings += 1
duplicate_keys.add(definition.key) if all_keys.include? definition.key
all_keys.add(definition.key)
duplicate_keys.add(row.key) if all_keys.include? row.key
all_keys.add(row.key)
keys_without_tags.add(definition.key) if definition.tags == nil or definition.tags.length == 0
keys_without_tags.add(row.key) if row.tags == nil or row.tags.length == 0
invalid_keys << definition.key unless definition.key =~ valid_key_regex
keys_with_python_only_placeholders << definition.key if definition.translations.values.any? { |v| Placeholders.contains_python_specific_placeholder(v) }
invalid_keys << row.key unless row.key =~ valid_key_regex
end
end
@ -263,14 +182,14 @@ module Twine
join_keys = lambda { |set| set.map { |k| " " + k }.join("\n") }
unless duplicate_keys.empty?
errors << "Found duplicate key(s):\n#{join_keys.call(duplicate_keys)}"
errors << "Found duplicate string key(s):\n#{join_keys.call(duplicate_keys)}"
end
if @options[:pedantic]
if keys_without_tags.length == total_definitions
errors << "None of your definitions have tags."
if keys_without_tags.length == total_strings
errors << "None of your strings have tags."
elsif keys_without_tags.length > 0
errors << "Found definitions without tags:\n#{join_keys.call(keys_without_tags)}"
errors << "Found strings without tags:\n#{join_keys.call(keys_without_tags)}"
end
end
@ -278,78 +197,67 @@ module Twine
errors << "Found key(s) with invalid characters:\n#{join_keys.call(invalid_keys)}"
end
unless keys_with_python_only_placeholders.empty?
errors << "Found key(s) with placeholders that are only supported by Python:\n#{join_keys.call(keys_with_python_only_placeholders)}"
end
raise Twine::Error.new errors.join("\n\n") unless errors.empty?
Twine::stdout.puts "#{@options[:twine_file]} is valid."
Twine::stdout.puts "#{@options[:strings_file]} is valid."
end
private
def output_encoding
@options[:encoding] || 'UTF-8'
end
def require_rubyzip
begin
require 'zip'
rescue LoadError
raise Twine::Error.new "You must run 'gem install rubyzip' in order to create or consume localization archives."
raise Twine::Error.new "You must run 'gem install rubyzip' in order to create or consume localization drops."
end
end
def determine_language_given_path(path)
code = File.basename(path, File.extname(path))
return code if @strings.language_codes.include? code
end
def formatter_for_format(format)
find_formatter { |f| f.format_name == format }
end
def find_formatter(&block)
formatters = Formatters.formatters.select(&block)
if formatters.empty?
return nil
elsif formatters.size > 1
raise Twine::Error.new("Unable to determine format. Candidates are: #{formatters.map(&:format_name).join(', ')}. Please specify the format you want using `--format`")
end
formatter = formatters.first
formatter.twine_file = @twine_file
formatter = Formatters.formatters.find &block
return nil unless formatter
formatter.strings = @strings
formatter.options = @options
formatter
end
def read_localization_file(path, lang = nil)
def read_string_file(path, lang = nil)
unless File.file?(path)
raise Twine::Error.new("File does not exist: #{path}")
end
formatter, lang = prepare_read_write(path, lang)
external_encoding = @options[:encoding] || Twine::Encoding.encoding_for_path(path)
formatter.read_file(path, lang)
end
IO.open(IO.sysopen(path, 'rb'), 'rb', external_encoding: external_encoding, internal_encoding: 'UTF-8') do |io|
io.read(2) if Twine::Encoding.has_bom?(path)
formatter.read(io, lang)
end
def write_string_file(path, lang)
formatter, lang = prepare_read_write(path, lang)
formatter.write_file(path, lang)
end
def prepare_read_write(path, lang)
if @options[:format]
formatter = formatter_for_format(@options[:format])
else
formatter = find_formatter { |f| f.extension == File.extname(path) }
end
formatter_for_path = find_formatter { |f| f.extension == File.extname(path) }
formatter = formatter_for_format(@options[:format]) || formatter_for_path
unless formatter
raise Twine::Error.new "Unable to determine format of #{path}. Try using `--format`."
raise Twine::Error.new "Unable to determine format of #{path}"
end
lang = lang || formatter.determine_language_given_path(path)
lang = lang || determine_language_given_path(path) || formatter.determine_language_given_path(path)
unless lang
raise Twine::Error.new "Unable to determine language for #{path}. Try using `--lang`."
raise Twine::Error.new "Unable to determine language for #{path}"
end
@twine_file.language_codes << lang unless @twine_file.language_codes.include? lang
@strings.language_codes << lang unless @strings.language_codes.include? lang
return formatter, lang
end

View file

@ -1,13 +1,19 @@
module Twine
class TwineDefinition
PLURAL_KEYS = %w(zero one two few many other)
class StringsSection
attr_reader :name
attr_reader :rows
def initialize(name)
@name = name
@rows = []
end
end
class StringsRow
attr_reader :key
attr_accessor :comment
attr_accessor :tags
attr_reader :translations
attr_reader :plural_translations
attr_reader :is_plural
attr_accessor :reference
attr_accessor :reference_key
@ -16,7 +22,6 @@ module Twine
@comment = nil
@tags = nil
@translations = {}
@plural_translations = {}
end
def comment
@ -27,59 +32,34 @@ module Twine
@comment
end
# [['tag1', 'tag2'], ['~tag3']] == (tag1 OR tag2) AND (!tag3)
def matches_tags?(tags, include_untagged)
if tags == nil || tags.empty? # The user did not specify any tags. Everything passes.
if tags == nil || tags.empty?
# The user did not specify any tags. Everything passes.
return true
elsif @tags == nil # This definition has no tags -> check reference (if any)
elsif @tags == nil
# This row has no tags.
return reference ? reference.matches_tags?(tags, include_untagged) : include_untagged
elsif @tags.empty?
return include_untagged
else
return tags.all? do |set|
regular_tags, negated_tags = set.partition { |tag| tag[0] != '~' }
negated_tags.map! { |tag| tag[1..-1] }
matches_regular_tags = (!regular_tags.empty? && !(regular_tags & @tags).empty?)
matches_negated_tags = (!negated_tags.empty? && (negated_tags & @tags).empty?)
matches_regular_tags or matches_negated_tags
end
return !(tags & @tags).empty?
end
return false
end
def translation_for_lang(lang)
translation = [lang].flatten.map { |l| @translations[l] }.compact.first
def translated_string_for_lang(lang)
translation = [lang].flatten.map { |l| @translations[l] }.first
translation = reference.translation_for_lang(lang) if translation.nil? && reference
translation = reference.translated_string_for_lang(lang) if translation.nil? && reference
return translation
end
def plural_translation_for_lang(lang)
if @plural_translations.has_key? lang
@plural_translations[lang].dup.sort_by { |key,_| TwineDefinition::PLURAL_KEYS.index(key) }.to_h
end
end
def is_plural?
!@plural_translations.empty?
end
end
class TwineSection
attr_reader :name
attr_reader :definitions
def initialize(name)
@name = name
@definitions = []
end
end
class TwineFile
class StringsFile
attr_reader :sections
attr_reader :definitions_by_key
attr_reader :strings_map
attr_reader :language_codes
private
@ -93,7 +73,7 @@ module Twine
def initialize
@sections = []
@definitions_by_key = {}
@strings_map = {}
@language_codes = []
end
@ -122,7 +102,7 @@ module Twine
File.open(path, 'r:UTF-8') do |f|
line_num = 0
current_section = nil
current_definition = nil
current_row = nil
while line = f.gets
parsed = false
line.strip!
@ -135,54 +115,42 @@ module Twine
if line.length > 4 && line[0, 2] == '[['
match = /^\[\[(.+)\]\]$/.match(line)
if match
current_section = TwineSection.new(match[1])
current_section = StringsSection.new(match[1])
@sections << current_section
parsed = true
end
elsif line.length > 2 && line[0, 1] == '['
key = match_key(line)
if key
current_definition = TwineDefinition.new(key)
@definitions_by_key[current_definition.key] = current_definition
current_row = StringsRow.new(key)
@strings_map[current_row.key] = current_row
if !current_section
current_section = TwineSection.new('')
current_section = StringsSection.new('')
@sections << current_section
end
current_section.definitions << current_definition
current_section.rows << current_row
parsed = true
end
else
match = /^([^:=]+)(?::([^=]+))?=(.*)$/.match(line)
match = /^([^=]+)=(.*)$/.match(line)
if match
key = match[1].strip
plural_key = match[2].to_s.strip
value = match[3].strip
value = match[2].strip
value = value[1..-2] if value[0] == '`' && value[-1] == '`'
case key
when 'comment'
current_definition.comment = value
current_row.comment = value
when 'tags'
current_definition.tags = value.split(',')
current_row.tags = value.split(',')
when 'ref'
current_definition.reference_key = value if value
current_row.reference_key = value if value
else
if !@language_codes.include? key
add_language_code(key)
end
# Providing backward compatibility
# for formatters without plural support
if plural_key.empty? || plural_key == 'other'
current_definition.translations[key] = value
end
if !plural_key.empty?
if !TwineDefinition::PLURAL_KEYS.include? plural_key
warn("Unknown plural key #{plural_key}")
next
end
(current_definition.plural_translations[key] ||= {})[plural_key] = value
end
current_row.translations[key] = value
end
parsed = true
end
@ -195,9 +163,9 @@ module Twine
end
# resolve_references
@definitions_by_key.each do |key, definition|
next unless definition.reference_key
definition.reference = @definitions_by_key[definition.reference_key]
@strings_map.each do |key, row|
next unless row.reference_key
row.reference = @strings_map[row.reference_key]
end
end
@ -212,26 +180,26 @@ module Twine
f.puts "[[#{section.name}]]"
section.definitions.each do |definition|
f.puts "\t[#{definition.key}]"
section.rows.each do |row|
f.puts "\t[#{row.key}]"
value = write_value(definition, dev_lang, f)
if !value && !definition.reference_key
Twine::stdout.puts "WARNING: #{definition.key} does not exist in developer language '#{dev_lang}'"
value = write_value(row, dev_lang, f)
if !value && !row.reference_key
puts "Warning: #{row.key} does not exist in developer language '#{dev_lang}'"
end
if definition.reference_key
f.puts "\t\tref = #{definition.reference_key}"
if row.reference_key
f.puts "\t\tref = #{row.reference_key}"
end
if definition.tags && definition.tags.length > 0
tag_str = definition.tags.join(',')
if row.tags && row.tags.length > 0
tag_str = row.tags.join(',')
f.puts "\t\ttags = #{tag_str}"
end
if definition.raw_comment and definition.raw_comment.length > 0
f.puts "\t\tcomment = #{definition.raw_comment}"
if row.raw_comment and row.raw_comment.length > 0
f.puts "\t\tcomment = #{row.raw_comment}"
end
@language_codes[1..-1].each do |lang|
write_value(definition, lang, f)
write_value(row, lang, f)
end
end
end
@ -240,8 +208,8 @@ module Twine
private
def write_value(definition, language, file)
value = definition.translations[language]
def write_value(row, language, file)
value = row.translations[language]
return nil unless value
if value[0] == ' ' || value[-1] == ' ' || (value[0] == '`' && value[-1] == '`')

View file

@ -1,3 +1,3 @@
module Twine
VERSION = '1.1.2-om'
VERSION = '0.8.1'
end

View file

@ -1,14 +0,0 @@
require 'twine_test'
class CommandTest < TwineTest
def prepare_mock_formatter(formatter_class, clear_other_formatters = true)
twine_file = Twine::TwineFile.new
twine_file.language_codes.concat KNOWN_LANGUAGES
formatter = formatter_class.new
formatter.twine_file = twine_file
Twine::Formatters.formatters.clear if clear_other_formatters
Twine::Formatters.formatters << formatter
formatter
end
end

14
test/command_test_case.rb Normal file
View file

@ -0,0 +1,14 @@
require 'twine_test_case'
class CommandTestCase < TwineTestCase
def prepare_mock_formatter(formatter_class)
strings = Twine::StringsFile.new
strings.language_codes.concat KNOWN_LANGUAGES
formatter = formatter_class.new
formatter.strings = strings
Twine::Formatters.formatters.clear
Twine::Formatters.formatters << formatter
formatter
end
end

BIN
test/fixtures/consume_loc_drop.zip vendored Normal file

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View file

@ -1,2 +0,0 @@
Üß`
da

View file

@ -1,12 +1,15 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Android Strings File -->
<!-- Generated by Twine <%= Twine::VERSION %> -->
<!-- Language: en -->
<resources>
<!-- SECTION: Section 1 -->
<!-- comment key1 -->
<string name="key1">value1-english</string>
<string name="key2">value2-english</string>
<!-- SECTION: Section 1 -->
<!-- comment key1 -->
<string name="key1">value1-english</string>
<string name="key2">value2-english</string>
<!-- SECTION: Section 2 -->
<string name="key3">value3-english</string>
<!-- comment key4 -->
<string name="key4">value4-english</string>
<!-- SECTION: Section 2 -->
<string name="key3">value3-english</string>
<!-- comment key4 -->
<string name="key4">value4-english</string>
</resources>

View file

@ -1,3 +1,9 @@
/**
* Apple Strings File
* Generated by Twine <%= Twine::VERSION %>
* Language: en
*/
/********** Section 1 **********/
/* comment key1 */

View file

@ -1,11 +1,10 @@
# Django Strings File
# Generated by Twine
# Language: en
msgid ""
msgstr ""
"Content-Type: text/plain; charset=UTF-8\n"
##
# Django Strings File
# Generated by Twine <%= Twine::VERSION %>
# Language: en
# --------- Section 1 --------- #
#--------- Section 1 ---------#
#. comment key1
# base translation: "value1-english"
@ -17,7 +16,7 @@ msgid "key2"
msgstr "value2-english"
# --------- Section 2 --------- #
#--------- Section 2 ---------#
# base translation: "value3-english"
msgid "key3"

View file

@ -1,5 +1,5 @@
## Flash Strings File
## Generated by Twine
## Generated by Twine <%= Twine::VERSION %>
## Language: en
## Section 1 ##

View file

@ -1,7 +1,7 @@
msgid ""
msgstr ""
"Language: en"
"X-Generator: Twine <%= Twine::VERSION %>"
"Language: en\n"
"X-Generator: Twine <%= Twine::VERSION %>\n"
# SECTION: Section 1

View file

@ -1,10 +0,0 @@
msgid ""
msgstr ""
"Language: en"
"X-Generator: Twine <%= Twine::VERSION %>"
# SECTION: Section
msgctxt "key"
msgid "foo \"bar\" baz"
msgstr "foo \"bar\" baz"

View file

@ -1,6 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Tizen Strings File -->
<!-- Generated by Twine -->
<!-- Generated by Twine <%= Twine::VERSION %> -->
<!-- Language: en -->
<string_table Bversion="2.0.0.201311071819" Dversion="20120315">
<!-- SECTION: Section 1 -->

View file

@ -1,164 +1,164 @@
require 'twine_test'
require 'twine_test_case'
class TestAbstractFormatter < TwineTest
class SetTranslation < TwineTest
class TestAbstractFormatter < TwineTestCase
class SetTranslation < TwineTestCase
def setup
super
@twine_file = build_twine_file 'en', 'fr' do
@strings = build_twine_file 'en', 'fr' do
add_section 'Section' do
add_definition key1: 'value1-english'
add_definition key2: { en: 'value2-english', fr: 'value2-french' }
add_row key1: 'value1-english'
add_row key2: { en: 'value2-english', fr: 'value2-french' }
end
end
@formatter = Twine::Formatters::Abstract.new
@formatter.twine_file = @twine_file
@formatter.strings = @strings
end
def test_set_translation_updates_existing_value
@formatter.set_translation_for_key 'key1', 'en', 'value1-english-updated'
assert_equal 'value1-english-updated', @twine_file.definitions_by_key['key1'].translations['en']
assert_equal 'value1-english-updated', @strings.strings_map['key1'].translations['en']
end
def test_set_translation_does_not_alter_other_language
@formatter.set_translation_for_key 'key2', 'en', 'value2-english-updated'
assert_equal 'value2-french', @twine_file.definitions_by_key['key2'].translations['fr']
assert_equal 'value2-french', @strings.strings_map['key2'].translations['fr']
end
def test_set_translation_escapes_newlines
@formatter.set_translation_for_key 'key1', 'en', "new\nline"
assert_equal 'new\nline', @twine_file.definitions_by_key['key1'].translations['en']
assert_equal 'new\nline', @strings.strings_map['key1'].translations['en']
end
def test_set_translation_adds_translation_to_existing_key
@formatter.set_translation_for_key 'key1', 'fr', 'value1-french'
assert_equal 'value1-french', @twine_file.definitions_by_key['key1'].translations['fr']
assert_equal 'value1-french', @strings.strings_map['key1'].translations['fr']
end
def test_set_translation_does_not_add_new_key
@formatter.set_translation_for_key 'new-key', 'en', 'new-key-english'
assert_nil @twine_file.definitions_by_key['new-key']
assert_nil @strings.strings_map['new-key']
end
def test_set_translation_consume_all_adds_new_key
formatter = Twine::Formatters::Abstract.new
formatter.twine_file = @twine_file
formatter.strings = @strings
formatter.options = { consume_all: true }
formatter.set_translation_for_key 'new-key', 'en', 'new-key-english'
assert_equal 'new-key-english', @twine_file.definitions_by_key['new-key'].translations['en']
assert_equal 'new-key-english', @strings.strings_map['new-key'].translations['en']
end
def test_set_translation_consume_all_adds_tags
random_tag = SecureRandom.uuid
formatter = Twine::Formatters::Abstract.new
formatter.twine_file = @twine_file
formatter.strings = @strings
formatter.options = { consume_all: true, tags: [random_tag] }
formatter.set_translation_for_key 'new-key', 'en', 'new-key-english'
assert_equal [random_tag], @twine_file.definitions_by_key['new-key'].tags
assert_equal [random_tag], @strings.strings_map['new-key'].tags
end
def test_set_translation_adds_new_keys_to_category_uncategoriezed
formatter = Twine::Formatters::Abstract.new
formatter.twine_file = @twine_file
formatter.strings = @strings
formatter.options = { consume_all: true }
formatter.set_translation_for_key 'new-key', 'en', 'new-key-english'
assert_equal 'Uncategorized', @twine_file.sections[0].name
assert_equal 'new-key', @twine_file.sections[0].definitions[0].key
assert_equal 'Uncategorized', @strings.sections[0].name
assert_equal 'new-key', @strings.sections[0].rows[0].key
end
end
class ValueReference < TwineTest
class ValueReference < TwineTestCase
def setup
super
@twine_file = build_twine_file 'en', 'fr' do
@strings = build_twine_file 'en', 'fr' do
add_section 'Section' do
add_definition refkey: 'ref-value'
add_definition key: :refkey
add_row refkey: 'ref-value'
add_row key: :refkey
end
end
@formatter = Twine::Formatters::Abstract.new
@formatter.twine_file = @twine_file
@formatter.strings = @strings
end
def test_set_translation_does_not_add_unchanged_translation
@formatter.set_translation_for_key 'key', 'en', 'ref-value'
assert_nil @twine_file.definitions_by_key['key'].translations['en']
assert_nil @strings.strings_map['key'].translations['en']
end
def test_set_translation_adds_changed_translation
@formatter.set_translation_for_key 'key', 'en', 'changed value'
assert_equal 'changed value', @twine_file.definitions_by_key['key'].translations['en']
assert_equal 'changed value', @strings.strings_map['key'].translations['en']
end
end
class SetComment < TwineTest
class SetComment < TwineTestCase
def setup
super
@twine_file = build_twine_file 'en' do
@strings = build_twine_file 'en' do
add_section 'Section' do
add_definition key: 'value'
add_row key: 'value'
end
end
end
def test_set_comment_for_key_does_not_update_comment
formatter = Twine::Formatters::Abstract.new
formatter.twine_file = @twine_file
formatter.strings = @strings
formatter.set_comment_for_key('key', 'comment')
assert_nil formatter.twine_file.definitions_by_key['key'].comment
assert_nil formatter.strings.strings_map['key'].comment
end
def test_set_comment_for_key_updates_comment_with_update_comments
formatter = Twine::Formatters::Abstract.new
formatter.twine_file = @twine_file
formatter.strings = @strings
formatter.options = { consume_comments: true }
formatter.set_comment_for_key('key', 'comment')
assert_equal 'comment', formatter.twine_file.definitions_by_key['key'].comment
assert_equal 'comment', formatter.strings.strings_map['key'].comment
end
end
class CommentReference < TwineTest
class CommentReference < TwineTestCase
def setup
super
@twine_file = build_twine_file 'en' do
@strings = build_twine_file 'en' do
add_section 'Section' do
add_definition refkey: 'ref-value', comment: 'reference comment'
add_definition key: 'value', ref: :refkey
add_row refkey: 'ref-value', comment: 'reference comment'
add_row key: 'value', ref: :refkey
end
end
@formatter = Twine::Formatters::Abstract.new
@formatter.twine_file = @twine_file
@formatter.strings = @strings
@formatter.options = { consume_comments: true }
end
def test_set_comment_does_not_add_unchanged_comment
@formatter.set_comment_for_key 'key', 'reference comment'
assert_nil @twine_file.definitions_by_key['key'].raw_comment
assert_nil @strings.strings_map['key'].raw_comment
end
def test_set_comment_adds_changed_comment
@formatter.set_comment_for_key 'key', 'changed comment'
assert_equal 'changed comment', @twine_file.definitions_by_key['key'].raw_comment
assert_equal 'changed comment', @strings.strings_map['key'].raw_comment
end
end

View file

@ -1,10 +1,10 @@
require 'twine_test'
require 'twine_test_case'
class CLITest < TwineTest
class CLITestCase < TwineTestCase
def setup
super()
super
@twine_file_path = File.join @output_dir, SecureRandom.uuid
@strings_file_path = File.join @output_dir, SecureRandom.uuid
@input_path = File.join @output_dir, SecureRandom.uuid
@input_dir = @output_dir
end
@ -13,455 +13,296 @@ class CLITest < TwineTest
@options = Twine::CLI::parse command.split
end
def parse_with(parameters)
raise "you need to implement `parse_with` in your test class"
class TestValidateStringsFile < CLITestCase
def test_command
parse "validate-strings-file #{@strings_file_path}"
assert_equal 'validate-strings-file', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
end
def test_pedantic
parse "validate-strings-file #{@strings_file_path} --pedantic"
assert @options[:pedantic]
end
def test_missing_parameter
assert_raises Twine::Error do
parse 'validate-strings-file'
end
end
def test_extra_parameter
assert_raises Twine::Error do
parse 'validate-strings-file strings extra'
end
end
end
def assert_help
parse_with '--help'
assert_equal @options, false
assert_match /Usage: twine.*Examples:/m, Twine::stdout.string
class TestGenerateStringFile < CLITestCase
def test_command
parse "generate-string-file #{@strings_file_path} #{@output_path}"
assert_equal 'generate-string-file', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
assert_equal @output_path, @options[:output_path]
end
def test_missing_parameter
assert_raises Twine::Error do
parse 'generate-string-file strings'
end
end
def test_validate
parse "generate-string-file #{@strings_file_path} #{@output_path} --validate"
assert @options[:validate]
end
def test_extra_parameter
assert_raises Twine::Error do
parse 'generate-string-file strings output extra'
end
end
def test_only_allows_one_language
assert_raises Twine::Error do
parse "generate-string-file strings output --lang en,fr"
end
end
end
def assert_option_consume_all
parse_with '--consume-all'
assert @options[:consume_all]
parse_with '--no-consume-all'
refute @options[:consume_all]
class TestGenerateAllStringFiles < CLITestCase
def test_command
parse "generate-all-string-files #{@strings_file_path} #{@output_dir}"
assert_equal 'generate-all-string-files', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
assert_equal @output_dir, @options[:output_path]
end
def test_missing_parameter
assert_raises Twine::Error do
parse "generate-all-string-files strings"
end
end
def test_validate
parse "generate-all-string-files #{@strings_file_path} #{@output_dir} --validate"
assert @options[:validate]
end
def test_extra_parameter
assert_raises Twine::Error do
parse "generate-all-string-files strings output extra"
end
end
end
def assert_option_consume_comments
parse_with '--consume-comments'
assert @options[:consume_comments]
parse_with '--no-consume-comments'
refute @options[:consume_comments]
class TestConsumeStringFile < CLITestCase
def test_command
parse "consume-string-file #{@strings_file_path} #{@input_path}"
assert_equal 'consume-string-file', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
assert_equal @input_path, @options[:input_path]
end
def test_missing_parameter
assert_raises Twine::Error do
parse "consume-string-file strings"
end
end
def test_extra_parameter
assert_raises Twine::Error do
parse "consume-string-file strings output extra"
end
end
def test_only_allows_one_language
assert_raises Twine::Error do
parse "consume-string-file strings output --lang en,fr"
end
end
end
def assert_option_developer_language
random_language = KNOWN_LANGUAGES.sample
parse_with "--developer-language #{random_language}"
assert_equal random_language, @options[:developer_language]
class TestConsumeAllStringFiles < CLITestCase
def test_command
parse "consume-all-string-files #{@strings_file_path} #{@input_dir}"
assert_equal 'consume-all-string-files', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
assert_equal @input_dir, @options[:input_path]
end
def test_missing_parameter
assert_raises Twine::Error do
parse "consume-all-string-files strings"
end
end
def test_extra_parameter
assert_raises Twine::Error do
parse "consume-all-string-files strings output extra"
end
end
end
def assert_option_encoding
parse_with '--encoding UTF16'
assert_equal 'UTF16', @options[:encoding]
class TestGenerateLocDrop < CLITestCase
def test_command
parse "generate-loc-drop #{@strings_file_path} #{@output_path} --format apple"
assert_equal 'generate-loc-drop', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
assert_equal @output_path, @options[:output_path]
end
def test_missing_parameter
assert_raises Twine::Error do
parse "generate-loc-drop strings --format apple"
end
end
def test_validate
parse "generate-loc-drop #{@strings_file_path} #{@output_path} --format apple --validate"
assert @options[:validate]
end
def test_extra_parameter
assert_raises Twine::Error do
parse "generate-loc-drop strings output extra --format apple"
end
end
def test_format_needed
assert_raises Twine::Error do
parse "generate-loc-drop strings output"
end
end
end
def assert_option_escape_all_tags
parse_with "--escape-all-tags"
assert @options[:escape_all_tags]
parse_with "--no-escape-all-tags"
refute @options[:escape_all_tags]
class TestConsumeLocDrop < CLITestCase
def test_command
parse "consume-loc-drop #{@strings_file_path} #{@input_path}"
assert_equal 'consume-loc-drop', @options[:command]
assert_equal @strings_file_path, @options[:strings_file]
assert_equal @input_path, @options[:input_path]
end
def test_missing_parameter
assert_raises Twine::Error do
parse "consume-loc-drop strings"
end
end
def test_extra_parameter
assert_raises Twine::Error do
parse "consume-loc-drop strings input extra"
end
end
end
def assert_option_format
random_format = Twine::Formatters.formatters.sample.format_name.downcase
parse_with "--format #{random_format}"
assert_equal random_format, @options[:format]
end
class TestParameters < CLITestCase
def parse_with(parameter)
parse 'validate-strings-file input.txt ' + parameter
end
def assert_option_include
random_set = [:all, :translated, :untranslated].sample
parse_with "--include #{random_set}"
assert_equal random_set, @options[:include]
end
def test_default_options
parse_with ''
expected = {command: 'validate-strings-file', strings_file: 'input.txt', include: "all"}
assert_equal expected, @options
end
def assert_option_single_language
random_language = KNOWN_LANGUAGES.sample
parse_with "--lang #{random_language}"
assert_equal [random_language], @options[:languages]
end
def test_create_folders
parse_with '--create-folders'
assert @options[:create_folders]
end
def assert_option_multiple_languages
random_languages = KNOWN_LANGUAGES.shuffle[0, 3]
parse_with "--lang #{random_languages.join(',')}"
assert_equal random_languages.sort, @options[:languages].sort
end
def test_consume_all
parse_with '--consume-all'
assert @options[:consume_all]
end
def assert_option_languages
assert_option_single_language
assert_option_multiple_languages
end
def test_consume_comments
parse_with '--consume-comments'
assert @options[:consume_comments]
end
def assert_option_output_path
parse_with "--output-file #{@output_path}"
assert_equal @output_path, @options[:output_path]
end
def test_untagged
parse_with '--untagged'
assert @options[:untagged]
end
def assert_option_quiet
parse_with '--quiet'
assert @options[:quiet]
parse_with '--no-quiet'
refute @options[:quiet]
end
def test_developer_language
random_language = KNOWN_LANGUAGES.sample
parse_with "--developer-lang #{random_language}"
assert_equal random_language, @options[:developer_language]
end
def assert_option_tags
# single tag
random_tag = "tag#{rand(100)}"
parse_with "--tags #{random_tag}"
assert_equal [[random_tag]], @options[:tags]
def test_single_language
random_language = KNOWN_LANGUAGES.sample
parse_with "--lang #{random_language}"
assert_equal [random_language], @options[:languages]
end
# multiple OR tags
random_tags = ["tag#{rand(100)}", "tag#{rand(100)}", "tag#{rand(100)}"]
parse_with "--tags #{random_tags.join(',')}"
sorted_tags = @options[:tags].map { |tags| tags.sort }
assert_equal [random_tags.sort], sorted_tags
def test_multiple_languages
random_languages = KNOWN_LANGUAGES.shuffle[0, 3]
parse_with "--lang #{random_languages.join(',')}"
assert_equal random_languages.sort, @options[:languages].sort
end
# multiple AND tags
random_tag_1 = "tag#{rand(100)}"
random_tag_2 = "tag#{rand(100)}"
parse_with "--tags #{random_tag_1} --tags #{random_tag_2}"
assert_equal [[random_tag_1], [random_tag_2]], @options[:tags]
# NOT tag
random_tag = "~tag#{rand(100)}"
parse_with "--tags #{random_tag}"
assert_equal [[random_tag]], @options[:tags]
end
def test_single_tag
random_tag = "tag#{rand(100)}"
parse_with "--tags #{random_tag}"
assert_equal [random_tag], @options[:tags]
end
def assert_option_untagged
parse_with '--untagged'
assert @options[:untagged]
parse_with '--no-untagged'
refute @options[:untagged]
end
def test_multiple_tags
random_tags = ([nil] * 3).map { "tag#{rand(100)}" }
parse_with "--tags #{random_tags.join(',')}"
assert_equal random_tags.sort, @options[:tags].sort
end
def assert_option_validate
parse_with "--validate"
assert @options[:validate]
parse_with "--no-validate"
refute @options[:validate]
end
end
def test_format
random_format = Twine::Formatters.formatters.sample.format_name
parse_with "--format #{random_format}"
assert_equal random_format, @options[:format]
end
class TestCLI < CLITest
def test_version
parse "--version"
def test_format_ignores_case
random_format = Twine::Formatters.formatters.sample.format_name
parse_with "--format #{random_format.upcase}"
assert_equal random_format, @options[:format]
end
assert_equal @options, false
assert_equal "Twine version #{Twine::VERSION}\n", Twine::stdout.string
end
def test_include
random_set = ['all', 'translated', 'untranslated'].sample
parse_with "--include #{random_set}"
assert_equal random_set, @options[:include]
end
def test_help
parse ""
assert_match 'Usage: twine', Twine::stdout.string
end
def test_include_ignores_case
random_set = ['all', 'translated', 'untranslated'].sample
parse_with "--include #{random_set.upcase}"
assert_equal random_set, @options[:include]
end
def test_invalid_command
assert_raises Twine::Error do
parse "not a command"
def test_output_path
parse_with "--output-file #{@output_path}"
assert_equal @output_path, @options[:output_path]
end
def test_file_name
random_filename = "#{rand(10000)}"
parse_with "--file-name #{random_filename}"
assert_equal random_filename, @options[:file_name]
end
def test_encoding
parse_with '--encoding UTF16'
assert_equal 'UTF16', @options[:output_encoding]
end
end
end
class TestGenerateLocalizationFileCLI < CLITest
def parse_with(parameters)
parse "generate-localization-file #{@twine_file_path} #{@output_path} " + parameters
end
def test_command
parse_with ""
assert_equal 'generate-localization-file', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
assert_equal @output_path, @options[:output_path]
end
def test_missing_argument
assert_raises Twine::Error do
parse "generate-localization-file #{@twine_file_path}"
end
end
def test_extra_argument
assert_raises Twine::Error do
parse_with "extra"
end
end
def test_options
assert_help
assert_option_developer_language
assert_option_encoding
assert_option_escape_all_tags
assert_option_format
assert_option_include
assert_option_single_language
assert_raises(Twine::Error) { assert_option_multiple_languages }
assert_option_quiet
assert_option_tags
assert_option_untagged
assert_option_validate
end
end
class TestGenerateAllLocalizationFilesCLI < CLITest
def parse_with(parameters)
parse "generate-all-localization-files #{@twine_file_path} #{@output_dir} " + parameters
end
def test_command
parse_with ""
assert_equal 'generate-all-localization-files', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
assert_equal @output_dir, @options[:output_path]
end
def test_missing_argument
assert_raises Twine::Error do
parse "generate-all-localization-files twine_file"
end
end
def test_extra_arguemnt
assert_raises Twine::Error do
parse_with "extra"
end
end
def test_options
assert_help
assert_option_developer_language
assert_option_encoding
assert_option_escape_all_tags
assert_option_format
assert_option_include
assert_option_quiet
assert_option_tags
assert_option_untagged
assert_option_validate
end
def test_option_create_folders
parse_with '--create-folders'
assert @options[:create_folders]
parse_with '--no-create-folders'
refute @options[:create_folders]
end
def test_option_file_name
random_filename = "#{rand(10000)}"
parse_with "--file-name #{random_filename}"
assert_equal random_filename, @options[:file_name]
end
end
class TestGenerateLocalizationArchiveCLI < CLITest
def parse_with(parameters)
parse "generate-localization-archive #{@twine_file_path} #{@output_path} --format apple " + parameters
end
def test_command
parse_with ""
assert_equal 'generate-localization-archive', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
assert_equal @output_path, @options[:output_path]
end
def test_missing_argument
assert_raises Twine::Error do
parse "generate-localization-archive twine_file --format apple"
end
end
def test_extra_argument
assert_raises Twine::Error do
parse_with "extra"
end
end
def test_options
assert_help
assert_option_developer_language
assert_option_encoding
assert_option_escape_all_tags
assert_option_include
assert_option_quiet
assert_option_tags
assert_option_untagged
assert_option_validate
end
def test_option_format_required
assert_raises Twine::Error do
parse "generate-localization-archive twine_file output"
end
end
def test_supports_deprecated_command
parse "generate-loc-drop #{@twine_file_path} #{@output_path} --format apple"
assert_equal 'generate-localization-archive', @options[:command]
end
def test_deprecated_command_prints_warning
parse "generate-loc-drop #{@twine_file_path} #{@output_path} --format apple"
assert_match "WARNING: Twine commands names have changed.", Twine::stdout.string
end
end
class TestConsumeLocalizationFileCLI < CLITest
def parse_with(parameters)
parse "consume-localization-file #{@twine_file_path} #{@input_path} " + parameters
end
def test_command
parse_with ""
assert_equal 'consume-localization-file', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
assert_equal @input_path, @options[:input_path]
end
def test_missing_argument
assert_raises Twine::Error do
parse "consume-localization-file twine_file"
end
end
def test_extra_argument
assert_raises Twine::Error do
parse_with "extra"
end
end
def test_options
assert_help
assert_option_consume_all
assert_option_consume_comments
assert_option_developer_language
assert_option_encoding
assert_option_format
assert_option_single_language
assert_raises(Twine::Error) { assert_option_multiple_languages }
assert_option_output_path
assert_option_quiet
assert_option_tags
end
end
class TestConsumeAllLocalizationFilesCLI < CLITest
def parse_with(parameters)
parse "consume-all-localization-files #{@twine_file_path} #{@input_dir} " + parameters
end
def test_command
parse_with ""
assert_equal 'consume-all-localization-files', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
assert_equal @input_dir, @options[:input_path]
end
def test_missing_argument
assert_raises Twine::Error do
parse "consume-all-localization-files twine_file"
end
end
def test_extra_argument
assert_raises Twine::Error do
parse_with "extra"
end
end
def test_options
assert_help
assert_option_consume_all
assert_option_consume_comments
assert_option_developer_language
assert_option_encoding
assert_option_format
assert_option_output_path
assert_option_quiet
assert_option_tags
end
end
class TestConsumeLocalizationArchiveCLI < CLITest
def parse_with(parameters)
parse "consume-localization-archive #{@twine_file_path} #{@input_path} " + parameters
end
def test_command
parse_with ""
assert_equal 'consume-localization-archive', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
assert_equal @input_path, @options[:input_path]
end
def test_missing_argument
assert_raises Twine::Error do
parse "consume-localization-archive twine_file"
end
end
def test_extra_argument
assert_raises Twine::Error do
parse_with "extra"
end
end
def test_options
assert_help
assert_option_consume_all
assert_option_consume_comments
assert_option_developer_language
assert_option_encoding
assert_option_format
assert_option_output_path
assert_option_quiet
assert_option_tags
end
def test_supports_deprecated_command
parse "consume-loc-drop #{@twine_file_path} #{@input_path}"
assert_equal 'consume-localization-archive', @options[:command]
end
def test_deprecated_command_prints_warning
parse "consume-loc-drop #{@twine_file_path} #{@input_path}"
assert_match "WARNING: Twine commands names have changed.", Twine::stdout.string
end
end
class TestValidateTwineFileCLI < CLITest
def parse_with(parameters)
parse "validate-twine-file #{@twine_file_path} " + parameters
end
def test_command
parse_with ""
assert_equal 'validate-twine-file', @options[:command]
assert_equal @twine_file_path, @options[:twine_file]
end
def test_missing_argument
assert_raises Twine::Error do
parse 'validate-twine-file'
end
end
def test_extra_argument
assert_raises Twine::Error do
parse_with 'extra'
end
end
def test_options
assert_help
assert_option_developer_language
assert_option_quiet
end
def test_option_pedantic
parse "validate-twine-file #{@twine_file_path} --pedantic"
assert @options[:pedantic]
parse "validate-twine-file #{@twine_file_path} --no-pedantic"
refute @options[:pedantic]
end
end

View file

@ -0,0 +1,27 @@
require 'command_test_case'
class TestConsumeLocDrop < CommandTestCase
def setup
super
options = {}
options[:input_path] = fixture 'consume_loc_drop.zip'
options[:output_path] = @output_path
options[:format] = 'apple'
@twine_file = build_twine_file 'en', 'es' do
add_section 'Section' do
add_row key1: 'value1'
end
end
@runner = Twine::Runner.new(options, @twine_file)
end
def test_consumes_zip_file
@runner.consume_loc_drop
assert @twine_file.strings_map['key1'].translations['en'], 'value1-english'
assert @twine_file.strings_map['key1'].translations['es'], 'value1-spanish'
end
end

View file

@ -1,33 +0,0 @@
require 'command_test'
class TestConsumeLocalizationArchive < CommandTest
def setup
super
@twine_file = build_twine_file 'en', 'es' do
add_section 'Section' do
add_definition key1: 'value1'
end
end
end
def new_runner(options = {})
options[:input_path] = fixture_path 'consume_localization_archive.zip'
options[:output_path] = @output_path
Twine::Runner.new(options, @twine_file)
end
def test_consumes_zip_file
new_runner(format: 'android').consume_localization_archive
assert @twine_file.definitions_by_key['key1'].translations['en'], 'value1-english'
assert @twine_file.definitions_by_key['key1'].translations['es'], 'value1-spanish'
end
def test_raises_error_if_format_ambiguous
assert_raises Twine::Error do
new_runner.consume_localization_archive
end
end
end

View file

@ -1,119 +0,0 @@
require 'command_test'
class TestConsumeLocalizationFile < CommandTest
def new_runner(language, file)
options = {}
options[:output_path] = File.join(@output_dir, file) if file
options[:input_path] = File.join(@output_dir, file) if file
FileUtils.touch options[:input_path]
options[:languages] = language if language
@twine_file = Twine::TwineFile.new
@twine_file.language_codes.concat KNOWN_LANGUAGES
Twine::Runner.new(options, @twine_file)
end
def prepare_mock_read_formatter(formatter_class)
formatter = prepare_mock_formatter(formatter_class)
formatter.expects(:read)
end
def test_deducts_android_format_from_output_path
prepare_mock_read_formatter Twine::Formatters::Android
new_runner('fr', 'fr.xml').consume_localization_file
end
def test_deducts_apple_format_from_output_path
prepare_mock_read_formatter Twine::Formatters::Apple
new_runner('fr', 'fr.strings').consume_localization_file
end
def test_deducts_jquery_format_from_output_path
prepare_mock_read_formatter Twine::Formatters::JQuery
new_runner('fr', 'fr.json').consume_localization_file
end
def test_deducts_gettext_format_from_output_path
prepare_mock_read_formatter Twine::Formatters::Gettext
new_runner('fr', 'fr.po').consume_localization_file
end
def test_deducts_language_from_input_path
random_language = KNOWN_LANGUAGES.sample
formatter = prepare_mock_formatter Twine::Formatters::Android
formatter.expects(:read).with(anything, random_language)
new_runner(nil, "#{random_language}.xml").consume_localization_file
end
class TestEncodings < CommandTest
class DummyFormatter < Twine::Formatters::Abstract
attr_reader :content
def extension
'.dummy'
end
def format_name
'dummy'
end
def read(io, lang)
@content = io.read
end
end
def new_runner(input_path, encoding = nil)
options = {}
options[:output_path] = @output_path
options[:input_path] = input_path
options[:encoding] = encoding if encoding
options[:languages] = 'en'
@twine_file = Twine::TwineFile.new
@twine_file.language_codes.concat KNOWN_LANGUAGES
Twine::Runner.new(options, @twine_file)
end
def setup
super
@expected_content = "Üß`\nda\n"
end
def test_reads_utf8
formatter = prepare_mock_formatter DummyFormatter
new_runner(fixture_path('enc_utf8.dummy')).consume_localization_file
assert_equal @expected_content, formatter.content
end
def test_reads_utf16le_bom
formatter = prepare_mock_formatter DummyFormatter
new_runner(fixture_path('enc_utf16le_bom.dummy')).consume_localization_file
assert_equal @expected_content, formatter.content
end
def test_reads_utf16be_bom
formatter = prepare_mock_formatter DummyFormatter
new_runner(fixture_path('enc_utf16be_bom.dummy')).consume_localization_file
assert_equal @expected_content, formatter.content
end
def test_reads_utf16le
formatter = prepare_mock_formatter DummyFormatter
new_runner(fixture_path('enc_utf16le.dummy'), 'UTF-16LE').consume_localization_file
assert_equal @expected_content, formatter.content
end
def test_reads_utf16be
formatter = prepare_mock_formatter DummyFormatter
new_runner(fixture_path('enc_utf16be.dummy'), 'UTF-16BE').consume_localization_file
assert_equal @expected_content, formatter.content
end
end
end

View file

@ -0,0 +1,53 @@
require 'command_test_case'
class TestConsumeStringFile < CommandTestCase
def new_runner(language, file)
options = {}
options[:output_path] = File.join(@output_dir, file) if file
options[:input_path] = File.join(@output_dir, file) if file
FileUtils.touch options[:input_path]
options[:languages] = language if language
@strings = Twine::StringsFile.new
@strings.language_codes.concat KNOWN_LANGUAGES
Twine::Runner.new(options, @strings)
end
def prepare_mock_read_file_formatter(formatter_class)
formatter = prepare_mock_formatter(formatter_class)
formatter.expects(:read_file)
end
def test_deducts_android_format_from_output_path
prepare_mock_read_file_formatter Twine::Formatters::Android
new_runner('fr', 'fr.xml').consume_string_file
end
def test_deducts_apple_format_from_output_path
prepare_mock_read_file_formatter Twine::Formatters::Apple
new_runner('fr', 'fr.strings').consume_string_file
end
def test_deducts_jquery_format_from_output_path
prepare_mock_read_file_formatter Twine::Formatters::JQuery
new_runner('fr', 'fr.json').consume_string_file
end
def test_deducts_gettext_format_from_output_path
prepare_mock_read_file_formatter Twine::Formatters::Gettext
new_runner('fr', 'fr.po').consume_string_file
end
def test_deducts_language_from_input_path
random_language = KNOWN_LANGUAGES.sample
formatter = prepare_mock_formatter Twine::Formatters::Android
formatter.expects(:read_file).with(anything, random_language)
new_runner(nil, "#{random_language}.xml").consume_string_file
end
end

View file

@ -1,256 +1,77 @@
require 'twine_test'
require 'twine_test_case'
class FormatterTest < TwineTest
class FormatterTest < TwineTestCase
def setup(formatter_class)
super()
@twine_file = build_twine_file 'en' do
add_section 'Section 1' do
add_definition key1: 'value1-english', comment: 'comment key1'
add_definition key2: 'value2-english'
add_row key1: 'value1-english', comment: 'comment key1'
add_row key2: 'value2-english'
end
add_section 'Section 2' do
add_definition key3: 'value3-english'
add_definition key4: 'value4-english', comment: 'comment key4'
add_row key3: 'value3-english'
add_row key4: 'value4-english', comment: 'comment key4'
end
end
@empty_twine_file = Twine::TwineFile.new
@strings = Twine::StringsFile.new
@formatter = formatter_class.new
@formatter.twine_file = @empty_twine_file
@formatter.strings = @strings
@formatter.options = { consume_all: true, consume_comments: true }
end
def assert_translations_read_correctly
1.upto(4) do |i|
assert_equal "value#{i}-english", @empty_twine_file.definitions_by_key["key#{i}"].translations['en']
assert_equal "value#{i}-english", @strings.strings_map["key#{i}"].translations['en']
end
end
def assert_file_contents_read_correctly
assert_translations_read_correctly
assert_equal "comment key1", @empty_twine_file.definitions_by_key["key1"].comment
assert_equal "comment key4", @empty_twine_file.definitions_by_key["key4"].comment
assert_equal "comment key1", @strings.strings_map["key1"].comment
assert_equal "comment key4", @strings.strings_map["key4"].comment
end
end
class TestAndroidFormatter < FormatterTest
def setup
super Twine::Formatters::Android
@escape_test_values = {
'this & that' => 'this &amp; that',
'this < that' => 'this &lt; that',
"it's complicated" => "it\\'s complicated",
'a "good" way' => 'a \"good\" way',
'<b>bold</b>' => '<b>bold</b>',
'<em>bold</em>' => '<em>bold</em>',
'<i>italic</i>' => '<i>italic</i>',
'<cite>italic</cite>' => '<cite>italic</cite>',
'<dfn>italic</dfn>' => '<dfn>italic</dfn>',
'<big>larger</big>' => '<big>larger</big>',
'<small>smaller</small>' => '<small>smaller</small>',
'<font color="#45C1D0">F</font>' => '<font color="#45C1D0">F</font>',
'<tt>monospaced</tt>' => '<tt>monospaced</tt>',
'<s>strike</s>' => '<s>strike</s>',
'<strike>strike</strike>' => '<strike>strike</strike>',
'<del>strike</del>' => '<del>strike</del>',
'<u>underline</u>' => '<u>underline</u>',
'<super>superscript</super>'=> '<super>superscript</super>',
'<sub>subscript</sub>' => '<sub>subscript</sub>',
'<ul>bullet point</ul>' => '<ul>bullet point</ul>',
'<li>bullet point</li>' => '<li>bullet point</li>',
'<br>line break' => '<br>line break',
'<div>division</div>' => '<div>division</div>',
'<span style="color:#45C1D0">inline</span>' => '<span style="color:#45C1D0">inline</span>',
'<p>para</p>' => '<p>para</p>',
'<p dir="ltr">para</p>' => '<p dir="ltr">para</p>',
'<b>%@</b>' => '&lt;b>%s&lt;/b>',
'<em>%@</em>' => '&lt;em>%s&lt;/em>',
'<i>%@</i>' => '&lt;i>%s&lt;/i>',
'<cite>%@</cite>' => '&lt;cite>%s&lt;/cite>',
'<dfn>%@</dfn>' => '&lt;dfn>%s&lt;/dfn>',
'<big>%@</big>' => '&lt;big>%s&lt;/big>',
'<small>%@</small>' => '&lt;small>%s&lt;/small>',
'<font color="#45C1D0>%@</font>' => '&lt;font color="#45C1D0>%s&lt;/font>',
'<tt>%@</tt>' => '&lt;tt>%s&lt;/tt>',
'<s>%@</s>' => '&lt;s>%s&lt;/s>',
'<strike>%@</strike>' => '&lt;strike>%s&lt;/strike>',
'<del>%@</del>' => '&lt;del>%s&lt;/del>',
'<u>%@</u>' => '&lt;u>%s&lt;/u>',
'<super>%@</super>' => '&lt;super>%s&lt;/super>',
'<sub>%@</sub>' => '&lt;sub>%s&lt;/sub>',
'<ul>%@</ul>' => '&lt;ul>%s&lt;/ul>',
'<li>%@</li>' => '&lt;li>%s&lt;/li>',
'<br>%@' => '&lt;br>%s',
'<div>%@</div>' => '&lt;div>%s&lt;/div>',
'<span style="color:#45C1D0">%@</span>' => '&lt;span style="color:#45C1D0">%s&lt;/span>',
'<p>%@</p>' => '&lt;p>%s&lt;/p>',
'<p dir="ltr">%@</p>' => '&lt;p dir="ltr">%s&lt;/p>',
'<a href="target">link</a>' => '<a href="target">link</a>',
'<a href="target">"link"</a>' => '<a href="target">\"link\"</a>',
'<a href="target"></a>"out"' => '<a href="target"></a>\"out\"',
'<a href="http://url.com?param=1&param2=3&param3=%20">link</a>' => '<a href="http://url.com?param=1&param2=3&param3=%20">link</a>',
'<q>escaped</q><![CDATA[]]>' => '&lt;q>escaped&lt;/q><![CDATA[]]>',
'<![CDATA[]]><q>escaped</q>' => '<![CDATA[]]>&lt;q>escaped&lt;/q>',
'<![CDATA[<q>unescaped</q>]]>' => '<![CDATA[<q>unescaped</q>]]>',
'<![CDATA[<q>unescaped with %@</q>]]>' => '<![CDATA[<q>unescaped with %s</q>]]>',
'<![CDATA[]]><![CDATA[<q>unescaped</q>]]>' => '<![CDATA[]]><![CDATA[<q>unescaped</q>]]>',
'<![CDATA[&]]>' => '<![CDATA[&]]>',
'<![CDATA[\']]>' => '<![CDATA[\']]>',
'<![CDATA["]]>' => '<![CDATA["]]>',
'<xliff:g></xliff:g>' => '<xliff:g></xliff:g>',
'<xliff:g>untouched</xliff:g>' => '<xliff:g>untouched</xliff:g>',
'<xliff:g id="42">untouched</xliff:g>' => '<xliff:g id="42">untouched</xliff:g>',
'<xliff:g id="1">first</xliff:g> inbetween <xliff:g id="2">second</xliff:g>' => '<xliff:g id="1">first</xliff:g> inbetween <xliff:g id="2">second</xliff:g>'
}
@escape_all_test_values = {
'<b>bold</b>' => '&lt;b>bold&lt;/b>',
'<i>italic</i>' => '&lt;i>italic&lt;/i>',
'<u>underline</u>' => '&lt;u>underline&lt;/u>'
}
end
def test_read_format
@formatter.read content_io('formatter_android.xml'), 'en'
def test_read_file_format
@formatter.read_file fixture('formatter_android.xml'), 'en'
assert_file_contents_read_correctly
end
def test_read_multiline_translation
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="foo">This is
a string</string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'This is\n a string', @empty_twine_file.definitions_by_key["foo"].translations['en']
end
def test_read_multiline_comment
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<!-- multiline
comment -->
<string name="foo">This is
a string</string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'multiline comment', @empty_twine_file.definitions_by_key["foo"].comment
end
def test_read_html_tags
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="foo">Hello, <b>BOLD</b></string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'Hello, <b>BOLD</b>', @empty_twine_file.definitions_by_key["foo"].translations['en']
end
def test_double_quotes_are_not_modified
content = <<-EOCONTENT
<?xml version="1.0" encoding="utf-8"?>
<resources>
<string name="foo">Hello, <a href="http://www.foo.com">BOLD</a></string>
</resources>
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_equal 'Hello, <a href="http://www.foo.com">BOLD</a>', @empty_twine_file.definitions_by_key["foo"].translations['en']
end
def test_set_translation_converts_leading_spaces
@formatter.set_translation_for_key 'key1', 'en', "\u0020value"
assert_equal ' value', @empty_twine_file.definitions_by_key['key1'].translations['en']
assert_equal ' value', @strings.strings_map['key1'].translations['en']
end
def test_set_translation_coverts_trailing_spaces
@formatter.set_translation_for_key 'key1', 'en', "value\u0020\u0020"
assert_equal 'value ', @empty_twine_file.definitions_by_key['key1'].translations['en']
assert_equal 'value ', @strings.strings_map['key1'].translations['en']
end
def test_set_translation_converts_string_placeholders
@formatter.set_translation_for_key 'key1', 'en', "value %s"
assert_equal 'value %@', @empty_twine_file.definitions_by_key['key1'].translations['en']
assert_equal 'value %@', @strings.strings_map['key1'].translations['en']
end
def test_set_translation_unescapes_at_signs
@formatter.set_translation_for_key 'key1', 'en', '\@value'
assert_equal '@value', @empty_twine_file.definitions_by_key['key1'].translations['en']
assert_equal '@value', @strings.strings_map['key1'].translations['en']
end
def test_set_translation_unescaping
@escape_test_values.each do |expected, input|
@formatter.set_translation_for_key 'key1', 'en', input
assert_equal expected, @empty_twine_file.definitions_by_key['key1'].translations['en']
end
@escape_all_test_values.each do |expected, input|
@formatter.set_translation_for_key 'key1', 'en', input
assert_equal expected, @empty_twine_file.definitions_by_key['key1'].translations['en']
end
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::Android.new
formatter.twine_file = @twine_file
assert_equal content('formatter_android.xml'), formatter.format_file('en')
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_android.xml'), output_content
end
def test_format_key_with_space
@ -265,19 +86,10 @@ class TestAndroidFormatter < FormatterTest
assert_equal "value\\u0020", @formatter.format_value('value ')
end
def test_format_value_string_placeholder
assert_equal "The file %s could not be found.", @formatter.format_value("The file %@ could not be found.")
end
def test_format_value_escaping
@escape_test_values.each do |input, expected|
assert_equal expected, @formatter.format_value(input)
end
@formatter.options.merge!({ escape_all_tags: true })
@escape_all_test_values.each do |input, expected|
assert_equal expected, @formatter.format_value(input)
end
def test_format_value_escapes_single_quotes
skip 'not working with ruby 2.0'
# http://stackoverflow.com/questions/18735608/cgiescapehtml-is-escaping-single-quote
assert_equal "not \\'so\\' easy", @formatter.format_value("not 'so' easy")
end
def test_format_value_escapes_non_resource_identifier_at_signs
@ -288,50 +100,6 @@ class TestAndroidFormatter < FormatterTest
identifier = '@android:string/cancel'
assert_equal identifier, @formatter.format_value(identifier)
end
def test_deducts_language_from_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("#{language}.xml")
end
def test_recognize_every_twine_language_from_filename
twine_file = build_twine_file "not-a-lang-code" do
add_section "Section" do
add_definition key: "value"
end
end
@formatter.twine_file = twine_file
assert_equal "not-a-lang-code", @formatter.determine_language_given_path("not-a-lang-code.xml")
end
def test_deducts_language_from_resource_folder
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("res/values-#{language}")
end
def test_deducts_language_and_region_from_resource_folder
assert_equal 'de-AT', @formatter.determine_language_given_path("res/values-de-rAT")
end
def test_does_not_deduct_language_from_device_capability_resource_folder
assert_nil @formatter.determine_language_given_path('res/values-w820dp')
end
def test_output_path_is_prefixed
assert_equal 'values-en', @formatter.output_path_for_language('en')
end
def test_output_path_with_region
assert_equal 'values-en-rGB', @formatter.output_path_for_language('en-GB')
end
def test_output_path_respects_default_lang
@formatter.twine_file.language_codes.concat KNOWN_LANGUAGES
non_default_language = KNOWN_LANGUAGES[1..-1].sample
assert_equal 'values', @formatter.output_path_for_language(KNOWN_LANGUAGES[0])
assert_equal "values-#{non_default_language}", @formatter.output_path_for_language(non_default_language)
end
end
class TestAppleFormatter < FormatterTest
@ -339,82 +107,17 @@ class TestAppleFormatter < FormatterTest
super Twine::Formatters::Apple
end
def test_read_format
@formatter.read content_io('formatter_apple.strings'), 'en'
def test_read_file_format
@formatter.read_file fixture('formatter_apple.strings'), 'en'
assert_file_contents_read_correctly
end
def test_deducts_language_from_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("#{language}.strings")
end
def test_recognize_every_twine_language_from_filename
twine_file = build_twine_file "not-a-lang-code" do
add_section "Section" do
add_definition key: "value"
end
end
@formatter.twine_file = twine_file
assert_equal "not-a-lang-code", @formatter.determine_language_given_path("not-a-lang-code.strings")
end
def test_deducts_language_from_resource_folder
language = %w(en de fr).sample
assert_equal language, @formatter.determine_language_given_path("#{language}.lproj/Localizable.strings")
end
def test_deducts_base_language_from_resource_folder
@formatter.options = { consume_all: true, consume_comments: true, developer_language: 'en' }
assert_equal 'en', @formatter.determine_language_given_path('Base.lproj/Localizations.strings')
end
def test_reads_quoted_keys
@formatter.read StringIO.new('"key" = "value"'), 'en'
assert_equal 'value', @empty_twine_file.definitions_by_key['key'].translations['en']
end
def test_reads_unquoted_keys
@formatter.read StringIO.new('key = "value"'), 'en'
assert_equal 'value', @empty_twine_file.definitions_by_key['key'].translations['en']
end
def test_ignores_leading_whitespace_before_quoted_keys
@formatter.read StringIO.new("\t \"key\" = \"value\""), 'en'
assert_equal 'value', @empty_twine_file.definitions_by_key['key'].translations['en']
end
def test_ignores_leading_whitespace_before_unquoted_keys
@formatter.read StringIO.new("\t key = \"value\""), 'en'
assert_equal 'value', @empty_twine_file.definitions_by_key['key'].translations['en']
end
def test_allows_quotes_in_quoted_keys
@formatter.read StringIO.new('"ke\"y" = "value"'), 'en'
assert_equal 'value', @empty_twine_file.definitions_by_key['ke"y'].translations['en']
end
def test_does_not_allow_quotes_in_quoted_keys
@formatter.read StringIO.new('ke"y = "value"'), 'en'
assert_nil @empty_twine_file.definitions_by_key['key']
end
def test_allows_equal_signs_in_quoted_keys
@formatter.read StringIO.new('"k=ey" = "value"'), 'en'
assert_equal 'value', @empty_twine_file.definitions_by_key['k=ey'].translations['en']
end
def test_does_not_allow_equal_signs_in_unquoted_keys
@formatter.read StringIO.new('k=ey = "value"'), 'en'
assert_nil @empty_twine_file.definitions_by_key['key']
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::Apple.new
formatter.twine_file = @twine_file
assert_equal content('formatter_apple.strings'), formatter.format_file('en')
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_apple.strings'), output_content
end
def test_format_key_with_space
@ -436,94 +139,49 @@ class TestJQueryFormatter < FormatterTest
super Twine::Formatters::JQuery
end
def test_read_format
@formatter.read content_io('formatter_jquery.json'), 'en'
def test_read_file_format
@formatter.read_file fixture('formatter_jquery.json'), 'en'
assert_translations_read_correctly
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::JQuery.new
formatter.twine_file = @twine_file
assert_equal content('formatter_jquery.json'), formatter.format_file('en')
end
def test_empty_sections_are_removed
@twine_file = build_twine_file 'en' do
add_section 'Section 1' do
end
add_section 'Section 2' do
add_definition key: 'value'
end
end
formatter = Twine::Formatters::JQuery.new
formatter.twine_file = @twine_file
refute_includes formatter.format_file('en'), ','
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_jquery.json'), output_content
end
def test_format_value_with_newline
assert_equal "value\nwith\nline\nbreaks", @formatter.format_value("value\nwith\nline\nbreaks")
end
def test_deducts_language_from_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("#{language}.json")
end
def test_deducts_language_from_extended_filename
language = KNOWN_LANGUAGES.sample
assert_equal language, @formatter.determine_language_given_path("something-#{language}.json")
end
def test_deducts_language_from_path
language = %w(en-GB de fr).sample
assert_equal language, @formatter.determine_language_given_path("/output/#{language}/#{@formatter.default_file_name}")
end
end
class TestGettextFormatter < FormatterTest
def setup
super Twine::Formatters::Gettext
end
def test_read_format
@formatter.read content_io('formatter_gettext.po'), 'en'
def test_read_file_format
@formatter.read_file fixture('formatter_gettext.po'), 'en'
assert_file_contents_read_correctly
end
def test_read_with_multiple_line_value
@formatter.read content_io('gettext_multiline.po'), 'en'
def test_read_file_with_multiple_line_value
@formatter.read_file fixture('gettext_multiline.po'), 'en'
assert_equal 'multiline\nstring', @empty_twine_file.definitions_by_key['key1'].translations['en']
assert_equal 'multiline\nstring', @strings.strings_map['key1'].translations['en']
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::Gettext.new
formatter.twine_file = @twine_file
assert_equal content('formatter_gettext.po'), formatter.format_file('en')
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_gettext.po'), output_content
end
def test_deducts_language_and_region
language = "en-GB"
assert_equal language, @formatter.determine_language_given_path("#{language}.po")
end
def test_deducts_language_from_path
language = %w(en-GB de fr).sample
assert_equal language, @formatter.determine_language_given_path("/output/#{language}/#{@formatter.default_file_name}")
end
def test_quoted_strings
formatter = Twine::Formatters::Gettext.new
formatter.twine_file = build_twine_file "not-a-lang-code" do
add_section "Section" do
add_definition key: "foo \"bar\" baz"
end
end
assert_equal content('formatter_gettext_quotes.po'), formatter.format_file('en')
end
end
class TestTizenFormatter < FormatterTest
@ -532,18 +190,20 @@ class TestTizenFormatter < FormatterTest
super Twine::Formatters::Tizen
end
def test_read_format
skip 'the current implementation of Tizen formatter does not support reading'
@formatter.read content_io('formatter_tizen.xml'), 'en'
def test_read_file_format
skip 'the current implementation of Tizen formatter does not support read_file'
@formatter.read_file fixture('formatter_tizen.xml'), 'en'
assert_file_contents_read_correctly
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::Tizen.new
formatter.twine_file = @twine_file
assert_equal content('formatter_tizen.xml'), formatter.format_file('en')
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_tizen.xml'), output_content
end
end
class TestDjangoFormatter < FormatterTest
@ -551,39 +211,17 @@ class TestDjangoFormatter < FormatterTest
super Twine::Formatters::Django
end
def test_read_format
@formatter.read content_io('formatter_django.po'), 'en'
def test_read_file_format
@formatter.read_file fixture('formatter_django.po'), 'en'
assert_file_contents_read_correctly
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::Django.new
formatter.twine_file = @twine_file
assert_equal content('formatter_django.po'), formatter.format_file('en')
end
def test_deducts_language_and_region
language = "en-GB"
assert_equal language, @formatter.determine_language_given_path("#{language}.po")
end
def test_deducts_language_from_path
language = %w(en-GB de fr).sample
assert_equal language, @formatter.determine_language_given_path("/output/#{language}/#{@formatter.default_file_name}")
end
def test_ignores_commented_out_strings
content = <<-EOCONTENT
#~ msgid "foo"
#~ msgstr "This should be ignored"
EOCONTENT
io = StringIO.new(content)
@formatter.read io, 'en'
assert_nil @empty_twine_file.definitions_by_key["foo"]
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_django.po'), output_content
end
end
@ -592,33 +230,16 @@ class TestFlashFormatter < FormatterTest
super Twine::Formatters::Flash
end
def test_read_format
@formatter.read content_io('formatter_flash.properties'), 'en'
def test_read_file_format
@formatter.read_file fixture('formatter_flash.properties'), 'en'
assert_file_contents_read_correctly
end
def test_set_translation_converts_placeholders
@formatter.set_translation_for_key 'key1', 'en', "value {#{rand(10)}}"
assert_equal 'value %@', @empty_twine_file.definitions_by_key['key1'].translations['en']
end
def test_format_file
def test_write_file_output_format
formatter = Twine::Formatters::Flash.new
formatter.twine_file = @twine_file
assert_equal content('formatter_flash.properties'), formatter.format_file('en')
end
def test_format_value_converts_placeholders
assert_equal "value {0}", @formatter.format_value('value %d')
end
def test_deducts_language_from_resource_folder
language = %w(en de fr).sample
assert_equal language, @formatter.determine_language_given_path("locale/#{language}/#{@formatter.default_file_name}")
end
def test_deducts_language_and_region_from_resource_folder
assert_equal 'de-AT', @formatter.determine_language_given_path("locale/de-AT/#{@formatter.default_file_name}")
formatter.strings = @twine_file
formatter.write_file @output_path, 'en'
assert_equal content('formatter_flash.properties'), output_content
end
end

View file

@ -1,140 +0,0 @@
require 'command_test'
class TestGenerateAllLocalizationFiles < CommandTest
def new_runner(create_folders, twine_file = nil, options = {})
default_options = {}
default_options[:output_path] = @output_dir
default_options[:format] = 'apple'
default_options[:create_folders] = create_folders
options = default_options.merge options
unless twine_file
twine_file = build_twine_file 'en', 'es' do
add_section 'Section' do
add_definition key: 'value'
end
end
end
Twine::Runner.new(options, twine_file)
end
class TestFormatterSelection < TestGenerateAllLocalizationFiles
def setup
super
Dir.mkdir File.join @output_dir, 'values-en'
end
def new_runner(options = {})
super(true, nil, options)
end
def test_returns_error_for_ambiguous_output_path
assert_raises Twine::Error do
new_runner(format: nil).generate_all_localization_files
end
end
def test_uses_specified_formatter_to_resolve_ambiguity
# implicit assert that this call doesn't raise an exception
new_runner(format: 'android').generate_all_localization_files
end
end
class TestDoNotCreateFolders < TestGenerateAllLocalizationFiles
def new_runner(twine_file = nil, options = {})
super(false, twine_file, options)
end
def test_fails_if_output_folder_does_not_exist
assert_raises Twine::Error do
new_runner.generate_all_localization_files
end
end
def test_does_not_create_language_folders
Dir.mkdir File.join @output_dir, 'en.lproj'
new_runner.generate_all_localization_files
refute File.exist?(File.join(@output_dir, 'es.lproj')), "language folder should not be created"
end
def test_prints_empty_file_warnings
Dir.mkdir File.join @output_dir, 'en.lproj'
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file).generate_all_localization_files
assert_match "Skipping file at path", Twine::stdout.string
end
def test_does_not_print_empty_file_warnings_if_quite
Dir.mkdir File.join @output_dir, 'en.lproj'
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file, quite: true).generate_all_localization_files
refute_match "Skipping file at path", Twine::stdout.string
end
end
class TestCreateFolders < TestGenerateAllLocalizationFiles
def new_runner(twine_file = nil, options = {})
super(true, twine_file, options)
end
def test_creates_output_folder
FileUtils.remove_entry_secure @output_dir
new_runner.generate_all_localization_files
assert File.exist? @output_dir
end
def test_creates_language_folders
new_runner.generate_all_localization_files
assert File.exist?(File.join(@output_dir, 'en.lproj')), "language folder 'en.lproj' should be created"
assert File.exist?(File.join(@output_dir, 'es.lproj')), "language folder 'es.lproj' should be created"
end
def test_prints_empty_file_warnings
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file).generate_all_localization_files
assert_match "Skipping file at path", Twine::stdout.string
end
def test_does_not_print_empty_file_warnings_if_quite
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file, quite: true).generate_all_localization_files
refute_match "Skipping file at path", Twine::stdout.string
end
end
class TestValidate < CommandTest
def new_runner(validate)
Dir.mkdir File.join @output_dir, 'values-en'
options = {}
options[:output_path] = @output_dir
options[:format] = 'android'
options[:validate] = validate
twine_file = build_twine_file 'en' do
add_section 'Section' do
add_definition key: 'value'
add_definition key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_does_not_validate_twine_file
prepare_mock_formatter Twine::Formatters::Android
new_runner(false).generate_all_localization_files
end
def test_validates_twine_file_if_validate
assert_raises Twine::Error do
new_runner(true).generate_all_localization_files
end
end
end
end

View file

@ -0,0 +1,76 @@
require 'command_test_case'
class TestGenerateAllStringFiles < CommandTestCase
class TestCreateFolders < CommandTestCase
def new_runner(create_folders)
options = {}
options[:output_path] = @output_dir
options[:format] = 'apple'
options[:create_folders] = create_folders
twine_file = build_twine_file 'en', 'es' do
add_section 'Section' do
add_row key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_fails_if_output_folder_does_not_exist
assert_raises Twine::Error do
new_runner(false).generate_all_string_files
end
end
def test_creates_output_folder
FileUtils.remove_entry_secure @output_dir
new_runner(true).generate_all_string_files
assert File.exists? @output_dir
end
def test_does_not_create_language_folders_by_default
Dir.mkdir File.join @output_dir, 'en.lproj'
new_runner(false).generate_all_string_files
refute File.exists?(File.join(@output_dir, 'es.lproj')), "language folder should not be created"
end
def test_creates_language_folders
new_runner(true).generate_all_string_files
assert File.exists?(File.join(@output_dir, 'en.lproj')), "language folder 'en.lproj' should be created"
assert File.exists?(File.join(@output_dir, 'es.lproj')), "language folder 'es.lproj' should be created"
end
end
class TestDeliberate < CommandTestCase
def new_runner(validate)
Dir.mkdir File.join @output_dir, 'values-en'
options = {}
options[:output_path] = @output_dir
options[:format] = 'android'
options[:validate] = validate
twine_file = build_twine_file 'en' do
add_section 'Section' do
add_row key: 'value'
add_row key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_does_not_validate_strings_file
prepare_mock_formatter Twine::Formatters::Android
new_runner(false).generate_all_string_files
end
def test_validates_strings_file_if_validate
assert_raises Twine::Error do
new_runner(true).generate_all_string_files
end
end
end
end

View file

@ -0,0 +1,74 @@
require 'command_test_case'
class TestGenerateLocDrop < CommandTestCase
def setup
super
options = {}
options[:output_path] = @output_path
options[:format] = 'apple'
@twine_file = build_twine_file 'en', 'fr' do
add_section 'Section' do
add_row key: 'value'
end
end
@runner = Twine::Runner.new(options, @twine_file)
end
def test_generates_zip_file
@runner.generate_loc_drop
assert File.exists?(@output_path), "language folder should not be created"
end
def test_zip_file_structure
@runner.generate_loc_drop
names = []
Zip::File.open(@output_path) do |zipfile|
zipfile.each do |entry|
names << entry.name
end
end
assert_equal ['Locales/', 'Locales/en.strings', 'Locales/fr.strings'], names
end
def test_uses_formatter
formatter = prepare_mock_formatter Twine::Formatters::Apple
formatter.expects(:write_file).twice.with() { |path, lang| FileUtils.touch path }
@runner.generate_loc_drop
end
class TestDeliberate < CommandTestCase
def new_runner(validate)
options = {}
options[:output_path] = @output_path
options[:format] = 'android'
options[:validate] = validate
twine_file = build_twine_file 'en' do
add_section 'Section' do
add_row key: 'value'
add_row key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_does_not_validate_strings_file
prepare_mock_formatter Twine::Formatters::Android
new_runner(false).generate_loc_drop
end
def test_validates_strings_file_if_validate
assert_raises Twine::Error do
new_runner(true).generate_loc_drop
end
end
end
end

View file

@ -1,85 +0,0 @@
require 'command_test'
class TestGenerateLocalizationArchive < CommandTest
def new_runner(twine_file = nil, options = {})
options[:output_path] = @output_path
options[:format] = 'apple'
unless twine_file
twine_file = build_twine_file 'en', 'fr' do
add_section 'Section' do
add_definition key: 'value'
end
end
end
Twine::Runner.new(options, twine_file)
end
def test_generates_zip_file
new_runner.generate_localization_archive
assert File.exist?(@output_path), "zip file should exist"
end
def test_zip_file_structure
new_runner.generate_localization_archive
names = []
Zip::File.open(@output_path) do |zipfile|
zipfile.each do |entry|
names << entry.name
end
end
assert_equal ['Locales/', 'Locales/en.strings', 'Locales/fr.strings'], names
end
def test_uses_formatter
formatter = prepare_mock_formatter Twine::Formatters::Apple
formatter.expects(:format_file).twice
new_runner.generate_localization_archive
end
def test_prints_empty_file_warnings
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file).generate_localization_archive
assert_match "Skipping file", Twine::stdout.string
end
def test_does_not_print_empty_file_warnings_if_quite
empty_twine_file = build_twine_file('en') {}
new_runner(empty_twine_file, quite: true).generate_localization_archive
refute_match "Skipping file", Twine::stdout.string
end
class TestValidate < CommandTest
def new_runner(validate)
options = {}
options[:output_path] = @output_path
options[:format] = 'android'
options[:validate] = validate
twine_file = build_twine_file 'en' do
add_section 'Section' do
add_definition key: 'value'
add_definition key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_does_not_validate_twine_file
prepare_mock_formatter Twine::Formatters::Android
new_runner(false).generate_localization_archive
end
def test_validates_twine_file_if_validate
assert_raises Twine::Error do
new_runner(true).generate_localization_archive
end
end
end
end

View file

@ -1,119 +0,0 @@
require 'command_test'
class TestGenerateLocalizationFile < CommandTest
def new_runner(language, file, options = {})
options[:output_path] = File.join(@output_dir, file) if file
options[:languages] = language if language
twine_file = Twine::TwineFile.new
twine_file.language_codes.concat KNOWN_LANGUAGES
Twine::Runner.new(options, twine_file)
end
def prepare_mock_format_file_formatter(formatter_class)
formatter = prepare_mock_formatter(formatter_class)
formatter.expects(:format_file).returns(true)
end
def test_deducts_android_format_from_output_path
prepare_mock_format_file_formatter Twine::Formatters::Android
new_runner('fr', 'fr.xml').generate_localization_file
end
def test_deducts_apple_format_from_output_path
prepare_mock_format_file_formatter Twine::Formatters::Apple
new_runner('fr', 'fr.strings').generate_localization_file
end
def test_deducts_jquery_format_from_output_path
prepare_mock_format_file_formatter Twine::Formatters::JQuery
new_runner('fr', 'fr.json').generate_localization_file
end
def test_deducts_gettext_format_from_output_path
prepare_mock_format_file_formatter Twine::Formatters::Gettext
new_runner('fr', 'fr.po').generate_localization_file
end
def test_deducts_django_format_from_output_path
prepare_mock_format_file_formatter Twine::Formatters::Django
new_runner('fr', 'fr.po').generate_localization_file
end
def test_returns_error_for_ambiguous_output_path
# both Gettext and Django use .po
gettext_formatter = prepare_mock_formatter(Twine::Formatters::Gettext)
gettext_formatter.stubs(:format_file).returns(true)
django_formatter = prepare_mock_formatter(Twine::Formatters::Django, false)
django_formatter.stubs(:format_file).returns(true)
assert_raises Twine::Error do
new_runner('fr', 'fr.po').generate_localization_file
end
end
def test_uses_specified_formatter_to_resolve_ambiguity
# both Android and Tizen use .xml
android_formatter = prepare_mock_formatter(Twine::Formatters::Android)
android_formatter.stubs(:format_file).returns(true)
tizen_formatter = prepare_mock_formatter(Twine::Formatters::Tizen, false)
tizen_formatter.stubs(:format_file).returns(true)
# implicit assert that this call doesn't raise an exception
new_runner('fr', 'fr.xml', format: 'android').generate_localization_file
end
def test_deducts_language_from_output_path
random_language = KNOWN_LANGUAGES.sample
formatter = prepare_mock_formatter Twine::Formatters::Android
formatter.expects(:format_file).with(random_language).returns(true)
new_runner(nil, "#{random_language}.xml").generate_localization_file
end
def test_returns_error_if_nothing_written
formatter = prepare_mock_formatter Twine::Formatters::Android
formatter.expects(:format_file).returns(false)
assert_raises Twine::Error do
new_runner('fr', 'fr.xml').generate_localization_file
end
end
class TestValidate < CommandTest
def new_runner(validate)
options = {}
options[:output_path] = @output_path
options[:languages] = ['en']
options[:format] = 'android'
options[:validate] = validate
twine_file = build_twine_file 'en' do
add_section 'Section' do
add_definition key: 'value'
add_definition key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_does_not_validate_twine_file
prepare_mock_formatter Twine::Formatters::Android
new_runner(false).generate_localization_file
end
def test_validates_twine_file_if_validate
assert_raises Twine::Error do
new_runner(true).generate_localization_file
end
end
end
end

View file

@ -0,0 +1,82 @@
require 'command_test_case'
class TestGenerateStringFile < CommandTestCase
def new_runner(language, file)
options = {}
options[:output_path] = File.join(@output_dir, file) if file
options[:languages] = language if language
strings = Twine::StringsFile.new
strings.language_codes.concat KNOWN_LANGUAGES
Twine::Runner.new(options, strings)
end
def prepare_mock_write_file_formatter(formatter_class)
formatter = prepare_mock_formatter(formatter_class)
formatter.expects(:write_file)
end
def test_deducts_android_format_from_output_path
prepare_mock_write_file_formatter Twine::Formatters::Android
new_runner('fr', 'fr.xml').generate_string_file
end
def test_deducts_apple_format_from_output_path
prepare_mock_write_file_formatter Twine::Formatters::Apple
new_runner('fr', 'fr.strings').generate_string_file
end
def test_deducts_jquery_format_from_output_path
prepare_mock_write_file_formatter Twine::Formatters::JQuery
new_runner('fr', 'fr.json').generate_string_file
end
def test_deducts_gettext_format_from_output_path
prepare_mock_write_file_formatter Twine::Formatters::Gettext
new_runner('fr', 'fr.po').generate_string_file
end
def test_deducts_language_from_output_path
random_language = KNOWN_LANGUAGES.sample
formatter = prepare_mock_formatter Twine::Formatters::Android
formatter.expects(:write_file).with(anything, random_language)
new_runner(nil, "#{random_language}.xml").generate_string_file
end
class TestDeliberate < CommandTestCase
def new_runner(validate)
options = {}
options[:output_path] = @output_path
options[:languages] = ['en']
options[:format] = 'android'
options[:validate] = validate
twine_file = build_twine_file 'en' do
add_section 'Section' do
add_row key: 'value'
add_row key: 'value'
end
end
Twine::Runner.new(options, twine_file)
end
def test_does_not_validate_strings_file
prepare_mock_formatter Twine::Formatters::Android
new_runner(false).generate_string_file
end
def test_validates_strings_file_if_validate
assert_raises Twine::Error do
new_runner(true).generate_string_file
end
end
end
end

View file

@ -1,84 +1,84 @@
require 'twine_test'
require 'twine_test_case'
class TestOutputProcessor < TwineTest
class TestOutputProcessor < TwineTestCase
def setup
super
@twine_file = build_twine_file 'en', 'fr' do
@strings = build_twine_file 'en', 'fr' do
add_section 'Section' do
add_definition key1: 'value1', tags: ['tag1']
add_definition key2: 'value2', tags: ['tag1', 'tag2']
add_definition key3: 'value3', tags: ['tag2']
add_definition key4: { en: 'value4-en', fr: 'value4-fr' }
add_row key1: 'value1', tags: ['tag1']
add_row key2: 'value2', tags: ['tag1', 'tag2']
add_row key3: 'value3', tags: ['tag2']
add_row key4: { en: 'value4-en', fr: 'value4-fr' }
end
end
end
def test_includes_all_keys_by_default
processor = Twine::Processors::OutputProcessor.new(@twine_file, {})
processor = Twine::Processors::OutputProcessor.new(@strings, {})
result = processor.process('en')
assert_equal %w(key1 key2 key3 key4), result.definitions_by_key.keys.sort
assert_equal %w(key1 key2 key3 key4), result.strings_map.keys.sort
end
def test_filter_by_tag
processor = Twine::Processors::OutputProcessor.new(@twine_file, { tags: [['tag1']] })
processor = Twine::Processors::OutputProcessor.new(@strings, { tags: ['tag1'] })
result = processor.process('en')
assert_equal %w(key1 key2), result.definitions_by_key.keys.sort
assert_equal %w(key1 key2), result.strings_map.keys.sort
end
def test_filter_by_multiple_tags
processor = Twine::Processors::OutputProcessor.new(@twine_file, { tags: [['tag1', 'tag2']] })
processor = Twine::Processors::OutputProcessor.new(@strings, { tags: ['tag1', 'tag2'] })
result = processor.process('en')
assert_equal %w(key1 key2 key3), result.definitions_by_key.keys.sort
assert_equal %w(key1 key2 key3), result.strings_map.keys.sort
end
def test_filter_untagged
processor = Twine::Processors::OutputProcessor.new(@twine_file, { tags: [['tag1']], untagged: true })
processor = Twine::Processors::OutputProcessor.new(@strings, { tags: ['tag1'], untagged: true })
result = processor.process('en')
assert_equal %w(key1 key2 key4), result.definitions_by_key.keys.sort
assert_equal %w(key1 key2 key4), result.strings_map.keys.sort
end
def test_include_translated
processor = Twine::Processors::OutputProcessor.new(@twine_file, { include: :translated })
processor = Twine::Processors::OutputProcessor.new(@strings, { include: 'translated' })
result = processor.process('fr')
assert_equal %w(key4), result.definitions_by_key.keys.sort
assert_equal %w(key4), result.strings_map.keys.sort
end
def test_include_untranslated
processor = Twine::Processors::OutputProcessor.new(@twine_file, { include: :untranslated })
processor = Twine::Processors::OutputProcessor.new(@strings, { include: 'untranslated' })
result = processor.process('fr')
assert_equal %w(key1 key2 key3), result.definitions_by_key.keys.sort
assert_equal %w(key1 key2 key3), result.strings_map.keys.sort
end
class TranslationFallback < TwineTest
class TranslationFallback < TwineTestCase
def setup
super
@twine_file = build_twine_file 'en', 'fr', 'de' do
@strings = build_twine_file 'en', 'fr', 'de' do
add_section 'Section' do
add_definition key1: { en: 'value1-en', fr: 'value1-fr' }
add_row key1: { en: 'value1-en', fr: 'value1-fr' }
end
end
end
def test_fallback_to_default_language
processor = Twine::Processors::OutputProcessor.new(@twine_file, {})
processor = Twine::Processors::OutputProcessor.new(@strings, {})
result = processor.process('de')
assert_equal 'value1-en', result.definitions_by_key['key1'].translations['de']
assert_equal 'value1-en', result.strings_map['key1'].translations['de']
end
def test_fallback_to_developer_language
processor = Twine::Processors::OutputProcessor.new(@twine_file, {developer_language: 'fr'})
processor = Twine::Processors::OutputProcessor.new(@strings, {developer_language: 'fr'})
result = processor.process('de')
assert_equal 'value1-fr', result.definitions_by_key['key1'].translations['de']
assert_equal 'value1-fr', result.strings_map['key1'].translations['de']
end
end

View file

@ -1,6 +1,6 @@
require 'twine_test'
require 'twine_test_case'
class PlaceholderTest < TwineTest
class PlaceholderTestCase < TwineTestCase
def assert_starts_with(prefix, value)
msg = message(nil) { "Expected #{mu_pp(value)} to start with #{mu_pp(prefix)}" }
assert value.start_with?(prefix), msg
@ -11,25 +11,23 @@ class PlaceholderTest < TwineTest
lucky = lambda { rand > 0.5 }
placeholder = '%'
placeholder += (rand * 20).to_i.to_s + '$' if lucky.call
placeholder += '-+0#'.chars.to_a.sample if lucky.call
placeholder += '-+ 0#'.chars.to_a.sample if lucky.call
placeholder += (0.upto(20).map(&:to_s) << "*").sample if lucky.call
placeholder += '.' + (0.upto(20).map(&:to_s) << "*").sample if lucky.call
placeholder += %w(h hh l ll L z j t).sample if lucky.call
placeholder += type || 'diufFeEgGxXocpaA'.chars.to_a.sample # this does not contain s or @ because strings are a special case
end
end
class ToAndroid < PlaceholderTest
class PlaceholderTest < TwineTestCase
class ToAndroid < PlaceholderTestCase
def to_android(value)
Twine::Placeholders.convert_placeholders_from_twine_to_android(value)
end
def test_replaces_simple_string_placeholder
assert_equal "some '%s' value", to_android("some '%@' value")
end
def test_replaces_complicated_string_placeholder
def test_replaces_string_placeholder
placeholder = placeholder('@')
expected = placeholder.dup
expected = placeholder
expected[-1] = 's'
assert_equal "some #{expected} value", to_android("some #{placeholder} value")
end
@ -43,11 +41,6 @@ class PlaceholderTest < TwineTest
assert_equal "some % value", to_android("some % value")
end
def test_does_not_modify_single_percent_signs_when_followed_by_space_and_format_letter
# Said differently: formartter parser should not recognize %a in "70% and"
assert_equal 'If 70% and 30% dog 80% end', to_android('If 70% and 30% dog 80% end')
end
def test_escapes_single_percent_signs_if_placeholder_present
assert_starts_with "some %% v", to_android("some % value #{placeholder}")
end
@ -79,13 +72,9 @@ class PlaceholderTest < TwineTest
to_android("some %d second %2$f")
end
end
def test_complicated_float_placeholders
assert_equal "%1$.0f%2$s (apparent: %3$.0f)", to_android("%.0f%@ (apparent: %.0f)")
end
end
class FromAndroid < PlaceholderTest
class FromAndroid < PlaceholderTestCase
def from_android(value)
Twine::Placeholders.convert_placeholders_from_android_to_twine(value)
end
@ -94,49 +83,4 @@ class PlaceholderTest < TwineTest
assert_equal "some %@ value", from_android("some %s value")
end
end
class ToFlash < PlaceholderTest
def to_flash(value)
Twine::Placeholders.convert_placeholders_from_twine_to_flash(value)
end
def test_replaces_placeholder
assert_equal "some {0} text", to_flash("some #{placeholder} text")
end
def test_replaces_string_placeholder
assert_equal "some {0} text", to_flash("some #{placeholder('@')} text")
end
def test_numbers_placeholders
assert_equal "some {0} more {1} text {2}", to_flash("some #{placeholder('@')} more #{placeholder('@')} text #{placeholder('@')}")
end
end
class FromFlash < PlaceholderTest
def from_flash(value)
Twine::Placeholders.convert_placeholders_from_flash_to_twine(value)
end
def test_maps_all_placeholders_to_string
assert_equal "some %@ more %@ text %@", from_flash("some {0} more {1} text {2}")
end
end
class PythonPlaceholder < PlaceholderTest
def test_negative_for_regular_placeholders
assert_equal false, Twine::Placeholders.contains_python_specific_placeholder(placeholder)
end
def test_positive_for_named_placeholders
inputs = [
"%(language)s has",
"For %(number)03d quotes",
"bought on %(app_name)s"
]
inputs.each do |input|
assert_equal true, Twine::Placeholders.contains_python_specific_placeholder(input)
end
end
end
end

58
test/test_strings_file.rb Normal file
View file

@ -0,0 +1,58 @@
require 'twine_test_case'
class TestStringsFile < TwineTestCase
class Reading < TwineTestCase
def setup
super
@strings = Twine::StringsFile.new
@strings.read fixture('twine_accent_values.txt')
end
def test_reading_keeps_leading_accent
assert_equal '`value', @strings.strings_map['value_with_leading_accent'].translations['en']
end
def test_reading_keeps_trailing_accent
assert_equal 'value`', @strings.strings_map['value_with_trailing_accent'].translations['en']
end
def test_reading_keeps_leading_space
assert_equal ' value', @strings.strings_map['value_with_leading_space'].translations['en']
end
def test_reading_keeps_trailing_space
assert_equal 'value ', @strings.strings_map['value_with_trailing_space'].translations['en']
end
def test_reading_keeps_wrapping_spaces
assert_equal ' value ', @strings.strings_map['value_wrapped_by_spaces'].translations['en']
end
def test_reading_keeps_wrapping_accents
assert_equal '`value`', @strings.strings_map['value_wrapped_by_accents'].translations['en']
end
end
class Writing < TwineTestCase
def test_accent_wrapping
@strings = build_twine_file 'en' do
add_section 'Section' do
add_row value_with_leading_accent: '`value'
add_row value_with_trailing_accent: 'value`'
add_row value_with_leading_space: ' value'
add_row value_with_trailing_space: 'value '
add_row value_wrapped_by_spaces: ' value '
add_row value_wrapped_by_accents: '`value`'
end
end
@strings.write @output_path
assert_equal content('twine_accent_values.txt'), output_content
end
end
end

47
test/test_strings_row.rb Normal file
View file

@ -0,0 +1,47 @@
require 'twine_test_case'
class TestStringsRow < TwineTestCase
def setup
super
@reference = Twine::StringsRow.new 'reference-key'
@reference.comment = 'reference comment'
@reference.tags = ['ref1']
@reference.translations['en'] = 'ref-value'
@row = Twine::StringsRow.new 'key'
@row.reference_key = @reference.key
@row.reference = @reference
end
def test_reference_comment_used
assert_equal 'reference comment', @row.comment
end
def test_reference_comment_override
@row.comment = 'row comment'
assert_equal 'row comment', @row.comment
end
def test_reference_tags_used
assert @row.matches_tags?(['ref1'], false)
end
def test_reference_tags_override
@row.tags = ['tag1']
refute @row.matches_tags?(['ref1'], false)
assert @row.matches_tags?(['tag1'], false)
end
def test_reference_translation_used
assert_equal 'ref-value', @row.translated_string_for_lang('en')
end
def test_reference_translation_override
@row.translations['en'] = 'value'
assert_equal 'value', @row.translated_string_for_lang('en')
end
end

View file

@ -1,111 +0,0 @@
require 'twine_test'
class TestTwineDefinition < TwineTest
class TestTags < TwineTest
def setup
super
@definition = Twine::TwineDefinition.new 'key'
end
def test_include_untagged
assert @definition.matches_tags?([[rand(100000).to_s]], true)
end
def test_matches_no_given_tags
assert @definition.matches_tags?([], false)
end
def test_matches_tag
@definition.tags = ['tag1']
assert @definition.matches_tags?([['tag1']], false)
end
def test_matches_any_tag
@definition.tags = ['tag1']
assert @definition.matches_tags?([['tag0', 'tag1', 'tag2']], false)
end
def test_matches_all_tags
@definition.tags = ['tag1', 'tag2']
assert @definition.matches_tags?([['tag1'], ['tag2']], false)
end
def test_does_not_match_all_tags
@definition.tags = ['tag1']
refute @definition.matches_tags?([['tag1'], ['tag2']], false)
end
def test_does_not_match_excluded_tag
@definition.tags = ['tag1']
refute @definition.matches_tags?([['~tag1']], false)
end
def test_matches_excluded_tag
@definition.tags = ['tag2']
assert @definition.matches_tags?([['~tag1']], false)
end
def test_complex_rules
@definition.tags = ['tag1', 'tag2', 'tag3']
assert @definition.matches_tags?([['tag1']], false)
assert @definition.matches_tags?([['tag1', 'tag4']], false)
assert @definition.matches_tags?([['tag1'], ['tag2'], ['tag3']], false)
refute @definition.matches_tags?([['tag1'], ['tag4']], false)
assert @definition.matches_tags?([['tag4', '~tag5']], false)
end
end
class TestReferences < TwineTest
def setup
super
@reference = Twine::TwineDefinition.new 'reference-key'
@reference.comment = 'reference comment'
@reference.tags = ['ref1']
@reference.translations['en'] = 'ref-value'
@definition = Twine::TwineDefinition.new 'key'
@definition.reference_key = @reference.key
@definition.reference = @reference
end
def test_reference_comment_used
assert_equal 'reference comment', @definition.comment
end
def test_reference_comment_override
@definition.comment = 'definition comment'
assert_equal 'definition comment', @definition.comment
end
def test_reference_tags_used
assert @definition.matches_tags?([['ref1']], false)
end
def test_reference_tags_override
@definition.tags = ['tag1']
refute @definition.matches_tags?([['ref1']], false)
assert @definition.matches_tags?([['tag1']], false)
end
def test_reference_translation_used
assert_equal 'ref-value', @definition.translation_for_lang('en')
end
def test_reference_translation_override
@definition.translations['en'] = 'value'
assert_equal 'value', @definition.translation_for_lang('en')
end
end
end

View file

@ -1,58 +0,0 @@
require 'twine_test'
class TestTwineFile < TwineTest
class Reading < TwineTest
def setup
super
@twine_file = Twine::TwineFile.new
@twine_file.read fixture_path('twine_accent_values.txt')
end
def test_reading_keeps_leading_accent
assert_equal '`value', @twine_file.definitions_by_key['value_with_leading_accent'].translations['en']
end
def test_reading_keeps_trailing_accent
assert_equal 'value`', @twine_file.definitions_by_key['value_with_trailing_accent'].translations['en']
end
def test_reading_keeps_leading_space
assert_equal ' value', @twine_file.definitions_by_key['value_with_leading_space'].translations['en']
end
def test_reading_keeps_trailing_space
assert_equal 'value ', @twine_file.definitions_by_key['value_with_trailing_space'].translations['en']
end
def test_reading_keeps_wrapping_spaces
assert_equal ' value ', @twine_file.definitions_by_key['value_wrapped_by_spaces'].translations['en']
end
def test_reading_keeps_wrapping_accents
assert_equal '`value`', @twine_file.definitions_by_key['value_wrapped_by_accents'].translations['en']
end
end
class Writing < TwineTest
def test_accent_wrapping
@twine_file = build_twine_file 'en' do
add_section 'Section' do
add_definition value_with_leading_accent: '`value'
add_definition value_with_trailing_accent: 'value`'
add_definition value_with_leading_space: ' value'
add_definition value_with_trailing_space: 'value '
add_definition value_wrapped_by_spaces: ' value '
add_definition value_wrapped_by_accents: '`value`'
end
end
@twine_file.write @output_path
assert_equal content('twine_accent_values.txt'), File.read(@output_path)
end
end
end

View file

@ -0,0 +1,61 @@
# encoding: utf-8
require 'command_test_case'
class TestValidateStringsFile < CommandTestCase
def setup
super
@options = { strings_file: 'input.txt' }
@twine_file = build_twine_file 'en' do
add_section 'Section 1' do
add_row key1: 'value1', tags: ['tag1']
add_row key2: 'value2', tags: ['tag1']
end
add_section 'Section 2' do
add_row key3: 'value3', tags: ['tag1', 'tag2']
add_row key4: 'value4', tags: ['tag2']
end
end
end
def random_row
@twine_file.strings_map[@twine_file.strings_map.keys.sample]
end
def test_recognizes_valid_file
Twine::Runner.new(@options, @twine_file).validate_strings_file
assert_equal "input.txt is valid.\n", Twine::stdout.string
end
def test_reports_duplicate_keys
@twine_file.sections[0].rows << random_row
assert_raises Twine::Error do
Twine::Runner.new(@options, @twine_file).validate_strings_file
end
end
def test_reports_invalid_characters_in_keys
random_row.key[0] = "!?;:,^`´'\"\\|/(){}[]~-+*=#$%".chars.to_a.sample
assert_raises Twine::Error do
Twine::Runner.new(@options, @twine_file).validate_strings_file
end
end
def test_does_not_reports_missing_tags_by_default
random_row.tags.clear
Twine::Runner.new(@options, @twine_file).validate_strings_file
end
def test_reports_missing_tags
random_row.tags.clear
assert_raises Twine::Error do
Twine::Runner.new(@options.merge(pedantic: true), @twine_file).validate_strings_file
end
end
end

View file

@ -1,69 +0,0 @@
# encoding: utf-8
require 'command_test'
class TestValidateTwineFile < CommandTest
def setup
super
@options = { twine_file: 'input.txt' }
@twine_file = build_twine_file 'en' do
add_section 'Section 1' do
add_definition key1: 'value1', tags: ['tag1']
add_definition key2: 'value2', tags: ['tag1']
end
add_section 'Section 2' do
add_definition key3: 'value3', tags: ['tag1', 'tag2']
add_definition key4: 'value4', tags: ['tag2']
end
end
end
def random_definition
@twine_file.definitions_by_key[@twine_file.definitions_by_key.keys.sample]
end
def test_recognizes_valid_file
Twine::Runner.new(@options, @twine_file).validate_twine_file
assert_equal "input.txt is valid.\n", Twine::stdout.string
end
def test_reports_duplicate_keys
@twine_file.sections[0].definitions << random_definition
assert_raises Twine::Error do
Twine::Runner.new(@options, @twine_file).validate_twine_file
end
end
def test_reports_invalid_characters_in_keys
random_definition.key[0] = "!?;:,^`´'\"\\|/(){}[]~-+*=#$%".chars.to_a.sample
assert_raises Twine::Error do
Twine::Runner.new(@options, @twine_file).validate_twine_file
end
end
def test_does_not_reports_missing_tags_by_default
random_definition.tags.clear
Twine::Runner.new(@options, @twine_file).validate_twine_file
end
def test_reports_missing_tags
random_definition.tags.clear
assert_raises Twine::Error do
Twine::Runner.new(@options.merge(pedantic: true), @twine_file).validate_twine_file
end
end
def test_reports_python_specific_placeholders
random_definition.translations["en"] = "%(python_only)s"
assert_raises Twine::Error do
Twine::Runner.new(@options, @twine_file).validate_twine_file
end
end
end

View file

@ -1,6 +1,6 @@
module TwineFileDSL
def build_twine_file(*languages)
@currently_built_twine_file = Twine::TwineFile.new
@currently_built_twine_file = Twine::StringsFile.new
@currently_built_twine_file.language_codes.concat languages
yield
result = @currently_built_twine_file
@ -10,37 +10,37 @@ module TwineFileDSL
def add_section(name)
return unless @currently_built_twine_file
@currently_built_twine_file_section = Twine::TwineSection.new name
@currently_built_twine_file_section = Twine::StringsSection.new name
@currently_built_twine_file.sections << @currently_built_twine_file_section
yield
@currently_built_twine_file_section = nil
end
def add_definition(parameters)
def add_row(parameters)
return unless @currently_built_twine_file
return unless @currently_built_twine_file_section
# this relies on Ruby preserving the order of hash elements
# this relies on Ruby 1.9 preserving the order of hash elements
key, value = parameters.first
definition = Twine::TwineDefinition.new(key.to_s)
row = Twine::StringsRow.new(key.to_s)
if value.is_a? Hash
value.each do |language, translation|
definition.translations[language.to_s] = translation
row.translations[language.to_s] = translation
end
elsif !value.is_a? Symbol
language = @currently_built_twine_file.language_codes.first
definition.translations[language] = value
row.translations[language] = value
end
definition.comment = parameters[:comment] if parameters[:comment]
definition.tags = parameters[:tags] if parameters[:tags]
row.comment = parameters[:comment] if parameters[:comment]
row.tags = parameters[:tags] if parameters[:tags]
if parameters[:ref] || value.is_a?(Symbol)
reference_key = (parameters[:ref] || value).to_s
definition.reference_key = reference_key
definition.reference = @currently_built_twine_file.definitions_by_key[reference_key]
row.reference_key = reference_key
row.reference = @currently_built_twine_file.strings_map[reference_key]
end
@currently_built_twine_file_section.definitions << definition
@currently_built_twine_file.definitions_by_key[definition.key] = definition
@currently_built_twine_file_section.rows << row
@currently_built_twine_file.strings_map[row.key] = row
end
end

View file

@ -1,12 +1,12 @@
require 'erb'
require 'minitest/autorun'
require "mocha/minitest"
require "mocha/mini_test"
require 'securerandom'
require 'stringio'
require 'twine'
require 'twine_file_dsl'
class TwineTest < Minitest::Test
class TwineTestCase < Minitest::Test
include TwineFileDSL
KNOWN_LANGUAGES = %w(en fr de es)
@ -23,26 +23,27 @@ class TwineTest < Minitest::Test
end
def teardown
FileUtils.remove_entry_secure @output_dir if File.exist? @output_dir
FileUtils.remove_entry_secure @output_dir if File.exists? @output_dir
Twine::Formatters.formatters.clear
Twine::Formatters.formatters.concat @formatters
super
end
def output_content
File.read @output_path
end
def execute(command)
command += " -o #{@output_path}"
command += " -o #{@output_path}"
Twine::Runner.run(command.split(" "))
end
def fixture_path(filename)
def fixture(filename)
File.join File.dirname(__FILE__), 'fixtures', filename
end
alias :f :fixture
def content(filename)
ERB.new(File.read fixture_path(filename)).result
end
def content_io(filename)
StringIO.new ERB.new(File.read fixture_path(filename)).result
ERB.new(File.read fixture(filename)).result
end
end

2
twine
View file

@ -1,3 +1,3 @@
#!/bin/sh
BASEDIR=$(dirname $0)
ruby -rrubygems -I $BASEDIR/lib $BASEDIR/bin/twine "$@"
ruby -rubygems -I $BASEDIR/lib $BASEDIR/bin/twine $@

View file

@ -5,8 +5,8 @@ Gem::Specification.new do |s|
s.name = "twine"
s.version = Twine::VERSION
s.date = Time.now.strftime('%Y-%m-%d')
s.summary = "Manage strings and their translations for your iOS, Android and other projects."
s.homepage = "https://github.com/scelis/twine"
s.summary = "Manage strings and their translations for your iOS and Android projects."
s.homepage = "https://github.com/mobiata/twine"
s.email = "twine@mobiata.com"
s.authors = [ "Sebastian Celis" ]
s.has_rdoc = false
@ -18,13 +18,11 @@ Gem::Specification.new do |s|
s.files += Dir.glob("test/**/*")
s.test_files = Dir.glob("test/test_*")
s.required_ruby_version = ">= 2.6"
s.add_runtime_dependency('rexml', "~> 3.2")
s.add_runtime_dependency('rubyzip', "~> 2.0")
s.required_ruby_version = ">= 2.0"
s.add_runtime_dependency('rubyzip', "~> 1.1")
s.add_runtime_dependency('safe_yaml', "~> 1.0")
s.add_development_dependency('rake', "~> 13.0")
s.add_development_dependency('rake', "~> 10.4")
s.add_development_dependency('minitest', "~> 5.5")
s.add_development_dependency('minitest-ci', "~> 3.0")
s.add_development_dependency('mocha', "~> 1.1")
s.executables = %w( twine )