Commit graph

138 commits

Author SHA1 Message Date
Anatoly Serdtcev
346fceb020 [generator:region] Speedup locality index building: parallel region object covering 2020-01-15 19:34:23 +03:00
Anatoly Serdtcev
895b474dd1 [generator:regions] Speedup locality index building: parallel regions stripping 2020-01-15 13:06:35 +00:00
Anatoly Serdtcev
143106cb1d [generator:regions] Parallel build regions tree 2020-01-15 13:04:35 +00:00
Anatoly Serdtcev
de9a1c60e1 [indexer] Build index in one pass 2020-01-15 12:49:45 +00:00
Anatoly Serdtcev
917387ce20 [generator:geo_objects] Speedup poi generation: parallel write into KV in node-file 2020-01-15 09:39:50 +00:00
Anatoly Serdtcev
73f4a75c72 [generator:geo_objects] Speedup null building processing 2020-01-13 18:08:51 +03:00
Anatoly Serdtcev
7a6a9eecf1 [generator:streets] Speedup streets building: split regions into arenas 2020-01-13 17:16:00 +03:00
Anatoly Serdtcev
9f8d0539f7 [indexer] Fix object covering 2020-01-13 17:14:44 +03:00
Anatoly Serdtcev
7305cc18c8 [generator:geo_objects] Build geo_objects index file without features files merge 2019-12-28 18:03:17 +03:00
Anatoly Serdtcev
1bcdf24d4f [generator:geo_objects] Build temporary index synchronously 2019-12-27 11:55:16 +03:00
Anatoly Serdtcev
04ec6f69bc [generator, indexer] Refact: rename LocalityIndex -> CoveringIndex 2019-12-27 11:54:35 +03:00
Anatoly Serdtcev
b924f170d0 [generator] Process locality object on memory for interval index building. 2019-12-19 19:16:55 +03:00
Anatoly Serdtcev
be7a96fd58 [generator:generator_tool] Erase geocoder tokidx generator 2019-12-17 18:28:11 +03:00
Anatoly Serdtcev
2a0c9dd451 [generator:geneartor_tool] Remove deprecated --allow_addressless_for_countries 2019-12-17 18:27:44 +03:00
Anatoly Serdtcev
1d343ab5f4 [geocoder] Refactor for code review 2019-12-17 14:06:56 +03:00
Anatoly Serdtcev
c572f3e57e [geocoder] Refactor for code review: rename *tokenIds -> *tokensPositions 2019-12-17 14:06:56 +03:00
Anatoly Serdtcev
73f20dd198 [geocoder] Fix tests 2019-12-17 14:06:56 +03:00
Anatoly Serdtcev
45609ef3cd [geocoder] Fix rank for house number candidate with extra tokens 2019-12-17 14:06:56 +03:00
Anatoly Serdtcev
eaf07a172d [generator] Delete unused class CalculateMidPoints 2019-12-17 13:06:28 +03:00
Anatoly Serdtcev
5ebb0ca11a [generator:locidx-building] Speedup: skip middle points 2019-12-16 18:44:38 +03:00
Anatoly Serdtcev
d5f93204b2 [generator:regions] Fix Budapest suburbs 2019-12-16 18:44:18 +03:00
Anatoly Serdtcev
03f036a647 [generator:geo_objects] Fix write jsonl file. 2019-12-16 16:15:25 +03:00
Anatoly Serdtcev
7e785156e9 [generator] Optimize region's info getter: exclude conversion string -> JSON 2019-12-13 18:58:05 +03:00
Anatoly Serdtcev
33467bf76b [generator:geo_objects] Add parallel KV write 2019-12-12 18:43:29 +03:00
Anatoly Serdtcev
b09d2df50e [generator:regions] Fix Hawai as country: ignore enclave ways 2019-12-10 14:06:05 +03:00
Anatoly Serdtcev
152eb7bd1d [generator] Speedup generation: features file parallel reading 2019-12-05 15:08:51 +03:00
Anatoly Serdtcev
6d388bc614 [geocoder] Fix for integration tests: stable order of equal rank candidates 2019-11-29 17:16:55 +03:00
Anatoly Serdtcev
fd63ea617c [geocoder] Fix correct rank for eponymous Town and Hamlet: calculate result ranks by kind field (in jsonl) 2019-11-27 15:37:31 +03:00
Anatoly Serdtcev
1711eb5454 [generator:features] Speedup features generation: reuse vector's 2019-11-27 15:32:15 +03:00
Anatoly Serdtcev
684443bcbf [geocoder] Add method for data version retrieve from jsonl-file 2019-11-25 14:02:54 +03:00
Anatoly Serdtcev
2d8ecec4d5 [generator:features] Speedup nodes reading: mmap() + MADV_WILLNEED usage 2019-11-25 12:45:09 +03:00
Anatoly Serdtcev
d56ec9ed08 [generator:preprocess] Fix read ahead the o5m file for asynchronously mode. 2019-11-25 12:34:57 +03:00
Anatoly Serdtcev
2ffce3cfde [generator] Refactor features generator: delete unused preload and force-reload modes 2019-11-25 12:34:57 +03:00
Anatoly Serdtcev
79ee32b690 [generator] Optimize features generatrion: share intermediate data for all translators 2019-11-25 12:34:57 +03:00
Anatoly Serdtcev
08474d499f [generator] Optimize features generatrion: async load intermediate data 2019-11-25 12:34:57 +03:00
Anatoly Serdtcev
214ec3520b [generator:preprocess] Optimize intermediate data reading 2019-11-22 15:55:31 +03:00
Anatoly Serdtcev
48b23eed46 [generator] Add "kind" field in jsonl to improve rank in forward geocoder 2019-11-22 13:44:31 +03:00
Anatoly Serdtcev
e8c8e74f7d [generator:preprocess] Optimize node file write: async write by mmap() 2019-11-22 12:14:20 +03:00
Anatoly Serdtcev
a88f11131f [generator] Use tcmalloc for efficient multithreading processing 2019-11-21 14:02:49 +03:00
Anatoly Serdtcev
4a302bee84 [generator] Optimize features generation: o5m parallel reading 2019-11-18 16:14:03 +03:00
Anatoly Serdtcev
a49738bd7c [generator:preprocess] Optimize o5m reading: parallel reading 2019-11-18 16:14:03 +03:00
Anatoly Serdtcev
22c08911d2 [generator] Add o5m chunk reading 2019-11-18 16:14:03 +03:00
Anatoly Serdtcev
7c4744f0ae [generator:preprocess] Optimize o5m reading: aggressive readahead 2019-11-11 18:09:16 +02:00
Maksim Andrianov
1c62760907
Merge pull request #45 from mapsme/generator.preprocess.o5m-readahead
[generator:preprocess] Optimize o5m reading: readahead usage
2019-11-11 15:31:03 +03:00
Anatoly Serdtcev
797457330e [generator:preprocess] Optimize o5m reading: readahead usage 2019-11-11 15:01:03 +03:00
Anatoly Serdtcev
7fd4934dd6 [generator:preprocess] Optimize o5m reading: parallel reading 2019-11-11 12:32:41 +03:00
Anatoly Serdtcev
7b111c1583 [generator] Fix stack overflow in o5m reader 2019-11-08 11:37:33 +03:00
Anatoly Serdtcev
7d331ef83e [generator] Add o5m chunk reading 2019-11-06 17:07:02 +03:00
Sergey Yershov
89194cb45b Fix xcode projects and remove unused files 2019-11-06 16:29:11 +03:00
Sergey Yershov
bd9b33759e Speedup o5m reading 2019-11-06 16:29:11 +03:00