Node id bigger than the allocated cache size #7199

Open
opened 2024-01-18 04:57:11 +00:00 by iBinh · 0 comments
iBinh commented 2024-01-18 04:57:11 +00:00 (Migrated from github.com)

Steps to reproduce

  1. Run python -m maps_generator --countries="Vietnam" --skip="Coastline"
  2. Got this error Found node with id 8590177076 which is bigger than the allocated cache size
  3. Detailed error:
Loading configuration from /home/bb/organicmaps/tools/python/maps_generator/var/etc/map_generator.ini
[2024-01-18 04:09:46,216] INFO env Start setup ...
[2024-01-18 04:09:46,216] INFO env Check for the osmctools binaries...
[2024-01-18 04:09:46,217] INFO env Found osmctools at /home/bb/organicmaps/../osmctools/osmconvert, /home/bb/organicmaps/../osmctools/osmupdate, /home/bb/organicmaps/../osmctools/osmfilter
[2024-01-18 04:09:46,217] INFO env Check generator tool ...
[2024-01-18 04:09:46,217] INFO env Looking for generator tool in /home/bb/organicmaps/../omim-build-release ...
[2024-01-18 04:09:46,217] INFO env Generator tool found - /home/bb/organicmaps/../omim-build-release/generator_tool
[2024-01-18 04:09:46,217] INFO env Build name is 2024_01_18__04_09_46.
[2024-01-18 04:09:46,218] INFO env Build path is /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46.
[2024-01-18 04:09:46,218] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46 ...
[2024-01-18 04:09:46,218] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data ...
[2024-01-18 04:09:46,219] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data/borders ...
[2024-01-18 04:09:46,219] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/draft ...
[2024-01-18 04:09:46,219] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/osm2ft ...
[2024-01-18 04:09:46,219] INFO env The following 1 maps will build: Vietnam.
[2024-01-18 04:09:46,219] INFO env Finish setup
[2024-01-18 04:09:46,220] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/logs ...
[2024-01-18 04:09:46,220] INFO stages Stage DownloadAndConvertPlanet: start ...
[2024-01-18 04:09:46,220] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/status ...
[2024-01-18 04:09:46,241] INFO file Trying to download /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf.
[2024-01-18 04:09:46,241] INFO file Trying to download /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf.md5 from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf.md5.
[2024-01-18 04:09:47,199] INFO file File /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf.md5 was downloaded from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf.md5.
[2024-01-18 04:13:39,478] INFO file File /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf was downloaded from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf.
[2024-01-18 04:13:53,529] INFO stages Stage DownloadAndConvertPlanet: finished in 0:04:07.309083
[2024-01-18 04:13:53,530] INFO stages Stage Preprocess: start ...
[2024-01-18 04:13:53,531] INFO env Create /home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 ...
[2024-01-18 04:13:53,560] INFO gen_tool Run generator tool [generator_tool version 2024.01.16-3-394d7d00-Linux]: /home/bb/organicmaps/../omim-build-release/generator_tool --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true
[2024-01-18 04:15:10,735] INFO stages Stage Preprocess: finished in 0:01:17.204467
[2024-01-18 04:15:10,736] ERROR env Uncaught exception: The launch of /home/bb/organicmaps/../omim-build-release/generator_tool failed.
Arguments used: --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true
See details in /home/bb/maps_build/2024_01_18__04_09_46/logs/Preprocess.log
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/bb/organicmaps/tools/python/maps_generator/__main__.py", line 250, in <module>
    main()
  File "/home/bb/organicmaps/tools/python/maps_generator/__main__.py", line 246, in main
    generate_maps(env, from_stage)
  File "/home/bb/organicmaps/tools/python/maps_generator/maps_generator.py", line 43, in generate_maps
    run_generation(env, stages, from_stage)
  File "/home/bb/organicmaps/tools/python/maps_generator/maps_generator.py", line 24, in run_generation
    generation.run(from_stage)
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/generation.py", line 79, in run
    self.run_stages()
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/generation.py", line 85, in run_stages
    stage(self.env)
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/stages.py", line 87, in __call__
    return self.apply(env, **self.args)
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/stages.py", line 199, in apply
    method(obj, env, *args, **kwargs)
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/stages_declaration.py", line 102, in apply
    steps.step_preprocess(env, **kwargs)
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/steps.py", line 95, in step_preprocess
    run_gen_tool(
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/gen_tool.py", line 161, in run_gen_tool
    GenTool(*args, **kwargs).run()
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/gen_tool.py", line 137, in run
    wait_and_raise_if_fail(self)
  File "/home/bb/organicmaps/tools/python/maps_generator/generator/exceptions.py", line 43, in wait_and_raise_if_fail
    raise BadExitStatusError(msg)
maps_generator.generator.exceptions.BadExitStatusError: The launch of /home/bb/organicmaps/../omim-build-release/generator_tool failed.
Arguments used: --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true
See details in /home/bb/maps_build/2024_01_18__04_09_46/logs/Preprocess.log

Log from /home/bb/maps_build/2024_01_18__04_09_46/logs/Preprocess.log

[2024-01-18 04:13:53,530] INFO stages Stage Preprocess: start ...
[2024-01-18 04:13:53,531] INFO env Create /home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 ...
[2024-01-18 04:13:53,560] INFO gen_tool Run generator tool [generator_tool version 2024.01.16-3-394d7d00-Linux]: /home/bb/organicmaps/../omim-build-release/generator_tool --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true
LOG TID(1) INFO   2.5278e-05 generator_tool/generator_tool.cpp:276 operator()(): Generating intermediate data ....
LOG TID(1) INFO       63.431 Reading OSM data from /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m
LOG TID(1) INFO      63.4311 generator/osm_source.cpp:237 GenerateIntermediateData(): Data source: /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m
TID(1) ASSERT FAILED
generator/intermediate_data.cpp:166
CHECK(id < m_data.size()) 8590177076 8589934592 Found node with id 8590177076 which is bigger than the allocated cache size
 0# 0x00005655273D5A70 in /home/bb/organicmaps/../omim-build-release/generator_tool
 1# 0x00007F13F68A1090 in /lib/x86_64-linux-gnu/libc.so.6
 2# gsignal in /lib/x86_64-linux-gnu/libc.so.6
 3# abort in /lib/x86_64-linux-gnu/libc.so.6
 4# 0x000056552732BFFB in /home/bb/organicmaps/../omim-build-release/generator_tool
 5# 0x00005655273D11F5 in /home/bb/organicmaps/../omim-build-release/generator_tool
 6# 0x00005655273D9B4C in /home/bb/organicmaps/../omim-build-release/generator_tool
 7# 0x000056552740A159 in /home/bb/organicmaps/../omim-build-release/generator_tool
 8# 0x000056552739A2EE in /home/bb/organicmaps/../omim-build-release/generator_tool
 9# 0x000056552737FBC9 in /home/bb/organicmaps/../omim-build-release/generator_tool
10# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
11# 0x00005655273952CE in /home/bb/organicmaps/../omim-build-release/generator_tool
[2024-01-18 04:15:10,735] INFO stages Stage Preprocess: finished in 0:01:17.204467

map_generator.ini file:

[Developer]
# Path to the `organicmaps` source code repository:
OMIM_PATH: ~/organicmaps
# A path with the generator_tool binary:
BUILD_PATH: ${Developer:OMIM_PATH}/../omim-build-release


[Main]
# A special small planet file will be downloaded if DEBUG is set to 1.
DEBUG: 0
# A main working directory. There is a subdirectory created for each generator run
# which contains the planet and other downloads, temporary build files, logs and completed MWMs.
MAIN_OUT_PATH: ${Developer:OMIM_PATH}/../maps_build
# Path for storing caches for nodes, ways, relations.
# If it's not set then caches are stored inside the directory of the current build.
CACHE_PATH: ${Main:MAIN_OUT_PATH}/cache


[Generator tool]
# Path to the data/ folder in the repository:
USER_RESOURCE_PATH: ${Developer:OMIM_PATH}/data
# Features stage only parallelism level. Set to 0 for auto detection.
THREADS_COUNT_FEATURES_STAGE: 0


[Osm tools]
# Path to osmctools binaries:
OSM_TOOLS_PATH: ${Developer:OMIM_PATH}/../osmctools
# If the binaries are not found neither in the configured path nor system-wide,
# then the tools are built from the sources:
OSM_TOOLS_SRC_PATH: ${Developer:OMIM_PATH}/tools/osmctools


[Logging]
# maps_generator's general (python output only) log file path and name.
# More detailed logs that include output of the `generator_tool` binary
# are located in the `logs/` subdir of a particular build directory,
# e.g. `maps_build/2023_06_04__20_05_07/logs/`.
LOG_FILE_PATH: ${Main:MAIN_OUT_PATH}/generation.log


[External]
# Planet file location. It should be a dump of OSM data in osm.pbf format.
# By default its an entire planet from "planet.openstreetmap.org".
# Or set it to a particular country/region extract from e.g. [Geofabrik](http://download.geofabrik.de/index.html).
# Note that an entire planet generation takes 40+ hours on a 256GB RAM server (and 1TB+ disk space).
# Stick to smaller extracts unless you have a machine this large.
# Here and further, its possible to specify either an URL (to be downloaded automatically)
# or a local file path like file:///path/to/file.
# A sample URL to download a latest OSM dump for North Macedonia:
PLANET_URL: https://download.geofabrik.de/asia/vietnam-latest.osm.pbf
# Location of the md5 checksum of the planet file:
PLANET_MD5_URL: ${External:PLANET_URL}.md5
# A base url to the latest_coasts.geom and latest_coasts.rawgeom files.
# For example, if PLANET_COASTS_URL = https://somesite.com/download/
# then the https://somesite.com/download/latest_coasts.geom url will be used to download latest_coasts.geom and
# the https://somesite.com/download/latest_coasts.rawgeom url will be used to download latest_coasts.rawgeom.
# Comment to skip getting the coastlines files.
# PLANET_COASTS_URL:
# Should be 'true' for an entire planet build to make a special routing section in World.mwm
# for alerting about absent regions without which the route can't be built.
NEED_BUILD_WORLD_ROADS: false
# Subway file location, see docs/SUBWAY_GENERATION.md if you want to generate your own file.
# Comment to disable subway layer generation.
#SUBWAY_URL: https://cdn.organicmaps.app/subway.json

# Location of the EXPERIMENTAL GTFS-extracted public transport transit files:
# TRANSIT_URL:

# Urls for production maps generation.
# UGC_URL:
# HOTELS_URL:
# PROMO_CATALOG_CITIES:
# POPULARITY_URL:
# FOOD_URL:
# FOOD_TRANSLATIONS_URL:
# SRTM_PATH:
# ISOLINES_PATH:
# UK_POSTCODES_URL:
# US_POSTCODES_URL:


[Stages]
# Set to 1 to update the entire OSM planet file (as taken from "planet.openstreetmap.org")
# via an osmupdate tool before the generation. Not for use with partial planet extracts.
NEED_PLANET_UPDATE: 0
# If you want to calculate diffs you need to specify where the old maps are.
DATA_ARCHIVE_DIR: ${Generator tool:USER_RESOURCE_PATH}
# How many versions in the archive to use for diff calculation:
DIFF_VERSION_DEPTH: 2


[Common]
# Default parallelism level for the most of jobs. Set to 0 for auto detection.
THREADS_COUNT: 0


[Stats]
# Path to rules for calculating statistics by type:
STATS_TYPES_CONFIG: ${Developer:OMIM_PATH}/tools/python/maps_generator/var/etc/stats_types_config.txt

System information:

  • Operating system and its version: Ubuntu 20.04.4 LTS
  • Organic Maps version: master branch
**Steps to reproduce** 1. Run `python -m maps_generator --countries="Vietnam" --skip="Coastline"` 2. Got this error `Found node with id 8590177076 which is bigger than the allocated cache size` 3. Detailed error: ``` Loading configuration from /home/bb/organicmaps/tools/python/maps_generator/var/etc/map_generator.ini [2024-01-18 04:09:46,216] INFO env Start setup ... [2024-01-18 04:09:46,216] INFO env Check for the osmctools binaries... [2024-01-18 04:09:46,217] INFO env Found osmctools at /home/bb/organicmaps/../osmctools/osmconvert, /home/bb/organicmaps/../osmctools/osmupdate, /home/bb/organicmaps/../osmctools/osmfilter [2024-01-18 04:09:46,217] INFO env Check generator tool ... [2024-01-18 04:09:46,217] INFO env Looking for generator tool in /home/bb/organicmaps/../omim-build-release ... [2024-01-18 04:09:46,217] INFO env Generator tool found - /home/bb/organicmaps/../omim-build-release/generator_tool [2024-01-18 04:09:46,217] INFO env Build name is 2024_01_18__04_09_46. [2024-01-18 04:09:46,218] INFO env Build path is /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46. [2024-01-18 04:09:46,218] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46 ... [2024-01-18 04:09:46,218] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data ... [2024-01-18 04:09:46,219] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data/borders ... [2024-01-18 04:09:46,219] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/draft ... [2024-01-18 04:09:46,219] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/osm2ft ... [2024-01-18 04:09:46,219] INFO env The following 1 maps will build: Vietnam. [2024-01-18 04:09:46,219] INFO env Finish setup [2024-01-18 04:09:46,220] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/logs ... [2024-01-18 04:09:46,220] INFO stages Stage DownloadAndConvertPlanet: start ... [2024-01-18 04:09:46,220] INFO env Create /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/status ... [2024-01-18 04:09:46,241] INFO file Trying to download /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf. [2024-01-18 04:09:46,241] INFO file Trying to download /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf.md5 from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf.md5. [2024-01-18 04:09:47,199] INFO file File /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf.md5 was downloaded from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf.md5. [2024-01-18 04:13:39,478] INFO file File /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.osm.pbf was downloaded from https://download.geofabrik.de/asia/vietnam-latest.osm.pbf. [2024-01-18 04:13:53,529] INFO stages Stage DownloadAndConvertPlanet: finished in 0:04:07.309083 [2024-01-18 04:13:53,530] INFO stages Stage Preprocess: start ... [2024-01-18 04:13:53,531] INFO env Create /home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 ... [2024-01-18 04:13:53,560] INFO gen_tool Run generator tool [generator_tool version 2024.01.16-3-394d7d00-Linux]: /home/bb/organicmaps/../omim-build-release/generator_tool --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true [2024-01-18 04:15:10,735] INFO stages Stage Preprocess: finished in 0:01:17.204467 [2024-01-18 04:15:10,736] ERROR env Uncaught exception: The launch of /home/bb/organicmaps/../omim-build-release/generator_tool failed. Arguments used: --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true See details in /home/bb/maps_build/2024_01_18__04_09_46/logs/Preprocess.log Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/bb/organicmaps/tools/python/maps_generator/__main__.py", line 250, in <module> main() File "/home/bb/organicmaps/tools/python/maps_generator/__main__.py", line 246, in main generate_maps(env, from_stage) File "/home/bb/organicmaps/tools/python/maps_generator/maps_generator.py", line 43, in generate_maps run_generation(env, stages, from_stage) File "/home/bb/organicmaps/tools/python/maps_generator/maps_generator.py", line 24, in run_generation generation.run(from_stage) File "/home/bb/organicmaps/tools/python/maps_generator/generator/generation.py", line 79, in run self.run_stages() File "/home/bb/organicmaps/tools/python/maps_generator/generator/generation.py", line 85, in run_stages stage(self.env) File "/home/bb/organicmaps/tools/python/maps_generator/generator/stages.py", line 87, in __call__ return self.apply(env, **self.args) File "/home/bb/organicmaps/tools/python/maps_generator/generator/stages.py", line 199, in apply method(obj, env, *args, **kwargs) File "/home/bb/organicmaps/tools/python/maps_generator/generator/stages_declaration.py", line 102, in apply steps.step_preprocess(env, **kwargs) File "/home/bb/organicmaps/tools/python/maps_generator/generator/steps.py", line 95, in step_preprocess run_gen_tool( File "/home/bb/organicmaps/tools/python/maps_generator/generator/gen_tool.py", line 161, in run_gen_tool GenTool(*args, **kwargs).run() File "/home/bb/organicmaps/tools/python/maps_generator/generator/gen_tool.py", line 137, in run wait_and_raise_if_fail(self) File "/home/bb/organicmaps/tools/python/maps_generator/generator/exceptions.py", line 43, in wait_and_raise_if_fail raise BadExitStatusError(msg) maps_generator.generator.exceptions.BadExitStatusError: The launch of /home/bb/organicmaps/../omim-build-release/generator_tool failed. Arguments used: --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true See details in /home/bb/maps_build/2024_01_18__04_09_46/logs/Preprocess.log ``` Log from `/home/bb/maps_build/2024_01_18__04_09_46/logs/Preprocess.log` ``` [2024-01-18 04:13:53,530] INFO stages Stage Preprocess: start ... [2024-01-18 04:13:53,531] INFO env Create /home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 ... [2024-01-18 04:13:53,560] INFO gen_tool Run generator tool [generator_tool version 2024.01.16-3-394d7d00-Linux]: /home/bb/organicmaps/../omim-build-release/generator_tool --threads_count=1 --data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --intermediate_data_path=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/intermediate_data --cache_path=/home/bb/organicmaps/../maps_build/cache/2024_01_18__04_09_46 --osm_file_type=o5m --osm_file_name=/home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m --node_storage=mem --user_resource_path=/home/bb/organicmaps/data --preprocess=true LOG TID(1) INFO 2.5278e-05 generator_tool/generator_tool.cpp:276 operator()(): Generating intermediate data .... LOG TID(1) INFO 63.431 Reading OSM data from /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m LOG TID(1) INFO 63.4311 generator/osm_source.cpp:237 GenerateIntermediateData(): Data source: /home/bb/organicmaps/../maps_build/2024_01_18__04_09_46/planet.o5m TID(1) ASSERT FAILED generator/intermediate_data.cpp:166 CHECK(id < m_data.size()) 8590177076 8589934592 Found node with id 8590177076 which is bigger than the allocated cache size 0# 0x00005655273D5A70 in /home/bb/organicmaps/../omim-build-release/generator_tool 1# 0x00007F13F68A1090 in /lib/x86_64-linux-gnu/libc.so.6 2# gsignal in /lib/x86_64-linux-gnu/libc.so.6 3# abort in /lib/x86_64-linux-gnu/libc.so.6 4# 0x000056552732BFFB in /home/bb/organicmaps/../omim-build-release/generator_tool 5# 0x00005655273D11F5 in /home/bb/organicmaps/../omim-build-release/generator_tool 6# 0x00005655273D9B4C in /home/bb/organicmaps/../omim-build-release/generator_tool 7# 0x000056552740A159 in /home/bb/organicmaps/../omim-build-release/generator_tool 8# 0x000056552739A2EE in /home/bb/organicmaps/../omim-build-release/generator_tool 9# 0x000056552737FBC9 in /home/bb/organicmaps/../omim-build-release/generator_tool 10# __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6 11# 0x00005655273952CE in /home/bb/organicmaps/../omim-build-release/generator_tool [2024-01-18 04:15:10,735] INFO stages Stage Preprocess: finished in 0:01:17.204467 ``` `map_generator.ini` file: ``` [Developer] # Path to the `organicmaps` source code repository: OMIM_PATH: ~/organicmaps # A path with the generator_tool binary: BUILD_PATH: ${Developer:OMIM_PATH}/../omim-build-release [Main] # A special small planet file will be downloaded if DEBUG is set to 1. DEBUG: 0 # A main working directory. There is a subdirectory created for each generator run # which contains the planet and other downloads, temporary build files, logs and completed MWMs. MAIN_OUT_PATH: ${Developer:OMIM_PATH}/../maps_build # Path for storing caches for nodes, ways, relations. # If it's not set then caches are stored inside the directory of the current build. CACHE_PATH: ${Main:MAIN_OUT_PATH}/cache [Generator tool] # Path to the data/ folder in the repository: USER_RESOURCE_PATH: ${Developer:OMIM_PATH}/data # Features stage only parallelism level. Set to 0 for auto detection. THREADS_COUNT_FEATURES_STAGE: 0 [Osm tools] # Path to osmctools binaries: OSM_TOOLS_PATH: ${Developer:OMIM_PATH}/../osmctools # If the binaries are not found neither in the configured path nor system-wide, # then the tools are built from the sources: OSM_TOOLS_SRC_PATH: ${Developer:OMIM_PATH}/tools/osmctools [Logging] # maps_generator's general (python output only) log file path and name. # More detailed logs that include output of the `generator_tool` binary # are located in the `logs/` subdir of a particular build directory, # e.g. `maps_build/2023_06_04__20_05_07/logs/`. LOG_FILE_PATH: ${Main:MAIN_OUT_PATH}/generation.log [External] # Planet file location. It should be a dump of OSM data in osm.pbf format. # By default its an entire planet from "planet.openstreetmap.org". # Or set it to a particular country/region extract from e.g. [Geofabrik](http://download.geofabrik.de/index.html). # Note that an entire planet generation takes 40+ hours on a 256GB RAM server (and 1TB+ disk space). # Stick to smaller extracts unless you have a machine this large. # Here and further, its possible to specify either an URL (to be downloaded automatically) # or a local file path like file:///path/to/file. # A sample URL to download a latest OSM dump for North Macedonia: PLANET_URL: https://download.geofabrik.de/asia/vietnam-latest.osm.pbf # Location of the md5 checksum of the planet file: PLANET_MD5_URL: ${External:PLANET_URL}.md5 # A base url to the latest_coasts.geom and latest_coasts.rawgeom files. # For example, if PLANET_COASTS_URL = https://somesite.com/download/ # then the https://somesite.com/download/latest_coasts.geom url will be used to download latest_coasts.geom and # the https://somesite.com/download/latest_coasts.rawgeom url will be used to download latest_coasts.rawgeom. # Comment to skip getting the coastlines files. # PLANET_COASTS_URL: # Should be 'true' for an entire planet build to make a special routing section in World.mwm # for alerting about absent regions without which the route can't be built. NEED_BUILD_WORLD_ROADS: false # Subway file location, see docs/SUBWAY_GENERATION.md if you want to generate your own file. # Comment to disable subway layer generation. #SUBWAY_URL: https://cdn.organicmaps.app/subway.json # Location of the EXPERIMENTAL GTFS-extracted public transport transit files: # TRANSIT_URL: # Urls for production maps generation. # UGC_URL: # HOTELS_URL: # PROMO_CATALOG_CITIES: # POPULARITY_URL: # FOOD_URL: # FOOD_TRANSLATIONS_URL: # SRTM_PATH: # ISOLINES_PATH: # UK_POSTCODES_URL: # US_POSTCODES_URL: [Stages] # Set to 1 to update the entire OSM planet file (as taken from "planet.openstreetmap.org") # via an osmupdate tool before the generation. Not for use with partial planet extracts. NEED_PLANET_UPDATE: 0 # If you want to calculate diffs you need to specify where the old maps are. DATA_ARCHIVE_DIR: ${Generator tool:USER_RESOURCE_PATH} # How many versions in the archive to use for diff calculation: DIFF_VERSION_DEPTH: 2 [Common] # Default parallelism level for the most of jobs. Set to 0 for auto detection. THREADS_COUNT: 0 [Stats] # Path to rules for calculating statistics by type: STATS_TYPES_CONFIG: ${Developer:OMIM_PATH}/tools/python/maps_generator/var/etc/stats_types_config.txt ``` **System information:** - Operating system and its version: Ubuntu 20.04.4 LTS - Organic Maps version: master branch
This repo is archived. You cannot comment on issues.
No labels
Accessibility
Accessibility
Address
Address
Android
Android
Android Auto
Android Auto
Android Automotive (AAOS)
Android Automotive (AAOS)
API
API
AppGallery
AppGallery
AppStore
AppStore
Battery and Performance
Battery and Performance
Blocker
Blocker
Bookmarks and Tracks
Bookmarks and Tracks
Borders
Borders
Bug
Bug
Build
Build
CarPlay
CarPlay
Classificator
Classificator
Community
Community
Core
Core
CrashReports
CrashReports
Cycling
Cycling
Desktop
Desktop
DevEx
DevEx
DevOps
DevOps
dev_sandbox
dev_sandbox
Directions
Directions
Documentation
Documentation
Downloader
Downloader
Drape
Drape
Driving
Driving
Duplicate
Duplicate
Editor
Editor
Elevation
Elevation
Enhancement
Enhancement
Epic
Epic
External Map Datasets
External Map Datasets
F-Droid
F-Droid
Fonts
Fonts
Frequently User Reported
Frequently User Reported
Fund
Fund
Generator
Generator
Good first issue
Good first issue
Google Play
Google Play
GPS
GPS
GSoC
GSoC
iCloud
iCloud
Icons
Icons
iOS
iOS
Legal
Legal
Linux Desktop
Linux Desktop
Linux packaging
Linux packaging
Linux Phone
Linux Phone
Mac OS
Mac OS
Map Data
Map Data
Metro
Metro
Navigation
Navigation
Need Feedback
Need Feedback
Night Mode
Night Mode
NLnet 2024-06-281
NLnet 2024-06-281
No Feature Parity
No Feature Parity
Opening Hours
Opening Hours
Outdoors
Outdoors
POI Info
POI Info
Privacy
Privacy
Public Transport
Public Transport
Raw Idea
Raw Idea
Refactoring
Refactoring
Regional
Regional
Regression
Regression
Releases
Releases
RoboTest
RoboTest
Route Planning
Route Planning
Routing
Routing
Ruler
Ruler
Search
Search
Security
Security
Styles
Styles
Tests
Tests
Track Recording
Track Recording
Translations
Translations
TTS
TTS
UI
UI
UX
UX
Walk Navigation
Walk Navigation
Watches
Watches
Web
Web
Wikipedia
Wikipedia
Windows
Windows
Won't fix
Won't fix
World Map
World Map
No milestone
No project
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: organicmaps/organicmaps-tmp#7199
No description provided.