[generator][search][routing] Long time Generator refactoring. #6537

Merged
vng merged 82 commits from vng-gen-1 into master 2023-11-22 15:39:59 +00:00
vng commented 2023-11-11 01:43:21 +00:00 (Migrated from github.com)

175919 lines are from OSM test data for the Generator.
The pure code lines number is ~ 8000

175919 lines are from OSM test data for the Generator. The pure code lines number is ~ 8000
biodranik (Migrated from github.com) approved these changes 2023-11-12 07:58:41 +00:00
biodranik (Migrated from github.com) left a comment

You can add any edits on top in a separate commit.

You can add any edits on top in a separate commit.
@ -74,0 +107,4 @@
std::vector<PtrT> vec;
vec.reserve(count);
biodranik (Migrated from github.com) commented 2023-11-11 16:59:23 +00:00

std::greater?

std::greater?
@ -74,0 +113,4 @@
vec.push_back(queue.top());
queue.pop();
}
biodranik (Migrated from github.com) commented 2023-11-11 17:00:14 +00:00

What if count == 0?

What if count == 0?
biodranik (Migrated from github.com) commented 2023-11-11 17:00:43 +00:00

Is this log needed?

Is this log needed?
biodranik (Migrated from github.com) commented 2023-11-11 07:09:24 +00:00
bool StartsWith(std::string_view s1, std::string_view s2)
```suggestion bool StartsWith(std::string_view s1, std::string_view s2) ```
biodranik (Migrated from github.com) commented 2023-11-11 07:09:42 +00:00
bool StartsWith(std::string_view s1, std::string_view s2);
```suggestion bool StartsWith(std::string_view s1, std::string_view s2); ```
biodranik (Migrated from github.com) commented 2023-11-11 17:02:08 +00:00

Remove?

Remove?
biodranik (Migrated from github.com) commented 2023-11-11 17:03:22 +00:00
  auto const i = m_addresses.find(id);
```suggestion auto const i = m_addresses.find(id); ```
biodranik (Migrated from github.com) commented 2023-11-11 17:04:37 +00:00
    auto const i = m_addresses.find(id);
```suggestion auto const i = m_addresses.find(id); ```
biodranik (Migrated from github.com) commented 2023-11-11 17:06:00 +00:00
    auto const it = mapping.find(base::asserted_cast<uint32_t>(fid));
```suggestion auto const it = mapping.find(base::asserted_cast<uint32_t>(fid)); ```
biodranik (Migrated from github.com) commented 2023-11-11 17:06:32 +00:00

std?

std?
@ -24,0 +71,4 @@
/// @note Mutable function!
template <class FnT> void ForEachLocality(FnT && fn)
{
for (auto & loc : m_data)
biodranik (Migrated from github.com) commented 2023-11-11 17:20:24 +00:00

Should it be initialized?

Should it be initialized?
biodranik (Migrated from github.com) commented 2023-11-12 07:15:58 +00:00
  /// @todo Do we need all Metadata for point/area World features? We delete meta for linear in ZeroParams.
```suggestion /// @todo Do we need all Metadata for point/area World features? We delete meta for linear in ZeroParams. ```
biodranik (Migrated from github.com) commented 2023-11-12 07:16:15 +00:00

TODO?

TODO?
biodranik (Migrated from github.com) commented 2023-11-12 07:18:06 +00:00

Is it a hack?

Is it a hack?
biodranik (Migrated from github.com) commented 2023-11-12 07:18:44 +00:00

Don't it make the search worse?

Don't it make the search worse?
biodranik (Migrated from github.com) commented 2023-11-12 07:28:53 +00:00
      feature::FeatureBuilder & feature = it->second;
```suggestion feature::FeatureBuilder & feature = it->second; ```
biodranik (Migrated from github.com) commented 2023-11-12 07:29:01 +00:00
      if (m_affiliation->GetAffiliations(feature).size() != 1)
```suggestion if (m_affiliation->GetAffiliations(feature).size() != 1) ```
biodranik (Migrated from github.com) commented 2023-11-12 07:29:11 +00:00
        features.emplace_back(&feature, wayId);
```suggestion features.emplace_back(&feature, wayId); ```
biodranik (Migrated from github.com) commented 2023-11-12 07:27:48 +00:00
  size_t constexpr kReserveSize = 4 * 1024;
```suggestion size_t constexpr kReserveSize = 4 * 1024; ```
@ -769,1 +881,3 @@
{
CHECK(calcOrg, ());
auto const org = calcOrg(p);
if (org && s_countriesChecker.IsTransformToState(*org))
biodranik (Migrated from github.com) commented 2023-11-12 07:33:13 +00:00

const?

const?
biodranik (Migrated from github.com) commented 2023-11-12 07:36:37 +00:00

Can clear be done in ApplyRules?

Can clear be done in ApplyRules?
@ -22,0 +29,4 @@
template <class Source> void Load(Source & src, RoadAccess::Type & ac)
{
uint8_t const res = ReadPrimitiveFromSource<uint8_t>(src);
CHECK_LESS(res, uint8_t(RoadAccess::Type::Count), ());
biodranik (Migrated from github.com) commented 2023-11-12 07:46:41 +00:00
template <class Source> RoadAccess::Type Load(Source & src)
```suggestion template <class Source> RoadAccess::Type Load(Source & src) ```
biodranik (Migrated from github.com) commented 2023-11-12 07:55:57 +00:00

Should this list be manually updated after adding other metadata fields?..

Should this list be manually updated after adding other metadata fields?..
vng (Migrated from github.com) reviewed 2023-11-12 12:30:42 +00:00
@ -74,0 +107,4 @@
std::vector<PtrT> vec;
vec.reserve(count);
vng (Migrated from github.com) commented 2023-11-12 12:30:41 +00:00

No, there is a l->second.

No, there is a l->second.
vng (Migrated from github.com) reviewed 2023-11-12 12:33:54 +00:00
vng (Migrated from github.com) commented 2023-11-12 12:33:54 +00:00

The main purpose of this function is LOG: PrintTop

The main purpose of this function is LOG: PrintTop
vng (Migrated from github.com) reviewed 2023-11-12 12:35:19 +00:00
vng (Migrated from github.com) commented 2023-11-12 12:35:19 +00:00

Good idea to get back in future.

Good idea to get back in future.
vng (Migrated from github.com) reviewed 2023-11-12 12:38:57 +00:00
@ -24,0 +71,4 @@
/// @note Mutable function!
template <class FnT> void ForEachLocality(FnT && fn)
{
for (auto & loc : m_data)
vng (Migrated from github.com) commented 2023-11-12 12:38:57 +00:00

Default initialization is ok here.

Default initialization is ok here.
vng (Migrated from github.com) reviewed 2023-11-12 12:41:42 +00:00
vng (Migrated from github.com) commented 2023-11-12 12:41:42 +00:00

Actually, no. But, will add todo if we will decide to process OSM's short_name.

Actually, no. But, will add todo if we will decide to process OSM's short_name.
vng (Migrated from github.com) reviewed 2023-11-12 12:46:05 +00:00
vng (Migrated from github.com) commented 2023-11-12 12:46:05 +00:00

We make 4 replacements only now, no problem here:

./World.log:LOG TID(1) WARNING   4818.22 generator/feature_sorter.cpp:267 operator()(): Replace USA;US;The States;United States with US for Osm Node 424317935
./World.log:LOG TID(1) WARNING   15721.9 generator/feature_sorter.cpp:267 operator()(): Replace P.R. China with PRC for Osm Node 424313582
./World.log:LOG TID(1) WARNING   15918.5 generator/feature_sorter.cpp:267 operator()(): Replace Russia;Россия with РФ for Osm Node 424314830
./World.log:LOG TID(1) WARNING   23349.4 generator/feature_sorter.cpp:267 operator()(): Replace Penna. with PA for Osm Node 316987717
We make 4 replacements only now, no problem here: ``` ./World.log:LOG TID(1) WARNING 4818.22 generator/feature_sorter.cpp:267 operator()(): Replace USA;US;The States;United States with US for Osm Node 424317935 ./World.log:LOG TID(1) WARNING 15721.9 generator/feature_sorter.cpp:267 operator()(): Replace P.R. China with PRC for Osm Node 424313582 ./World.log:LOG TID(1) WARNING 15918.5 generator/feature_sorter.cpp:267 operator()(): Replace Russia;Россия with РФ for Osm Node 424314830 ./World.log:LOG TID(1) WARNING 23349.4 generator/feature_sorter.cpp:267 operator()(): Replace Penna. with PA for Osm Node 316987717 ```
vng (Migrated from github.com) reviewed 2023-11-12 13:03:49 +00:00
vng (Migrated from github.com) commented 2023-11-12 13:03:48 +00:00

ApplyRules is called in other places so decided not to change. Maybe I can make it a little better.

ApplyRules is called in other places so decided not to change. Maybe I can make it a little better.
vng (Migrated from github.com) reviewed 2023-11-12 13:05:27 +00:00
@ -22,0 +29,4 @@
template <class Source> void Load(Source & src, RoadAccess::Type & ac)
{
uint8_t const res = ReadPrimitiveFromSource<uint8_t>(src);
CHECK_LESS(res, uint8_t(RoadAccess::Type::Count), ());
vng (Migrated from github.com) commented 2023-11-12 13:05:27 +00:00

Function's overloading is used here with common notation: void Load(Source &, T &);

Function's overloading is used here with common notation: void Load(Source &, T &);
vng (Migrated from github.com) reviewed 2023-11-12 13:06:54 +00:00
vng (Migrated from github.com) commented 2023-11-12 13:06:54 +00:00

Well, not obvious, but I didn't have a better solution.

Well, not obvious, but I didn't have a better solution.
biodranik (Migrated from github.com) approved these changes 2023-11-12 18:19:44 +00:00
pastk requested changes 2023-11-12 19:45:01 +00:00
@ -873,0 +1040,4 @@
{"motor_vehicle", "private", [&flags] { flags[Flags::MotorVehicle] = -1; }},
{"motor_vehicle", "!", [&flags] { flags[Flags::MotorVehicle] = -1; }},
{"motor_vehicle", "yes", [&flags] { flags[Flags::MotorVehicle] = 1; }},
{"motor_vehicle", "designated", [&flags] { flags[Flags::MotorVehicle] = 1; }},

Leads to double logging, e.g.

LOG TID(3) WARNING   20.2459 indexer/feature_data.cpp:456 FinishAddingTypesEx(): Exceeded max types count: amenity-recycling-container recycling-cans recycling-cardboard recycling-glass_bottles recycling-green_waste recycling-paper recycling-plastic recycling-plastic_bottles recycling-scrap_metal
LOG TID(3) WARNING   20.2461 generator/osm2type.cpp:1323 GetNameAndType(): Exceeded types count for: Osm Node 5264249349 Types: amenity-recycling-container recycling-cans recycling-cardboard recycling-glass_bottles recycling-green_waste recycling-paper recycling-plastic recycling-plastic_bottles recycling-scrap_metal
Leads to double logging, e.g. ``` LOG TID(3) WARNING 20.2459 indexer/feature_data.cpp:456 FinishAddingTypesEx(): Exceeded max types count: amenity-recycling-container recycling-cans recycling-cardboard recycling-glass_bottles recycling-green_waste recycling-paper recycling-plastic recycling-plastic_bottles recycling-scrap_metal LOG TID(3) WARNING 20.2461 generator/osm2type.cpp:1323 GetNameAndType(): Exceeded types count for: Osm Node 5264249349 Types: amenity-recycling-container recycling-cans recycling-cardboard recycling-glass_bottles recycling-green_waste recycling-paper recycling-plastic recycling-plastic_bottles recycling-scrap_metal ```
@ -409,29 +427,35 @@ void FeatureParams::SetRwSubwayType(char const * cityName)
}
}

Did you overwrite it by accident? It was changed in d01929a3ee

I'm worried some other more important changes could be reverted elsewhere..

Did you overwrite it by accident? It was changed in d01929a3ee973c38121446a1e73646aee115fb02 I'm worried some other more important changes could be reverted elsewhere..
vng (Migrated from github.com) reviewed 2023-11-12 19:53:47 +00:00
@ -409,29 +427,35 @@ void FeatureParams::SetRwSubwayType(char const * cityName)
}
}
vng (Migrated from github.com) commented 2023-11-12 19:53:47 +00:00

Should be deleted at all, seems like a mistake in a merge conflict.

Should be deleted at all, seems like a mistake in a merge conflict.
pastk reviewed 2023-11-12 20:15:43 +00:00
@ -759,0 +851,4 @@
auto const countryId = m_infoGetter->GetRegionCountryId(pt);
if (!countryId.empty())
{
auto const country = storage::GetTopmostParentFor(*m_countryTree, countryId);

nit whitespace

nit whitespace
@ -759,0 +873,4 @@
// 2. Check valid capital. We support only place-city-capital-2 (not town, village, etc).
if (isCapital && !value.empty())
value = "city";

"stronger" here and below

"st**r**onger" here and below
pastk reviewed 2023-11-14 14:03:47 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());

what cases did you want to fix with that change?

this could work for some cases, but could be controversial/detrimental in others...
e.g. imagine amenity-parking covered=yes (which implies it has some kind of roof, it doesn't imply its an underground parking), then this change will make it layer=-1 and it'll disappear if it was mapped over e.g. a park or industrial landuse etc.

what cases did you want to fix with that change? this could work for some cases, but could be controversial/detrimental in others... e.g. imagine `amenity-parking covered=yes` (which implies it has some kind of roof, it doesn't imply its an underground parking), then this change will make it `layer=-1` and it'll disappear if it was mapped over e.g. a park or industrial landuse etc.
pastk reviewed 2023-11-14 14:10:50 +00:00

is it possible it'd create a duplicate tag if it existed already?

is it possible it'd create a duplicate tag if it existed already?
pastk reviewed 2023-11-14 15:30:36 +00:00
@ -162,3 +153,1 @@
// TODO : re-checks geom limit rect size via IsDrawableForIndexGeometryOnly() which was checked already in CalculateMidPoints.
if (fb.IsDrawableInRange(scales::PatchMinDrawableScale(i > 0 ? m_header.GetScale(i - 1) + 1 : 0),
scales::PatchMaxDrawableScale(level)))
bool const isLine = fb.IsLine();

why did remove the message?
just looking at the code its not obvious it checks for geometry types (need to look into isLine()/isArea() impl)

why did remove the message? just looking at the code its not obvious it checks for geometry types (need to look into isLine()/isArea() impl)

won't it be better to make coasts' trg0 fallback to trg1 always?
or explicitly keep only 3 geom levels for coasts (expand trg0 to [1-5] instead of [1-3])

won't it be better to make coasts' trg0 fallback to trg1 always? or explicitly keep only 3 geom levels for coasts (expand trg0 to `[1-5]` instead of `[1-3]`)

maybe just leave a todo comment for now if you agree

maybe just leave a todo comment for now if you agree

wrong indent left

wrong indent left
pastk reviewed 2023-11-14 16:05:40 +00:00
@ -62,20 +76,38 @@ bool RelationTagsWay::IsAcceptBoundary(RelationElement const & e) const
{
// Do not accumulate boundary types (boundary=administrative) for inner polygons.

Would be good to add some examples of "skipped useful linear tags", otherwise its not clear what is it about.

nit: "according to the the drule's geometry type"

Would be good to add some examples of "skipped useful linear tags", otherwise its not clear what is it about. nit: "according to **the the** drule's geometry type"
pastk reviewed 2023-11-14 16:33:27 +00:00
@ -130,2 +130,4 @@
}
/// @todo Should we also take into account "none", "false" here?

remove this newline, otherwise its not very clear to which block the comment belongs

remove this newline, otherwise its not very clear to which block the comment belongs
pastk reviewed 2023-11-14 16:33:36 +00:00

maybe it'd be better to remove all useless ;name;int_name; in one go, as single changes like this can confuse people?

maybe it'd be better to remove all useless `;name;int_name;` in one go, as single changes like this can confuse people?
pastk reviewed 2023-11-14 17:32:55 +00:00

why smoothness is ignored for trunks & motorways?
there could be even unpaved ones

why smoothness is ignored for trunks & motorways? there could be even unpaved ones
vng (Migrated from github.com) reviewed 2023-11-14 22:33:45 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());
vng (Migrated from github.com) commented 2023-11-14 22:33:45 +00:00

Probably this one organicmaps/organicmaps#2564
Can't say for sure if it is really needed.

Probably this one https://git.omaps.dev/organicmaps/organicmaps/issues/2564 Can't say for sure if it is really needed.
vng (Migrated from github.com) reviewed 2023-11-14 22:34:02 +00:00
vng (Migrated from github.com) commented 2023-11-14 22:34:02 +00:00

Maybe, but no problem here.

Maybe, but no problem here.
vng (Migrated from github.com) reviewed 2023-11-14 22:35:09 +00:00
@ -162,3 +153,1 @@
// TODO : re-checks geom limit rect size via IsDrawableForIndexGeometryOnly() which was checked already in CalculateMidPoints.
if (fb.IsDrawableInRange(scales::PatchMinDrawableScale(i > 0 ? m_header.GetScale(i - 1) + 1 : 0),
scales::PatchMaxDrawableScale(level)))
bool const isLine = fb.IsLine();
vng (Migrated from github.com) commented 2023-11-14 22:35:09 +00:00

Seems like merging conflicts. Well, check body and context is more than enough here, no? "масло масленное" otherwise.

Seems like merging conflicts. Well, check body and context is more than enough here, no? "масло масленное" otherwise.
vng (Migrated from github.com) reviewed 2023-11-14 22:44:28 +00:00
@ -162,3 +153,1 @@
// TODO : re-checks geom limit rect size via IsDrawableForIndexGeometryOnly() which was checked already in CalculateMidPoints.
if (fb.IsDrawableInRange(scales::PatchMinDrawableScale(i > 0 ? m_header.GetScale(i - 1) + 1 : 0),
scales::PatchMaxDrawableScale(level)))
bool const isLine = fb.IsLine();
vng (Migrated from github.com) commented 2023-11-14 22:44:28 +00:00

Well, trg0 -> 1 maybe better, but maybe a bit slow. With increments, current simplification zooms are:
{5, 6, 8, 10}.

Well, trg0 -> 1 maybe better, but maybe a bit slow. With increments, current simplification zooms are: {5, 6, 8, 10}.
vng (Migrated from github.com) reviewed 2023-11-15 00:21:04 +00:00
@ -62,20 +76,38 @@ bool RelationTagsWay::IsAcceptBoundary(RelationElement const & e) const
{
// Do not accumulate boundary types (boundary=administrative) for inner polygons.
vng (Migrated from github.com) commented 2023-11-15 00:21:04 +00:00

Just after the comment, we skip tags inheritance for the multipolygon relation (). Added barrier as an exception, but suppose that there are more tags like this, when ways should inherit from parent relation.

Just after the comment, we skip tags inheritance for the multipolygon relation (). Added barrier as an exception, but suppose that there are more tags like this, when ways should inherit from parent relation.
vng (Migrated from github.com) reviewed 2023-11-15 00:24:34 +00:00
@ -130,2 +130,4 @@
}
/// @todo Should we also take into account "none", "false" here?
vng (Migrated from github.com) commented 2023-11-15 00:24:34 +00:00

Looks good here IMO, since the next 2 separated blocks below process "no".

Looks good here IMO, since the next 2 separated blocks below process "no".
vng (Migrated from github.com) reviewed 2023-11-15 00:26:48 +00:00
vng (Migrated from github.com) commented 2023-11-15 00:26:48 +00:00

"surface" tag is not ignored, but smoothness is controversial in the examples I saw. There are a lot of them in Ukraine, where somebody tagged all main roads like smoothness=intermediate and the app prefers nearby primary/secondary roads without tags, which are definitely not better than trunk/motorway.

"surface" tag is not ignored, but smoothness is controversial in the examples I saw. There are a lot of them in Ukraine, where somebody tagged all main roads like smoothness=intermediate and the app prefers nearby primary/secondary roads without tags, which are definitely not better than trunk/motorway.
vng (Migrated from github.com) reviewed 2023-11-15 00:30:58 +00:00
vng (Migrated from github.com) commented 2023-11-15 00:30:58 +00:00

The purpose is to keep readable diff in this not obvious block.

The purpose is to keep readable diff in this not obvious block.
pastk reviewed 2023-11-15 06:29:34 +00:00

the diff is bogus also, that's how I did notice this mistake :)
and the result is obviously very bad readability and comprehension-wise:
image

the diff is bogus also, that's how I did notice this mistake :) and the result is obviously very bad readability and comprehension-wise: ![image](https://github.com/organicmaps/organicmaps/assets/18434508/7f7e3d1f-0c86-4e68-9b75-b4ea52e1766e)
pastk reviewed 2023-11-15 06:35:14 +00:00

This sounds a bit like mismapping and OM helping to legalise it :)
I think its fine if OM's behavior is consistent with other routers. Is it?

And in any case add a comment describing a reason for that exception please.

This sounds a bit like mismapping and OM helping to legalise it :) I think its fine if OM's behavior is consistent with other routers. Is it? And in any case add a comment describing a reason for that exception please.
pastk reviewed 2023-11-15 10:06:48 +00:00
@ -228,1 +221,4 @@
LOG(LDEBUG, ("Area: too small or degenerate 1st (outer) polygon of", polys.size(),
", points count", points.size(), "at scale", i, DebugPrint(fb)));
continue;
}

with the latest commit the most of this method's lines are changed anyway, so maybe its better to re-indent the block properly;
or

if (fb.IsPoint())
  return;
with the latest commit the most of this method's lines are changed anyway, so maybe its better to re-indent the block properly; or ``` if (fb.IsPoint()) return; ```
pastk reviewed 2023-11-15 10:12:49 +00:00
@ -162,3 +153,1 @@
// TODO : re-checks geom limit rect size via IsDrawableForIndexGeometryOnly() which was checked already in CalculateMidPoints.
if (fb.IsDrawableInRange(scales::PatchMinDrawableScale(i > 0 ? m_header.GetScale(i - 1) + 1 : 0),
scales::PatchMaxDrawableScale(level)))
bool const isLine = fb.IsLine();

take out of the loop too?

take out of the loop too?

so it makes trg0 coasts in country mwms unnecessary as they duplicate trg3 in WorldCoasts :) might save a few kilobytes someday in the future :)

so it makes trg0 coasts in country mwms unnecessary as they duplicate trg3 in WorldCoasts :) might save a few kilobytes someday in the future :)
pastk reviewed 2023-11-15 11:01:08 +00:00
pastk left a comment
Owner

The last commit is a nice change, makes the code easier to understand!

The last commit is a nice change, makes the code easier to understand!
@ -162,3 +153,1 @@
// TODO : re-checks geom limit rect size via IsDrawableForIndexGeometryOnly() which was checked already in CalculateMidPoints.
if (fb.IsDrawableInRange(scales::PatchMinDrawableScale(i > 0 ? m_header.GetScale(i - 1) + 1 : 0),
scales::PatchMaxDrawableScale(level)))
bool const isLine = fb.IsLine();

also am I right that fb.IsCoastCell(); is true not only for coastline features but for any other feature residing in the same "cell"?

also am I right that `fb.IsCoastCell();` is true not only for coastline features but for any other feature residing in the same "cell"?
@ -205,0 +175,4 @@
if (level <= scales::GetUpperWorldScale())
++level;
if (i == 0)
++level;

is it possible there are several outer polygons?
or they're split into individual features?

is it possible there are several outer polygons? or they're split into individual features?

Just to make it more obvious

            // A polygon's closed outline has 3 points and the 4th should be same as the first.
            CHECK_GREATER_OR_EQUAL(points.size(), 4, ());
            // We don't need the last point equal to the first further on.
            points.pop_back();
Just to make it more obvious ``` // A polygon's closed outline has 3 points and the 4th should be same as the first. CHECK_GREATER_OR_EQUAL(points.size(), 4, ()); // We don't need the last point equal to the first further on. points.pop_back(); ```

I agree. We can't store several versions of inner triangles so no point to try simplify and process outlines for lower zooms.

I agree. We can't store several versions of inner triangles so no point to try simplify and process outlines for lower zooms.

Makes sense! But its a good idea indeed to first try log such cases and see (might be cases when bad simplification made the outer degenerate while inners came out fine?).

Makes sense! But its a good idea indeed to first try log such cases and see (might be cases when bad simplification made the outer degenerate while inners came out fine?).

and similarly below when processing inner polygons

and similarly below when processing inner polygons
vng (Migrated from github.com) reviewed 2023-11-15 11:42:53 +00:00
vng (Migrated from github.com) commented 2023-11-15 11:42:52 +00:00

OSM routers work fine here. Mapping also satisfies OSM rules, I think. The big problem here (and we have TODOs and issues) is that we try to fit all OSM surface and smoothness tags into 4 OM types:

/// @todo Should make some compare constrains (like in CarModel_TrackVsGravelTertiary test)
/// to better fit these factors with reality. I have no idea, how they were set.
VehicleModel::SurfaceInitList const kCarSurface = {
  // {{surfaceType, surfaceType}, {weightFactor, etaFactor}}
  {{"psurface", "paved_good"}, {1.0, 1.0}},
  {{"psurface", "paved_bad"}, {0.6, 0.7}},
  {{"psurface", "unpaved_good"}, {0.4, 0.7}},
  {{"psurface", "unpaved_bad"}, {0.2, 0.3}}
};
OSM routers work fine here. Mapping also satisfies OSM rules, I think. The big problem here (and we have TODOs and issues) is that we try to fit all OSM surface and smoothness tags into 4 OM types: ``` /// @todo Should make some compare constrains (like in CarModel_TrackVsGravelTertiary test) /// to better fit these factors with reality. I have no idea, how they were set. VehicleModel::SurfaceInitList const kCarSurface = { // {{surfaceType, surfaceType}, {weightFactor, etaFactor}} {{"psurface", "paved_good"}, {1.0, 1.0}}, {{"psurface", "paved_bad"}, {0.6, 0.7}}, {{"psurface", "unpaved_good"}, {0.4, 0.7}}, {{"psurface", "unpaved_bad"}, {0.2, 0.3}} }; ```
vng (Migrated from github.com) reviewed 2023-11-15 11:45:25 +00:00
@ -162,3 +153,1 @@
// TODO : re-checks geom limit rect size via IsDrawableForIndexGeometryOnly() which was checked already in CalculateMidPoints.
if (fb.IsDrawableInRange(scales::PatchMinDrawableScale(i > 0 ? m_header.GetScale(i - 1) + 1 : 0),
scales::PatchMaxDrawableScale(level)))
bool const isLine = fb.IsLine();
vng (Migrated from github.com) commented 2023-11-15 11:45:25 +00:00

No, only for coastlines. I think we can (try to) remove this check since we increased the coast's precision.

No, only for coastlines. I think we can (try to) remove this check since we increased the coast's precision.
vng (Migrated from github.com) reviewed 2023-11-15 11:47:36 +00:00
@ -205,0 +175,4 @@
if (level <= scales::GetUpperWorldScale())
++level;
if (i == 0)
++level;
vng (Migrated from github.com) commented 2023-11-15 11:47:36 +00:00

Yes, that is the point! Outer polygon is 1st, others are inners and I can't remember why it was done like this.

We now split multiple outers into several FeatureBuilders. And it is also controversial logic :) because we have several park features (for example) with names where it is one logical feature.

Yes, that is the point! Outer polygon is 1st, others are inners and I can't remember why it was done like this. We now split multiple outers into several FeatureBuilders. And it is also controversial logic :) because we have several park features (for example) with names where it is one logical feature.
pastk reviewed 2023-11-15 13:06:50 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());

I think its better to remove it for now.
This tag is very tricky and depending on the main feature type it should be processed differently.

I think its better to remove it for now. This tag is very tricky and depending on the main feature type it should be processed differently.
pastk reviewed 2023-11-15 13:59:45 +00:00

We don't need building-address added to entrance* types at all.
Housenumbers (if present) are displayed above entrance's icons just like with any other POI.

We don't need `building-address` added to `entrance*` types at all. Housenumbers (if present) are displayed above entrance's icons just like with any other POI.

and actually ATM presence of building-address hides entrance's icons in vehicle style, because building-address has a higher prio there (we wanted to prioritize housenumbers in nav mode compared to other micromapping)

and actually ATM presence of `building-address` hides entrance's icons in vehicle style, because `building-address` has a higher prio there (we wanted to prioritize housenumbers in nav mode compared to other micromapping)

ref/rank should be cleared too? only house should remain, right?

`ref`/`rank` should be cleared too? only `house` should remain, right?
RedAuburn reviewed 2023-11-15 14:06:59 +00:00

it'd also be good to add the was: prefix, around 90k uses roughly

it'd also be good to add the `was:` prefix, around 90k uses roughly
pastk reviewed 2023-11-15 14:10:56 +00:00
@ -478,6 +480,7 @@ void FeatureBuilder::SerializeAccuratelyForIntermediate(Buffer & data) const

But you made it displayed in PP in f7db63b959, no?

But you made it displayed in PP in f7db63b959aad49840a2ab145518aed729f66bfb, no?

fixed typos:

  /// @todo Remove temporary string with the new cpp standard.
  /// Localized brands are not working as expected now because we store raw names from OSM, not brand IDs.
fixed typos: ``` /// @todo Remove temporary string with the new cpp standard. /// Localized brands are not working as expected now because we store raw names from OSM, not brand IDs. ```
pastk reviewed 2023-11-15 14:26:14 +00:00

Actually looking at https://wiki.openstreetmap.org/wiki/Lifecycle_prefix we should avoid displaying other prefixed POIs names like

  • proposed:
  • planned:
  • construction:
  • ruins:
  • demolished:
  • removed:
  • razed:
  • destroyed:
Actually looking at https://wiki.openstreetmap.org/wiki/Lifecycle_prefix we should avoid displaying other prefixed POIs names like - proposed: - planned: - construction: - ruins: - demolished: - removed: - razed: - destroyed:
vng (Migrated from github.com) reviewed 2023-11-16 00:59:58 +00:00
vng (Migrated from github.com) commented 2023-11-16 00:59:58 +00:00

Ok, did you test it locally? Remove adding for "entrance" and see how it is drawn. Because all our maps generation was with this change.

Ok, did you test it locally? Remove adding for "entrance" and see how it is drawn. Because all our maps generation was with this change.
vng (Migrated from github.com) reviewed 2023-11-16 01:11:27 +00:00
@ -478,6 +480,7 @@ void FeatureBuilder::SerializeAccuratelyForIntermediate(Buffer & data) const
vng (Migrated from github.com) commented 2023-11-16 01:11:27 +00:00

Yes, we remove duplicates. Don't store a brand if it is equal to a name.

Yes, we remove duplicates. Don't store a brand if it is equal to a name.
vng (Migrated from github.com) reviewed 2023-11-16 01:39:10 +00:00
vng (Migrated from github.com) commented 2023-11-16 01:39:10 +00:00

Will add "was:". The other list is not about POIs, IMO ..

Will add "was:". The other list is not about POIs, IMO ..
vng (Migrated from github.com) reviewed 2023-11-16 03:00:20 +00:00
vng (Migrated from github.com) commented 2023-11-16 03:00:20 +00:00

organicmaps/organicmaps#6537/commits/b3c4104bbdec795da24773da1bedf22b472a47b4

https://git.omaps.dev/organicmaps/organicmaps/pulls/6537/commits/b3c4104bbdec795da24773da1bedf22b472a47b4
pastk reviewed 2023-11-16 11:19:57 +00:00

I double-checked - it works fine!

I double-checked - it works fine!
vng (Migrated from github.com) reviewed 2023-11-16 13:05:48 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());
vng (Migrated from github.com) commented 2023-11-16 13:05:48 +00:00

Now I remember, we had an issue with a building and a highway. Most of the covered tags are for the highways under buildings.
You can check locally here for example: https://www.openstreetmap.org/#map=19/48.14226/11.55020
Before and after. Probably, many things has changed since our styles refactoring, but anyway ..

Now I remember, we had an issue with a building and a highway. Most of the covered tags are for the highways under buildings. You can check locally here for example: https://www.openstreetmap.org/#map=19/48.14226/11.55020 Before and after. Probably, many things has changed since our styles refactoring, but anyway ..
pastk reviewed 2023-11-16 14:05:58 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());

For this particular case adding layer=-1 changes nothing in the main 3D buildings mode.
But hides the "covered" roads completely in 2D.

Which is not nice neither for comprehension nor for navigation, especially given that some footways are rendered over the building still (they don't have covered=yes).

IMHO in this case we should show covered roads as tunnels (but w/o a "Tunnel" title as it'll be misleading).

For this particular case adding `layer=-1` changes nothing in the main 3D buildings mode. But hides the "covered" roads completely in 2D. Which is not nice neither for comprehension nor for navigation, especially given that some footways are rendered over the building still (they don't have `covered=yes`). IMHO in this case we should show covered roads as tunnels (but w/o a "Tunnel" title as it'll be misleading).
vng (Migrated from github.com) reviewed 2023-11-16 14:33:44 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());
vng (Migrated from github.com) commented 2023-11-16 14:33:44 +00:00

Yes, hiding highways is not good. Same as I don't like hiding highways under the bridge.

Yes, hiding highways is not good. Same as I don't like hiding highways under the bridge.
vng (Migrated from github.com) reviewed 2023-11-16 21:37:37 +00:00
@ -588,0 +638,4 @@
// Check surface.
if (surface.empty())
{
CHECK(!smoothness.empty(), ());
vng (Migrated from github.com) commented 2023-11-16 21:37:37 +00:00

Removed

Removed
biodranik (Migrated from github.com) approved these changes 2023-11-18 09:49:11 +00:00
biodranik (Migrated from github.com) left a comment

@vng how to review your newer changes? It would be easier if they came in separate commits. Now I can't review everything again. Approving blindly.

@vng how to review your newer changes? It would be easier if they came in separate commits. Now I can't review everything again. Approving blindly.
biodranik (Migrated from github.com) approved these changes 2023-11-18 12:37:00 +00:00
biodranik (Migrated from github.com) left a comment

ACK. LGTM

ACK. LGTM
biodranik (Migrated from github.com) commented 2023-11-18 12:36:36 +00:00

const

const
pastk reviewed 2023-11-18 15:17:55 +00:00
@ -873,0 +1040,4 @@
{"motor_vehicle", "private", [&flags] { flags[Flags::MotorVehicle] = -1; }},
{"motor_vehicle", "!", [&flags] { flags[Flags::MotorVehicle] = -1; }},
{"motor_vehicle", "yes", [&flags] { flags[Flags::MotorVehicle] = 1; }},
{"motor_vehicle", "designated", [&flags] { flags[Flags::MotorVehicle] = 1; }},

Please add total types count, like it was before (otherwise have to count manually for each line - very inconvenient).
LOG(LWARNING, ("Exceeded max types count, got total", m_types.size(), ":", TypesToString(m_types)));

Please add total types count, like it was before (otherwise have to count manually for each line - very inconvenient). `LOG(LWARNING, ("Exceeded max types count, got total", m_types.size(), ":", TypesToString(m_types)));`
pastk reviewed 2023-11-18 15:19:16 +00:00
@ -409,29 +427,35 @@ void FeatureParams::SetRwSubwayType(char const * cityName)
}
}

But total types count was very useful, please add to the new log line.

But total types count was very useful, please add to the new log line.
pastk reviewed 2023-11-18 15:33:28 +00:00

Typo in "juctions".
Overall its hard to understand this comment. I had to read three times :)
My understanding: in England there are many junction=circular which are oneway for sure (how do we know? looking at aerial images?), but they are not tagged as such, hence we force oneway=yes explicitly?

Typo in "juctions". Overall its hard to understand this comment. I had to read three times :) My understanding: in England there are many `junction=circular` which are oneway for sure (how do we know? looking at aerial images?), but they are not tagged as such, hence we force `oneway=yes` explicitly?
vng (Migrated from github.com) reviewed 2023-11-18 18:34:47 +00:00
vng (Migrated from github.com) commented 2023-11-18 18:34:47 +00:00

Yes, we had several user claims. I checked some more "circulars" manually. They are obvious oneway (connected roads are mapped as oneway). I didn't find any circular that is not oneway. Even so, false-positive (put oneway while it is not) doesn't break routing. While false-negative makes really bad routes.

Yes, we had several user claims. I checked some more "circulars" manually. They are obvious oneway (connected roads are mapped as oneway). I didn't find any circular that is not oneway. Even so, false-positive (put oneway while it is not) doesn't break routing. While false-negative makes really bad routes.
pastk reviewed 2023-11-19 20:20:12 +00:00
@ -1028,0 +1302,4 @@
// Зеленоград
m_rects.emplace_back(37.119113, 55.944925, 37.273608, 56.026874);
// Add new {lon, lat} city bboxes here.

nit: please put typesCount = under the if

nit: please put `typesCount =` under the if
vng (Migrated from github.com) reviewed 2023-11-19 22:31:24 +00:00
@ -1028,0 +1302,4 @@
// Зеленоград
m_rects.emplace_back(37.119113, 55.944925, 37.273608, 56.026874);
// Add new {lon, lat} city bboxes here.
vng (Migrated from github.com) commented 2023-11-19 22:31:24 +00:00

But the purpose is to know the initial exceeded types count. Under the "if" it will be already truncated = 8.

But the purpose is to know the initial exceeded types count. Under the "if" it will be already truncated = 8.
pastk reviewed 2023-11-20 08:05:08 +00:00
@ -1028,0 +1302,4 @@
// Зеленоград
m_rects.emplace_back(37.119113, 55.944925, 37.273608, 56.026874);
// Add new {lon, lat} city bboxes here.

Right, I've missed it

Right, I've missed it
pastk approved these changes 2023-11-20 08:06:49 +00:00
pastk left a comment
Owner

How do you think to manage experimental generator changes from now on?
Maybe dedicated a generator branch for this?

How do you think to manage experimental generator changes from now on? Maybe dedicated a `generator` branch for this?
This repo is archived. You cannot comment on pull requests.
No reviewers
No labels
Accessibility
Accessibility
Address
Address
Android
Android
Android Auto
Android Auto
Android Automotive (AAOS)
Android Automotive (AAOS)
API
API
AppGallery
AppGallery
AppStore
AppStore
Battery and Performance
Battery and Performance
Blocker
Blocker
Bookmarks and Tracks
Bookmarks and Tracks
Borders
Borders
Bug
Bug
Build
Build
CarPlay
CarPlay
Classificator
Classificator
Community
Community
Core
Core
CrashReports
CrashReports
Cycling
Cycling
Desktop
Desktop
DevEx
DevEx
DevOps
DevOps
dev_sandbox
dev_sandbox
Directions
Directions
Documentation
Documentation
Downloader
Downloader
Drape
Drape
Driving
Driving
Duplicate
Duplicate
Editor
Editor
Elevation
Elevation
Enhancement
Enhancement
Epic
Epic
External Map Datasets
External Map Datasets
F-Droid
F-Droid
Fonts
Fonts
Frequently User Reported
Frequently User Reported
Fund
Fund
Generator
Generator
Good first issue
Good first issue
Google Play
Google Play
GPS
GPS
GSoC
GSoC
iCloud
iCloud
Icons
Icons
iOS
iOS
Legal
Legal
Linux Desktop
Linux Desktop
Linux packaging
Linux packaging
Linux Phone
Linux Phone
Mac OS
Mac OS
Map Data
Map Data
Metro
Metro
Navigation
Navigation
Need Feedback
Need Feedback
Night Mode
Night Mode
NLnet 2024-06-281
NLnet 2024-06-281
No Feature Parity
No Feature Parity
Opening Hours
Opening Hours
Outdoors
Outdoors
POI Info
POI Info
Privacy
Privacy
Public Transport
Public Transport
Raw Idea
Raw Idea
Refactoring
Refactoring
Regional
Regional
Regression
Regression
Releases
Releases
RoboTest
RoboTest
Route Planning
Route Planning
Routing
Routing
Ruler
Ruler
Search
Search
Security
Security
Styles
Styles
Tests
Tests
Track Recording
Track Recording
Translations
Translations
TTS
TTS
UI
UI
UX
UX
Walk Navigation
Walk Navigation
Watches
Watches
Web
Web
Wikipedia
Wikipedia
Windows
Windows
Won't fix
Won't fix
World Map
World Map
No milestone
No project
No assignees
3 participants
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: organicmaps/organicmaps-tmp#6537
No description provided.