Incorporate ESA WorldCover 2021 landcover data #8571

New issue

Open

opened 2024-06-26 14:20:36 +00:00 by rami-dv · 0 comments

rami-dv commented

2024-06-26 14:20:36 +00:00

(Migrated from github.com)

Incorporate ESA WorldCover 2021 landcover data

OpenStreetMap data, while having excellent global coverage for some data themes such as roads, it is lacking in other themes such as landcover. Outside of Europe and select other regions, landcover data simply does not exist and the map in those regions is bare.

In order to provide the best map for users, including being able to style the maps more beautifully with a complete landcover background layer, we should consider incorporating global landcover from the ESA WorldCover 2021 dataset, into OM.

What is ESA WorldCover 2021?

An independently-validated data product of the European Space Agency using compiled Sentinel-1 and Sentinel-2 data, providing 10 landcover classes* at 10m resolution for the vast majority of Earth's surface, licensed as CC-BY 4.0

More information can be found here: https://worldcover2021.esa.int/documentation
Visualize the data here, it's really good quality: https://worldcover2021.esa.int/viewer

*Landcover classes: "Tree cover", "Shrubland", "Grassland", "Cropland", "Built-up", "Bare / sparse vegetation”, “Snow and Ice”, “Permanent water bodies”, “Herbaceous Wetland”, “Mangrove”, “Moss and lichen"

What data processing is necessary to get the data into Organic Maps?

Admittedly, I am not a pro at the Organic Maps data processes, however I am a pro at this particular dataset.

The source data is tiled raster chunks covering the entire globe. In order to be pre-processed for OM, the rasters would be downsampled to a suitable resolution for each zoom level, vectorized, and then a topographically-correct polygon simplification would be applied.

Depending on how OM needs the data to be formatted for its tiling process, it could be provided in many different ways, including as an OSM pbf file.

This repo here is an example of this downsampling/vectorization/simplification processes of the dataset (though not topologically correct)

How could the data be incorporated into Organic Maps?

> As part of the world basemap which is packaged with the app

Certainly in this case it would only go up to a certain zoom. But it would be nice to not have such a "blank" map at the beginning.

> Splice in Worldcover OSM PBF file into source planet file, before OM map tiling process, provide data as part of country / regional map chunks.

This way fits easily with the current OM processes as it's just a preprocessing step for the planet file and there's no code changes necessary within OM.

However, by making the source data "OSM-flavoured", we lose some granularity as OSM doesn't have all the landclasses that WorldCover has (Mangrove, Moss & Lichen...)

In addition, this has the issue of the inefficiency of providing the same landcover data for every data release.

> Have a separate data source for WorldCover included in OM

This would be the best but I'm unclear on how difficult it would be to implement. Since the underlying data doesn't change, there could be an option to download WorldCover data regionally, and that data stays with the phone and app updates.

The data would be provided in OSM PBF at each zoom level, and it could be fed to the same processes as the current OM map build, only it would produce a separate, landcover-only dataset.

I defer to those more experts in the code to understand how much of a lift this option would be.

Other considerations

> In places like Europe, with a dedicated OSM community and great OSM landcover coverage, replacing landcover there with ESA WorldCover might make some users upset.

This can be mitigated by not replacing landcover with WorldCover data but just adding it as another option that can be overlaid with OSM on top, so users can have the best of both worlds--worldcover where there's no OSM, and OSM where it exists.

> Make downloads bigger

This will certainly happen, as there are many many areas with no landcover data and those will now be filled. It's much of a bigger concern if the landcover would be included as part of the periodic data releases than as a one-time download.

In addition, depending on the download size constraints, the pre-processing pipeline of the data can be adjusted to yield either higher or lower resolution data (higher or lower data size), depending on constraints. Certainly, though the dataset is to 10m resolution, we don't need all that resolution by any means.

> Data quality: is it that good?

I've done extensive work with this dataset, and my answer is a resounding "yes!" it's just as good if not much better than OSM and there's worldwide coverage.

## Incorporate ESA WorldCover 2021 landcover data OpenStreetMap data, while having excellent global coverage for some data themes such as roads, it is lacking in other themes such as landcover. Outside of Europe and select other regions, landcover data simply does not exist and the map in those regions is bare. In order to provide the best map for users, including being able to style the maps more beautifully with a complete landcover background layer, we should consider incorporating global landcover from the ESA WorldCover 2021 dataset, into OM. ### **What is ESA WorldCover 2021?** An independently-validated data product of the European Space Agency using compiled Sentinel-1 and Sentinel-2 data, providing 10 landcover classes* at 10m resolution for the vast majority of Earth's surface, licensed as CC-BY 4.0 More information can be found here: https://worldcover2021.esa.int/documentation Visualize the data here, it's really good quality: https://worldcover2021.esa.int/viewer *Landcover classes: "Tree cover", "Shrubland", "Grassland", "Cropland", "Built-up", "Bare / sparse vegetation”, “Snow and Ice”, “Permanent water bodies”, “Herbaceous Wetland”, “Mangrove”, “Moss and lichen" ### **What data processing is necessary to get the data into Organic Maps?** Admittedly, I am not a pro at the Organic Maps data processes, however I am a pro at this particular dataset. The source data is tiled raster chunks covering the entire globe. In order to be pre-processed for OM, the rasters would be downsampled to a suitable resolution for each zoom level, vectorized, and then a topographically-correct polygon simplification would be applied. Depending on how OM needs the data to be formatted for its tiling process, it could be provided in many different ways, including as an OSM pbf file. [This repo here](https://wipfli.github.io/esa-worldcover-polygons/) is an example of this downsampling/vectorization/simplification processes of the dataset (though not topologically correct) ### **How could the data be incorporated into Organic Maps?** #### > As part of the world basemap which is packaged with the app Certainly in this case it would only go up to a certain zoom. But it would be nice to not have such a "blank" map at the beginning. #### > Splice in Worldcover OSM PBF file into source planet file, before OM map tiling process, provide data as part of country / regional map chunks. This way fits easily with the current OM processes as it's just a preprocessing step for the planet file and there's no code changes necessary within OM. However, by making the source data "OSM-flavoured", we lose some granularity as OSM doesn't have all the landclasses that WorldCover has (Mangrove, Moss & Lichen...) In addition, this has the issue of the inefficiency of providing the same landcover data for every data release. #### > Have a separate data source for WorldCover included in OM This would be the best but I'm unclear on how difficult it would be to implement. Since the underlying data doesn't change, there could be an option to download WorldCover data regionally, and that data stays with the phone and app updates. The data would be provided in OSM PBF at each zoom level, and it could be fed to the same processes as the current OM map build, only it would produce a separate, landcover-only dataset. I defer to those more experts in the code to understand how much of a lift this option would be. ### **Other considerations** #### > In places like Europe, with a dedicated OSM community and great OSM landcover coverage, replacing landcover there with ESA WorldCover might make some users upset. This can be mitigated by not replacing landcover with WorldCover data but just adding it as another option that can be overlaid with OSM on top, so users can have the best of both worlds--worldcover where there's no OSM, and OSM where it exists. #### > Make downloads bigger This will certainly happen, as there are many many areas with no landcover data and those will now be filled. It's much of a bigger concern if the landcover would be included as part of the periodic data releases than as a one-time download. In addition, depending on the download size constraints, the pre-processing pipeline of the data can be adjusted to yield either higher or lower resolution data (higher or lower data size), depending on constraints. Certainly, though the dataset is to 10m resolution, we don't need all that resolution by any means. #### > Data quality: is it that good? I've done extensive work with this dataset, and my answer is a resounding "yes!" it's just as good if not much better than OSM and there's worldwide coverage.

👍 1

This repo is archived. You cannot comment on issues.