organicmaps/organicmaps-tmp

Archived

Fork 1

Don't publish edits immediately (but in batches/on intervals) to protect privacy #1325

New issue

Open

opened 2021-09-20 13:43:24 +00:00 by keunes · 6 comments

keunes commented

2021-09-20 13:43:24 +00:00

(Migrated from github.com)

When travelling abroad or walking around and about in my own neighborhood I try to make map changes to increase map accuracy or detail (e.g. opening hours, amenities).

These changes are uploaded to & applied on OSM immediately. Therefore my edits leave a digital trace on OSM (esp my user page) of where I am and have been, including time.

To kinda obfuscate traces it would be great if changes would be uploaded in batches (e.g. of 5) or in time intervals (e.g. every week).

It would also be great if there were some mechanism to pause uploading of edits. This can be useful when travelling, for example, so edits are published only when returned at home. The police advices to do the same for holiday pictures to avoid criminals being able to figure out when the house is 'unguarded'.

Of course the maximum waiting period shouldn't be too long, and also any outstanding changed should be uploaded before new maps are downloaded (or well before, so that the changes are included in the map updates).

Thanks again for your wonderful work!

When travelling abroad or walking around and about in my own neighborhood I try to make map changes to increase map accuracy or detail (e.g. opening hours, amenities). These changes are uploaded to & applied on OSM immediately. Therefore my edits leave a digital trace on OSM (esp my user page) of where I am and have been, including time. To kinda obfuscate traces it would be great if changes would be uploaded in batches (e.g. of 5) or in time intervals (e.g. every week). It would also be great if there were some mechanism to pause uploading of edits. This can be useful when travelling, for example, so edits are published only when returned at home. The police advices to do the same for holiday pictures to avoid criminals being able to figure out when the house is 'unguarded'. Of course the maximum waiting period shouldn't be too long, and also any outstanding changed should be uploaded before new maps are downloaded (or well before, so that the changes are included in the map updates). Thanks again for your wonderful work!

👍 11 👀 1

biodranik commented

2021-09-21 21:25:06 +00:00

(Migrated from github.com)

The algorithm should be smart, but take into account that:

We cannot delay upload for too long to avoid merge conflicts, and to make sure that changes will not be lost (what if you are going into a long offline route?).
OSM people do not like when there are many edits in a changeset and there is a big distance between edits. They usually monitor their area and edits in different cities uploaded in one batch trigger different people in between (inside the edit rectangle) to review/validate changes.

It would be great to avoid manual upload control to have it simple and safe for most people. Any ideas are welcome.

The algorithm should be smart, but take into account that: 1. We cannot delay upload for too long to avoid merge conflicts, and to make sure that changes will not be lost (what if you are going into a long offline route?). 2. OSM people do not like when there are many edits in a changeset and there is a big distance between edits. They usually monitor their area and edits in different cities uploaded in one batch trigger different people in between (inside the edit rectangle) to review/validate changes. It would be great to avoid manual upload control to have it simple and safe for most people. Any ideas are welcome.

👍 3

keunes commented

2021-09-22 06:59:09 +00:00

(Migrated from github.com)

Thanks a lot @biodranik for those comments.

We cannot delay upload for too long to avoid merge conflicts

That can happen anyway, right? E.g., between OM users, and between an OM and an online (OSM) editor (the former doesn't know about edits by the latter).

But to avoid conflicts, I think we need two maximums.

I don't submit too many edits, so a period (time limit) of 2 weeks would protect my privacy quite well. There's low risk of conflicts because the low number of data points I'm editing.

However, conflict chance increases as more edits are made. For users that do many edits, I'd suggest we also upload changes if there's 10 edits are waiting.

When upload happens, a 'last upload' date should be updated, to ensure the time-limit is respected (not too short, as that could accidentally lead to the privacy issue that's currently there).

We cannot delay upload for too long … to make sure that changes will not be lost (what if you are going into a long offline route?).

That's difficult to predict. I'm hardly ever offline (without connection) for a long time, but others (e.g., in remote areas) might be offline most of the time.

I'm also not sure in which cases edits might be 'lost'. There is of course the situation where no more edits can be made to the map because a new map is available. At that point the edits by the user could still be valid – there's just a higher chance of conflicts.

To avoid conflicts I'd propose to upload edits as structured notes if new maps are available. (Similar to structured change proposal notes described in organicmaps/organicmaps#1323 (comment).)

I'd suggest a notification/dialog if upload is due and there's no connection. The user can then temporarily ensure internet access, or look for internet access (if a physical location needs to be reached). Edits waiting to be uploaded, can then be uploaded in the background when an internet connection is available.
If 4 days after the threshold (be it time or number of changes) has been reached they still weren't uploaded, they could be converted to notes, as above.

OSM people do not like when there are many edits in a changeset and there is a big distance between edits

I wasn't aware of this, but makes sense. Each edit currently is its own changeset, right? Is it feasible/possible to group edits & notes by commune/municipality (with one changeset per municipality)? I guess that'd require the storage of extra metadata, though, for each edit/note.

So, to summarise, I would propose:

Queue edits & notes until either of two thresholds is reached, and then upload them in one go:
- Time since last (successful) upload: 2 weeks
- Number of queued edits/notes: 10
On every successful upload, update the 'last upload' date
If upload is due and if, or as soon as, connection is available, do the upload in the background
If no connection available, notify the user of outstanding upload (offer option to turn off aeroplane mode, if active)
If upload is due but not possible within 4 days, turn edits into structured notes
Group edits/notes into 1 changeset per municipality (send out multiple changesets at the same time if necessary)

Thanks a lot @biodranik for those comments. > We cannot delay upload for too long to avoid merge conflicts That can happen anyway, right? E.g., between OM users, and between an OM and an online (OSM) editor (the former doesn't know about edits by the latter). But to avoid conflicts, I think we need two maximums. I don't submit too many edits, so a period (time limit) of 2 weeks would protect my privacy quite well. There's low risk of conflicts because the low number of data points I'm editing. However, conflict chance increases as more edits are made. For users that do many edits, I'd suggest we also upload changes if there's 10 edits are waiting. When upload happens, a 'last upload' date should be updated, to ensure the time-limit is respected (not too short, as that could accidentally lead to the privacy issue that's currently there). > We cannot delay upload for too long … to make sure that changes will not be lost (what if you are going into a long offline route?). That's difficult to predict. I'm hardly ever offline (without connection) for a long time, but others (e.g., in remote areas) might be offline most of the time. I'm also not sure in which cases edits might be 'lost'. There is of course the situation where no more edits can be made to the map because a new map is available. At that point the edits by the user could still be valid – there's just a higher chance of conflicts. To avoid conflicts I'd propose to upload edits as structured notes if new maps are available. (Similar to structured change proposal notes described in https://git.omaps.dev/organicmaps/organicmaps/issues/1323#issuecomment-922734384.) I'd suggest a notification/dialog if upload is due and there's no connection. The user can then temporarily ensure internet access, or look for internet access (if a physical location needs to be reached). Edits waiting to be uploaded, can then be uploaded in the background when an internet connection is available. If 4 days after the threshold (be it time or number of changes) has been reached they still weren't uploaded, they could be converted to notes, as above. > OSM people do not like when there are many edits in a changeset and there is a big distance between edits I wasn't aware of this, but makes sense. Each edit currently is its own changeset, right? Is it feasible/possible to group edits & notes by commune/municipality (with one changeset per municipality)? I guess that'd require the storage of extra metadata, though, for each edit/note. **So, to summarise, I would propose:** * Queue edits & notes until either of two thresholds is reached, and then upload them in one go: * Time since last (successful) upload: 2 weeks * Number of queued edits/notes: 10 * On every successful upload, update the 'last upload' date * If upload is due and if, or as soon as, connection is available, do the upload in the background * If no connection available, notify the user of outstanding upload (offer option to turn off aeroplane mode, if active) * If upload is due but not possible within 4 days, turn edits into structured notes * Group edits/notes into 1 changeset per municipality (send out multiple changesets at the same time if necessary)

biodranik commented

2021-09-22 21:27:31 +00:00

(Migrated from github.com)

Edits could be lost by reinstalling the app, by losing the phone, breaking it, dropping it from the mountain/into the lake etc.

In my opinion, 2 weeks delay is too much. Active notifications are also disturbing and it's better to avoid them.

What about an option "Delay OSM uploads for better privacy" option into settings or OSM profile page, and add a random 1-7 days delay?

Thanks! There is another important point to consider: many of our users, especially OSMers, want their changes to be available ASAP because they want to see it online at home, and maybe even correct and update tags which are not editable from OM. Edits could be lost by reinstalling the app, by losing the phone, breaking it, dropping it from the mountain/into the lake etc. In my opinion, 2 weeks delay is too much. Active notifications are also disturbing and it's better to avoid them. What about an option "Delay OSM uploads for better privacy" option into settings or OSM profile page, and add a random 1-7 days delay?

👍 3 ❤️ 2

keunes commented

2021-09-23 06:49:35 +00:00

(Migrated from github.com)

What about an option "Delay OSM uploads for better privacy" option into settings or OSM profile page, and add a random 1-7 days delay?

Right, such option could work, yeah. If implemented I think it should be enabled by default (for new installs), so the app protects the privacy by default.

I see the appeal of randomness, but this means in some cases your privacy might not be protected. And the thing is: viewers of an online profile won't know that the upload date is random. From larger changesets (which is a logical consequence of longer collection period, for occasional contributors), however, it is clear that the person cannot have been there at that time (due to the physical constraints around being at multiple locations at one point in time).

Edits could be lost by reinstalling the app, by losing the phone, breaking it, dropping it from the mountain/into the lake etc.

That last one I very much feared during my current holiday 😁 Though having your phone on aeroplane mode (or being in a remote area w/out connection) also creates that risk.

There is another important point to consider: many of our users, especially OSMers, want their changes to be available ASAP because they want to see it online at home, and maybe even correct and update tags which are not editable from OM.

Alternatively (to a setting), on the OSM profile page a 'Submit changes and notes now' button could be added. Then no new setting needs to be added (why fewer options are better). Very active contributors can use the button if they want to see and expand their changes when they get home (if you're out and about and make a change it's not likely you'll immediately check OSM online). Would introduce another button to tap, though.

In my opinion, 2 weeks delay is too much.

For those power editors that create many edits in different locations, a short timespan is fine.

For me, a delay of one or 4 days doesn't make much sense, though. I submit only two or three edits per month at home. With such small delay, you can still draw conclusions like this: "This week he went to the hairdresser. After (probably over the weekend) he went to the train station and visited city X. The next week he went for an ice-cream in a neighbouring town, probably by bike because there's edits along the route." (It tells the order of events.)

When on holiday, I make more edits. If published the same week, it's still obvious I'm not at home at that very time.

Are there any stats on how many people make how many changes through OM? We should be considering both power editors and occasional contributors like me. But I'm curious about the balance as it might (should) impact the decisions/conclusions we would make here.

Remember also that power editors don't risk losing many many edits, as (in addition to the time threshold) I proposed to also have a number-threshold. Whichever is first applies. So power editors' waiting time would be much shorter than mine.

Active notifications are also disturbing and it's better to avoid them.

Totally agree. I just proposed it as a backup mechanism for contributors that might not be connected as often as most folks in European and American (urban) areas. (This is not to say that in all other parts of the world there's less connectivity; these are only the areas I am sure to expect connectivity.) Those with high connectivity I wouldn't expect to see the notification.

I should've written the feature in short as follows:

If no connection has been available even after 24 hours following scheduled upload, notify the user of outstanding upload (offer option to turn off aeroplane mode, if active)

> What about an option "Delay OSM uploads for better privacy" option into settings or OSM profile page, and add a random 1-7 days delay? Right, such option could work, yeah. If implemented I think it should be enabled by default (for new installs), so the app protects the privacy by default. I see the appeal of randomness, but this means in some cases your privacy might not be protected. And the thing is: viewers of an online profile won't know that the upload date is random. From larger changesets (which is a logical consequence of longer collection period, for occasional contributors), however, it is clear that the person cannot have been there at that time (due to the physical constraints around being at multiple locations at one point in time). > Edits could be lost by reinstalling the app, by losing the phone, breaking it, dropping it from the mountain/into the lake etc. That last one I very much feared during my current holiday 😁 Though having your phone on aeroplane mode (or being in a remote area w/out connection) also creates that risk. > There is another important point to consider: many of our users, especially OSMers, want their changes to be available ASAP because they want to see it online at home, and maybe even correct and update tags which are not editable from OM. Alternatively (to a setting), on the OSM profile page a 'Submit changes and notes now' button could be added. Then no new setting needs to be added ([why fewer options are better](http://neugierig.org/software/blog/2018/07/options.html)). Very active contributors can use the button if they want to see and expand their changes when they get home (if you're out and about and make a change it's not likely you'll _immediately_ check OSM online). Would introduce another button to tap, though. > In my opinion, 2 weeks delay is too much. For those power editors that create many edits in different locations, a short timespan is fine. For me, a delay of one or 4 days doesn't make much sense, though. I submit only two or three edits per month at home. With such small delay, you can still draw conclusions like this: "This week he went to the hairdresser. After (probably over the weekend) he went to the train station and visited city X. The next week he went for an ice-cream in a neighbouring town, probably by bike because there's edits along the route." (It tells the order of events.) When on holiday, I make more edits. If published the same week, it's still obvious I'm not at home at that very time. Are there any stats on how many people make how many changes through OM? We should be considering both power editors and occasional contributors like me. But I'm curious about the balance as it might (should) impact the decisions/conclusions we would make here. Remember also that power editors don't risk losing many many edits, as (in addition to the time threshold) I proposed to _also_ have a number-threshold. Whichever is first applies. So power editors' waiting time would be much shorter than mine. > Active notifications are also disturbing and it's better to avoid them. Totally agree. I just proposed it as a backup mechanism for contributors that might not be connected as often as most folks in European and American (urban) areas. (This is not to say that in all other parts of the world there's less connectivity; these are only the areas I am sure to expect connectivity.) Those with high connectivity I wouldn't expect to see the notification. I should've written the feature in short as follows: * If no connection _has been_ available _even after 24 hours following scheduled upload_, notify the user of outstanding upload (offer option to turn off aeroplane mode, if active)

biodranik commented

2021-09-24 03:37:35 +00:00

(Migrated from github.com)

I don't think that the privacy option should be a default, otherwise it will break existing users' patterns.

Also, OSM editing literally requires that you should see that place and be there. So it is not ideally private by design.

BTW, you can log out of OSM, make any number of edits, and then login back again to upload. I think it should work now as a workaround for the privacy issues.

I don't think that the privacy option should be a default, otherwise it will break existing users' patterns. Also, OSM editing literally requires that you should see that place and be there. So it is not ideally private by design. BTW, you can log out of OSM, make any number of edits, and then login back again to upload. I think it should work now as a workaround for the privacy issues.

keunes commented

2021-10-02 21:09:11 +00:00

(Migrated from github.com)

I don't think that the privacy option should be a default, otherwise it will break existing users' patterns.

It doesn't break existing user's patterns if introduced as a default only for new installs (?) I didn't mean to suggest to change existing installations.

OSM editing literally requires that you should see that place and be there

Didn't know, but sure, makes sense. Doesn't mean it has to be 'live' though :)

it should work now as a workaround for the privacy issues

It is a bit cumbersome, but good idea. Will do that for now, thanks.

> I don't think that the privacy option should be a default, otherwise it will break existing users' patterns. It doesn't break existing user's patterns if introduced as a default only for new installs (?) I didn't mean to suggest to change existing installations. > OSM editing literally requires that you should see that place and be there Didn't know, but sure, makes sense. Doesn't mean it has to be 'live' though :) > it should work now as a workaround for the privacy issues It is a bit cumbersome, but good idea. Will do that for now, thanks.

This repo is archived. You cannot comment on issues.