Migration from OAuth1 to OAuth2 for OSM API #7333
No reviewers
Labels
No labels
Accessibility
Address
Android
Android Auto
Android Automotive (AAOS)
API
AppGallery
AppStore
Battery and Performance
Blocker
Bookmarks and Tracks
Borders
Bug
Build
CarPlay
Classificator
Community
Core
CrashReports
Cycling
Desktop
DevEx
DevOps
dev_sandbox
Directions
Documentation
Downloader
Drape
Driving
Duplicate
Editor
Elevation
Enhancement
Epic
External Map Datasets
F-Droid
Fonts
Frequently User Reported
Fund
Generator
Good first issue
Google Play
GPS
GSoC
iCloud
Icons
iOS
Legal
Linux Desktop
Linux packaging
Linux Phone
Mac OS
Map Data
Metro
Navigation
Need Feedback
Night Mode
NLnet 2024-06-281
No Feature Parity
Opening Hours
Outdoors
POI Info
Privacy
Public Transport
Raw Idea
Refactoring
Regional
Regression
Releases
RoboTest
Route Planning
Routing
Ruler
Search
Security
Styles
Tests
Track Recording
Translations
TTS
UI
UX
Walk Navigation
Watches
Web
Wikipedia
Windows
Won't fix
World Map
No milestone
No project
No assignees
6 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: organicmaps/organicmaps#7333
Loading…
Add table
Reference in a new issue
No description provided.
Delete branch "osm-oauth2-migration"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This PR doesn't make any UI changes.If user was logged in to OSM account then he would see new alert asking to re-login. Old authentication secrets are removed. New OAuth2 token is requested.string token
andstring secret
to authenticate each API callstring oauthToken
to authenticate API callsoauth_consumer_key
,oauth_token
,oauth_signature_method
,oauth_signature
, etc. See specficationAuthorization: Bearer {oauthToken}
Questions
Q1. Users should re-login once after installing new OM version
Should we notifiy users somehow? Or they'll get notification why try to edit some POI.
A1: User would see alert asking to relogin.
Q2. What is time-to-live for OAuth2 token?
I couldn't find this info in OSM documentation. Installed OM on my Android to check when API call will fail.
Q3. Unused Facebook and Google authentications
Found some unsed code to login using Facebook or Google (probably with WebView). Should we keep it? Did it ever worked?
Q4. How to manage secrets properly?
I used my personal OSM account
s.trump@gmail.com
to register OAuth2 applications for prod and dev environments.Someone should replace constants inAlready handled byprivate_default.h
andeditor/osm_auth.cpp
with official ones and prepare secret values for prod env.@rtsisyk
Update 2024-03-10
If user was previously logged in then after installing new version he/she will see a dialog. This dialog informs him/her that old authentication is not working and he/she needs to re-login.
Update 2024-03-20
Note: text on the alert has "here" link. But it's not a link. It's just a text with blue color. When user taps on this
UITextView
a listener navigates to URL. To make part of text look blue I use two localizable strings instead of one:{alert_reauth_message}
{alert_reauth_message_ios} + ' ' + {alert_reauth_link_text_ios}
Please login to OpenStreetMap to automatically upload all your map edits. Learn more <a href="https://git.omaps.dev/organicmaps/organicmaps/issues/6144">here</a>.
Please login to OpenStreetMap to automatically upload all your map edits. Learn more
+ space +here
{alert_reauth_link_text_ios}
string manually. Handle tap event explicitlyCurrent implementation of
bool HttpClient::RunHttpRequest()
doesn't support handling of redirects.Actual behaviour:
osm.org/oauth2/authorize
om://oauth2/osm/callback?code=xQpf58...
om://oauth2/osm/callback?code=xQpf58...
Expected behaviour:
osm.org/oauth2/authorize
om://oauth2/osm/callback?code=xQpf58...
HttpClient::handle_redirects
field istrue
then do not follow redirect and give back response from server.To handle redirect I had to implement
RedirectDelegate
. Is it made correctly?For some reason tests on MacOS hangs with
dispatch_get_main_queue()
function. Replaced withdispatch_get_current_queue()
. Looks like main queue != current queue. It doesn't break iOS.But
dispatch_get_current_queue()
function is deprecated. What is propper fix here?Because of my change tests are failing at assertion:
How to fix it on MacOS? Remove assert?
HTTP code is thread-safe only if it's called on the same thread. Instead of removing the assert, the callbacks logic should be checked, why different thread is used in different
OnWrite
calls.Looks like there is no such thing as main thread in our test apps.
If you go to downloader_test.cpp you can see calls to
QCoreApplication::exec()
.I think, main queue is created only after
QCoreApplication::exec()
call. But it doesn't work forosm_auth_tests.cpp
. I'm getting error:This is how good stack trace looks like when running

platform_tests
:dispatch_async(dispatch_get_main_queue(), ...)
really dispatches call in main queue. Need to do simillar forosm_auth_tests
.Are these oauth2? Can we reuse them later for faster user registration, or should it be rewritten from scratch?
My proposal is to keep the existing OAuth1 code for some time while we test this new OAuth2 implementation on Android first. Would it be possible to keep two versions?
Please use refresh tokens: https://oauth.net/2/refresh-tokens/
Let's remove it via a separate PR.
Yes, please use private_default.h for now. I will take care of it later.
requested review from
@rtsisyk
OAuth2 bearer tokens on osm.org don't expire at the moment, hence there's also no refresh token available. However, for the sake of being future proof, it's a good idea to consider refresh tokens. Maybe they will be opt-in at some point in the future.
--> https://github.com/openstreetmap/openstreetmap-website/blob/master/config/initializers/doorkeeper.rb#L213
Notification is definitely needed, maybe in the case of not uploaded edits only.
It's not reset now, but it would be great to properly detect the case when it's reset, and at least offer to relogin again. Ideally use what Roman mentioned.
Don't remove it. We want to use it for faster user registration in OSM in the future, when we reuse OSM login instead of implementing OM auth.
In the same way as they're managed now. We'll set them properly later.
What is the point of keeping died code? It will be broken during edits because it can't be run. The old code can always be restored from git when it is needed.
Nobody will look for some older code in git history without knowing that some code existed before. Commented code is very explicit and acts as a TODO comment.
@biodranik
I found a workaround for MacOS: see commit #6c8171f8.For MacOS each test should be wrapped in lambda and executed in a separate thread while main thread is used for main queue.
Do you have any ideas how to improve this solution? Introduce macro for this case?
Not only is there currently no refresh token because OAuth2 tokens on OSM are valid forever, according to the OSM website maintainers are of the opinion that expiring OAuth2 tokens would not really serve a useful purpose in the case of OSM because they can already be revoked by the user (source). So, it is very unlikely that OSM OAuth2 tokens will ever expire and that they will ever be refresh tokens.
That however does still mean that the case that the OAuth2 token suddenly is not accepted by OSM anymore needs to be handled gracefully. As before with OAuth 1.0a, I figure, because it is also possible for the user to revoke these tokens.
In StreetComplete, I blanket handle such cases by logging out the user (i.e. clearing the token) when any authorization-related HTTP error code is returned on any call to the OSM API (HTTP 401 or HTTP 403).
That doesn't look like a good solution. A better approach is needed. Maybe hide relevant logic in the test's main, like it was already done for some tests, or check if the code can be corrected/rewritten properly.
I've heard today that OAuth1 in osm.org will be disabled on June 1 this year. I'm withdrawing my request for support for both versions. Please go ahead 🚀
mentioned in issue #7422
@biodranik
I changedHttpSessionManager
behaviour in commit #ec7ece7eInstead of using main queue for all callbacks it now uses custom queue.
Map download functionality works on iOS simulator. Tests
oam_auth_tests
does not fail.Need to fix
platform_tests
👆 Update: that solution doesn't actually work.
We can't replace main queue with custom queue because there is no guaranty that the same thread will be used. When we post task to a queue with
dispatch_async(...)
it could be handled from different threads. See https://stackoverflow.com/q/52389851So we should either keep main queue and use some workarounds for unit-tests or get rid of queues and use threads in
HttpSessionManager
class.Is this patch ready for beta?
I have done firsts tests on Android, and it works properly;
✅ Connection to OSM
✅ Update POI (I can't test to create a new POI, I use the developer OSM website)
@rtsisyk
feature is ready to merge. Tests are failing. Maybe PR #7405 will fix MacOS testsI also made a change to
osm_auth_tests
. Asked@biodranik
if my workaround is OK: commentReview: Changes requested
Is there any UX migration for existing users? How do they know that they should login again?
What happens with Google Play users who were logged in but where OSM login dialog is disabled?
The current HttpClient/Downloader is one big mess, unfortunately.
m_handleRedirects
solution looks good to me.Thanks for adding this!!!
Review: Approved
Thank you for this substantial contribution. I've tested it on both Android and iPhone, and it functions perfectly. Excellent job!
I agree with the comment that it would be nice to notify the user about the need for re-authentication. However, I don't see it as a blocker for this PR. The situation that the authentication token expires, and the user needs to authenticate is quite common. The app didn't handle such case before this PR and doesn't handle with this PR. Therefore, the existing state of affairs is preserved. My recommendation is to file a separate ticket for this issue and address it somewhere in the future, based on priorities.
approved this merge request
Credentials have been added to master.
requested review from
@biodranik
It is not a common situation with OM and mapsme. The token has NEVER expired before. That is the reason why the app didn't handle such case.
It is crucially important to support a proper UX migration.
@strump
did you test what happens in the following scenarios? Are there any notifications on iOS and Android?Make sure you're logged in qith oauth1
Edit something on the current master but don't upload it to OpenStreetMap.org.
Update the app and run it with new oauth2
Any notifications about not uploaded features/missing login?
Edit something.
Any notifications?
Make sure that you're logged in with oauth1 and there are no edits.
Update the app
Any notifications?
Edit something
Any notifications?
Since when did increasing the entropy become a better solution? ;)
This change should be reverted, it is wrong.
This test workaround is very ugly, are there any better options?
Are all these constants still used?
Aren't these controllers needed for Google/FB OSM auth?
Please check indents and code style (e.g. {}) everywhere. Maybe they are wrongly set in XCode.
handleRedirects
is unclear. Does it mean that HttpClient handles redirects automatically? Or the user should handle it manually?followRedirects
is more clear.Are these vars still used?
Can
self
implement the delegate in a simpler way?Should status code be checked too?
Review: Changes requested
Was maps downloading tested on the Simulator and the iOS device?
Please generate translations using ./tools/python/translate.py
translate.py
requires DeepL API key. DeepL requires card number to prove you are a human. But Ukrainian cards are not accepted.Could someone translate
alert_reauth_message
string? Or I can do it manually through web UIReplaced
handleRedirects
withfollowRedirects
all over the code.If someone was already logged in, then he is not a noob anymore ) Maybe remove it?
Which other dialog can be displayed at that time?
@kirylkaveryn
is it possible to reuse the translation above, with the embedded HTTP URL, without manual strings splitting? NSMutableAttributedString should support clickable links with "Selectable" text field, right?@strump
strings may not split fine in different languages. Is it possible to dynamically find and cut the HTML tag on iOS?Can this 3party library be completely removed?
m_authenticityToken
No need to split these lines in XCode. Longer lines are easier to read in some cases.
Doesn't it look ugly? :)
Won't changing the default value break some other existing requests?
Wrong indent
Meaning: will it be simpler if instead of passing an external delegate, it will be implemented in this class, and
delegate:self
will be passed to the function?nit
Should the PR be rebased?
Review: Commented
Thanks! We're almost there.
@kirylkaveryn can you please review and test the iOS code?
Any volunteers to test Android?
m
file. But be careful while moving this line from one file to another with the outlets! Sometimes it can duplicate and produce crashes (just deleting the property or renaming may produce the same error). So it's better to move the line, clean-up the outlet in the InterfaceBuilder, and re-create the outlet.retain
should not be used because this property attribute is from the MRC (Manual Reference Counting) world.For IBOutlets
@property (nonatomic, weak) IBOutlet
should be used in most cases.Usually for links
NSLinkAttributeName
is used with the URL asvalue
to make a tappable link inside the text fully handled by the system.So the link for the only
here
can be added without explicit link opening.This method does nothing and should not be used.
https://developer.apple.com/documentation/foundation/userdefaults/1414005-synchronize
Can be removed
Replaced
!m_handleRedirects
withm_followRedirects
.Fixed formatting
Fixed argument name
Renamed
m_authenticity_token
tom_authenticityToken
Fixed formatting
Fixed
Ok, removed.
In
http_client_apple.mm
I replacedwith
For Android it's tricky. In
android/app/src/main/cpp/app/organicmaps/util/HttpClient.cpp
there was a line:Meaning that
m_handleRedirects
field is used asfollowRedirects
field which is wrong. But HTTP Client in Java works in such way that it doesn't follow redirect to unknown protocols.So if
followRedirects = true
and HTTP Response redirect toom://oauth2/osm/callback?code=P7dyHlhtepaYT
then Java doesn't follow this redirect and everything works fine. So I changed code to:It should work correctly now.
Tests are green, so this flag is handled correctly.
Fixed formatting
Fixed
Rebased to master
Fixed formatting
Fixed according to suggestion
We don't use
liboauthcpp
lib for OAuth2. In OM it's used to calculate SHA1.I think we can remove it in PR #7658
I'm not very good at string processing with
NSString
. I created stringsalert_reauth_message_ios
andalert_reauth_link_text_ios
by splittinhalert_reauth_message
. Maybe we can do the same from Objective-C code.Update: suggested change gives compilation error:
Reverting ...
Removed calls to
NSUserDefaults.synchronize
method all over the codeRemoved calls to
NSUserDefaults.synchronize
method all over the code@biodranik
Yes it's possible. But to set different attributes for different texts we should retrieve the link's range from the base string and update attrs for this range. So it's easier to have 2 strings rather than retrieve one string from another, update attrs, and combine the back.Changed link handling with
NSLinkAttributeName
Please postpone merging of this PR until April's release is released. No need to rush. We will merge it in Apr (~1 week from now) to have more time for beta-testing.
Review: Approved
I have tested it on an iPhone11Pro (device) and it works well. Thank you!
approved this merge request
mentioned in issue #867
It's time to merge. Let's address all leftovers and pending comments via separate PRs.
@strump
what's left? Can you please create a separate issue for it, if necessary?mentioned in issue #6144
mentioned in merge request !354
mentioned in issue #5740