organicmaps/organicmaps-tmp

Archived

Fork 1

WIP: [android] User-selectable folder for bookmarks - adds online backup (NextCloud, Google Drive, etc.) #1871

Draft

pastk wants to merge 1 commit from pastk/pastk-storage into master

pastk commented

2022-01-16 22:59:02 +00:00

Member

This is a quick & dirty proof-of-concept code to see if its possible to keep bookmarks in an arbitrary user-selectable folder since Android 11 scoped storage enforcing.

Despite what was written in organicmaps/organicmaps#85 (comment) the app can see new files in user-selected directory.

So the code uses ACTION_OPEN_DOCUMENT_TREE intent to allow a user to select a directory.

go in Settings -> Save maps to...
tap on maps size header, a system dir picker will appear
choose/create a directory, e.g. OrganicMaps in internal storage
long tap on the size header and monitor the logs (filter by "SAF")
the app will create a file in that dir
and then list all files it can see in that dir and print first line of their contents
it will overwrite contents of the files also (to check they're writable)
add more files in the dir using a file manager, text editor, etc.
long tap again and see if the files are discovered and writable

The app will remember the chosen uri on device reboots.
I've tested it on emulated Android 11 & 5.0 and on hardware 9 - everything works so far!
Or did I miss something?

@biodranik @rtsisyk

This is a quick & dirty proof-of-concept code to see if its possible to keep bookmarks in an arbitrary user-selectable folder since Android 11 scoped storage enforcing. Despite what was written in https://git.omaps.dev/organicmaps/organicmaps/issues/85#issuecomment-846880014 the app can see new files in user-selected directory. So the code uses `ACTION_OPEN_DOCUMENT_TREE` intent to allow a user to select a directory. - go in Settings -> Save maps to... - tap on maps size header, a system dir picker will appear - choose/create a directory, e.g. OrganicMaps in internal storage - long tap on the size header and monitor the logs (filter by "SAF") - the app will create a file in that dir - and then list all files it can see in that dir and print first line of their contents - it will overwrite contents of the files also (to check they're writable) - add more files in the dir using a file manager, text editor, etc. - long tap again and see if the files are discovered and writable The app will remember the chosen uri on device reboots. I've tested it on emulated Android 11 & 5.0 and on hardware 9 - everything works so far! Or did I miss something? @biodranik @rtsisyk

👍 5

rtsisyk reviewed 2022-01-16 22:59:02 +00:00

biodranik commented

2022-01-16 23:15:54 +00:00

(Migrated from github.com)

Интересно. А если накопировать в эту папку внешних файлов, аппа их увидит (хотя бы после перезапуска)? Раньше в этом была проблема тоже.

pastk commented

2022-01-17 05:47:29 +00:00

Author

Member

Интересно. А если накопировать в эту папку внешних файлов, аппа их увидит (хотя бы после перезапуска)? Раньше в этом была проблема тоже.

Я так понимаю, что в этом и была основная проблема, именно поэтому я решил перепроверить, т.к. согласно докам всё должно видеться.
В моём тесте внешние файлы приложение сразу видит, никакого перезапуска не надо.

> Интересно. А если накопировать в эту папку внешних файлов, аппа их увидит (хотя бы после перезапуска)? Раньше в этом была проблема тоже. Я так понимаю, что в этом и была основная проблема, именно поэтому я решил перепроверить, т.к. согласно докам всё должно видеться. В моём тесте внешние файлы приложение сразу видит, никакого перезапуска не надо.

biodranik commented

2022-01-17 07:22:03 +00:00

(Migrated from github.com)

А эти пермишны не истекают со временем? Если аппу не запускать пару месяцев?

pastk commented

2022-01-17 10:11:26 +00:00

Author

Member

А эти пермишны не истекают со временем? Если аппу не запускать пару месяцев?

В доках ничего такого нет.
"To preserve access to files across device restarts and create a better user experience, your app can "take" the persistable URI permission grant that the system offers"
"Caution: Even after calling takePersistableUriPermission(), your app doesn't retain access to the URI if the associated document is moved or deleted. In those cases, you need to ask permission again to regain access to the URI."

> А эти пермишны не истекают со временем? Если аппу не запускать пару месяцев? В доках ничего такого нет. "To preserve access to files across device restarts and create a better user experience, your app can "take" the persistable URI permission grant that the system offers" "Caution: Even after calling takePersistableUriPermission(), your app doesn't retain access to the URI if the associated document is moved or deleted. In those cases, you need to ask permission again to regain access to the URI."

pastk commented

2022-01-17 12:16:18 +00:00

Author

Member

The biggest issue / challenge here is that those permissions work only with SAF/DocumentsContract API.
Java File API is not supported as well as native calls like fopen.

It should be possible to pass file descriptors obtained via SAF API to the native code, but some operations will need java callbacks anyway and a big question is how make it work for iOS at the same time. So it looks like a lot of headache.

But there are some other options.
Don't use a user-selected dir as the main/only bookmark storage, but consider it rather as a mirror or sync target.
Bookmark files are not too many and are of relatively small size, so won't be a big issue to have two copies of them and sync them.
The app could sync user dir on every start or maybe on every file change even (if fast enough).

Maybe even better to sync it one way only, i.e. use as a backup. Because we do have mass import of bookmarks feature already and its something that is not done too often (so not a big difference between copying files into a dir or using a special app's import feature for that).
But compared to a hypothetical all bookmarks export feature having a synced bookmarks dir has its advantages:

always up-to-date (or very little lag)
bookmarks are not lost if the app is uninstalled
power users will be able to setup more advanced backup/sync solutions like syncthing

So we can call it "bookmarks backup folder" or "sync bookmarks into folder" feature. And it'll be complimentary to the cloud sync/backup feature when it comes.

The biggest issue / challenge here is that those permissions work only with SAF/DocumentsContract API. Java File API is not supported as well as native calls like fopen. It should be possible to pass file descriptors obtained via SAF API to the native code, but some operations will need java callbacks anyway and a big question is how make it work for iOS at the same time. So it looks like a lot of headache. But there are some other options. Don't use a user-selected dir as the main/only bookmark storage, but consider it rather as a mirror or sync target. Bookmark files are not too many and are of relatively small size, so won't be a big issue to have two copies of them and sync them. The app could sync user dir on every start or maybe on every file change even (if fast enough). Maybe even better to sync it one way only, i.e. use as a backup. Because we do have mass import of bookmarks feature already and its something that is not done too often (so not a big difference between copying files into a dir or using a special app's import feature for that). But compared to a hypothetical all bookmarks export feature having a synced bookmarks dir has its advantages: - always up-to-date (or very little lag) - bookmarks are not lost if the app is uninstalled - power users will be able to setup more advanced backup/sync solutions like syncthing So we can call it "bookmarks backup folder" or "sync bookmarks into folder" feature. And it'll be complimentary to the cloud sync/backup feature when it comes.

❤️ 1

biodranik commented

2022-01-18 11:37:26 +00:00

(Migrated from github.com)

Local backup to something like Dropbox/google drive folder can solve online backup too.
But we should carefully choose how and when to "sync" files, to avoid excess battery draining, for example, when user walks in an offline hike for several days.
Also, it should be a simple-setup button, probably in Bookmarks. Maybe we should implement a submenu for the "Export all" button which can show "Locally" together with "As KMZ".

Local backup to something like Dropbox/google drive folder can solve online backup too. But we should carefully choose how and when to "sync" files, to avoid excess battery draining, for example, when user walks in an offline hike for several days. Also, it should be a simple-setup button, probably in Bookmarks. Maybe we should implement a submenu for the "Export all" button which can show "Locally" together with "As KMZ".

👍 1

pastk commented

2022-01-18 12:22:33 +00:00

Author

Member

Update: actually its easy to do if the app will sync in version-named subfolders by default, e.g. a newly installed version will start syncing into a new subfolder, so the previous copy will be safe. But it could look confusing to the users from UX POV.

It would be great also to make an extra copy of the bookmarks into a separate subfolder before each app update - to avoid bookmark data being screwed because of some regression. I don't know if its technically possible at all though. Update: actually its easy to do if the app will sync in version-named subfolders by default, e.g. a newly installed version will start syncing into a new subfolder, so the previous copy will be safe. But it could look confusing to the users from UX POV.

This repo is archived. You cannot comment on pull requests.

No reviewers

rtsisyk