Android DOES remember permissions for folders that you have opened previously through the picker (although the app does have to code for that); and you can reuse the URLs for files that you have received through the picker, as long as the permissions are still intact. (You can lose them if the app is used infrequently).

Life would be so much easier if the Android File Picker UI weren't so incredibly awful. Has to be the worst piece of UI design I have ever seen. Incredibly difficult to use even if you know exactly what you want.