Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package ID changed #4128

Open
KaKi87 opened this issue Dec 10, 2024 · 24 comments
Open

Package ID changed #4128

KaKi87 opened this issue Dec 10, 2024 · 24 comments
Assignees
Labels
Milestone

Comments

@KaKi87
Copy link

KaKi87 commented Dec 10, 2024

Describe the bug
The package ID for the APK at https://download.kiwix.org/release/kiwix-android/kiwix.apk was org.kiwix.kiwixmobile, but now it's org.kiwix.kiwixmobile.standalone.

Expected behavior
The package ID wouldn't have changed.

Steps to reproduce the behavior:

  1. Have an existing installation of Kiwix via APK ;
  2. Try upgrading it, either manually from the APK link, or automatically via Obtainium ;
  3. Manually, see it offers installing a new app instead of upgrading an existing one / Automatically, see the package ID mismatch error.

Screenshots

Manually Automatically

Environment

  • Version of Kiwix Android : 3.11.1
  • Device : Xiaomi Mi 8
  • OS version : 14

Logs
None


Thanks

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Dec 10, 2024

@KaKi87 Thanks for reporting the bug. We have intentionally changed the package ID for the website and nightly APKs to fix #3933 in #3951. You can upgrade your existing app by downloading the new version of our app from the release section https://github.com/kiwix/kiwix-android/releases. Please download the kiwix-universal-release.apk if you are not aware of your device's architecture.

@KaKi87
Copy link
Author

KaKi87 commented Dec 10, 2024

Although adding APKs to GitHub releases is very much welcome, changing the package ID of an existing download URL is a breaking change, for everyone's upgrade workflow.

In order to fix that while preserving the fix for #3933, I would like to suggest :

  • creating a new website URL pointing to the new package ID APK ;
  • making the website link point to that URL ;
  • making the existing URL point back to the previous package ID APK.

Thank you

@kelson42
Copy link
Collaborator

kelson42 commented Dec 11, 2024

We don't publish updates for the old APK id, so I don't see the point of changing anything... We don't offer any garanty about APK filenames or URLs either so... but yes this is a breaking change we had to make to avoid confusions in the past.

@KaKi87
Copy link
Author

KaKi87 commented Dec 11, 2024

We don't publish updates for the old APK id

So you're saying that I have to either upgrade using Google, or re-download dozens of gigabytes of data, or remain on an outdated version ?

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Dec 11, 2024

@KaKi87 You are currently using the full version of the application(It has the capability to load sideloaded ZIM files from your storage), and the Google playStore version has the limitation of loading the sideloaded ZIM files. In the PS version, we just introduced file picker functionality for loading the sideloaded ZIM files from your storage. But in this approach, we are copying/moving the ZIM file from your storage to Kiwix's app-specific directory which can be deleted upon data clear or app uninstall. So upgrading from google playStore is not fit for you since there is some limitation.

IMHO you should use our GitHub release section to upgrade your existing application to a newer version. Since this is the full version of our application and it has the same package ID as you have installed. So in this case, you don't have to redownload your existing ZIM files.

or re-download dozens of gigabytes of data,

May I know where the existing ZIM files are placed in your storage? If they are not inside Kiwix's app-specific directory Android/data/org.kiwix.kiwixmobile/... then you don't need to worry about re-downloading ZIM files. Since our full version of the application can reload these ZIM files from your storage. Or if there is some ZIM files available in this directory, so I recommend that you, move your those ZIM files from another location using a PC(Just for the safer side). Since the device's file manager does not have access to open this directory.

@KaKi87
Copy link
Author

KaKi87 commented Dec 11, 2024

you should use our GitHub release section to upgrade your existing application to a newer version

Oh, right, you said that, but I forgot about it because Kelson contradicted you by saying We don't publish updates for the old APK id 🤔

May I know where the existing ZIM files are placed in your storage? If they are not inside Kiwix's app-specific directory [...] then you don't need to worry about re-downloading ZIM files.

I downloaded from the app itself so I suspect that's where they are.

@MohitMaliFtechiz
Copy link
Collaborator

Oh, right, you said that, but I forgot about it because Kelson contradicted you by saying We don't publish updates for the old APK id

@KaKi87, @kelson42 was talking about the website APK.

I downloaded from the app itself so I suspect that's where they are.

Okay, if you downloaded them from the application, then the ZIM files should be in storage/0/Kiwix/.. since you were using the full version of the APK. You can be sure by checking the ZIM files by going inside the file manager and finding the Kiwis folder; the ZIM files should be there inside that.

Image

@kelson42 kelson42 modified the milestones: 3.13.0, 3.14.0 Dec 13, 2024
@KaKi87
Copy link
Author

KaKi87 commented Dec 18, 2024

Oh, right, you said that, but I forgot about it because Kelson contradicted you by saying We don't publish updates for the old APK id

@KaKi87, @kelson42 was talking about the website APK

Well, now, the GitHub releases APK changed too 😭

Okay, if you downloaded them from the application, then the ZIM files should be in storage/0/Kiwix/.. since you were using the full version of the APK.

I just checked, and they're not : they're in /storage/emulated/0/Android/data/org.kiwix.kiwixmobile/files/Kiwix/.

Thanks

@IzzySoft
Copy link
Contributor

IzzySoft commented Dec 18, 2024

And for that reason, this new release won't reach IzzyOnDroid either (where the RB verification builder complains not finding any APK by the defined file name patter, so no RB check either).

Edit: I have to disable update checks for now, "demoting" Kiwix to monthly checks with our repo (to avoid downloading the same not-matching file over and again) and setting RB to manual, as I won't have much time to check and fix things before the end of the year.

@MohitMaliFtechiz
Copy link
Collaborator

MohitMaliFtechiz commented Dec 19, 2024

I just checked, and they're not : they're in /storage/emulated/0/Android/data/org.kiwix.kiwixmobile/files/Kiwix/.

@KaKi87 Please move these files from this folder to another storage location before uninstalling the APK.

Well, now, the GitHub releases APK changed too 😭

Sorry for the inconvenience. Now the official APK is org.kiwix.kiwixmobile.standalone and we will publish this APK everywhere, so please download this APK and remove the previous one. You will continue getting updates for this APK. For the saved bookmarks: You can export your existing bookmark from the settings screen, and after installing the new version of our APK import them from the settings. There are options for exporting and importing the bookmarks.

@MohitMaliFtechiz
Copy link
Collaborator

@IzzySoft As we know, our official APK is org. kiwis.kiwixmobile.standalone. We will stick with this new APK and publish updates for it everywhere.

Can you please configure your CD to recompile the project and publish the app on IzzyOnDroid instead of taking the apk from the GitHub release section? Or can we create our own CD which can publish the application on IzzyOnDroid. If both options are not possible please publish the application with the new appId with some kind of indication for the migration.

@KaKi87
Copy link
Author

KaKi87 commented Dec 19, 2024

Sorry for the inconvenience.

That's not an "inconvenience", that's unacceaptable. Breaking users' updates like this ? That's huge. Especially compared to the original issue and the fix I suggested.

You killed a fly with a bazooka.

@kelson42
Copy link
Collaborator

@IzzySoft @KaKi87 We had to change the appid because apps become different (because of Google Policies), keeping naimg them with the same id started to really (legitimately) confuse users. This is indeed a breaking change, and we are sorry for the inconvenience, but we have no control on Google Policies and we still want to provide other ways to users to download/install Kiwix. All the rest, URL or filename are and should not be machine readable stuff.

@MohitMaliFtechiz The only think I ask myself is if, at the end, if would not be smarter to rename kiwix-3.14.0.apk in org. kiwix.kiwixmobile.standalone_3.14.0.apk. It would be semantically better and would have been a way to clearer about that change. What do you think?

@KaKi87
Copy link
Author

KaKi87 commented Dec 30, 2024

We had to change the appid because apps become different (because of Google Policies)

Okay, but Google can only make requirements about the ID of the app you're submitting to the Play Store, not the one you're publishing on GitHub & kiwix.org, so why changing those too ?

@kelson42
Copy link
Collaborator

We had to change the appid because apps become different (because of Google Policies)

Okay, but Google can only make requirements about the ID of the app you're submitting to the Play Store, not the one you're publishing on GitHub & kiwix.org, so why changing those too ?

Already told you: they work differently in significant manner and we didn't want to have users mixing things (for example by having the play store interfering with a version which was installed manually).

@KaKi87
Copy link
Author

KaKi87 commented Dec 30, 2024

So you didn't change the Play Store app ID too ?

@parkerlreed
Copy link

So you didn't change the Play Store app ID too ?

No as the name change was as to NOT conflict with the Play Store listing.

https://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile

@KaKi87
Copy link
Author

KaKi87 commented Dec 30, 2024

I thought both were changed.

Well, then I vote for the Play Store ID to be changed and the APK to be changed back.

@IzzySoft
Copy link
Contributor

This gets more confusing everytime I look at it:

We had to change the appid because apps become different (because of Google Policies)

suggests to have something to do with Google (and their PlayStore). I'm not aware what policy should force existing listed apps to have their packageName/applicationId changed. And on the question "So you didn't change the Play Store app ID too ?" you reply:

No as the name change was as to NOT conflict with the Play Store listing.

What policy was it then? There are thousands of apps listed with the same packageName on Play and at the same time in other places like F-Droid.org, IzzyOnDroid, Amazon, whatever. So why confuse users even more by a different package name for the same app, forcing them to uninstall/reinstall, loosing their settings on the way and having to go through extra hoops to be able to receive updates again?

Can you please configure your CD to recompile the project and publish the app on IzzyOnDroid instead of taking the apk from the GitHub release section? Or can we create our own CD which can publish the application on IzzyOnDroid. If both options are not possible please publish the application with the new appId with some kind of indication for the migration.

You cannot push APKs to either IzzyOnDroid or F-Droid.org, we pull them. Changing the packageName means it's a different app, so you force us to extra hoops setting up such a migration, too. Yes, the last variant of what you suggest there is possible – but is it also feasible? Estranging everyone around just for some unnamed Google policy (or Google abuse practices, not respecting the "installer" property)? You're sure this is not just a misunderstanding? If it's Google interfering, why not change the package name there, where the fault lies? Why instead causing trouble for those already avoiding Google for good reasons – instead of putting it with the troublemaker?

Apologies if my writing sounds harsh – it's not meant to be, but indeed serious asking and strongly suggesting to consider.

@kelson42
Copy link
Collaborator

This gets more confusing everytime I look at it:

We had to change the appid because apps become different (because of Google Policies)

suggests to have something to do with Google (and their PlayStore). I'm not aware what policy should force existing listed apps to have their packageName/applicationId changed. And on the question "So you didn't change the Play Store app ID too ?" you reply:

Nobody is forced, our will, reasons have been given.

No as the name change was as to NOT conflict with the Play Store listing.

What policy was it then? There are thousands of apps listed with the same packageName on Play and at the same time in other places like F-Droid.org, IzzyOnDroid, Amazon, whatever. So why confuse users even more by a different package name for the same app, forcing them to uninstall/reinstall, loosing their settings on the way and having to go through extra hoops to be able to receive updates again?

Here too, reasons have been given, it was our case as long as the app were the same. Honestly, here we have chosen to impact the less users as possible.

Can you please configure your CD to recompile the project and publish the app on IzzyOnDroid instead of taking the apk from the GitHub release section? Or can we create our own CD which can publish the application on IzzyOnDroid. If both options are not possible please publish the application with the new appId with some kind of indication for the migration.

You cannot push APKs to either IzzyOnDroid or F-Droid.org, we pull them. Changing the packageName means it's a different app, so you force us to extra hoops setting up such a migration, too. Yes, the last variant of what you suggest there is possible – but is it also feasible? Estranging everyone around just for some unnamed Google policy (or Google abuse practices, not respecting the "installer" property)? You're sure this is not just a misunderstanding? If it's Google interfering, why not change the package name there, where the fault lies? Why instead causing trouble for those already avoiding Google for good reasons – instead of putting it with the troublemaker?

Yes we are sure, this has been discuss in length for at least 12 months in different issues.

Apologies if my writing sounds harsh – it's not meant to be, but indeed serious asking and strongly suggesting to consider.

This is all considered, this is exactly because PS is hard to live with that we have done that move.

@IzzySoft
Copy link
Contributor

because PS is hard to live with

Oh, that one I'd sign. It's why I left it behind about 10 years ago. Never regretted that step 😜

OK, going to initiate the switch. While on it, the scanner reports:

SigningBlock blobs:
-------------------
0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)

Easily avoided with a minor adjustment to your build.gradle:

android {
    dependenciesInfo {
        // Disables dependency metadata when building APKs.
        includeInApk = false
        // Disables dependency metadata when building Android App Bundles.
        includeInBundle = false
    }
}

For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains. More details can be found e.g. here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.

As the APK is not intended for Google anyway, could you please do that? Further, if there's a way to only build a single ABI (currently, 5 ABIs are built and we simply drop 4 of them), a hint would be appreciated; I could not find any ABI filter setup in your build.gradle.kts (could probably be done here by removing all those not wanted using e.g. sed?).

OK, that worked out – but the app still is no longer RB due to differences in classes.dex, like

     #17              : (in Lorg/kiwix/kiwixmobile/di/components/DaggerKiwixComponent$KiwixComponentImpl;)
-      name          : 'provideFat32Checker$3_13_0_standaloneProvider'
+      name          : 'provideFat32Checker$kiwix_standaloneProvider'
       type          : 'Ljavax/inject/Provider;'
       access        : 0x0001 (PUBLIC)
     #18              : (in Lorg/kiwix/kiwixmobile/di/components/DaggerKiwixComponent$KiwixComponentImpl;)
-      name          : 'provideLocationManager$3_13_0_standaloneProvider'
+      name          : 'provideLocationManager$kiwix_standaloneProvider'
       type          : 'Ljavax/inject/Provider;'

This pattern repeats multiple times, with your build having a 3_13_0_ (i.e. the version) instead of simply kiwix in the names. I've looked at your workflow but found no hints. Here's my build recipe if you want to compare:

        build:
          - GRADLE_VERSION="$(sed -rn 's/distributionUrl.*gradle-([0-9.]+).*\.zip/\1/p' gradle/wrapper/gradle-wrapper.properties)"
          - sed -r 's/return baseVersionCode \+ daysDifference/return 231255/' -i buildSrc/src/main/kotlin/VersionCodeGenerator.kt
          - sed -r 's/val abiCodes.+$/val abiCodes = mapOf("armeabi-v7a" to 5)/ ; s/isUniversalApk = true/isUniversalApk = false/' -i buildSrc/src/main/kotlin/plugin/AppConfigurer.kt
          - git clone https://github.com/obfusk/gradlew.py.git
          - gradlew.py/gradlew.py --version $GRADLE_VERSION -v assembleStandalone -PdisableSigning

Any hints on how to fix that?

Thanks in advance!

@MohitMaliFtechiz
Copy link
Collaborator

@MohitMaliFtechiz The only think I ask myself is if, at the end, if would not be smarter to rename kiwix-3.14.0.apk in org. kiwix.kiwixmobile.standalone_3.14.0.apk. It would be semantically better and would have been a way to clearer about that change. What do you think?

@kelson42 I agree with this. Also, our GitHub release has this new name which indicates this change.

Image

@MohitMaliFtechiz
Copy link
Collaborator

OK, going to initiate the switch. While on it, the scanner reports:
SigningBlock blobs:


0x504b4453 (DEPENDENCY_INFO_BLOCK; GOOGLE)
Easily avoided with a minor adjustment to your build.gradle:
android {
dependenciesInfo {
// Disables dependency metadata when building APKs.
includeInApk = false
// Disables dependency metadata when building Android App Bundles.
includeInBundle = false
}
}
For some background: that BLOB is supposed to be just a binary representation of your app's dependency tree. But as it's encrypted with a public key belonging to Google, only Google can read it – and nobody else can even verify what it really contains. More details can be found e.g. here: Ramping up security: additional APK checks are in place with the IzzyOnDroid repo.
As the APK is not intended for Google anyway, could you please do that?

@IzzySoft We have disabled this in #4057, and will merge soon.

Further, if there's a way to only build a single ABI (currently, 5 ABIs are built and we simply drop 4 of them), a hint would be appreciated; I could not find any ABI filter setup in your build.gradle.kts (could probably be done here by removing all those not wanted using e.g. sed?).

For this, we can use the direct command to generate the one abi APK like ./gradlew assembleRelease -Pandroid.injected.build.abi=armeabi-v7a, it will generate the given abi APK and ignore the other abi APKs.

@IzzySoft
Copy link
Contributor

We have disabled this in #4057, and will merge soon.

Thanks! LGTM (as at IzzyOnDroid we only care about the APK, and the AAB is only going to PlayStore, it's fine to leave it in with the latter).

For this, we can use the direct command to generate the one abi APK

Wonderful, thanks – noted and will be used when trying the next build. So the only remaining issue is $3_13_0 in your APK vs $kiwix in the one resulting from our builds (file name of the attached APK can be adjusted here easily – and I agree the suggested naming with kiwix.*apk is better, as long as you keep the variants clearly recognizable that is 😉).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants