Allow app to sync data across devices

[iSoron]

It would be nice if Loop could synchronize your habit data across multiple devices. This feature has been one of the most frequently requested so far. It also opens many interesting possibilites, such as syncing data with a future desktop or web client.

Since this feature raises many privacy and security concerns, here is my non-comprehensive list of requirements:

1. It should be completely opt-in, and the app should still be fully functional with this feature disabled.
2. If there is software running on a server to make this feature work, this software should be fullly open-source and self-hostable.
3. It goes without saying, but the whole thing should just work. In particular, it must gracefully handle the case where multiple devices make changes while offline, then go online. The solution must be able to merge the offline changes, not simply discard all the changes made all clients, except one.

Because of requirement 2, most commonly used solutions such as Parse, Firebase or Play Games Services are out of the picture. Requirement 3 probably eliminates file-based sync solutions, such as Syncthing, Google Drive or Dropbox.

[iSoron]

I am currently experimenting with custom back-end solution, implemented in node.js, on the branch feature/sync, but it's not usable yet. I will post here once it works somewhat properly.

Other ideas are very welcome, by the way.

[NiklasMM]

Do you want to host that backend yourself and make it available to all users? Or would that be a self-hosted solution, which everyone could set-up for themselves?

[iSoron]

Hi @NiklasMM, I will probably host an instance myself and make it available for all users. I might start accepting donations to keep the server running. In any case, you will also be able to run a private instance of the server yourself.

[quark6789]

Would it be possible to use git for this? Have each device make its own little repo with the data in it, and then either have each sync with a server, or possibly sync directly with each other (decentralized).

For merging, only conflicts for the same habit and day would be an issue. I think choosing the most recent change would be a fine solution.

[colonelpanic8]

THIS. git is quite good at managing conflicts as well.

[iSoron]

@PunctualEmoticon If you are suggesting to create a local git repository on each phone with the database file, then I think we would run into the same problems as using Syncthing or Dropbox; namely, there is not enough granularity for git to merge things automatically. Only if we split the database into multiple files, would git (or Syncthing/Dropbox) be able to help us, but this doesn't strike me as a very good solution.

[quark6789]

Yes, that's what I was suggesting.
But unlike those file syncing services, git isn't limited to just checking files as a whole. It looks at the contents of the files (if they're ultimately text files), and can resolve changes automatically, unless the same line has been modified on both branches since the branches last split.
Source: http://www.drdobbs.com/tools/three-way-merging-a-look-under-the-hood/240164902

[xdmx]

You may consider the ability to sync it with a nextcloud/owncloud server, that'd be nice

[fwuensche]

I believe you don't want to have any server structure on your side. So, as you suggested above, maybe we could try to use Dropbox.

Take a look at 1password for example. It works like a charm across my devices (mobile and laptop) and it's always in sync by using Dropbox as "database". Do you believe that might be feasible?

[LeonardoGentile]

Another nice thing about 1password is the ability to sync over wifi.
I don't use dropbox to sync, but the app has a self-hosted sync server embedded in the desktop app. When connected to the same wifi I can sync from desktop to mobile and vice-versa

[kyriakosbrastianos]

What about webdav? I would like to share and use data with apps like nextcloud tasks & calendar and qownnotes: https://github.com/pbek/QOwnNotes
QOwnNotes is great on desktop and syncs great with nextcloud tasks so u get web interface too and loop habits is amazing on android. A collaboration will be great with these services/apps.

[omarbenhamid]

What about using cloudant, it is à cloud service based on opensource couchdb technology ?
We could use https://github.com/cloudant/sync-android library. It should transparently fit the 3 requirements.

[MaluNoPeleke]

An optional Play Games Services support would be nice, even if it doesn't meet all requirements.

[recheej]

What about a web based solution? Send data back and forth between the web server. The web server stores the information in a database.

[mr-flibble]

Hello, just an idea:
Since now application uses sqlite, would it not be too time-consuming to allow a user to attach an application to their own mysql postgresql server? The application would be in the online only mode but would be available from multiple devices.

[jharder01]

I'm in favor of self hosted options as they are in-line with the spirit of this project. Granted, a shared, public sync server may help instead awareness of the app.

Additionally, I would love to be able to sync my Android Loop data locally to a personal Grafana/InfluxDB server for data logging and visualization.

[namdnguyen]

Where is the application sqlite database, not backup files, stored on Android? I'm having trouble finding it.

[olegivo]

I think should be a core functional with high level sync logic and "plugin" modules for sync with concrete data sources such as WebDav or Dropbox or custom backend and so on.

[spiraldancing]

Quick thoughts on this feature idea. Anki / Ankiweb might offer a good model to base the design on. Also, uHabit already allows for data backup/restore ... a simple(r) first-draft solution would be expanding that tool to handle a relatively seamless behind-the-scenes automatic backup-restore process through a Cloud storage location of the user's choice, be it Dropbox or a self-hosted Nextcloud or whatever. If it isn't already, data backup file should/oughta be encrypted for this.

... my dos centavos.

[VA-MMXXII]

There appear to be a number of great synching options out there. I've used ​the open source Tasks.org app on Android (https://play.google.com/store/apps/details?id=org.tasks&hl=en_US&gl=US) and it offers a number of ways to synch. Maybe one or more of them would allow Loop users to use their phones or their tablets (or maybe even their desktop) to check off habits throughout the day?

[Improver2000]

Same here! Tasks sync works perfect! Implementation would be wonderful :-)

[fel1]

I've noticed that many of the Habits that I've added are more like to-do's/chores, which are repeatable. I like the app functionality and simplicity so much, that now many of my daily basic to-do's also live here. I.e. "Take out the trash", "Feed the dog" etc.

It would be more than amazing to be able to share some habits (tasks) with my wife so either of us could mark them as done on any of our devices. This would need some sort of indicator of who (which user) ticked off the habit. If some sort of "Users" function could be added then a file-based sync solution might work well, depending on the implementation. What do you think?

[recheej]

Hey @iSoron just checking the status of this one. Are you still looking for help with it?

[woodsea9]

I would love this to be done so my habits could be shared

[pflajszer]

+1 from me. I'd be very keen on either an exposed API, webhooks to subscribe to, or even some automated db exports.

[SimonDedman]

Hi, idk how useful this is but I recently noticed Joplin uses 3rd party services for backend sync Inc Dropbox, nextcloud, WebDAV, and many others. I wonder if you could copy their approach/code, they're open source?

[HelloLudger]

https://remotestorage.io/ could be used for this feature .

[AnsahMohammad]

Hey any update on this ?,

About crossplatform, i was thinking how about writing Flutter code which can be used to build for other platforms (ios, linux, windows, web etc..), and it'll be a lot easier. Native for android can be maintained along with the flutter for rest.

About the database i think "supabase" is a good open source alternative

[astingengo]

I would like to see this implemented to be able to have then web interface. Thank you for your amazing work

[kusk]

What about just changing the path in /uhabits-core/src/jvmMain/java/org/isoron/uhabits/core/Constants.kt#L21 to something like /sdcard/Download or some other path that another sync app can handle? I changed the code ran it in a emulator and seems to work fine. However cant build a apk because of some config. Anyone else would be able to build it with those changes?

From:
const val DATABASE_FILENAME = "uhabits.db"

To:
const val DATABASE_FILENAME = "/sdcard/Downloads/uhabits.db"

[KyleSCraig]

Have a read of the note I made when building a few months ago and see if it unblocks you.

Did not get any links when trying to sink gradle, ultimately had to fix it by adding to the end of settings.gradle.kts:
plugins {
id("org.gradle.toolchains.foojay-resolver-convention") version "0.4.0"
}

[EDLLT]

That's actually the simplest and best idea

[adharsh]

That's a great idea! I threw together a quick implementation here:
#178 (comment)

It doesn't address point 3 in the original post tho, but it might be a step in the right direction

[EDLLT]

I found an open source tracker called ActivityWatch, their syncing mechanism is smart as it is service independent(leaving the user choose whatever method they find convenient whether that's gdrive, rsync, syncthing, ... through creating some sort of isolated directory structure.

I think uhabits could implement something similar
https://github.com/ActivityWatch/aw-server-rust/tree/master/aw-sync