Updating anki-bundled past 2.1.16+

[genedan]

I tried updating anki-bundled to 2.1.26, and while I was able to get the server started and do an initial upload, downloading a full sync failed.

Some observations:

1. You'll need to install rust and other dependencies and then compile anki, the instructions of which are in README.development.

2. The python modules are now in anki-bundled/anki/pylib/anki, so I tried changing line 5 in ankisyncd/init.py from:

sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), "anki-bundled"))

to

sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(__file__)), "anki-bundled/pylib"))

3. If you try running anki-sync-server at this point, you'll get an error saying that there is no module named ankirspy. You'll need to install it, but the build hash of ankirpsy needs to match anki 2.1.26, so

pip3 install ankirspy==2.1.26

4. If you try running anki-sync-server again, you'll get an error saying 'EnumTypeWrapper' object has no attribute INTERVALS. In anki-bundled/anki/pylib/anki/rsbackend.py, line 168:

FormatTimeSpanContext = pb.FormatTimeSpanIn.Context

pb.FormatTimeSpanIn.Context has no attribute INTERVALS, but pb.FormatTimeSpanIn does, so I tried changing it to:

FormatTimeSpanContext = pb.FormatTimeSpanIn

5. Now you can start anki-sync-server and upload some cards. However trying to download a collection from the server to a new profile fails with the error:

'_Collection' object has no attribute 'reopen'

According to this commit:

ankitects/anki@fa12213#diff-286ad1cf4f79bc4880ecbcc7c6a09062

It seems that reopen() has been moved from _Collection to class AnkiQt in /anki/qt/aqt/main.py

I'm not exactly sure what to do now, but that's where I'm at. I figure it's worth figuring out, since with all the changes that have happened since 2.1.16 and the introduction of the rust backend, etc., it seems like a major change to the way the collection is being synced.

[genedan]

I've been able to get it to work with 2.1.23, in anki-bundled:

1. git checkout 2.1.23

2. Install rust dependencies

3. make check will reveal an unused import "use unicode_normalization::is_nfc" in /src/media/sync.rs, so remove this line

4. compile the rust code, you may need to install the dependency rename.

5. The compilation process outputs ankispry in a folder called dist, install ankispry

6. Update init.py in ankisyncd to update the changed path of the python libraries.

7. fix the issue with INTERVALS in rsbackend.py

It seems to sync now, so far 2.1.23 is the most recent version I've been able to update anki-bundled with. You should be able to use version 2.1.26 as the client without any problem. AnkiDroid versions past 2.10 seem to require https for some users.

[genedan]

Well, the reopen function has been moved back into the collection class, so that solves the problem in the first post:

ankitects/anki@c49c378

But just about everything else from the storage module has been moved into collection as well.

[AntonOfTheWoods]

Thanks for your work on this @genedan ! I'm going to have a go at getting this working up to anki@master. Do you have any more advice?

[genedan]

@AntonOfTheWoods, the latest commit I have gotten it to work with is ankitects/anki@77cf7dd, after which I think the media check fails.

Even bigger changes have happened since I last checked in. It looks like the entire sync.py has been removed:

https://github.com/ankitects/anki/blob/master/pylib/anki/sync.py

I tried using the latest alpha as a client and I can't log in.

[AntonOfTheWoods]

@genedan , yep, I think this may be the end of the road for me, at least for a while. I would need to learn Rust and I don't really have the bandwidth at the moment to try and learn Rust and architect around whatever changes upstream might decide are necessary. He definitely doesn't care about other people's server implementations, that is for certain.

I am using ankisyncd as a part of a platform I am building for my PhD, but I can just take the latest compatible versions of the desktop and droid clients and enforce those. Actually, there are some advantages, basically I have certain model formats that can't be changed, so I will likely rip a few parts out while I am there. I am basing now on 2.1.16, before the rust set in...

From here I guess the plan for ankisyncd will be to radically simplify things if possible. If we think about it on a higher level, if it is just being used as a syncer and not a client, then all it needs is to get the diffs from the clients and then integrate those into the DB. The DB structure doesn't need to mirror the client either. Neither does (should?) it need to use SQLite. But that is probably a completely different project...

[genedan]

I think the good news is the rust is compiled into libraries that Python can use, so I would think that the server can be written in Python as long as it can properly process client requests.

Unfortunately, there's not any documentation on ankirspy or any of the other compiled modules, hopefully that will change once (if?) Anki gets a bit more stable.

There was a class called httpsyncer that Damien provided which anki-sync-server seems to be based off of. That's gone now...next thing for me to figure out is to see how much of it we can use.

We definitely need a new init.py addon file for newer clients, as the constants that it modified are no longer available.