Data Synchronization in NASPA Zyzzyva and NASPA Zyzzyva Mobile

Wait, there’s a mobile version of NASPA Zyzzyva?

Yes! NASPA Zyzzyva Mobile (NZM) was introduced in 2021 as a mobile companion to the familiar NASPA Zyzzyva (NZ) desktop application. Both are currently at version 3.4.2. While they look very different on the surface, they share common quiz, search, and judge functions, and they are fully compatible in how they record your saved study data. For more information about NZM, please see NASPA Zyzzyva Mobile.

What is data synchronization in Zyzzyva, and why would I want to do it?

Zyzzyva data synchronization is mostly for users with more than one device that they use for word study. If that describes you, and you don’t synchronize your study data between devices, then nothing is coordinated, because those installations live totally separate lives. (That may be good enough, for example, if you only use the desktop application for adjudication, or if you’ve been exceptionally clever and moved your study data to a USB drive that you plug into whichever device you’re using.) Data synchronization makes multiple Zyzzyva installations seem like one, so you can switch devices and pick up where you left off in your study.

People who use Zyzzyva data synchronization usually have one of two purposes: Either they routinely move back and forth between a home computer and a mobile device, or they use NZM to run quizzes that they had to create in NZ because NZM’s set of search conditions is too limited. (NZM can run those, even though its user interface can’t create them.)

How does data synchronization work? Is it like a backup?

Zyzzyva synchronizes your study data with servers operated by a cloud-storage provider. If you use only one device, but you’re concerned about losing years’ worth of study data if it’s lost or stolen, data synchronization is one way to do it, but there are other ways: On a computer, you can train an Internet service’s program to back up your study data to their servers on a regular schedule. On a mobile device, Zyzzyva’s data are part of the larger set of user data that is covered by backup features included in the operating system.

When reading the following, it’s important to remember that your Zyzzyva installations will be syncing with a cloud-storage provider, not directly with each other. (At no time is your Zyzzyva session “live on the Internet”!) Consequently, syncing your data between devices always requires an “up” sync followed by a “down” sync.

What Zyzzyva data can I synchronize?

In both the desktop application (NASPA Zyzzyva) and the mobile app (NASPA Zyzzyva Mobile), every user has a “data directory” in local storage to hold all of your study data — everything but the program itself. When you run it for the first time, the program creates this directory in a location that belongs to you (named Zyzzyva in your home folder, if you’re on a computer; it’s hidden on mobile devices).

For the purposes of synchronization between devices, this data directory is one big blob — you can either sync its entire contents or nothing at all. Still, it’s good to know what’s in there: The quiz subfolder contains your quiz statistics and cardbox data, along with any quiz specifications that you created and saved. The search subfolder contains any search specifications that you created and saved. The words subfolder contains any word lists that you saved, perhaps from the result of a search. The judge subfolder contains logs of your adjudications. The lexicons subfolder contains lexicon databases, if you’ve opted to build them, and any Custom lexicons you may have created in NZM.

Is synchronization only for mobile users?

First, let’s distinguish between “passive” synchronization, which requires the assistance of another program to sync data with a cloud-storage provider, and “active” synchronization, which is something that Zyzzyva manages on its own using network services.

As of version 3.4.0 and later, both NZ and NZM support active synchronization with a cloud service, in a similar fashion. This is the configuration that we recommend for most users. Active sync only occurs when you trigger it, avoiding most redundant network traffic.

Historically, NASPA Zyzzyva, the desktop application, has always been compatible with passive sync using Dropbox or another cloud-storage provider; it only required that you move the data directory to a location that the provider’s installed application targets by default. This works very well between computers, but it generates a lot of network traffic, because files in the data directory change every time you answer a quiz question or judge a word. Also, because passive sync happens in the background, it can be hard to know when it’s complete. Despite those issues, many users are happy using the provider’s installed application, so NZ continues to support passive sync.

The mobile app (NASPA Zyzzyva Mobile and its predecessor, the pre-2014 Zyzzyva iOS app) only supports active sync, and it does not use or benefit from the installation of any cloud-storage sync program, thus, no passive sync.

Whatever your scenario, Zyzzyva works fine while you’re offline, and you can sync any changes to its data when you’re back online.

What cloud-storage providers can I use, and is it safe?

For passive sync when using NZ on a computer, any cloud provider will do, including Dropbox, Google Drive, iCloud Drive, and OneDrive. For active sync, Zyzzyva currently only supports Dropbox. Dropbox offers free storage space up to 2 GB per account, and active-sync clients such as Zyzzyva don’t count toward the maximum of three connected Dropbox program installations per account.

Zyzzyva’s active-sync engine only requests access to its own “app folder” in your Dropbox storage, so it can’t see any other files that you may have stored there or otherwise affect your cloud-storage setup.

How exactly do I synchronize my Zyzzyva data directory across my desktops / laptops and mobile devices?

For the recommended active-sync configuration, consult the online help in NZ and NZM to get your devices linked to a sync provider and take note of its advice about operation. The passive-sync alternative requires some technical knowledge and careful attention to the step-by-step instructions here.

In most cases, users have a desktop / laptop installation with valuable data before installing NZM or installing NZ on another computer. In this scenario, it is important to get that desktop / laptop installation fully synced with the provider first (whether actively or passively), then perform the new installation, then sync it with the provider immediately before doing any quizzing there, so as not to risk damaging those valuable data. Once they’re synced as a group, you can stop worrying about such conflicts. These steps are outlined in the instructions linked above.

If instead you have two existing installations that both have valuable data and aren’t synced, it won’t be practical to merge them, and syncing them will force you to choose one over the other, so be careful!

Will synchronization use up my mobile-data plan?

Probably not, but if you’re uncertain, use your mobile device’s metering features to watch how much data each sync is using. The initial sync could use as much as 400 MB of data, so maybe you’ll want to do that one while you’re on wi-fi (and not on your friend’s tethered hotspot!).

What could possibly go wrong?

Zyzzyva’s sync engine is very cautious, and it hasn’t been known to “lose” data on its own, but it’s good to be conscious of what can go wrong without realizing it in the moment.

Anything that changes at one end of the sync relationship will be reflected at the other, either later or almost immediately, depending on whether the sync is active or passive. That includes routine updates to file contents, but also deletions and replacements. When Zyzzyva’s active-sync engine sees changes to a file at both ends, it will favor the end whose change seems more recent, if asked to consider both. If it already had knowledge of the file from a previous sync, it’ll prompt you to choose a side — should it consider changes from one end, both, or neither, or just cancel the sync? This will usually steer you to make a safe decision that you won’t need to correct later.

The scarier scenario is where the sync engine sees a file for the first time, but it exists in both locations with different contents and last-modified dates; this is very common when a user is syncing the data directories of two Zyzzyva installations for the first time. In such a case, and especially when the sync engine tells you that there are many files in conflict, read the prompts carefully and consider the consequences, as it will happily overwrite an older quiz-stats/cardbox database that you wanted to keep with a brand-new but almost empty one from a new installation where you just happened to run one quiz before syncing. (Ouch!) Remember, changes are handled at the file level, so don’t expect database changes to be merged at the record level.

Why does synchronization take so long?

With passive sync, you probably won’t notice how long it is taking, and it will pick up where it left off if it was interrupted. Most of the time, active sync is very quick, because the active-sync engine is conscious of the prior state of each file and only syncs files that have really changed. However, the first sync can take a long time (minutes to hours, depending on the size of your data directory and the speed of your network connection). Watch the file-countdown indicator if you aren’t sure that anything useful is happening; you can extend the timeout setting if you are getting a lot of prompts asking if you want to cancel the sync.

On mobile devices, especially iOS devices, it’s important not to put the app into the background or put the device to sleep during a long sync, because the operating system will suspend the app before long, and the sync will have to be restarted — so just plug it in and let the sync complete.

How long a sync takes depends greatly on the size of the files involved. Of all files in the data directory, lexicon databases are by far the largest (typically 60 MB), but they’ll only change if you rebuild them, so consider that factor if it seems that a sync is taking a long time.

Why do my data take up so much space?

The format of Zyzzyva’s data directory is the same on computers and on mobile devices. Unless you’ve built lexicon databases (optional, but recommended for many reasons as explained in the help files), the size of that directory is generally small. Where the pain can start on mobile devices with limited storage is with lexicon databases, either ones you built locally or ones that were synced in from another device where storage wasn’t a concern.

Lexicon databases typically use 300 to 400 bytes of storage for each word in the lexicon. This can easily put the size of the database over 60 MB, and you can multiply that by the number of lexicons. Some of those lexicons, you probably don’t even use. So, to cut down on the space consumption on your mobile devices, you may want to delete some of them. There isn’t a way to do this through NZ or NZM, so you’ll have to delete the files yourself; the best way is to log on to the cloud-storage provider’s website, find the Zyzzyva data directory and then its lexicons subdirectory, where you will find files with names like NSWL2020.db. (To avoid creating those files in the first place, use Zyzzyva’s settings to unload those lexicons before asking it to build or rebuild lexicon databases.)

Why am I seeing data synchronization conflicts, even after the initial sync?

Zyzzyva’s active-sync engine is very cautious, as mentioned above, so it may report file conflicts when they don’t exist. For example, if you initiate a routine sync from your mobile device and put it to sleep, all of the changed files may reach the cloud-storage provider, but the app may be suspended before it gets final confirmation of that, so it won’t update its sync status; the next time you ask it to sync, both ends of the sync relationship think that they have an updated version of the file, which could indicate a conflict. Only after you tell it how to resolve the conflict will the engine go to the trouble of computing the cryptographic hash of the file’s contents and decide that there’s no actual change to be synced.

A sync just squished my cardbox! What do I do now?

Oh dear. Let’s hope you have another copy somewhere. If you have a device that hasn’t yet synced with the cloud-storage provider, you may be able to preserve an older copy of the affected database file and feed it back into the sync process after the damaged file has finished propagating. Or, if your computer operating system or cloud-storage provider offers a way to recover previous versions of files, you can restore it that way, and let that restored version propagate among your devices.

Can I make active sync automatic?

Passive sync is by its nature automatic, as long as the cloud-storage provider’s program is operating and your devices are online. Active sync only occurs when you trigger it, but there are options you can set to make it happen automatically when Zyzzyva starts or quits — or, on a mobile device, whenever you bring it to the foreground or put it in the background. (On iOS devices, this sync-on-quit task has about 30 seconds to complete before the operating system will suspend the app, aborting the sync; however, this is usually more than enough time for it to complete, and you can redo the sync later if it gets cut off.)

Can I sync my data with other users?

No. When you set up data synchronization, either passive or active, you must log on to a cloud-storage provider using a specific account. Unless you share those account credentials with others, the Zyzzyva data directory stored in that location is not accessible to them. (Outside of the sync scheme, though, there’s nothing stopping you from sharing those files with other users.)

I use a Custom lexicon, and nothing seems to sync properly. What can I do?

Custom lexicons are problematic for data synchronization. While each named lexicon is assumed to be identical between installations, the one lexicon named Custom may not be, so it would be bad practice for Zyzzyva to maintain one quiz-stats/cardbox database and one set of adjudication logs in its data directory that are synced between devices with different Custom lexicons.

NZ lets you load a Custom lexicon from any location in local storage; NZM only lets you create one interactively, storing it in the lexicons subfolder of the data directory under a timestamped name. For now, they don’t agree on where to store a lexicon database that you may build from that Custom lexicon. The only way to marry these implementations so that they sync properly involves some serious Unix wizardry, so we don’t recommend it. A solution to this problem will have to wait for a later version where all lexicons are separately loadable and distinguished between installations by analyzing their file contents, so that the Custom lexicon is no longer a special case.

My data directories won’t sync across NZ/NZM installations, though they should. Why is this happening?

We’ve seen this happen to users who set up a passive-sync link to Dropbox by creating a folder named Apps/Zyzzyva in Dropbox storage and changing their NZ preferences to use the locally synced copy of that folder as their data directory, and only then tried to enable active sync on that or another device. When the active-sync engine tries to sync with that folder in the cloud, it is refused access and creates its own Apps/Zyzzyva (1) folder instead; the two Zyzzyva installations don’t sync with each other.

The problem, here, is that Dropbox strongly enforces access control between its client apps; if an app didn’t create the folder, it can’t access it. First, you’ll have to break all of your active-sync links, because they’re pointing to the wrong place. Then, take a backup copy of your “good” (passively linked) data directory, delete both folders (Apps/Zyzzyva and Apps/Zyzzyva (1)) in the cloud, and, once those deletions have propagated to all passively linked devices, link one device to Dropbox using active sync so that our Dropbox client identity can recreate the Apps/Zyzzyva folder in its own right; then, once the “good” device is linked to Dropbox again (actively or passively), restore the backup copy to that device’s data directory location, and finally sync all devices to resolve any conflicts that may result from differences between their data directories.

I tried to link NZM to Dropbox using “Sign in with Google”. Why doesn’t that work?

NZM uses an embedded browser (“webview”) to ask for authorization when you ask it to link to a cloud-storage provider. The resulting logon page may include a “Sign in with Google” button, which probably won’t work. That’s because Google’s identity service doesn’t support browsers that don’t store cookies (I wonder why?). Better just to log on with an account and password — you should only have to do this once per device.