Property values being cached for longer than they should

[wetneb]

Originally posted by @Jheald at OpenRefine/OpenRefine#5202:

I was working through some reconciliations suggested by OpenRefine, manually adding a P10873 (Mapping Museums ID) on Wikidata if the reconciliation was correct, or creating a new item there & updating the reconciliation on my OR sheet if it wasn't.

To confirm that I had indeed updated everything correctly, I then used "add columns from reconciled values" to generate an updated column of the MM ids from wikidata on my OR sheet.

In most cases this was successful, and the MM id that I had added was successfully returned. However for six rows no MM id was returned, even though it was now present on Wikidata.

Additionally, when I tried to return P31 values for the six, one edit I had made 24 hours previously was not reflected, also another I had made 9 hours earlier.

I posted an account of this to the Open Refine+Wikidata telegram channel, and somebody there tried to reproduce the issue, but got completely up-to-date values.

I tried again this morning, and found two rows now returned up-to-date values (including one of the P31 changes that hadn't previously been reflected); but not the other 4 rows, which still returned no MM id, nor the other P31 change.

Re-starting OpenRefine and re-reconciling the six rows as a new project did not change this.

To Reproduce

Steps to reproduce the behavior:

1. Load data-file below as a new project
2. Reconcile on Q-ids vs Wikidata
3. Retrieve P10873 and P31 from Wikidata using "add columns from reconciled values"

Current Results

As described above, the first two rows now work for me; however the remaining 4 rows do not retrieve the MM id, and "Wallace monument" does not retrieve the P31 = "monument"

Expected Behavior

All 6 rows should retrieve MM ids, and the P31 values shown in the screenshot immediately below

Screenshots

Success from New Zealand:

Continued failure in Scotland:

Versions

• Operating System: Windows 10
• Browser Version: Chrome version 104.0.5112.81
• JRE or JDK Version: openjdk version "17.0.4" 2022-07-19 ; OpenJDK Runtime Environment Temurin-17.0.4+8 (build 17.0.4+8) ; OpenJDK 64-Bit Server VM Temurin-17.0.4+8 (build 17.0.4+8, mixed mode, [sharing)]
• OpenRefine: OpenRefine 3.6.0

Datasets

Additional context

[wetneb]

I am currently unable to find the time to work on this but would be happy to give access to the production service to whoever wants to debug this redis caching issue.

[thadguidry]

Cache Consistency is known to affect many systems and redis as they describe in redis blog post.

But you don't need Enterprise Redis to solve it, just register a client to react to Expired events and other ways.
Or with 4.0+ versions use the "hammer" and just timely do a FLUSHDB ASYNC or FLUSHALL ASYNC

But generally, yeah, just update the cache when the data changes. or only massively changes, depending on needs.