Provide guidance when we fail to download data files

[birtles]

We get a lot of error reports of failed downloads which are either:

• 403 access denied, or
• some HTML being injected into the JSON file

So many, in fact, that I recently changed the error reporting mechanism to use a different channel for these types of issues.

In general, they seem to come from firewalls, proxies, and virus checkers (I've seen "ESET" come up in some of the reported error messages). I've heard from people in Russia that they have trouble connecting to the 10ten.life domain where we serve the data files from.

Perhaps there's some way to work around this such as using a more well-known domain to proxy the data files?

For now, I think we could probably detect some of these cases and show a message in the database status part of the options page suggesting adding data.10ten.life (or 10ten.life) to your firewall/virus checker. I know of at least one user who was able to download the data files after adding 10ten.life to their proxy.

[JoselleAstrid]

Hi, I'm not sure if there's another preferred place to add my user experience on this - please let me know if there is.

I'm running on Firefox + Linux, and at the bottom of the extension's Preferences tab, it says Update failed: Version file https://data.10ten.life/jpdict/reader/version-en.json not accessible (NetworkError when attempting to fetch resource.) When I try to access that URL directly on my PC or phone, it looks like there are two difficulties:

1. The browsers I've tried can't read a valid SSL config for data.10ten.life. For example, Firefox gets SSL_ERROR_RX_RECORD_TOO_LONG, Vivaldi gets ERR_SSL_PROTOCOL_ERROR, and my phone browser just says the site doesn't support HTTPS. No such problems when accessing the parent domain 10ten.life though.

2. My home ISP in the US, Spectrum, blocks data.10ten.life, saying it's a suspicious site which may contain unsafe content. My mobile provider, T-Mobile, does not block it though. Again, no such issue with just 10ten.life.

[birtles]

Hi @JoselleAstrid! Thanks, that's super useful information. I don't know why your browsers/ISP are having trouble with the SSL config. For (1), are you referring to connecting via your phone over data (e.g. 4G/5G) as opposed to through wifi?

I wonder if it's because there's no homepage at https://data.10ten.life. Maybe I should add a blank page there and then work out how to get it off Spectrum's block list.

It's also odd because I get a lot of "failed download" telemetry pings sent to data.10ten.life so it seems like some browsers/ISPs are rejecting downloading the JSONL data file but allowing a PUT request to the telemetry endpoint 🤷

Surely someone out there knows the dark arts of ISP blocking behavior.

[birtles]

Ok, I did some digging around and it seems most likely that hose SSL errors you're encountering are due to Spectrum inserting a security block.

I've tweaked the site to add a root homepage, robots.txt etc. and submitted a false positive report to Spectrum so hopefully they can fix their list in the coming days.

Thank you so much for reporting this!

[JoselleAstrid]

For (1), are you referring to connecting via your phone over data (e.g. 4G/5G) as opposed to through wifi?

The SSL error was actually the first thing I saw. I only saw the Spectrum block page when I was able to say "don't automatically redirect me to HTTPS". For some reason I wasn't able to get Firefox to stop doing that, but when I directly did wget or curl on the URL starting with http://, I did get the Spectrum block page. The other way I got the Spectrum page was using my mobile browser through WiFi, since I was able to tell it to continue without HTTPS. When it was my mobile browser through 4G/5G, I got a valid version-en.json, and trying the data.10ten.life homepage got some kind of failed download.

Trying again now, mobile browser through 4G/5G gets a data.10ten.life homepage saying:

This domain hosts machine-readable data files used by the 10ten Japanese Reader browser extension.

I assume that's the homepage you just added then. Thanks for the quick response, and I'm glad the report was helpful! Hope Spectrum unblocks the subdomain soon. This extension has been really helpful to me over the years, so I'm looking forward to being able to update it regularly again without a VPN or similar.

[birtles]

Thank you! Yes, that all fits with what I understand.

The Spectrum false positive report form said it can take up to 5 days to process the request so with any luck it might just start working by the end of the week.

[birtles]

@JoselleAstrid Spectrum have updated their block list now to remove data.10ten.life. They say it might take up to 48 hours for the change to take effect. Just wondering if it's working for you now?

[JoselleAstrid]

Yes, it's working now - successfully updated dictionary data through my home WiFi. Thank you very much!

[birtles]

Thank you!