dynamic loading of assets (full-offline support) (WIP)

[foxsouns]

todo here now.

this is the pull request for dynamic loading of assets in webretro. for discussion, and a general "problem tracker". goals and stuff are above.

[foxsouns]

@BinBashBanana need some help on this: could you explain, to the best of your knowledge, how the cores are loaded in this? i need to know where it is hitching on to get past where i am at.

[foxsouns]

just realized that, by default, index.html doesnt even work locally (with internet), meanwhile i have a menu working (without internet). so, at least i have progress towards something functioning.

[BinBashBanana]

I'm already working on this for the next release (v6.6), see #52. Since cores are 2-6 MB (per core), I don't want to bundle them all in one file, but instead download them once and use the Cache web API for offline use. I will bundle the essential assets (javascript, css, etc) into the offline html file.

To answer your question, each core has 2 parts; the .js file and the .wasm file. The .js file is loaded from base.js, and the .wasm file is loaded using fetch from the .js file.

[foxsouns]

@BinBashBanana still going to work on this to a functional state on my own: i need something to spend my free time on anyways. i will adapt my end goal to that though: ends up with less work on your end for v6.6.

[BinBashBanana]

Ok!

[foxsouns]

Got core: snes9x
stderr: failed to asynchronously prepare wasm: [object ProgressEvent]
stderr: [object ProgressEvent]
Succesfully read ROM file "Super Mario World [USA, headered].smc"

..it never even loads the core
at least i know for sure now, time to work on that

[BinBashBanana]

This happens when you load the core from a file:// url.

[foxsouns]

@BinBashBanana: This happens when you load the core from a file:// url.

Which is, indeed, what my purpose here is: any idea on why they do not function? Thinking about potential workarounds: I guess I could try to implement indexdb loading to get around it, but that seems like a handful that I don't exactly want to attack until necessary.

[BinBashBanana]

Most browsers disable XHR/fetch to file:// urls, for security reasons. You could theoretically embed the .wasm file into the .js file and it could work. Or see here.

When doing quick local testing, another option than a local webserver is to bundle everything into a single file, using -s SINGLE_FILE (as then no XHRs will be made to file:// URLs).

[BinBashBanana]

(Doing this may increase load times, because modern WASM streaming won't work)

[foxsouns]

Ah, I see. Everything works from Github Pages: now I guess I have to figure out how to actually use indexeddb, now. And.. also do that with the BIOS files, too.

...Or just embed the BIOS files in the repo.. but that's a bit storage-wasteful. probably just do the above, as much as that may suck to program.

[BinBashBanana]

This is why I want to use a service worker cache, it will make it much easier for the asset bundle, cores, and BIOSes.

In v6.6 I plan to make a feature that uses the service worker to store cores for offline usage. It will have a "manager" so that you can choose which cores to cache.

Originally posted by @BinBashBanana in #52 (comment)

[BinBashBanana]

Although, now that I think about it, service workers may not work on file:// urls either. I'll check.

[BinBashBanana]

Just a heads up, this is not dynamic linking. Dynamic linking is something else I have planned for the future, see here.

[foxsouns]

Just a heads up, this is not dynamic linking. Dynamic linking is something else I have planned for the future, see here.

sorry: must have mixed up "linking" with "loading" somewhere, haha. to clarify: i always meant "loading". oops.

[foxsouns]

@BinBashBanana: thinking of different ways this could be implimented: i have a few i could work on, and wanted your opinion on it.

possible approaches

• no dl frontend at all, download cores/bios files on first load (results in no visible change to the user, but only way to remove them is via deleting site cache).
• dl frontend is unobtrusive, available at the bottom of the core-select page. downloads are preformed automatically (but doable within menu; deletion as well). user doesnt have to learn everything, but the tools are there. cores are always listed, regardless of installed status.
• dl frontend is obtrusive: available immediately upon page load. user has to select assets before progressing to the core-select page (which will only include downloaded cores). forces user to understand the new way before progressing.

[BinBashBanana]

I think automatically downloading everything would be a bad idea, so I want to have a manager (one of the modal windows) for it. Unobtrusive would probably be best (?)

[foxsouns]

@BinBashBanana this is roughly what i had in mind (ignore the quick paint art)

[BinBashBanana]

Something like that yeah. There are no per-core assets, besides the BIOS files.

So it could look like:

• Asset bundle
• Core: Beetle PSX HW
  • BIOS: Beetle PSX HW
• Core: Nestopia UE
• Core: Genesis Plus GX
  • BIOS: Genesis Plus GX
• Core: Snes9x

[foxsouns]

right, that makes sense
was thinking shaders were game-specific for some reason, lol
here's the site as is: going to be fun robbing, er, probably what i can salvage from the sram manager code, and then implementing service worker backend stuff, and generating downloadable cores and bios stuff from a list..

[foxsouns]

i havent forgotten about this: life has only taken priority for a bit. it will probably stay like that for a bit more.. taking a couple of days to myself, to recover and focus on myself, and also thanksgiving stuff.

(and now, to close this off with a very Stallman-esk "Happy Hacking!")

[foxsouns]

sorry about going kind of stale on this. life has been rough, and i haven't been feeling very motivated. this is still something i am willing to work on, it's just going to take a bit.

[BinBashBanana]

It's all good. I frequently feel demotivated as well.