Performance investigation of blitting to 2d context

[DavidPeicho]

Great library!

I don't really like to pollute issues, but I guess it's a good question to ask.

I see that you keep a single canvas with a WebGL context, that you blit per scene to canvas with 2d context.

Another way of doing it would be to have a fullscreen canvas in the DOM, on which each scene would blit at the good position using scissoring. My guess is that you didn't go this way because it breaks the rendering order of the elements in the DOM.

However, did you investigate on the performance impact this call has compared to blitting directly into a fullscreen WebGL context?

I am trying to setup a quick test to see the speed difference, but if you have already done that test it would be amazing if you could share the results.

[cdata]

@DavidPeicho the way we optimize contexts is indeed not ideal, but it is necessary. If I understand your proposed alternative correctly, we would need to measure the layout of <model-viewer> elements in the DOM (possibly on every frame) in order to know where to draw them on the full screen canvas. Is that correct?

We have plans on the books to optimize our current approach by sometimes not blitting at all, and sometimes using OffscreenCanvas + ImageBitmapRenderingContext. All in due time.

[cdata]

Of course, any investigation you do into this would be interesting to read. Although I can already tell you that the way we do things today is slow compared to basic alternatives.

We also compensate a bit right now by not rendering unless something actually changed in the scene.

[DavidPeicho]

@cdata That's indeed correct.

The only problem with this approach, is that the final blitting is on only one canvas, so for a WebComponent library, that kind of break a lot of things in the DOM (how could someone draw a div, the webgl stuff above, and then another div above the canvas?).

If I understand you first link, your goal is too simply have at many WebGL context as you can without breaking the tab (so 16 if I am correct)?

The second link looks interesting but I am not sure what will be the blitting cost.

[cdata]

@DavidPeicho if I understand ImageBitmapRenderingContext correctly, the primary optimization it offers is that image data does not need to be copied when it is transferred so it saves memory.

If I understand you first link, your goal is too simply have at many WebGL context as you can without breaking the tab (so 16 if I am correct)?

Yes, the intention of this approach is that if we detect < N (where N is some number less than 16) <model-viewer> elements are in use on the page, we would just create that many WebGL contexts instead of the current approach. We have to be careful with this because the page might be doing other WebGL things.

so for a WebComponent library, that kind of break a lot of things in the DOM

Yes, <model-viewer> strives to be coherent - even complementary - with the DOM. This means abiding by notions of layout, stacking context, focus, interaction events etc. It shouldn't lead to any surprises when the browser goes to composite the page.

[DavidPeicho]

So I guess the best shot seems really to be ImageBitmapRenderingContext, but as stated it's early stage right now

[DavidPeicho]

Ok, I know the fullscreen canvas is not a viable option for you, but I decided to give a shot to a test.
I will check that the tests are fully working tomorrow after some rest. Because obviously if they aren't, my current results & assumptions are biased.

Here is a JSFiddle that you can tweak to test with WebGL Scissoring blitting, and with Context2D drawImage blitting. (Sorry the code isn't super clean, but I had to go fast).

btw: The viewers are created dynamically, but you may have to change manually the size of the instanciated viewers, I have been a bit lazy on that.

Below are the results on my full HD screen and my Quadro P2000:

LOOP_PER_RUN 5000 18 viewers (250 * 250 viewers)

  Starting test with context2d...
  TEST: blitContext2d, run in 233784.29500001948ms

  Starting test with WebGL...
  TEST: fullWebGL, run in 83561.55999994371ms

LOOP_PER_RUN 5000 6 viewers (250 * 250 viewers)

  Starting test with context2d...
  TEST: blitContext2d, run in 85846.12000000197ms

  Starting test with WebGL...
  (index):238 TEST: fullWebGL, run in 83501.52499997057ms

LOOP_PER_RUN 5000 6 viewers (480 * 352 viewers)

  Starting test with context2d...
  TEST: blitContext2d, run in 83355.6249999674ms

  Starting test with WebGL...
  TEST: fullWebGL, run in 83380.114999949ms

LOOP_PER_RUN 5000 2 viewers (768 * 564 viewers)

  Starting test with context2d...
  TEST: blitContext2d, run in 83346.50499990676ms

  Starting test with WebGL...
  fullWebGL, run in 83365.46999996062ms

Actually, the results are way better than what I expected. But again, this is based on the assumption I didn't make any mistake in the tests, which I will look again more thoroughly tomorrow.

It may be a good idea to tests on relatively different GPUs too.

You will obviously not use the fullscreen canvas technique, but at least with these results show that you can be pretty confident with the blitting using drawImage. It would be great to indeed compare that with the ImageBitmapRenderingContext technique.

[cdata]

That's interesting, thanks for looking into that. It is disheartening but not surprising to see the drawImage approach being up to an order of magnitude slower. It might be cool to have samples for all of the approaches, including context-per-element, if this is an area you are interested in spending time :)

[DavidPeicho]

I can look into that maybe yes. I already implemented the scissoring version some time ago (the test above has it too, but it's not 100% working with the scrolling), but there is actually a really good usage of it on threejsfundamentals. That's actually a tutorial with code showing how to implement it, so that's neat!

I also thought about the fact that you don't often have more than 3 - 4 viewers visible simultaneously. meaning that the context2d blitting is actually a pretty good idea, especially if ImageBitmapRenderingContext gives a blitting boost later (which may not be the case though).

[cdata]

@DavidPeicho that's a good observation. Early on, we speculated that our worst case scenario is probably on the order of ~10. That's actually why we built this example in the first place 😄

[DavidPeicho]

Yes, I guess that kind of conclude the experiment.
Is it slower? obviously yes (and not in a crazy way)
Is it a real problem? I guess not

[DavidPeicho]

Hi @cdata,

I actually investigated it a bit more. One thing that's an issue actually is not the blitting time of drawImage, but rather the fact that two canvases with two context2d can create massive browser software re-paint.

In this example, I only have two canvases on which I draw using the same technique you use.

I profiled quickly your tests page, and you actually have the same problem as I do:

However, in your animation examples, it's actually funny because the browser re-paint & compositing seem to happen mainly on the GPU, saving you a lot of time that hasn't been saved in my sample.

I profiled quickly your samples page:

I am not entirely sure of my sketch, because I am not sure what's happening after the first two parts on the GPU, but I imagine it's more linked to browser compositing and repainting.

I just wanted to make you aware of that (maybe you already are actually), I don't know what would be the best choice, it's still a tradeoff between creating WebGL contexts that including memory duplication, or spending time doing browser painting....

[cdata]

Thanks for doing additional research here 👍

I actually noticed recently (like in the last week) that my MacBook Pro is experiencing really high repaint times in certain scenarios. I wonder if there were recent rendering changes that may be putting us on a bad path.

I did some testing recently to get a back-of-the-napkin comparison between our current strategy, naive strategy (e.g., one context per element) and the bitmaprenderer approach. In general, perhaps seems to be improved for both of those scenarios. I didn't pay close attention to paint/composite costs though, I should go back and see.

[DavidPeicho]

It's not really related to your code. The browser basically decides when it's worth to do GPU repaint / compositing. For instance, I have a little sample of my own, in which when the canvas has a width that is > 50% of its parent size, everything is good and the painting is fast (on the GPU). However, if I put the canvas width at 30% of the parent size, I get a massive 4ms paint per frame....

If you could monitor before and after, and do profiling of each, I have to admit that would be awesome! Last time my mistake was to not look at the browser potential reflow / repaint.

[cdata]

Sorry, yes, I meant rendering changes in Chrome 😄

Yeah, we want to establish some kind of performance testing regime. It can be tricky to measure properly in a way that is consistent (VM resources can vary a lot depending on when / where it is provisioned etc.).

[DavidPeicho]

Ha ok, I wasn't sure haha. As a sidenote, my code also runs much more slowly in Firefox, so even if some changes happened to Chrome recently, that I guess that's still a general performance problem.

Indeed, having formal performance benchmarks will be hard. Let me know if you need help for that.