SDK method to update a Sprite animation frame with new image?

Is there an SDK method to update a Sprite animation frame with a new image (from a blob, data URI, JS array, etc.)

I am looking for alternatives/workarounds to a current stuttering issue seen when using the Sprite Action Load Image from URL.

This would inevitably involve a WebGL texture upload, which has a high performance cost. Is there a way to render the image source directly?

I think I found at least one improvement to use more of the C3 SDK, I can use the below to get the rcTex coordinates:

getLeft()
getTop()
getRight()
getBottom()

I think the best approach would be to use the same approach as the Text plugin, which similarly renders via a canvas. The steps are roughly:

1. Draw content to a 2D canvas (text for the Text plugin, or in your case a Spine rendering)
2. Create a new empty texture managed by the plugin
3. Whenever the canvas size changes, destroy the texture and create a new one to match the new size
4. Whenever the canvas content changes, upload it to the texture
5. Finally draw the texture normally with a quad

The key points are the texture is owned and managed by the plugin itself, so it's not trying to overwrite existing content used by other parts of the engine; it uses a direct WebGL texture upload so there's no unnecessary compress/decompress cycle; and there is support in the engine for this approach already (as Text uses it already). I also must point out that you cannot safely mix raw WebGL calls and Construct renderer calls, as Construct's renderer queues WebGL commands using a batching system (meaning WebGL calls will happen in an unexpected order) and heavily relies on caching and ignoring redundant changes (and changing WebGL state outside of the renderer with calls like bindTexture will corrupt the internal state of the renderer and probably break the game). This is why there is no documented/supported way to get the WebGL context directly - it's for good reason! I've added to my todo list to document the necessary renderer calls to support that approach.

I just updated the WebGLRenderer docs to cover new methods CreateDynamicTexture(), UpdateTexture() and DeleteTexture(). These should be enough to implement the approach I outlined above. If not let me know and I can look in to covering more of the API surface. These methods should already be supported in the runtime, but note if you want to use them in the editor they won't be supported until the next release.

Ashley Thanks for the great response, details of the engine and suggested method for implementation. I will take a look at implementing something similar to the text method and thanks for the additional documentation that you just provided. I appreciate the comment on the resize and creating a new texture based on that - very helpful in making this all more robust.

Thanks Ashley this worked well, it seems to perform reasonably well (100 instances at 320x240 still ran at 60 fps, I think I will be more in the 1-10 range.) I am also glad to be using only the C3 SDK and no 'raw' webgl which could get out of sync with the engine gl batching and optimizations. Thanks for the help and the docs. I will post the addon (and Spine example which uses it) after some more testing.

Here's my updated Draw() function.

 Draw(renderer)
 {
 if (this._elementId == "") return; // elementID not set, can't draw the element

 var myCanvas = document.getElementById(this._elementId)
 // Create texture if it does not exist (could this be done in constructor or setElementId action)
 if (this._elementTexture === null)
 {
 this._elementTexture = renderer.CreateDynamicTexture(myCanvas.width,myCanvas.height,{mipMap:false})
 // console.log(this._elementTexture)
 }

 const wi = this.GetWorldInfo();
 const quad = wi.GetBoundingQuad();
 const rcTex = new C3.Rect(0,0,1,1) // Possible to get from this._texture instead? Not needed, not spritesheeted?

 renderer.UpdateTexture(myCanvas, this._elementTexture, {})

 renderer.SetTexture(this._elementTexture)

 if (this._runtime.IsPixelRoundingEnabled())
 {
 const ox = Math.round(wi.GetX()) - wi.GetX();
 const oy = Math.round(wi.GetY()) - wi.GetY();
 tempQuad.copy(quad);
 tempQuad.offset(ox, oy);
 renderer.Quad3(tempQuad, rcTex);
 }
 else
 {
 renderer.Quad3(quad, rcTex);
 }
 }

Ashley This is working very well on Desktop setup and Igor has actually created a nice Spine Addon which uses this new method. I have found it to work well on Desktop.

However, on mobile (specifically iOS/iPhone), the UpdateTexture() takes a long time, specifically the gl.texSubImage2D() call. After some research, this is not surprising due to the PowerVR GPU tiled rendering method and also the GPU's BGRA pixel format.

I am wondering how this is handled for the case for the Text plugin? Is it just reliant on the developer to not update Text plugins that often on a mobile system, or do you have another optimization for cases like a timer that might update a Text plugin every frame?

Yeah, texture uploads are slow. Text does not do anything specific to handle this, but often text objects do not change every frame, so they just render like sprites and largely get away with it. Large text objects that change text every tick have been reported to be slow before, but there's not much to be done about it, so we just advise to avoid that case if possible, or use SpriteFonts which render directly without any texture uploads.

Since a Spine animation presumably can be expected to always change contents every frame, then it will hit you harder. There's very little that can be done to optimise it, other than not using texture uploads at all. That is why my very first suggestion was to find a way to render it directly, so this problem would be bypassed. The main advantage of using texture uploads is that it's probably much quicker to get it working, since you mentioned it looked difficult to set up the library to render directly... so to completely solve this, you're probably just going to have to bite the bullet and do all that work.

Well, I guess you can render using raw WebGL calls, but it's not ideal. Construct's renderer has its own batching and state tracking system, so if you make any changes at all to the WebGL state, you will need to save the old state, do any rendering with custom WebGL calls, then restore the old state again. This is necessary to ensure the WebGL state is consistent with what Construct's engine expects it to be (if you change state and leave it, you're pulling the rug out under the engine and will break stuff). Even then, changing loads of WebGL state is slow. So if you go down this route, your next performance bottleneck will be rendering lots of animations simultaneously, as it hammers saving and restoring loads of WebGL state repeatedly during a frame. Additionally if you make raw WebGL calls you bypass the rest of the renderer, which makes it impossible to use things like custom WebGL shader effects, essentially removing the ability to use that powerful (and very useful artistically) feature.

The ideal solution would be to render everything in a plugin Draw() call using IWebGLRenderer. This fully integrates with Construct supporting all features like effects, and works with the existing renderer code and batching system, so you won't need to do things like save and restore masses of WebGL state every draw, and takes advantage of its sophisticated batching system to ensure maximum performance. The renderer is a fairly high-level API and can do many of the things the standard canvas2d can, so it should be feasible - but you might have to change a lot of code, since it works differently.