GDevelop 5.6 Adds Real-Time 3D Editor

Maybe it is worth considering to make the C3 editor extendable by 3rd party developers instead.

The problem is, stated like this, and to do a good job of it, is an absolutely huge, open-ended project with some pretty nightmarish maintenance considerations. For example in my view to do this properly, we'd need to expose to addons the entire editor UI library, such as tree controls, list views, data tables etc. so editor addons can make use of the same look-and-feel of the rest of the editor and not have to reinvent the wheel for things like a custom editor window with a tree view in it. Every API that addons can use then obligates us to effectively support it forever to avoid breaking backwards compatibility - especially as breaking something that causes the editor to crash on startup has a far higher impact than something that crashes the runtime on startup. In the long-term that can then become a drag on our ability to upgrade the core editor features.

We could bypass that by just giving addons a window to put any old HTML in, but then you likely end up with a bunch of ugly addons with janky looking UI. Implementing quality user interfaces is actually extremely difficult and a great deal of work, and I don't think it's realistic to expect addons to do that for each and every addon. There are also likely going to be complicated compatibility problems anyway - the editor is designed to support only the built-in UI, and is not designed to handle arbitrary HTML injected by addons. There may well be glitches and bugs that happen because an addon inserts some HTML or CSS that does not integrate with the editor. People then assume it's our responsibility to fix such things, which ends up being a nightmare. HTML and CSS also don't have much in the way of encapsulation - although it's better now than it was a few years ago - and so we risk ending up in an SDKv1 type situation with regular breakages as future Construct updates change things.

Then if you want to go in to further specifics, like customizing the Layout View, integrating with the undo system, etc. then you open further cans of worms with compatibility, API design and so on.

So on that basis I would say rather than making an editor system to allow others to implement a 3D editor view, it is actually significantly less work and much easier to support for us to just implement the 3D editor view ourselves.

Other than that, a better approach is to figure out what the smallest possible set of editor features you'd need to have fairly good flexibility for some useful editor features, and request that. For example to address the specific problem of being able to import complex content to the editor, like Spriter animations, we built the Custom Importer API with methods like AddDragDropFileHandler (see the customImporterPlugin sample for a demo). It is far more feasible to support specific cases like that than go for a whole open-ended editor addon system.

[...] That's a fairly significant amount of work to implement, but it's doable.

[...] Construct is a pretty mature product now. Perhaps risk the rest of the product suffering or getting left behind is not that significant a concern any more. So perhaps we could more seriously consider doing more work on 3D features.

[...] Let us know what you think! - Ashley

This is an exciting response. I'm happy with the Scirra team being open to exploring 3D further.

I’d like to ask Ashley about the feasibility of adding ThreeJS, BabylonJS, or PlayCanvas as alternative renderers. It may seem like an absurd suggestion, but hear me out... Although it would require considerable effort, it could be less than developing and maintaining each new feature from scratch, as you are currently doing. In short, it could provide a strong (WebGPU-ready) foundation to build upon.

Since most of Construct's built-in plugins render either quads or simple meshes (such as 9-patch), now might be the right time to consider making such a change.

This could be introduced smoothly for those who opt in, as you did with WebGPU OR as a special layer type limited to objects of certain plugins.

for example:

I experimented by extending Construct's official 3D Engine example using TypeScript and subclassing to recreate, synchronize objects, and utilize textures from Construct's spritesheets in a 3D Library (similar to ThreeJS)... including Construct's 3D Camera. It turned out to be relatively straightforward. I am happy to share the complete working example along with the source code. (It worked so well that it might become my default workflow for basic 3D games)

Play on Construct Arcade

Reason GDevelop has this all of a sudden is because they basically just implemented Three.js. - joelmayer

This!

Why is it important to leverage an existing 3D library (like ThreeJS)? Otherwise, Scirra and the community would need to reinvent the wheel for each feature, such as bone animation, WebGPU, particles, mesh distortion, Spine, lights, shadows, and so on, whether natively or via community addons.

From my experiment, it seems easier to enable Construct-built-in plugins to work in a ThreeJS environment than to develop all the upcoming features from scratch in Construct's internal 3D context.

To begin with:

• Load 3D models (glb, gltf);
• Normal 3D view in the editor;
• Object rotation on all axes;
• Changing textures in runtime;
• 3D animation support;
• Enable/disable nodes;
• 3D Physics;

That alone would be wonderful.

- artheads

This reinforces my suggestion of integrating a 3D Library. For example, the MakeIt3D plugin was developed in a few months using the approach I described and includes the features you mentioned (plus Spine, real-time lights, shadows).

MakeIt3D achieved this by utilizing ThreeJS and synchronizing Construct's internal state with the 3D library's state [video]. This serves as the strongest proof-of-concept we could have. I think it would be great if Scirra could adopt a similar approach, especially if they can ensure smooth integration with the editor.

Integrating an existing 3D engine like ThreeJS is not a straightforward choice and has some tricky trade-offs.

Absolutely. I completely agree with you on this. I understand it's not simple or straightforward. My proof of concept was "straightforward" as a testing ground, but of course, a hypothetical official support involves much more and is far, far, far from simple.

[...] it's probably a major project to switch to another engine wholesale. Making a high-performance 2D engine also has specific requirements that 3D engines don't usually need or care about, so it may be that it cannot be done without a major performance degradation. [...] In the long term that means the limitations of ThreeJS become the limitations of Construct.

That's why I've been thinking of this as a special/dedicated layer (like the HTML one) with restrictions that don't clash with the existing high-performance 2D/basic3D engine. The limitations of ThreeJS would become at most the limitations on an opt-in layer type.

Such a layer could support Sprites, 3D shapes, and addon/SDK2-based objects (which could interface with ThreeJS's context). This would limit the surface of potential bugs by being somewhat "isolated" from Construct's current capabilities.

Users using this "beta" layer would understand that they are in a "third-party" territory (as with HTML layers).

Taking on board a third-party dependency for such an important component is also fairly risky in the long-term. The intentions of the ThreeJS maintainers may not align with our needs. [...]

Indeed. I agree with you again. Dependencies can cause significant issues, and many people underestimate how problematic it is to have excessive dependencies beyond their control.

It's important to "pick our battles", and so far, you have done a good job of picking stable dependencies that are "invisible" to us users. For instance, the way you handle minification, the Monaco editor, TypeScript, etc. So I absolutely respect your decision.

Since you said you're open to tackling 3D "a bit more", then perhaps there's a good compromise to be had with an "isolated layer" and a well-chosen external dependency.

I suspect a hybrid approach like that might be the way to go. As the 3D engine example illustrates there are also already ways you can include third-party 3D engines wholesale, so that's something you can explore as well, and perhaps there are also ways we can tweak the engine to better support that.

This is precisely what I'm proposing. Some QoL around the concept demonstrated by the 3D engine, leading the way to what an "ideal surface area" would be, can go a long way.

A special layer that instantiates a ThreeJS context and has supports a subset of plugins (Sprites, 3D shapes, and the SDK2-based scripting interface to access the ThreeJS context) seems to be the most balanced and less risky approach I can think of.

PS: Please think about this with an open mind, like how you did with the TypeScript request back in the day. =)

edit: I'm happy to submit a feature request for this on github, but it likely won't get many +1 votes since it's more about laying the technical groundwork rather than a feature itself.

This decision should go beyond just the vote count. It needs a thorough analysis by you and your team. So, yeah... I trust your judgment even if you doubt this approach.

I'm not sure that would work well in practice. Suppose someone wants to drop a 9-patch object or Text object on to a 3D layer powered by a third-party library like ThreeJS.

🙂

(or maybe this can be handled in a way that resembles how HTML Elements are handled when outside of an HTML Layer [AKA: logically in one layer, but rendered in a "virtual[for the lack of a better term]" layer)

That probably involves two sets of rendering code, depending on what they are rendering in, which is a pain for maintenance.

Indeed, inevitably. Therefore, the way to opt-in to make an object available in the special 3D layer could be to implement some "glue code interface" to instantiate, update, and maintain an instance in that library. I think 99% of the addon builders would be more than Ok with that.

It could be that object types that support such a special layer have to implement an onCreateRich3D(runtime, threejsScene), onUpdateRich3D(runtime, threejsScene), onDestroyRich3D(runtime, threejsScene) interface. Where `threejsScene` is provided by Construct. (oversimplification to clarify the idea.)

For the native plugins, having just Sprites, Tiled Backgrounds, and 3D Shapes supporting the special 3d layer would be sufficient.

I want to highlight that, in theory, the code for Sprites and Tiled Backgrounds is the "straightforward" (POC/non-production-grade) code I wrote [Construct Arcade Link]. Also, the author of MakeIt3D successfully wrote the code necessary for native Sprite animations to function. While clearly not production-ready code, these two examples demonstrates that such code may not be as much of a liability as it appears. (it will be some pain for maintenance, like everything... but perhaps less painful than the increasing 3d feature gap)

but now you have zones with different feature sets that aren't compatible, and you can't do something like decide to make an existing 2D layer in to a 3D one.

To some extent, this is the case with current 3D/2D layers: Once a project includes any 3D camera rotation, changing a layer from 3D to 2D will likely break the project and require some refactoring.

Still, it wouldn't be completely incompatible. Typically, the 3D layers consist of Sprites, 3D Objects, and Tiled Backgrounds. As long as these objects implement the onCreateRich3D(), etc interface, these layers could be transformed into a "special 3D layer" while maintaining their original functionality, enabling customers to switch seamlessly between standard 3D and "special 3D" layer types.

Ideally, an error message should prevent such a switch if the layer includes objects that aren't compatible with the "special 3d layer".

In my experience users do not understand such things. Instead - not unreasonably - they expect things in a commercial product that they paid for to just work.

I think this type of approach would be better explored via third-party addons, so it's not in the core product and can more easily be treated as an "beta" type technology that we don't support. So perhaps that's another direction we could explore, via exposing APIs in the addon/scripting system.

You're right. Users expect things to "just work," but I struggle to see how this is much different than the 3D/2D/HTML layer distinction users already deal with.

Sorry for the stubbornness... This is just some brainstorming on creating a foundation for addon developers to converge when choosing to leverage a third-party library. I truly believe this would help the Construct community quickly gain impressive 3D capabilities.

I wonder what all the talented addon developers would accomplish if they could just converge around a common `THREE.Scene` instance (AKA: the special layer), knowing that all they have to do is call `add()/getObjectById()` and manage their objects. (again, oversimplification)

I agree with Ashley. With HTML5 we‘re already dependent on enough 3rd party mercy (Microsoft Webview i.e.) so if they really want to do 3D then better do it properly and with their own tech with which we get the flexibility that‘s expected and makes their lives easier in the long run to support it.

EDIT: Oy, what‘s with those cheeky downvotes lol