Is there a method to access engine internals from JS blocks?

People provided you some solution to fix the main issue that you're raising and that we're trying to understand (basically protecting people from using risky 3rd party addons).

An other idea to solve this :

• The addon developer check a simple bool if the addon is considered as "risky".
• A pop-up warning saying "If you use [insert risky addons names] in your project, it could break your project in a future update and Scirra won't help you to fix the issues related to your 3rd party addons" with a link to a new documentation page explaining everything you explaind here. ( you could list a bunch of potential issues here such as the Android publishing requirements).
• The warning popup could have a button "make a backup" to let the user save an alternative c3p of his project before opening this.
• You could pop up that warning each time a project with a risky addon is updated to a newer version, and suggesting to open the last version the project was saved instead (so people could stick to this version)

This kind of thing would do the trick, there are actually easy solutions to the issue you keep mentionning. It just requires a bit of good will.

However what you're about to do WILL absolutely create a disaster. It could potentially break all the most advanced projects currently developped using C3 + thousands of more simple projects using 5 years of community work

It would be throwing off those 5-6 years of collective work, for litteraly no rewards for the community who put so much effort in it.

As C3 users and a community we know the long dark times Construct went through when it hurted its own powerusers / 3rd party devs with some decisions. At least, the last times there were clear benefits for us in the balance : Construct 2 => Construct 3, worker mode support, but those were still painful trade-offs.

How should we expect you to create a better Addon SDK (merging it with Scripting API) when we know getting obviously missing Addon SDK and Scripting API from you, related to already existing features is such a painful, time-consuming and hopeless process ?

Here is just one of the most recent example I personnaly experienced : github.com/Scirra/Construct-bugs/issues/7735

Skymen also wrote a blog post mentionning that exact issue 2 years ago : construct.net/en/blogs/skymen-13/flexbox-weird-characters-1590

(If anyone wants to know how bad an obfuscated Runtime would be, the part of the blogpost about the already obfuscated Editor API is insightful btw)

Also if 3rd party addon dev, relied on Scripting Interfaces, it would be even worse than the current limited Addon SDK in some ways because it would mean that we'll have to develop plugins and behavior TOTALLY differently than the way Scirra is developing vanilla addons.

Anyway C3 is moving in the opposite direction of Game Engine history here with the rise of Open Tech. (by that i mean having an Open Philosophy not necessarily Open Source)

A bunch of game engines/frameworks targeting web are growing pretty fast with a similar Open philosophy such as Defold or Phaser.

Game-maker very recently OPEN-SOURCED their runtime.

Godot has become the biggest Engine in term of itch releases, social media interaction, exponential growth of commercial releases and is relying on a Open philosophy letting devs customize their experience both at Edit and at Runtime. They probably have the most capable 2D engine and each Godot user has a guarantee to keep the full ownership of their own work indefinitely, no worries about unipersonal decision that could destroy years of work (+100% free, 100% open source). They are also planning to enhance their web exports a lot by providing options to reduce the build size and manage load times efficiently.

The 3rd party dev communities are big and prolific in all those engines, this is probably one of their biggest and most resilient strength.

From a gamedev perspective, C3 advantages over its competitor are already narrowing, and this single decision would makes things worse really fast.

Welding down the engine cover so nobody can get to it is however a step too far in the other direction.

Obfuscating is equivalent to welding down the cover... It's actively trying to avoid anyone even looking into the engine.

The worst case scenarios given seem rather exaggerated. When looking at the addon download counts, 3rd party plugins aren't used THAT broadly. That 3rd party addons break at some point is to be expected, using documented or undocumented features, and i'm sure the majority of users understands that from the get go. While teachers can teach their students the principle very easily. But this issue of users misunderstanding what a 3rd party addon truly is and how it's supported could be further mitigated (actively and passively) through multiple ways by Scirra (as suggested already).

So even if you've been burned before or learned your lesson, why go into the extreme?!

I can't wrap my head around how possibly breaking 3rd party addons or projects every once in a while (which can be fixed by the community/users) can be compared to such an extreme counter-measurement. Which obviously does much more damage to the product than breaking all 3rd party addons altogether could ever do.

We're all aware of the warning and its consequences, and everyone is entirely happy to comply with that as much as possible. If there were feasible alternative workarounds, we would have done it in the first place, it's not like we've been given a choice.

Personally i would love to see Construct deprecating things (and breaking old projects) much more in exchange for greater enhancements. Or at least replacing certain features in a similar fashion to what you've done with the new functions. But i understand why you're trying to avoid just that.

Either way, appreciate the open conversation here.

Let's hope for the best.

While we are arguing for days why we don't want Construct to be even further black boxed, the creator and technical lead of Godot just started a Youtube series where he explain in details the Engine internals.

He posted the second episode a few hours ago.

Anyway a few questions to understand :

I am talking about encapsulation, not obfuscation.

So you guarantee the runtime will never be obfuscated like the Editor SDK ? Just better encapsulation but everything will still be readable for us ?

(Would still make us dependant on the feature request for everything but at the very least we could read the code to get inspired for some stuff)

We will likely start moving to a better designed SDK in the future, and we will try over the coming months and years (as this will be a very long term project) to make good faith efforts to mitigate

This is probably what you meant, but isn't the best way to deal with it is to FIRST make improvements to the SDK with an active cooperation with 3rd party dev during several months/years and only then starting to progressively encapsulating ? Is it what you're planning to do ? Improvements to SDK first, encapsulation second ?

Releasing the feature requests towards better Addon-Making that are currently hibernating on the feature suggestion platform would be a good start to show your good will btw

Linking a few of them, starting from the most popular on the current platform (some of them are just repost from older suggestions platforms)

1. Add support in vertex shader to output world position (would unlocks new possibilities for 3D shaders)
2. Debugger: more granular Behavior and Plugin processing
3. Missing Editor/Runtime Hierarchy methods to create UI Systems
4. Allow behaviors to have object type/link/info properties.
5. Add scripting interfaces for Custom Actions
6. Effects SDK: down and up-sampling

Ashley, I think you have made your position much more clear in the last few responses, so thank you for taking the time to do that.

Out of respect for your position, I will do my best to shift over to only documented API.

I don't think anyone here is arguing that encapsulation is bad.

I think as is being pointed out, hacking always, eventually, inevitably, ends in disaster. I don't think anyone disagrees with that, and I don't think anyone is making tools thinking they will never have to update them at some point, maybe entirely!

I think the point is this; and it isn't incompatible with your position:

The current sdk api is too limited to realize certain types of projects. I would say it affects most projects, if you discount simple arcade games and prototypes, or learning projects.

So long as the encapsulation occurs alongside (and preferably after the efforts to expand the api), that would be great! It would be welcome!

But so long as you have to resort to internal engine hacks to achieve what official behaviors achieve, then the sdk is objectively too limited. The official behaviors, simple as some are, are demonstrative of the official sdk limitations.

I understand there are some api that need to stay internal for flexibility reasons. But I also see many that just look like oversights - missing for no reason other than lack of priority.

As you say, industry standard is too use encapsulation, but every other tool has a larger more robust api.

Just! Please don't obfusicate the internals, just to enforce "good practices"! As I see it, at best that is like the fire authority deciding to torch every building not up to code, in order to prevent the eventual fires that could happen.

The internals provide a very useful learning tool, even if not abused, I would still like that access. If I couldn't see the internals in c2, I never would have been able to make any behaviors. The same thing goes for c3. The current documentation and example list is just far too limited and the internals become a very useful learning ground. Being able to pop the lid and have a peak really helps to improve understanding and inform practices.

Thinking about UE/Unity model more...

Ashley if you do take the big step and encapsulate everything except API in the future. There may be a business/revenue possibility for Scirra.

Add a license to get an unecapsulated version of C3. This license could explicitly state that there is no additional support for licensees that work with undocumented features / API. So, it will be very explicit about where the responsibility lies. Also it will _limit_ the number of devs who actually have this type of access to ones that accept the responsibility and also show their commitment via a payment.

Now to make this have real added value vs the situation we have today where addons can already use undocumented features and make it worth the extra license payment. With the above knowledge about licensees responsibility of using undocumented features and possible feature breakage. This additional license would also provide a C3 editor with code that is _not_ obfuscated. Which would allow advanced devs and addon makers to look at doing more advanced editor customization to improve their workflows.

So, you would then have a limited subset of devs with the resources to both do / hire out for more involved customization than the SDK allows, ability to pay for the extra license and explicit knowledge from the license agreement about the possibility that they may need to freeze their C3 editor and runtime version due to the customization they have done.

While we are arguing for days why we don't want Construct to be even further black boxed, the creator and technical lead of Godot just started a Youtube series where he explain in details the Engine internals.

Once again, repeating myself, source code is not an API. We are talking here about an extensibility system where a first-party codebase interacts with separately installed third-party modules through a defined interface with encapsulation. There's that, and then there's getting a copy of the entire codebase in a game engine, which is something entirely different.

So you guarantee the runtime will never be obfuscated like the Editor SDK ? Just better encapsulation but everything will still be readable for us ?

I haven't been talking about obfuscation here, only encapsulation. I am not sure where you got the obfuscation point from. Even so, as the warning has always said - we reserve the right to change undocumented features at any time. So the very purpose of the warning is to say "we might do that, and take the possibility seriously".

isn't the best way to deal with it is to FIRST make improvements to the SDK with an active cooperation with 3rd party dev during several months/years and only then starting to progressively encapsulating ?

No, because the current SDK is unmaintainable and risks disaster, and the longer it is used, the greater the risk.

Add a license to get an unecapsulated version of C3.

I think this thread has shown, if we say "don't do XYZ, it's unsupported and we won't help you", in the end people do it anyway and then still expect us to help them. We had a warning specifically reserving the right to change and remove all undocumented features at any time and say "tough luck" to those affected, but in the end, in practice, doing so is not feasible because of the backlash, and we are going to have to go to great lengths to avoid the consequences of doing the very thing we warned we might do - an outcome the warning was also meant to avoid. In practice saying "this is not supported" is not realistic as at the end of the day customers facing disaster end up begging us for help anyway, and we usually still try to help them because are we just going to refuse to help them finish a project they had been working on for years? So the commercial reality of running a business like ours is you end up helping everyone anyway, even if they did things you specifically warned against; the only way to prevent that happening is to enforce it with measures like encapsulation that make it as hard as possible to get to the internals and prevent anyone doing the unsupported stuff in the first place.

Some companies provide source code. Good luck to them - it sounds like a nightmare to me. Perhaps they just have enough staff to support the desperate customers who get themselves in to a huge mess, or maybe they really do just say "tough luck", but I don't know how they would get away with that in practice. If you decide you absolutely require access to source code for your game, then perhaps Construct is not the right tool for you - there are tools out there which provide that, and it's not something we intend to do for Construct. We do intend to have a public, documented, supported API with encapsulation though, which anyone can do anything they like with, and we promise to support indefinitely.

Once again, repeating myself, source code is not an API.

Yeah i know I just point out that here, you're about to dedicate your limitated ressource and work hard to lock everything for your users and to do the opposite of what your community wants.

Meanwhile, the creator of Godot, the most promising alternative for your disappointed advanced users (free forever, open-source, enabling total customization, with the most active creators and 3rd party dev community, getting updated at light speed based on actual user issues) is teaching anyone how the internals of this engine is working

I haven't been talking about obfuscation here, only encapsulation. I am not sure where you got the obfuscation point from

The obfuscation interrogation comes from several post you wrote in this thread, such as this one which is the first one that worried us.

> Question: Or is it that there is a plan to obfuscate all internals at some point?

Answer: This could well happen, and it could mean whatever internals you are accessing become permanently unavailable.

At least 5 person asked you about it after that to understand concretly what you meant by that and it's still not clear.

Does encapsulation solve any concerns we raised ? To me encapsulation means the exact same outcome we want to avoid. (and it's likely you would do both Encapsulation AND obfuscation).

Encapsulation OR obfuscation (or both of them) means we depend on you for anything and makes the engine 10x less powerful for its advanced users, it removes a bunch of control over our own work. Besides that i'm not comfortable knowing that I don't have ownership of my own work, into which I'm putting all my soul.

> Question: This is probably what you meant, but isn't the best way to deal with it is to FIRST make improvements to the SDK with an active cooperation with 3rd party dev during several months/years and only then starting to progressively encapsulating ? Is it what you're planning to do ? Improvements to SDK first, encapsulation second ?

Answer: No, because the current SDK is unmaintainable and risks disaster, and the longer it is used, the greater the risk.

It seems that promise you made a few post ago to do your best to make the transition as smooth as possible didn't last very long. You just want to remove us what makes our projects feasible without even working on a decent solution to mitigate it first ?

I think this thread has shown, if we say "don't do XYZ, it's unsupported and we won't help you", in the end people do it anyway and then still expect us to help them

We're not asking for your help, we're asking you to not depend on you to get help and to not obstruct our way to make our games.

When it comes to 3rd party dev/community initiative and how Scirra reacts to it, what should be an obvious win-win often becomes a lose-lose