Addon SDK v2

Post reply
1 favourites
From the Asset Store
DATA+ (Addon Pack)
$22.50 USD
Data+ is the best Data Management solution for Construct 3. It contains 4 Addons (Plugin & Behavior).
Overboy's avatar
Overboy

  • In Construct 3 r391, we're introducing a new addon SDK, which changes the way plugin and behavior runtime scripts are written. This post outlines why we need to do this and how the roll-out of the v2 SDK and phase-out of the v1 SDK will work.

    Unmaintainability of existing SDK

    Construct's original addon SDK is not sustainable. Plugins and behaviors can access internal details of the runtime, mainly because the JavaScript programming language has historically had weak encapsulation features. Despite our SDK documentation warning not to do this, a number of addons have been published that do this anyway. While accessing internal details might seem like a good option to get things done in the short term, in the long term it causes disaster, ending up ruining customer's projects as addons permanently break. This is because in the long run, all internal details of the engine eventually are changed, replaced or upgraded, and this usually breaks addons that access internal details; the changes may then be impossible to reimplement as the internal details have fundamentally changed, or it may simply be that years later the original developer is no longer around and so won't update it. In the long run this harm can outweigh any good the addon did in the first place. This problem is already occuring, with the latest Construct update breaking a few addons, it has already been a severe problem in the past, and it will likely only get worse with time.

    Introducing SDK v2

    We aim to solve this problem and ensure customer's projects can continue working even in the long term without maintenance by the addon developer. We're doing this by introducing a new and redesigned addon SDK v2. This uses encapsulation to ensure addons only use a long-term supported API, which is the industry-standard approach, including with most other game development tools. Ultimately we will retire the addon SDK v1, but phasing it out will be a long process. This is to allow addon developers time to migrate, as well as co-operating with addon developers to ensure the v2 SDK covers all features where reasonable and feasible. However developers should be aware some internal features they were using may not have an API created for them, as we do not want those features to be accessed by outside developers, as it is not maintainable to do so; in this case, unfortunately the addon developer must carry the responsibility for ignoring the warning in our SDK documentation, and they will have to retire their addon and customers will have to move to alternative features or workarounds. We will do our best to avoid such cases wherever possible, but the warning in the SDK documentation was there for a reason, and these addons were likely to end up permanently broken in the long run anyway.

    SDK v2 design

    Construct already has a broad, long-term supported JavaScript API in its JavaScript coding feature. This was introduced in 2019 and was specially designed with modern encapsulation features to ensure it is robust in the long term and avoid the same maintenance problems that plague the addon SDK v1. Rather than creating a separate API, the addon SDK v2 is implemented in terms of the existing JavaScript APIs, unifying both the addon SDK and the scripting feature, so code is written the same way for both.

    This approach has several key benefits, including:

    • Adding a new API makes it available to both the scripting feature and addon SDK at the same time, making it quicker to improve both. This also avoids the current problem of the documented SDK methods lagging behind the script API features (such as with access to collision APIs).
    • In the long run, new API improvements for the SDK will make the scripting feature more powerful, and vice versa. There have already been some API additions for the v2 SDK that benefit the scripting feature.
    • The API only needs to be documented once, rather than having two sets of API documentation for runtime methods.
    • As addons are implemented directly on scripting classes, there is no need for a separate script interface class - the addon class is the script interface.
    • It should allow for writing addons with TypeScript, similar to how TypeScript can be used to write code in projects.
    • People already familiar with Construct's JavaScript coding feature will find it much easier to develop addons, and vice versa.

    Note that this only affects runtime scripts for plugins and behaviors. Effects and themes are unmodified in the v2 SDK. Editor scripts and JSON files for plugins and behaviors have only very minor changes, and DOM scripts and wrapper extensions do not need any modification at all.

    Getting started with the v2 SDK

    The following resources are available to help addon developers write new addons using the Addon SDK v2, or port existing addons to the Addon SDK v2.

    SDK samples

    All the sample plugins and behaviors in the addon SDK have been ported to the v2 SDK. There is also a new 'basic bullet' sample behavior with the v2 SDK. You can find these on the Construct Addon SDK GitHub repository, under behavior-sdk/v2 and plugin-sdk/v2. You can also see in the commit list a set of commits to update each sample plugin and behavior to the addon SDK v2, such as this commit for updating the singleGlobalPlugin sample. These show diffs that highlight exactly what needs to be changed to update an addon.

    Documentation reference

    The Addon SDK documentation now covers the Addon SDK v2. A full reference on the APIs specific to the Addon SDK v2 is now available in the Addon SDK interfaces section of Construct's scripting reference.

    Porting guide

    The Addon SDK documentation now includes a guide on Porting to the Addon SDK v2 to help addon developers upgrade existing addons to SDK v2.

    Roll-out timeline

    Replacing an SDK is a complex project, and also requires addon developers update all their addons, and then all customers install the updated addons. We realize this is potentially a lot of work for addon developers. Unfortunately this is a necessary part of the project, but we are providing a long period of time before the phase-out of the old SDK to allow plenty of time for this work to take place.

    The timeline is subject to change, but we are guaranteeing at least the following time periods before any action is taken - so if it changes it will only be to delay steps on this timeline.

    • Milestone 1 (we are here): beta release of the addon SDK v2 (now available in r391).
    • Milestone 2: full release of the addon SDK v2. This will be reached when the addon SDK v2 is supported in a stable release of Construct, and all documentation is complete.
    • Milestone 3: Construct will start showing compatibility warnings when opening a project that uses SDK v1 addons, prompting the user to update them, or contact the addon developer for an update. This will be at least 6 months after milestone 2.
    • Milestone 4: Construct will remove support for SDK v1 addons. The addons will no longer be loaded in the editor, and projects using them will fail to open and report that the addons are not supported and need updating. This will be at least 6 months after milestone 3 (and so at least 12 months after milestone 2).

    Therefore addon developers have, as a minimum, over a year to update their addons (one year starting from the next stable release). We will review this schedule over time and milestones may be delayed if necessary, but we are publishing this as a guide for addon developers to plan with on the assumption that this schedule will be adhered to.

    Conclusion

    Construct will ultimately drop support for the Addon SDK v1, so addon developers should consider starting updating their plugins and behaviors to use the Addon SDK v2. We realise this could be a significant amount of work for some developers, but we strongly believe this is a necessary step to provide customers confidence that their projects will continue to work in the long term. Improving customer's trust in third-party addons should also encourage adoption of third-party addons, as there will be less cause for customers to avoid them entirely for fear of compatibility problems. Finally, we believe the v2 SDK is better designed and easier to maintain, and by unifying the addon SDK and scripting features, will allow both to be improved much more quickly than otherwise. We ask for addon developer's patience and co-operation as we go through this process.

    Tagged:

  • In case anyone is interested in getting more context about this, many addon developers voiced their concerns about this change in this thread:

    Thread about why Addon SDK2 is a massive regression in what's achievable with Construct (making it less powerful and less trustworthy, for no valid reason) :

    construct.net/en/forum/construct-3/scripting-51/method-access-engine-181244

    Construct will remove support for SDK v1 addons. The addons will no longer be loaded in the editor, and projects using them will fail to open

    This post is even more worrying than what we were all thinking in this previous thread, because not only everything will be hidden/made inaccessible on purpose, making the engine 10x less powerful and making many of its limitations impossible to overcome (no matter how skilled you are at eventsheets/JS/AddonDev),

    but it looks like you also plan to break every single addon ever made for C3 (6 years of collective work making the engine way more powerful and being used in thousands of projects) in approximately 1 year, while you always promised that at least the documented features of Addon SDK V1 would be supported forever.

    Here is what you said on the thread I linked above, 2 months ago :

    The exception to all this is the documented, supported APIs. If you use those, we promise to support them indefinitely

    You can of course stick to the documented APIs and do anything you like, and we promise to support that indefinitely

    So are those promises not worth anything anymore ?

  • People already familiar with Construct's JavaScript coding feature will find it much easier to develop addons, and vice versa.

    I have my doubts about that. It would be easier to not handle 10 different files to make an addon. But i will cope with it, as always.

  • I try to empathise, I really do, but I don't see how this benefits us as C3 users.

    I always looked at addons as the "if you can't do this in events, then an addon can get around it".

    But this SDK change would be more like "if you can't do this events, and if you can't do this as an addon, then either change your vision for your game, or submit a suggestion and wait X months/years to maybe see it accepted or rejected".

    I have always had a good time dealing with addon devs, very quick to fix things on a new C3 release, and also this grows the community as there are wonderful addon devs that feel proud to serve the community.

    Even addon devs that left, like with ProUI, the community worked hard to contact them and they kindly open sourced it and now it's being maintained by the addon devs of this community.

    I have no clue who will take on porting ProUI, or if they even can. I don't use this personally but it's a great example of how hard working the addon devs are and how much they care, but it sounds like it is inevitable that their hands will be tied entirely.

    I wanted to look at skymen's FMOD plugin soon, but now I have no idea if I should risk it. Infact thinking long term, I don't know if I should be removing all addons from my project for safety. Whereas over the years, if an addon broke, it was usually fixed before I asked the addon dev.

    Was also thinking to pay addon devs for specific things, but again, I have no idea what to do anymore.

    I think the best thing that could happen would be a major change to how suggestions are handled. Then the whole "this is impossible to achieve in events or the new addon sdk" at least has a chance. But I understand Scirra's position and small team and such, I don't see this able to be changed. Suggestions are a luck based time based thing, can't save some money and pay Scirra to make a suggestion (but we could save money and pay addon devs).

    The absolute top fear, is, there's not a big addon dev community for C3, and I pray they see positivity in this sdk change and find value in it, rather than them giving up.

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads

  • So instead of a few addons that might break, you're breaking every single addon instead? Come on Scirra! This will kill my project I've been working on for 2 years. All addons I currently use work perfectly even if old, and now all of them will be useless. I don't know what to do anymore...

  • it looks like you plan to break every single addon ever made for C3

    It is unfortunate that all addons need updating. But we already went through this with moving from C2 to C3, where all addons needed rewriting, and then again with moving from the C2 runtime to the C3 runtime, where all addon runtime scripts needed rewriting again. I know this creates a lot of work and I don't like doing it, but I strongly believe it is necessary to avoid compatibility disasters, as the current approach is not sustainable.

    So are those promises not worth anything anymore ?

    All the documented APIs in the addon SDK v1 are supported in the addon SDK v2. So they are still supported, but addon developers need to move to the new SDK to continue using them.

    But this SDK change would be more like "if you can't do this events, and if you can't do this as an addon, then either change your vision for your game, or submit a suggestion and wait X months/years to maybe see it accepted or rejected".

    This is how it works in every other engine, because you generally can't access the internal engine in any other software. It was only really possible in Construct because of JavaScript's weak encapsulation. In other engines, people use workarounds or other approaches. You should do that in Construct too - in fact that's what people should have done in the first place, instead of resorting to accessing internal details of the engine, which causes compatibility disasters.

  • For me, it's great news that the plugin system is being renewed. The current base dates back to an evolution of Construct 2, so we're talking about technology that is around 10 years old. It's a very solid system, but it was starting to show some issues and "old" patterns. The move to TypeScript is an excellent choice, and the provided support period is commendable. I am confident that the support and documentation will be sufficient to facilitate the transition.

    I understand that there will be some resistance from the community, as I see here in the forum. However, the given timeframe seems more than adequate to adapt the plugins. I'm sure additional tools will be developed, just as they were for Construct 2, to assist with the conversion of plugins from the old to the new system.

    I have some years of experience in Construct and software development, and I know that Ashley and his team have a wealth of experience as well ;)

  • Will Scirra provide assistance to help with porting from V1 to V2, besides the very simple SDK examples?

    Is there a possibility to make a 'wrapper' around V2 SDK to support previous V1 SDK addons or at least for the vast majority of the V1 SDK?

    Can Scirra create a script or tool to translate the majority of an addon (and add comments or highlights where the translation is not possible via automatic translation).

    Ashley - please take a collaborative look at the SDK V2 requests - realize that they are not trying to prescribe your SDK V2 design, they are requests for functionality - which could be implemented in different ways. If the request does not map directly to you design, can you please consider other methods or alternative approaches to the functionality that can be used. We want to work together with you as you promised and make SDK V2 powerful and useful for the new addons which use V2.

  • I'm sure additional tools will be developed, just as they were for Construct 2, to assist with the conversion of plugins from the old to the new system.

    It sure would be nice! I wonder what ethereal entity makes these tools.

    In general, I agree with the concept that the SDK v2 will be better in the long run, but it is very dismissive to look at what I and many other devs have spent years working on being basically rendered useless and say "eh I'm sure it's gonna be fine :)"

    I do promise I will do my best to port my own addons, cooperate as much as I can with Scirra, and help write any tools that could help with porting, but also there are 282 addons that need to be ported right now, just counting the ones listed on Scirra's addon exchange page (so excluding the ones on github, on itch and the ones made in private) and last time it took us multiple years to port like 100 of them from C2 to C3.

    I used to maintain a database of what was and wasn't ported back in the day, and looking at the list there are a good amount of C2 addons that were never ported fully to C3 (with C3runtime)

    wasthataddonported.surge.sh

  • Well... not like we can stop you.

    as well as co-operating with addon developers to ensure the v2 SDK covers all features where reasonable and feasible

    I do hope this will happen. And just to get it started (and it already was mentioned in this thread)...

    It would be easier to not handle 10 different files to make an addon.

    Imo this would be my priority. Because if it's not done now it's never gonna be done. Can it be done? I dunno, but man it's really annoying to juggle all these files. 13 of them... really? I remember whenever I made addons I actively ignored like 8 of them apart from changing the addon id. Surely this can be boiled down to maybed 5 files (including language files). To me it seems too convoluted as to what's happening. Why's there two instance.js for example, what's the deal with that?

    BTW if anyone wants to port my addons feel free. No idea if I have the urge to do the porting myself tbh...

  • Looking at some of the sample v2 addons, I notice that this is removed: 

    GetScriptInterfaceClass()
	{
		return self.IMyDrawingPluginInstance;
	}

    My hope is that is because the scripting interfaces for addons will be 'automatically' generated somehow. Is that correct, or is there something else happening?

  • having it auto generated would be great

  • I'm not an addon dev so I can't speak on a technical level, but I still see something odd here...

    Ashley you've put so much work and effort into making C3 extensible, but then you completely disrespect addon devs time or concerns. Like... I do not understand who all this work you are doing is for?

    It can't be for the addon devs, because everyone that's still left is in this thread complaining about it, and not wanting this to happen.

  • My proposition is, Ashley should every month add one popular plugin (from third addon list) and make them as part of construct 3

  • This is how it works in every other engine, because you generally can't access the internal engine in any other software.

    In general, for game engine software, Unity, Unreal, Godot all have plans that allow for their internal engine source to be accessed, changed and rebuilt as needed for addons or different functionality by more experienced game developers.

Post reply
Return to board index
Jump to:
Active Users
There are 1 visitors browsing this topic (0 users and 1 guests)