Public API far too limited - impossible to do much of anything - experimental API suggestion

github.com/Scirra/Construct-feature-requests/issues/173

At the end of the day, scripting is suggested to be able to solve needs that the event system can't.

But...

There is limited api access for basically EVERYTHING.

The devs know this, but choose to ignore it for other priorities, and that makes sense in their position. But if Scirra has to access private internals to get a behavior or plugin to work, its because the public API is far too limited. You can't recreate a huge swathe of vanilla behaviors, let alone create better versions of them with the current public api.

This isn't good if you care about creating anything unique, or have higher standards - I'm looking at you pathfinding, platformer, and 8way.

This makes Construct an engine that supports ONLY out of the box behaviors. Creating sophisticated behaviors in events is poor in performance if you keep them abstract or wish to scale, and using JS to recreate all the wheels construct won't expose is stupid when other engines do you the favor of allowing you access. Plus, to beat a dead horse, the event system is objectively one of the best things about construct, in terms of speed of design. Wading into scripting editor side essentially defeats the advantages of working with construct and, objectively, there are much better engines out there if you are coding.

The issue as I see it is that Scirra wishes to keep things internal to allow them to be able to a.) refactor at will when needed, and b.) not break peoples projects in the process, which is a great goal and I understand. But the flipside is that while construct is very stable, there is so little exposed that the promised "stability" is nearly pointless as you have to use the undocumented api to accomplish anything.

All and all, I understand I am in a minority here.

I am at a cross roads. I can keep building up my brittle collection of behaviors built on so many undocumented api calls...

Many undocumented features have remained very stable over the years, and I'm okay correcting the odd compatibility issue here and there - But what if they become inaccessible or obfuscated in the future - for no other reason than simply scirra wants it to be? Ashley has hinted at that possibility - though that would pointlessly break many 3rd party addons that have necessarily used internal features.

This isn't an issue if you are making small, uncomplicated games, or are happy with the provided boxed solutions. I am not, and I am not.

How many other people have simply ignored the official advice and built behaviors on undocumented API?

I personally don't feel like I have a choice. Either I use the unofficial api, or I don't use construct for this project... again...

And to be clear, my project isn't fancy. I just want a custom platformer that doesn't resolve collisions the same way construct does, is extendable and well written in "SOLID" fashion, and one that isn't pigeon holed into using the solid attribute, along with better oneway jump through handing, slope performance, and a different physics integration model that accurately deals with variable accelerations over time. This is all pretty basic needs stuff AND construct has everything built in to handle this... just not exposed. What is exposed through the custom behavior makes alot of hard-coded assumptions about what is needed instead of just letting the user decide.

For example: PushOutSolid, which is an undocumented function that custom movement behavior uses, has a hard coded max distance for resolves, a fractional nudge algorithm that iterates a hard coded 4 times, and only works with solids, when in fact, along with other internal functions, you can do less work and achieve sub-pixel perfect collision resolutions without limiting it to solids only or hard-coding the details.

Construct has trapped itself in a place where it is afraid to expose api because it needs to stay flexible, but insisting on remaining backwards compatible for decades means things can't advance.

---

Perhaps a new class of api should be released. A publicly documented "experimental" api. This would allow construct to sanction the use of more internal features, but with the caveat that they can change in the future, with "fair" warning.

Basically, you can use them, but they are subject to documented change.

Thoughts anyone?

There is a fundamental tradeoff between a small well-supported API which is reliable in the long term, and a large API which keeps breaking in the long term. The worst case scenario is to refer to undocumented engine internals. This can break at any time and we will not provide support if this happens due to use of undocumented internals.

Long, painful experience has shown the downside to just letting developers loose on the codebase results in unmaintainable software. This point is obvious to experienced software developers who have maintained a platform other developers use for the long term, and is the reason why decades ago the industry came up with concepts like encapsulation. It's not always obvious to people who run in to the limitations of the public API. The risk is that you end up in a situation where every single release of Construct breaks hundreds of projects and causes constant chaos with rolling waves of breakages on every release. Normal users do not care whose fault it is when things break. They blame us, blame the product, leave negative reviews, and tell other people not to use it because it sucks and keeps breaking all the time. Sometimes addon developers even blame us because they do not understand the tradeoff. Avoiding that means stopping improving the product: no more bug fixes and no more features, so eventually the product ossifies and becomes uncompetitive. This has happened to us to some degree in the past, and we've seen it appear to happen to a couple of competitors over the years, sometimes with catastrophic consequences. Once you fall in to development hell it's already too late. The best approach is to avoid that outcome happening in the first place, at all costs.

Calling a set of APIs "experimental" does not mitigate this risk. Someone might use those APIs, leave the community, then years later we change them, lots of people's projects break, and then we are forced to reverse the change, and accept we are no longer able to improve the product in the way we want to.

The cost is a small and documented public API that we promise to support indefinitely. Relative to the risk of the failure of the entire product, that is not such a high cost to pay.

We can and do increase the public API over time at the request of addon developers, but we have to think carefully about every addition - we know we are committing to still support that in 10 years time, possibly long after any developers who wrote code using it have left the community. We already do this kind of thing, as Construct 2 first came out in 2011, and we are still maintaining backwards compatibility with projects and from back then (this blog post includes a section on how adding a single seemingly obvious action led to significant work to maintain backwards compatibility, which illustrates the kind of complex work we occasionally do with the engine to avoid mass breakage). Even the C3 runtime alone is about 6 years old now, and we are still committed to supporting addons using the public API from ~2018 and will still be doing so in 2030 and beyond. When you've felt the pain of decisions made a decade ago, it gives you a new and deeper perspective on how to best manage an SDK.