Construct 2 and custom files

In response to the conversation on twitter between Ashley and konjak - I thought I would add my thoughts and we could discuss it on the forum.

This is been one of my main concerns about C2 using HTML5. I like to make custom files for things like levels since I like to do some unconventional stuff sometimes. With CC it was easy, there were actions for saving and loading arrays and hash tables. C2, not so much because of the web format - security limitations obviously don't allow that sort of thing. That presents a frustrating challenge for developers.

Let's assume I'm trying to make a level editor. I could save the instance data (position, animframe, etc) to local storage as a JSON string, but there are several problems with that.

1, local storage only works up to 5MB of data. I've heard chrome stores its data in a different way, basically resulting in only having 2.5MB. So what happens when the level data is above that amount? The browser throws an error to the console. That's it. The only way to tell if this happens is to have the console open and watch it when you save. So you could be working, forget to check, and lose your edits. It would be nice if there were 'save successful' and 'save unsuccessful' conditions so we could display an error message in game.

If there was a way to raise a browser's local storage limit, I would set it to a gigabyte and not have to worry about it at all, and it would solve most of the problems, except for...

2, the data in local storage is stored in a very insecure way. What if there are multiple users of that machine, and one of them decides to clear the local storage? Bam, all the levels are erased.

What I'm doing in response to the two problems mentioned above is using the file saver plug in to save the JSON string to disk as a file to make sure it gets saved correctly. The problem with this is I get a gazillion copies of the file that I have to keep cleaning up because it won't overwrite the previous file. That's a mild annoyance, though.

Also, I tried using the array's new save feature, but chrome put up a 'popup blocked' message. Even allowing popup windows, though, the right cliking and selecting save as requires a whole lot more clicks each and every time I want to save. You might want to check out how the file saver plug in does it - one action and it saves.

3, then the're the problems resulting from not having enough space if you need more than 2.5MB. Mainly, the loading problem. Once I've got the data out of the program and on to disk, I can't get it back into the program at runtime without using the file loader plug in, which requires clicking a button, selecting the file from the window that appears, then clicking OK - every single time I preview. Very annoying. That also obviously won't work when the user is actually playing the game. Even if local storage allowed more data, that data can't be transferred when exporting.

So then before exporting, I have to put the data directly into C2. To do that, I have to save to disk, open it in a word processor, replace all the quotes with double quotes because construct won't accept it as is if I paste it as an expression, and then copy it and paste it back into C2.

To get around having to do all that I've been trying to come up with some sort of method to save SOME of the level data in local storage - the stuff that's been currently worked on - and only needing to load from disk when switching from one major area to another - but then I need to do that even when playtesting the game and not editing it, and I still need to redo the process in the paragraph above every time I want to update the exported levels. I don't really have an elegant solution for this.

The process, honestly, is just frustrating. It was so simple in CC, one action to save and one action to load. If it wasn't for the local storage limits, it wouldn't be as much of a problem, at least when editing.

I realize that there is a lot about this that you can't do anything about, but having conditions to check if saving to local storage was successful or unsuccessful would be at least some help to know if I've gone over the 2.5 MB limit, and maybe an expression to know the amount stored/available, because on a one-person machine, everything before that point is fine for editing. I don't know what could be done to simplify the process for incorporating the data back into C2 though.

I know it's frustrating, it's a lot to do with browsers obviously having security restrictions. However, it can be made better, and I hope to improve it soon - and there's a Filesystem API in the works for HTML5 which should solve this completely. (If I understand right Filesystem basically gives the browser access to a virtual folder on your computer, so you can read/write files to disk as you please with a much higher storage limit. I haven't had much time to check it out though.) However support is patchy, especially with writing files, which right now appears to only be supported in Chrome.

Obviously in CC the work flow is nice and simple: load a file direct from disk, and save direct back to disk.

From Construct 2, there's the added problem of the preview server. This is another thing I want to improve, but currently it only serves a list of files known in advance to be part of the Construct 2 engine. So unfortunately currently there's no way to add an external file to your project, and AJAX it from the preview server. This is on my todo list and should make it a little bit better.

In C2 the ultimate process I imagine is:

1. You create a level editor, storing to (for example) the Array object.

2. You design a few levels, and download each of them to disk using the 'Download' action. (Hopefully this can be made to work smoother, with just one click - the file saver plugin looked like it used an interesting library which might help.)

3. You add the downloaded files to your project. They appear in the Project Bar under a 'Files' folder.

4. These files are exported with your project, and are also served up by the preview server.

5. This means you can test your project in preview using the AJAX object. For example, AJAX request "level3.json", and when it completes, load it in to the Array object then load the level.

6. It also means on export all your levels are stored as separate files. Bonus: when published, the game does not need to download all levels before it starts running. You can dynamically load levels with AJAX during the game.

7. If smartly designed, you could add new levels to your game by simply uploading more .json level files.

Obviously this is not all possible yet, but it's the goal I'm aiming for. I think it's a good strategy because it circumvents the browser storage completely - you just download and upload files as ordinary files on disk. That sidesteps all quota limitations and avoids patchy browser support. It has the added benefit it's highly visible which files you're dealing with. If you save a bunch of files to the browser in preview with WebStorage or some File Writer, how do you get them back? Unless you write special code to download them, it's not easy to know where the browser has put that data. If you download it to disk though, you always know where everything is. It gets stored with your project, so it will be copied, backed up etc. along with the project, whereas data saved in the browser is easily lost if you switch browser or change computer.

It's kind of low-tech compared to using the fancy new APIs, but I think it will work better in practice.

So what do you think of that? Do you think it will solve everything or is it missing something still?

That would make it smoother and use less RAM, but it still has the problem that it would require reimporting the level files every time they're updated.

As far as I understand it, C2 couldn't just automatically grab the file from a path when exporting because browsers don't allow overwriting files unless it's through a file selection window - which could be used instead of the one click solution, I suppose, but that's a lot more clicks every time you save... but I guess it's the best option available. Could C2 do it that way then? Have the user point to a file that is grabbed at export rather than stored in the .capx itself? That could be helpful for updating other assets as well, like music. You wouldn't have to reimport after updating the file, it would be incorporated automatically at export.

It's a shame browsers don't have a 'developer mode' of some sort to unrestrict access for a page if the user sets it to. That would make this stuff way easier...

maybe something can be added to the spec, where a file created by your app can always be updated/deleted by your app?

I think the aim sounds very good indeed.

Would it also be possible with this goal, to write to that json file when the exported project is running in browser/mobile?

Because that would also make it possible to use save-points for games as well. I dont think it would be a issue if you need to export, lets say, 3 json array files with the project that are gonna be used as three save slots in the game, since Ive seen that in a lot of Nintendo DS games.

Arima - if you use a folder-based project, I imagine you could just right click-save as in the browser, and save directly over the file in your project. That avoids any reimporting.

so php is really the best way for the moment to write and read files,

you can write new files and overwrite existing ones, without dialogs,

what i would want to do is when you execute, it instantly loads up te last file you worked on, so you adjust something in construct then execute and you continue working on the level, saving would overwrite current level by shortcut, and loading different level could be textbox + enterkey

Construct 2 games are essentially websites. Server-side scripting is probably going to remain the most stable and secure way to save and load data for a long time.

so php is really the best way for the moment to write and read files,

you can write new files and overwrite existing ones, without dialogs,

what i would want to do is when you execute, it instantly loads up te last file you worked on, so you adjust something in construct then execute and you continue working on the level, saving would overwrite current level by shortcut, and loading different level could be textbox + enterkey

Most people don't have server-side scripting though, or don't know how to set it up. So downloading files is still a better option IMO.