How do I store a large (250K+ keys) dictionary so I don't need to load it from a file every runtime?

I'm making a Bookworm Adventure clone and need to store a large English language dictionary. I'm currently reading in a JSON file using AJAX and loading a dictionary object at startup. This seems costly to do every time the game loads. Is there a way to store a dictionary so I can load it once and freeze it across sessions?

I'll look into leveraging local storage for this. Is this the best solution? Would I be better using XML and searching the XML to validate at runtime? The JSON file that I prepared to import via AJAX is around 18MB. I know most modern browsers will support at least that for local storage so I'm more worried about runtime performance.

This game will not be able to reach the web as it will be built into a standalone arcade cabinet w/o internet access. I could host a local web server via Apache and fetch data from there if it will improve performance.

How you load the file doesn't affect the performance of accessing it once loaded. Where you load it from probably doesn't affect the load performance (it still has to parse the same data). And to be honest if it's 18mb that's not really that big - modern CPUs are so fast they'll probably make light work of that!

In short, don't worry about performance until it's actually a measurable problem.

Thanks for all of the help so far. I've successfully loaded my JSON file into local storage, though I noticed if I use my complete JSON file (250K+ records) Chrome chokes. I'll need to split the files and take multiple requests.

Now that I have my keys loaded into local storage, I'm attempting to compare if a key exists against a string. I'm using the following:

Local storage > compare key

= Equal To

String: curWord

curWord exists and I can see it in the debugger. But this even doesn't fire, even though the key exists in local storage. I can't use a top level Key Exists event because I need this to trigger on an input, in my case a key press event.

The value inside curWord must be exactly the same as the KeyName you are trying to reach.

It must have the exact same case (cAsE is not equal to CaSe).

Consider posting your project focusing on this aspect for investigation.

I'm forcing the key to uppercase when it is loaded into local storage and also forcing my compare string to uppercase. Looks like it should be OK based on the debugger output, but it's not working. I'm sure it's something simple I'm overlooking.

I've uploaded a copy of my capx file here:

filehosting.org/file/details/779033/SpellQuest.capx

Thanks again for the assistance.

I'm certainly open to those approaches but since this is static data, is there no way to populate a dictionary before runtime? I see there is an editor for Construct3 but there must be an underlying structure of the data that could be hand edited and stored in the project, circumventing the need to read a file in the first place, no?

One huge mistake in your project is event #4 - it's executed on every tick. You should move it to sub-event under event #3.

I would definitely try splitting this database into about 1000 smaller files and forget about local storage. Simply read them with AJAX.

OK thanks. I made that quick change and it does improve performance drastically. I can easily split the files, but like I said in my previous post, why do I need to read this at runtime?

I noticed that if I open the XML for my layout I can see the instance of the dictionary:

<instance type="Dictionary" uid="41" />

Can I not append my data here manually? If I can, what's the correct XML schema to follow for dictionary items?

I don't think you properly understand: the data must be read in to memory at some point. You can shuffle around the data and store it in different places, but some time before you start reading from the dictionary, it's gotta be in memory. To get it in memory means loading it. It sounds like you're trying to find a way to avoid loading it. You can't avoid that - you can put it somewhere it may load a little faster or more conveniently, but the only way to entirely avoid the load step would be to burn the data in to your RAM sticks in the factory... which I don't think you'll be doing!

I'm not dense and I've written software applications for many years. I understand that the data must be in memory at runtime, but I don't see why I can't prepopulate a dictionary.

If I want a static dictionary in C# I don't need to read the data from a file at runtime every time to populate the dictionary. I can populate the dictionary at design time and it is then loaded into memory at runtime. This is all I'm asking to do in Construct to avoid an unnecessary file read/parse every time the application runs.

In C# that dictionary would be compiled, so I understand it's not an apples to apples comparison. Setting the keys as events is essentially what I'm looking to do, but without having to set them manually. 250,000 keys would take a long time to add events for. Is there a way to either:

1. Edit the events in a source view so I can write a script that outputs the necessary code

2. Define the dictionary by directly editing the embedded data in the Construct project file

I don't see this as any different than setting instance variables at design time. I'm setting the keys and values of a dictionary at design time, but for a large quantity of data, so I'd prefer to not do it by hand with events.

I've got multiple dictionaries working using AJAX to read JSON files and it does what I need to at this point, I just feel like this is a sledgehammer of a solution for data that only needs to be set once in the entire lifetime of the project.