Fastest way to retrieve text data

I have text file which I've added to my project, containing 10,000 surnames, which are seperated onto individual lines. I'm reading them into a single string using Ajax, which is very fast, but then I'm trying to find the fastest way to store and read the lines.

At the moment, the fastest I've found is to keep them in a single string (also more convenient) and when I want a single line I can get it with tokenat(mystring,number,newline). This is pretty fast, but still causes a noticeable pause if I do it several thousand times (which I need to).

I've tried parsing them into an array or as individual lines into a listbox, but both of those are much slower.

Any other options I could try?

Hey,

Nice original case :)

The question is about the method to store, yes, but mostly access the data.

The text storage you are using is fast, and you should keep it in my opinion, for low number access (1-100/sec).

If you need to pull a lot of data at once, like a thousand names in a tick, I would try to go for a database. SQL could help select the right names, and give you back a list in the same format (1 name per line) but you could just use ALL the names, and not seek the first, then the second, ... effectively speeding the algorithm a lot. (explode is a lot faster than 1000 seek ^^)

But isn't there another way to use/reuse the names ? Why do you have to pull thousands of names at a time ? Could you imagine load those 1k names over 1 to 2 seconds, displaying a loading bar during this ?

By far the fastest way is to copy them to an array then look up in to the array. Copying to the array might take a moment, but then reading the array will be instant, even with thousands of lookups.

It's a bit of a dilemma because I'm only doing it once when the game is setting up. It's a Soccer management game so it just needs to create about 5000 players then it's done. I know it's not the sort of thing that C2 is really built for but I haven't actually found anything that C2 really struggles with compared to other languages (well it's better at some things and not so good at others but can do pretty much anything).

So anyway, I load the names in, then create the players then I won't need the file again. Loading it into an array takes so long (about 5 seconds) that it's actually slower than just taking them directly from the string, and since it won't be needed again, that's probably the best method. In fairness, it's only about 2 seconds but could be more on a slower computer. Having said that, it would only happen at the beginning of a game with a 'Please wait while database is created' type message and I'm sure nobody will mind 2 or 3 seconds at that point.

While I'm here, auto-resizing for text objects would be great :)

So pre-load them in to an array, save the array as JSON, then load the JSON in to array on startup. That should be really fast.

Edit: text objects (along with everything else) are already automatically scaled depending on the 'fullscreen in browser' project property.

Ah I get it, you mean recreate the text file as a JSON in the first place and use that? Thanks, I'll try it.

What I meant about text objects is, they're not scaled to the text inside them, so you either have to give them plenty of spare room to fit any potential text, or do the 'Wait 0.1 seconds, scale to textwidth/textheight' thing, which seems a bit fiddly and inefficient to me. Also, I'm having a situation where I want to right-align some text, but if I use that method, they flicker as they're created with a standard-sized box then there's the 0.1 second pause, then the box is resized correctly so they move over to their correct position. It would be nice if the box was automatically sized to the size of the text.

Actually I'm not sure what you meant after all. I thought you meant I could get the file, save it as JSON, then save the file somewhere. Then for my main program, I could add that as a file which can then be loaded in as JSON.

If you mean preload as in, do all of this at the start of the program to be manipulated later, it still means that at some point in my game, there needs to be a big 5 second pause while the program is unresponsive (that's how long it takes to parse it into an array).