I'm trying to make line-of-sight work with multiple enemies. This is proving to be a tremendous pain in the ass. Where to begin?
This is my goal: I need the player character to see enemies within a cone of vision at indefinite range, blocked by solid obstacles. I need the enemies to see the player character likewise.
Okay, so, originally, I set LOS to the player sprite, and quickly realized it could only handle one target at a time. I then scrapped the LOS behavior and programmed a sprite, 'Beam', to set its position and angle to the player sprite and increase its length constantly, until it collided with a solid or enemy, at which point it would set its length to the distance between the player and the object.
In this manner, the Beam object acted as an impromptu LOS that could target multiple enemies. However, it was not a cone, so I could only see enemies immediately in front of my character. Even if I made the Beam object wider, it would still limit its size to the first object it collided with, essentially decreasing its accuracy (increasing the smallest gap it could see through). I tried to procedurally generate a conic spread of these objects, which worked...but killed the framerate.
I scrapped the Beam object and tried something different. I removed the player's LOS behavior and gave the enemies LOS behavior instead. I created a 'Compass' object, a long, thin sprite that was created when an enemy established LOS with the player. Compass set its position to the player but set its angle toward the enemy. The idea was to make the enemy visible to the player when the player sprite's angle matched the Compass' angle, give or take X degrees (X being the player's cone of vision, in degrees). This caused some thoroughly crazy issues, as the Compass objects refused to sync up with their proper enemies, even when identical private variables were assigned to help them identify each other. It was a mess, and ended up butchering the framerate as badly as the previous attempt. Back to the drawing board.
I scrapped the Compass system and went back to the LOS behavior. I created a C-shaped 'Cloak' sprite and made it solid. I set its position and angle to the player sprite, with the 'open' end of the C facing forward. Now the LOS for all enemies was blocked by the Cloak except when the player was facing the enemy, so that it could be seen through the open end. This effectively established the player's cone of vision. Now I just had to find a way to allow the enemies to target the player, regardless of which way the player was facing. I created a second object for each enemy, a 'Radar' sprite, which I also gave LOS behavior. I changed the LOS setting from 'solid' to 'custom' and told it to acknowledge only walls (but not the Cloak) as obstacles. This worked...but, again, killed the framerate after running for a few minutes. I'm not sure why custom LOS settings would do this, but they did.
But I was not about to give up so easily. I decided to create a ring-shaped sprite, 'Bubble', which would likewise follow the player sprite, but sit outside of the Cloak. This should allow the Radar objects to detect it without custom settings, right? Right? WRONG. The presence of the Cloak messes with their ability to detect the Bubble, even though it's sitting outside of the Cloak. I thought it might be an issue of LOS accuracy, so I increased the accuracy to 1 pixel. This only helped marginally; whereas before, the enemies would ignore the Bubble until the player sprite turned to face them, they now track the Bubble in very abrupt, choppy movements, unless the player sprite is facing them (in which case, they track smoothly).
In desperation, I scrapped the Cloak and Bubble objects and made a gigantic triangular sprite, 'Vision', large enough to cover the entire screen (the game does not scroll, so it only had to be large enough to cover 640x480). I set Vision's position and angle to the player sprite, allowing enemies to target the player with their regular LOS behavior and allowing the player to see them when they had LOS and Vision was overlapping them. This likewise worked...and, of course, kicked the sh*t out of the framerate.
These are the three final products of this God-forsaken endeavor. I can only hope there is something fundamental I'm overlooking that will solve this problem. I tried putting the Cloak on a different layer than the Radar sprites, but this didn't seem to work. I've never worked with multiple layers extensively, so there may be something I'm missing there. That's really the only direction I can see this going successfully, but I don't know enough to say.