Reduce touch input lag for real-time music instrument app?

  • Im investigating using construct to develop a simple real-time instrument music app. However Ive noticed that touch input lag can vary wildly between devices from minimum estimated 40/50ms on newer higher end phones to, on some older or cheaper phones, more than maybe 200ms (making a real-time music app unfeasible for many devices).

    I still may go ahead as 40/50ms is getting into the realm of usable but is there anything I can do to reduce it further?

    For example, I am assuming "on touch object" or "on any touch start" is the best way to go rather than "on tap" (as this may be waiting for a second tap.)

    Are there any other things I should be considering. Is there any code I can add in that optimizes touch input processing etc.

    Can stuff like mentioned in this article be of any benefit?

    developers.google.com/web/updates/2013/12/300ms-tap-delay-gone-away

    any advice is appreciated cheers.

    edit. sorry I just realized Im am being a little dumb as the lag could also be on the audio processor end or other software hardware. Essentially, I am looking to reduce the time as much as possible between touch event and a play sound/ change sound action.

    in most games even a 200ms delay may not be noticeable but for a music/sound app any delay is very perceivable and longer delays like 100ms + make real-time playing almost impossible. Im told that Ios processes sound much faster than android but I have not tested any ios devices.

    anyone have experience in this area?

  • I would suggest using the following settings for now:

    - turn off Worker mode

    - set compositing mode to 'Low latency'

    - make sure Audio latency hint is 'Interactive' (it should be the default)

    I don't think there's much more you can do than that right now.

  • Appreciate that Ashley.

    I didnt even know that last two settings existed.

    I have implemented and do believe there is a slight improvement.

    Thanks!

  • Ashley

    Using desynced compositing for apks results in black-screen/load fail.

    I assume this is a bug?

    So will raise.

  • Just to follow up here, ive become a little obsessed with touch event to audio response time (which I will call lag from now on) for the last few days.

    I have found the following.

    1. If I have nothing updating on the screen there is a lot less lag
    2. On phones, APK are less laggy than visiting the website , even without the desync option (looks like the APK allows the phone work harder than the website version. there is a doubling of CPU utilization from 2% in the browser to 4% when running the APK)
    3. An android 5 lollipop build is less laggy than an android 8 Oreo build.

    I have no way to measure but basically the android 5 lollipop build (at least on my phone cubot power) is getting a reasonable enough response time to make a real time instrument viable, at least for slow medium speed playing.

    I just have two questions.

    1. On phone. Is it not possible to make the website version as snappy as the APK version?
    2. With the APK, Why is the lollipop build snappier than the oreo build? I thought it should be the opposite, and is there any drawback using the old lolipop build?
  • Another thing I found that drastically reduces Touch to sound generation lag.

    It seems that , by default, android, on the phones Ive tested at least,

    puts pretty much all apps into a

    "Battery Optimized" List

    if you go to battery setting and move your app to the "Non Optimized List"

    then bang! you get some huge reduction in lag as the phone cpu is not being throtled

    especially evident when battery is below 50-60%

  • How are you going about making your real-time instrument? Are you just playing an audio file when the user taps an object? Or something more advanced?

  • NetOne Some interesting findings, I would be curious to see what the results would be like over a wider range of devices.

    As for your questions:

    1. Are you using Worker mode or DOM mode for your game? Worker mode doesn't work in an Android app, and has some latency for certain interactions that must be performed on the main thread ( audio, mouse and touch to name a few ).

    2. It's important to understand that we are actually targeting APK 28 ( Android 9 ) and the value that is set is the minimum supported device version. I'm not exactly sure what optimisations are done for this but my expectations are it affects how your app interacts with dynamically linked libraries. These are software libraries which are already on your device, so they don't have to be included in the APK. Setting a lower version will likely include additional libraries in the APK that are absent on older devices, or may use older API methods that behave differently in terms of performance. There's an outside chance that one APK ends up with more performant memory layout and allocation, but that's something which is going to be very difficult to test and tweak.

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads
  • Nepeo

    I am currently uploading an app and going to upload a proper demo in the next few days so will get a better picture soon!

    and not using worker mode as advised by Ashley. also it looks like chrome devs have fixed the desycro issue (https://github.com/Scirra/Construct-3-bugs/issues/3369) with APKs for a future release but from reading the notes Im still not sure that that means that desyncro option will be available in APKS in future?

    The final APK I made I went with Android 8 as I felt that this would block out a lot phones that would be older less powerful so more laggy.

    Fib nothing sophisticated at all, just bog standard sample player, im playing a set of samples over a range of frequencies in this case its not when button is pressed (like a piano) but as Im trying to emulate a flute / whistle its doing a little calculation to figure out which is the highest not pressed button (i.e. open hole) then playing the sample at frequency accordingly. Then just added reverb and an LP filter to take off any harsh high ends.

    Cant really do anything sophisticated off the bat as we dont have access to wave creation part of the audio API in construct or the modular routing of tracks/ effects which is 1) why I am a little excited that it may be possible using scripting and 2) I will raise a suggestion for them to be added in as I think music instruments / synthesizers / sequencers etc etc could be a cool avenue for Construct.

  • NetOne

    "Battery Optimized" List

    if you go to battery setting and move your app to the "Non Optimized List"

    then bang! you get some huge reduction in lag as the phone cpu is not being throtled

    especially evident when battery is below 50-60%

    I believe you can solve this with the https://www.npmjs.com/package/cordova-plugin-powermanagement.

    I used that with my Mobile Sleep addon to overcome that problem, though I think you can still use that using the Scripting Feature.

    That is because of a limitation of Cordova applications not being detected by device manufacturers as games, so they assume Cordova apps as office or general applications, hence the slowdown. Especially when dimmed or low battery.

Jump to:
Active Users
There are 1 visitors browsing this topic (0 users and 1 guests)