How do I download and play an MP3 generated via an API request?

Not favoritedFavorited Favorited 0 favourites
  • 15 posts
From the Asset Store
Forget about default textbox restrictions, you can create sprites atop of the textbox
  • Hi,

    I’m trying to use Eleven Labs text-to-speech API to generate and then play MP3s.

    I request in their default format – “output_format=mp3_44100_128” using an AJAX request.

    But I’m not sure how to play it.

    I’m assuming I need to pass it to a Binary Data object first and have tried:

    • BinaryData – set from base64 (this produces no data)
    • AJAX – Set response binary -> Destination = BinaryData (also produces no data)
    • BinaryData – set from text (this does produce data)

    (If I download this via Browser – Invoke Download for string = BinaryData.GetAllText, MIME type = “audio/mpeg), it downloads something, but it won’t play)

    I’ve then tried:

    • Audio – Add remote URL = BinaryData (type = audio/mpeg) and added a tag
    • Audio – play tag

    But this returns:

    Failed to load audio 'blob:https://preview.construct.net/<some UID>': EncodingError: Unable to decode audio data

    Any ideas? I don’t really want to use JavaScript but importantly, I need to be able to use C3’s effects – specifically the audio analyser so I can animate a sprite based on the audio’s level.

  • If an API returns a URL directly to an MP3 file, then you should just be able to put that URL directly in the 'Add remote URL' action.

  • Unfortunately they don't provide a URL - just audio to download in one of a myriad of formats! It's this API:

    elevenlabs.io/docs/api-reference/text-to-speech/convert

    I'll keep experimenting, because it IS returning something!

  • If I read the API docs right, it looks like the response is just a raw binary download of the generated audio file. In that case it would probably be best to download it to a Binary Data object, using 'Set response binary' before the request. Then get the URL from the Binary Data object for the 'Add remote URL' action in the Audio object.

  • That’s what I’ve tried – both:

    • AJAX Set Response to BinaryData
    • BinaryData Set buffer to UTF-8 text AJAX.LastData

    (Which I believe are equivalent)

    If I then download it, I can see some relevant headers when opened in a text editor, but can’t play it in e.g. VLC. And Audio –> Add remote URL = BinaryData.GetURL -> Play returns “EncodingError: Unable to decode audio data”

    I’ve been working with ChatGPT on this and it gave me some JavaScript which sends the request, successfully plays the response and sets a var to the URL of the response. When that URL is then used in Audio – Add remote URL, it DOES work.

    My ultimate aim is to use the Audio Analyser effect to animate a sprite (to sync lip animation), so this approach works, but agree that it _should_ work using native C3 events.

    This is ChatGPT's JavaScript:

    fetch("https://api.elevenlabs.io/v1/text-to-speech/fexRy0lZ3HSLmYNrlI0v?output_format=mp3_44100_128", {
     method: "POST",
     headers: {
     "xi-api-key": "<my_secret_key",
     "Content-Type": "application/json"
     },
     body: JSON.stringify({
     text: "Construct 3 Rocks.",
     model_id: "eleven_multilingual_v2"
     })
    })
    .then(response => response.blob())
    .then(blob => {
     const audioURL = URL.createObjectURL(blob);
    
     // OPTION A: Play using JS
     const audio = new Audio(audioURL);
     audio.play().then(() => console.log("Audio is playing")).catch(console.error);
    
     // OPTION B: Pass to Construct's Audio plugin
     runtime.globalVars.AudioDataURL = audioURL; // You must create this global text variable
    })
    .catch(err => {
     console.error("Failed to fetch ElevenLabs audio:", err);
    });
  • BinaryData Set buffer to UTF-8 text AJAX.LastData

    That's wrong - it will incorrectly interpret the binary data of the audio file as UTF-8 text and most likely corrupt it. You don't need to do that to load the AJAX data in to the Binary Data object. When you use the 'Set response binary' action it does it automatically when the request completes.

    The Binary Data object's GetURL expression does pretty much the same thing as your JavaScript code, so it looks like it should then work.

  • I was just trying that to see what happened because:

    Also results in "EncodingError: Unable to decode audio data"

  • Hi! First, are you trying to use a custom voice with a free account?

    Also, check the Network tab to see what the blob looks like.

  • Try with a default voice first and do this:

    In the JS function store the blob in the global var

    runtime.globalVars.AudioDataURL = URL.createObjectURL(blob);

    Then in the event sheet:

    -> AJAX: Set response binary to BinaryData

    -> AJAX: Request AudioDataURL (tag "mysound")

    + AJAX: On "mysound" completed

    -> System: Set AudioDataURL to BinaryData.GetURL

    -> Audio: Add remote URL AudioDataURL (type "audio/mpeg; codecs=mp3") as name "voice"

    -> Audio: Play "voice" not looping from Sounds at 0 dB (stereo pan 0, tag "")

  • I'm using a paid Eleven Labs account.

    The JavaScript above works fine - sending the request, playing the response in JS, passing the URL of the response to a global variable, then in the event sheet, setting the Audio plugin's "Add remote URL" to that var and playing it (so I can insert effects).

    This is everything I need, but I was trying to do it all in event sheets, but something's not working in the events in the screenshot above. If I download AJAX.LastData, I see stuff like this in the header:

    ID3 #TSSE  Lavf60.16.100

    But VLC won't play it and presumably also why I get "EncodingError: Unable to decode audio data" when trying to play it in the event sheet using the actions in the screenshot.

  • I was just trying that to see what happened because:

    Also results in "EncodingError: Unable to decode audio data"

    The image also shows incorrect logic. The Binary Data object remains empty until the AJAX request is completed. That means you must send the request and wait for 'On completed' to trigger before the Binary Data object contains the downloaded data. Only then can you use the Binary Data GetURL expression with 'Add remote URL'. In the actions pictured, you obtain the Binary Data URL before the AJAX request has completed, so you are giving the Audio object a URL pointing to empty data, which it will then fail to decode audio from.

  • I'm using a paid Eleven Labs account.

    The JavaScript above works fine - sending the request, playing the response in JS, passing the URL of the response to a global variable, then in the event sheet, setting the Audio plugin's "Add remote URL" to that var and playing it (so I can insert effects).

    This is everything I need, but I was trying to do it all in event sheets, but something's not working in the events in the screenshot above. If I download AJAX.LastData, I see stuff like this in the header:

    ID3 #TSSE  Lavf60.16.100

    But VLC won't play it and presumably also why I get "EncodingError: Unable to decode audio data" when trying to play it in the event sheet using the actions in the screenshot.

    I've just tested it, and my events example work fine.

  • Sorry - should have screenshot more - I only run those events on the click of a button which is only enabled once the AJAX request had completed. I'll try to adapt my project to use the free version of Eleven Labs so anyone can try.

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads
  • Sorry - should have screenshot more - I only run those events on the click of a button which is only enabled once the AJAX request had completed.

    It still looks wrong though - using 'Set response binary' has no effect once the AJAX request has completed. You must use that before sending the request for it to load in to the Binary Data object automatically when the request completes.

  • ...using 'Set response binary' has no effect once the AJAX request has completed. You must use that before sending the request for it to load in to the Binary Data object automatically when the request completes.

    Argh! Sorry - didn't read the manual thoroughly - it's working now that I've set that before the request. Thanks everyone - I now have a fully working native event sheet solution!

Jump to:
Active Users
There are 0 visitors browsing this topic (0 users and 0 guests)