When exporting a debug APK from Construct, the app runs in a WebView using the Cordova framework to communicate with the app that contains the WebView. It's not the same as a web browser, and so some things like opening links work differently, especially when trying to open a new tab/window: as it's not a browser it doesn't have the ability to use tabs or different windows, so it just opens in the same WebView.
Construct's Browser object works around this, so if you can use the 'Open URL in new window' action, it will work correctly.
If you are using JavaScript coding, you can also make use of the Cordova InAppBrowser plugin: calling cordova.InAppBrowser.open(url, "_system")
will open the given URL in the system browser, which is what the Browser object does. (If you do this, add the Browser object to the project, as it includes the InAppBrowser plugin.)
If you are clicking links inside an iframe showing other web content, it is much more difficult, as the web content inside the iframe is not under the control of Construct or Cordova and so neither of these things will work. You will have to do some customization in Android Studio as you were already looking at, but this is beyond the scope of what I can help with.
Of course, a much easier solution is: publish a web app and skip the Android app completely. Then it always runs in a web browser and links will open normally.