I was thinking pretty much the same thing as — Like what was done in Oot. With younger Link, the world took on the more colorful and childlike aspects of Zelda lore. Once older Link came into the picture, the world got a little darker, the stakes a little higher, the edges a little sharper, and danger was at last apparent.
There's a number of terms for this: to reference TV Tropes, it's somewhere between 'Growing the Beard', 'Shoo Out the Clowns', and the 'Cerebus Syndrome'. Also, and perhaps most accurately, there's the classic concept of the bildungsroman; or, the quest of the child to becoming the adult. Anime does this a lot -- Evangelion, Full Metal Alchemist, etc, ad infinitum...<img src="smileys/smiley2.gif" border="0" align="middle" />
Ultimately, you have a concept worth exploring, but in the end, the best thing you can do is go with your gut. If you don't like the direction you are taking things, it'll be hard to get your fans to go for it.