It would be a great setting to have in the Alexa app.
I can see an issue having two different volume controls while the device itself only has one set of volume buttons, but I think if you made the Alexa voice volume control as part of the app only and made adjustments relative to the skill volume, it would work out.
So, consider the following use case - I'm playing music at my preferred volume level, but when I ask Alexa a question, I can barely hear her. So, I open the app and turn up Alexa's voice volume to my preference. Now, let's say I want to turn down my overall volume because it's late at night. I can do that using the device buttons or voice command and the Alexa voice volume adjustment I made earlier will scale with the new volume setting, even whisper mode.