Any ideas on how to implement this kind of thing?
Unfortunately Alexa does not support Jarvis voice out of the box. Supported voices can be found here.
In your case, the only solution, would be to generate audio files for all responses using Jarvis from polly and then use those audio files in the response, even though this would be quite time consuming.
All I can suggest for now is to create a feature request in amazon alexa uservoice website to support more polly voices.
If you mean to have Jarvis's voice be the voice of your skill, you could look at something along these lines
https://aws.amazon.com/polly/ along with other services that do the same thing. You basically send your ssml string to the service, and you are given back the audio stream that you send to the skill.
The nice thing about this approach is your "voice" can be used on other platforms as well, keeping your brand in sync.