I don't think instrumentation is the real challenge here. First, let's suppose that we send back a piano score. Usually a musical starts with a piano score because that's how it's composed and most vocal rehearsals and auditions will be performed with just a piano. I suspect Hamilton may have had some prerecorded music for some auditions or rehearsals, but perhaps not if clapping, snapping, stamping, etc. were used for mostly percussions sections of the score.
Even though we've sent back a piano score, pianos were different then (let's assume we're talking about the year 1800 or so). There were probably a few pianos in the Colonies at the time, but they were likely built in Europe and shipped over at great expense, and were almost certainly not the most advanced examples. That means they would be much quieter than the modern piano and have a smaller range of notes. A lot of the low bass notes just wouldn't be playable, and the dynamic range would be very disappointing to modern ears - but we can only guess at whether Hamilton rendered much more quietly would be a flop or not for that reason.
Finally, the real challenge would likely be that much of the music would confuse musicians of the time - at least at first. After working on it for a while, some people probably hate it and consider it unplayable or unsingable, others might be excited by it's bold "new" musical concepts, and still others might just give up in frustration.
If we compare the Bach and Mozart piano works that would have been popular at the time with a Hamilton piano score, one thing we would see right aware is that by comparison, Hamilton is quite minimalist, more than Beethoven's 5th, which was a bit controversial at its debut. Even the numbers that aren't extremely sparse, such as "You'll Be Back", would likely seem dull and repetitive in comparison with other piano music of the time. Also, "You'll Be Back" is played and notated with 1/8th note (quaver) swing, which I'm not aware of being a concept 200 years ago. Either it would have to be rewritten with the swing notated (and that would sure frustrate musicians of 1800) or it would be played without swing. That said, removing the swing might make it a lot more palatable to the pianist and singers of 1800 - and pretty dry to us.
One number that I think would survive the test of (reverse) time is "Dear Theodosia", which has a piano/keyboard part that would work on almost any keyboard instrument and is similar to the popular music of the time. Also, the lyrics, if adjusted slightly to handle the changes in the language, would likely be just as poignant. One major lyrical challenge would be adjusting the idiom "you'll blow us all away".
Other numbers might get anyone willing and able to perform it into serious social or even legal trouble. I don't know what would happen to someone busting out with:
Brrrap! Brrrrap! I am Hercules Mulligan
Up in it, lovin' it, yes heard your mother said "come again!"
Lock up your daughters and horses, of course
It's hard to have intercourse over four sets of corsets.
Suspicion of demon possession wouldn't be too surprising, IMHO.