Peel the oranges and lemons, keeping the peels. Juice the oranges and lemons. Combine sugar and citrus peels in the bottom of a punch bowl. Muddle together until sugar starts to clump together. Let sit for about 2 hours (while not necessary, this does add a little complexity).
Brew the tea for about 30 minutes, remove loose tea or tea bags and allow to cool. Add the juice of the peeled fruit, tea and bourbon. Stir. Top with champagne just before serving and stir gently.
Top with freshly grated nutmeg and serve.
*If you cant grate fresh nutmeg, dont use the prepackaged powdered stuff. It will turn your punch into a paste. If fresh isn't possible, garnish with citrus wheels or fresh fruit.