Lay the bread slices on baking sheets in a single layer and bake in a 300°F (149°C) oven for 10 minutes.
Once the bread is removed, heat the oven to 350°F (177°C) degrees.
Generously butter a 9- by 13-inch baking dish (or any 3.5–4-quart) or spray with nonstick spray.
Whisk brown sugar, cinnamon, nutmeg, and salt in a bowl. Set aside.
Whisk the eggs, milk, and vanilla extract in a medium bowl, preferably with a spout. Set aside.
Sprinkle 3 tablespoons of brown sugar mixture evenly over the bottom of the prepared dish. Place 6 bread slices (use bread heels here) in an even layer at the bottom of the dish and sprinkle with 3 tablespoons of sugar mixture.
Place 6 bread slices in a single layer over the first layer. Sprinkle with 3 tablespoons sugar mixture. Place the remaining 6 bread slices over the previous layer.
Pour the milk mixture over the bread and press lightly to submerge.
Brush the top layer of brioche bread with the remaining 4 tablespoons of melted butter. Sprinkle it with the remaining heaping 3 tablespoons sugar mixture. Bake until the casserole is slightly puffed and golden brown for about 45 minutes or until the center appears set and is no longer runny. Transfer the casserole to a wire rack, and let cool for 15 minutes. Casserole deflates slightly as it cools.
Sprinkle with confectioners’ sugar and serve.