Preheat the oven to 350°F. Line a 24-count muffin pan with 16 liners (if using a 12-count pan, you can work in batches)
Mix the graham cracker crumbs, nutmeg, melted butter, and granulated sugar together until combined, adding more butter if needed. It should be the consistency of wet sand.
Press about 1 tablespoon of the crust mixture into each liner and pack it down. Pre-bake the crusts for 4-6 minutes, then remove from the oven.
Beat the cream cheese until smooth. Gradually add egg yolks, then the sweetened condensed milk, salt, vanilla extract, and key lime juice. Mix until smooth and combined.
Disperse the filling evenly into each crust. Bake for 15 minutes or until the edges are set, but the centers slightly jiggle.
Let the mini pies cool to room temperature, then refrigerate for at least 2 hours.
When serving, decorate each mini pie with whipped cream and fresh lime zest on top.