Preheat your oven to 375°F and line a baking sheet with parchment paper.
In a large bowl, beat the butter, granulated sugar, light brown sugar, creamy peanut butter, vanilla, and egg until the mixture is smooth and creamy.
Stir in the all-purpose flour, baking soda, baking powder, and salt until just combined. Avoid overmixing to keep the dough tender.
Scoop tablespoon-sized balls of dough and place them about 2 inches apart on the prepared baking sheet.
Bake for about 11-13 minutes. The cookies should be puffy and light. They may look soft in the center, but will firm up as they cool.
Allow the cookies to cool on the baking sheet for a few minutes before transferring them to a wire rack to cool completely.
In a bowl, mix 6 tablespoons creamy peanut butter with the powdered sugar, vanilla extract, and milk until smooth and spreadable. Spread or pipe a generous layer onto one cookie and top with another to form a sandwich. Repeat with the remaining cookies.