Preheat oven and prepare baking sheet. Preheat your oven to 350 degrees F and line baking sheets with parchment paper. Set aside.
Mix dry ingredients. In a medium-size bowl, add cocoa, baking soda and salt. Mix well to combine.
Cream butter and sugar. In the bowl of a standing mixer, add butter and sugar. Cream together until light and creamy.
Combine dry and wet ingredients. To the same bowl, add egg, milk and vanilla as well as the dry ingredient mixture. Mix well to combine, but do not overmix the batter!
Drop batter. Using an ice cream scoop, drop batter about 2" apart on the parchment-lined baking sheets.
Bake. Transfer baking sheets to the oven and bake for 10 to 12 minutes (or until they spread and begin to rise but are still soft).
Press in marshmallow. Remove the cookies from the oven and press a marshmallow half into the top of each.
Return to oven. Place the cookies back in the oven for 2 more minutes to gently set the marshmallow into the cookie.
Let cool. Transfer baking sheets to a cooling rack while you make the frosting.
Make the chocolate frosting. In a mixing bowl, whisk together the confectioner's sugar, butter, cocoa, milk and vanilla until smooth.
Frost cookies. Once the cookies have cooled, spread about 1 tablespoon of frosting over each cookie to hide the marshmallow surprise, enjoy!