Heat a pan over medium-high heat. Add the sugar, cocoa powder, vanilla, and milk, and whisk until smooth. Bring to a boil, then remove from heat.
Let the pan cool.
Line a loaf pan with plastic wrap, or use popsicle molds. Pour mixture into pan.
Freeze for about 5 hours, until firm enough to stick popsicle sticks in. Gently slide each popsicle stick into chocolate, leaving about 2 inches (5 cm) in between each stick. Continue with the next row. There should be about 8 sticks per row, 16 popsicle sticks total.
Continue to freeze for another 4 hours, until completely frozen.
Remove the loaf pan from the freezer, and pull the plastic wrap upward releasing the chocolate. You may need to use a spatula to help release chocolate from pan.