In a bowl attached to a stand mixer (or with a hand mixer), cream together butter and sugar until light and creamy. Add in eggs one at a time, beating well after each addition. Add vanilla extract, lemon zest and rosemary and beat until smooth. With mixer on low speed, add in flour and mix just until mixture forms a soft dough–do not overmix.
Press dough into disk shape and wrap in plastic wrap. Refrigerate for at least 30 minutes or up to several days.
Preheat oven to 325 degrees F. Roll out chilled dough onto lightly floured surface to 1/4″ thickness. Using a cookie cutter, cut dough into desired shape and carefully transfer to a parchment-lined baking sheet. Bake cookies for 10-12 minutes or until edges are just lightly golden. Be sure to not overbake. Remove from oven and let cool on sheet for 5 minutes before transferring to wire rack to cool completely.
In small bowl, mix together confectioners’ sugar and lemon juice until combined. Add water to thin glaze one tablespoon at a time until desired consistency is reached.
Place parchment paper under cooling rack to catch icing that drips down. Using a spoon, pour glaze over cooled cookies, letting excess drip off onto parchment. Sprinkle with chopped rosemary. Let stand on rack 10-15 minutes or until glaze is set. Store in airtight container in refrigerator.