Preheat oven to 325 degrees F. Prepare an 8x4-inch loaf pan by spraying it with nonstick spray.
In a medium bowl, whisk together flour, cinnamon, nutmeg, cloves, baking soda, baking powder, and salt.
To the bowl of your stand mixer, fitted with the paddle attachment, add butter and sugar. Beat together until pale and combined, about 3 minutes.
Beat in egg until just combined. Add pumpkin and vanilla extract. Beat again.
Pour flour mixture into the wet batter. Beat until just combined and there is no dry flour remaining.
Scoop batter into prepared pan. Place pan in oven and bake for 65 to 75 minutes, or until a toothpick inserted in the middle comes out clean with just a few crumbs.
Allow bread to cool for 10 to 15 minutes in the pan before removing it and placing it on a wire cooling rack to cool completely.
As the bread cools, prepare the frosting: Add cream cheese, butter, and vanilla to the bowl of your stand mixer. Beat until smooth and combined, about 1 minute. Add powdered sugar, milk, and salt. Beat again until light and creamy.
Once bread has cooled, spread cream cheese frosting over top. Then, slice and serve.