Heat the oven to 425°F. Rub potato halves with olive oil and 1/2 teaspoon salt and place cut-side down on a rimmed sheet. Bake for 40 to 50 minutes until very tender with browned cut faces.
Meanwhile, cook bacon in a soup pot over medium heat for 8 minutes until crisp. Transfer bacon to a plate, leaving 1 tablespoon fat in the pot and discarding excess. Cook onion in the fat for 7 minutes.
Add broth, pepper and remaining salt to the onion and bring to a gentle simmer. Cut half the baked potatoes into 3/4-inch skin-on chunks; scoop the flesh from the other half and coarsely mash it, discarding those empty skins.
Stir mashed flesh and skin-on chunks into the broth. Simmer gently for 8 to 10 minutes, stirring carefully to keep some chunks intact.
Turn off the heat and wait until bubbling stops. Whisk buttermilk in a bowl with 1 cup hot potato broth added gradually, then stir the tempered mixture into the pot.
Stir in two-thirds of the Monterey Jack until melted. Divide soup among 6 bowls and top each with remaining Jack, crisp bacon and chopped chives; do not boil after adding buttermilk.