Heat oven to 425°F. Rub potato halves with oil and 1/2 teaspoon salt; place cut-side down on a rimmed sheet. Bake 40 to 50 minutes until very tender and cut faces brown.
Meanwhile, cook bacon in a soup pot over medium 8 minutes until crisp. Transfer to a plate; discard all but 1 tablespoon fat. Cook onion in fat 7 minutes.
Add broth, pepper, and remaining salt; gently simmer. Cut half the baked potatoes into 3/4-inch skin-on chunks. Scoop and coarsely mash remaining flesh; discard empty skins.
Add mashed flesh and chunks to broth. Gently simmer 8 to 10 minutes, stirring carefully to retain some chunks.
Turn off heat; wait until bubbling stops. Gradually whisk 1 cup hot broth into buttermilk; stir tempered mixture into soup.
Stir in two-thirds of the Jack until melted. Divide among 6 bowls; top with remaining Jack, bacon, and chives. Do not boil after adding buttermilk.