Start by shucking the fresh corn, ensuring to remove all the silk. Cut the kernels off the cobs and set them aside. Don't forget to keep the cobs for flavor.

In a large pot, heat olive oil and butter over medium heat. Add onion and celery. Sauté for about 2 minutes until softened.

Stir in the minced garlic, kosher salt, black pepper, thyme, and cayenne pepper. Cook for another minute.

Pour in the vegetable broth and add the reserved corn cobs. Bring the mixture to a simmer and cook for 10 to 15 minutes.

Carefully remove the corn cobs and add diced yellow potatoes. Simmer for another 10 minutes.

Add the corn kernels and milk. Stir well, reduce heat, and let it simmer for about 5 minutes.

Taste and adjust seasoning, then stir in chopped chives. Serve warm.