In a large pot or Dutch oven, heat the olive oil over medium heat. Add the diced onion, sliced carrots, and sliced celery. Sauté until the vegetables are softened, about 5 minutes.
Add the minced garlic and cook for an additional 1 minute until fragrant.
Pour in the chicken broth and bring to a boil. Add the shredded chicken, dried thyme, dried rosemary, salt, and pepper. Reduce the heat and let the mixture simmer for about 10 minutes.
While the broth is simmering, prepare the dumplings. In a mixing bowl, combine gluten-free all-purpose flour, baking powder, and salt. Add the cubed cold butter and use a pastry cutter or your fingers to work it into the flour until it resembles coarse crumbs.
In another bowl, whisk together the milk and egg. Gradually add the wet ingredients to the dry ingredients, stirring until just combined.
Once the broth has simmered, drop spoonfuls of the dumpling mixture onto the surface of the simmering broth. Cover the pot with a lid and let the dumplings cook for about 15 minutes without lifting the lid.
Check that the dumplings are cooked through by inserting a toothpick into the center of one. Remove from heat and let it sit for a few minutes before serving.