Preheat oven to 350 F. Line a 9 inch round cake pan with parchment paper and grease with butter.
Cream eggs and brown sugar in a large bowl with an electric mixer. Set aside.
Add fennel, cardamom, and butter to a small saucepan and brown the butter: Simmer on medium heat and whisk butter on until it foams/water evaporates, at least 5 minutes.
Transfer spiced butter to a small bowl or glass measuring cup. Add oil and let cool for 5 minutes. In the meantime, mix flour, baking powder, and salt in a medium bowl.
While mixing the creamed eggs and sugar, stream in the spiced butter and oil into the batter and mix until fully combined. Add half the dry ingredients into the wet ingredients and mix with a rubber spatula. Add milk and then add the remaining half of dry ingredients and gently stir until just combined.
Sprinkle remaining 1 Tbsp brown sugar onto the bottom of your lined and greased cake pan. Arrange thinly sliced apples or any fruit on the base. Pour batter into the cake pan and bake for 25 min. After 25 minutes, place a sheet of aluminum foil loosely over the cake to prevent excessive browning. Bake for another 10-15 minutes, or until edges of the cake release from the sides of the pan.
Remove from the oven and let cool in cake pan for 10 minutes. Flip out onto a serving plate and enjoy with cream or creme fraiche.