You’re indoors in the sense that you’re protected from the weather and the elements, and the cave could even have some kind of covering or entrance area that could be considered a door or doorway. People have built homes in caves.
Is caving an outside, inside activity?
Now that’s a good question. I believe it’s still outdoors, indoors has the meaning of “comfort from your own home” rather than meaning inside a closed off area. Knitting is an indoor activity, hiking is outdoors
Lots of sports are played indoors that isn’t inside someone’s home.