Hi Sampat,
While I can’t say that I’ve had a similar experience, I can imagine what a struggle it might have been (and still may be!) for you.
I was raised in a secular home as a child. My father was openly agnostic with spiritual leanings and my mother seemed to have no interest whatsoever in spiritual ideas and questions. And yet they weren’t opposed to my going to church with friends if I had stayed the night at their house, so I got a small education in Southern Baptist ways.
I do think that growing up in a secular home freed me from any challenges that might have arisen as I got older and questioned religious dogmas or grappled with my own spiritual beliefs. I was really blessed in this lifetime to be able to pursue my yogic path without any doubts or difficulties.