I'm not surprised that some of a pastor's children may choose to follow a path of delinquency. This is mostly because I've seen enough bad churches to recognize that not all churhes proclaim true, biblical teaching. Also, not everyone who goes to church takes their walk with God seriously. Some of them may not even be Christians at all. This can even apply to a pastor.
God warns us that false teachers will appear in the later days. The entire book of Jude talks about this, as well as 2nd Peter chapter 2. Hypocrisy is rapant and easy.
My sister once told me about a church baseball team which had a few players who were mean and who cursed on a regular basis. Of course, that's not good Christian conduct, and it could be a symptom that God is not foundational to such a person's life. If that happens to be the case, then it wouldn't be surprising to find that such a person's children are similar, in nature.
Going to church doesn't make a person a Christian. Anyone can treat the affair like belonging to a simple club, and could even have no sincere relationship with Christ. It can even happen with those who speak from the pulpit. That's why God has warned us about such people in Jude, 2nd Peter chapter 2, as well as other places in his word.
I'll admit, that as a person who isn't social, I went through a period in my life when going to church was merely a way for me to give my tithes. As most Christians do, I pray and read the Bible when I'm not at church. I've also been a Christian for a very long time. So, after a while, you get to a point where church sermons may not be as informative as they were when you first decided to follow Christ. This is because you've become familiar enough with the Bible that there's probably no subject, that a preacher can preach on, that you haven't already heard about, or studied, in your private time of prayer and worship. So going to church becomes less of a place to learn about God, and if you're not social, you don't really value time with other members. So it may be possible to feel as if going isn't critical to your relationship with Christ. However, I have since learned that going is always important. At the very least, it's an easy way to identify with Christ and his other followers.
If anyone were to ever observe you from a distance, what would they make of you? Would they recognize you as one who identifies with Christ? If you're a Christian, you should never be ashamed of Christ. It makes sense to be quick to identify with him and his people, regardless of who may, or may not, be watching. We should be unambiguous and clear. We should choose to stand with Jesus, since he said that if we deny him, he will also deny us. It never makes sense to be a closet Christian.
“Those who declare publicly that they belong to me, I will do the same for them before my Father in heaven. But those who reject me publicly, I will reject before my Father in heaven."
-Matthew 10:32-33
Unfortunately, because we are all sinners, none of us will ever be perfect representations of Christ. It is, of course, the goal, of every Christian, to be a perfect representative of Christ. But unlike him, we're not perfect. For this reason, it's crucial that we turn to Jesus often, pray often, and consciously choose to be close to him. Christ has stated clearly that, without him, we can do nothing. The following portion of scripture states what Jesus says about this, specifically.
“I am the real vine, and my Father is the gardener. He breaks off every branch in me that does not bear fruit, and he prunes every branch that does bear fruit, so that it will be clean and bear more fruit. You have been made clean already by the teaching I have given you. Remain united to me, and I will remain united to you. A branch cannot bear fruit by itself; it can do so only if it remains in the vine. In the same way you cannot bear fruit unless you remain in me.
“I am the vine, and you are the branches. Those who remain in me, and I in them, will bear much fruit; for you can do nothing without me. Those who do not remain in me are thrown out like a branch and dry up; such branches are gathered up and thrown into the fire, where they are burned. If you remain in me and my words remain in you, then you will ask for anything you wish, and you shall have it. My Father's glory is shown by your bearing much fruit; and in this way you become my disciples. I love you just as the Father loves me; remain in my love. If you obey my commands, you will remain in my love, just as I have obeyed my Father's commands and remain in his love.
“I have told you this so that my joy may be in you and that your joy may be complete. My commandment is this: love one another, just as I love you. The greatest love you can have for your friends is to give your life for them. And you are my friends if you do what I command you. I do not call you servants any longer, because servants do not know what their master is doing. Instead, I call you friends, because I have told you everything I heard from my Father. You did not choose me; I chose you and appointed you to go and bear much fruit, the kind of fruit that endures. And so the Father will give you whatever you ask of him in my name. This, then, is what I command you: love one another."
-John 15:1-17