•        Highlights

•        Rabbi Shmuley Boteach, Michael Jackson friend, calls him "tortured soul"

•        Jackson was wonderful father, Boteach says, but was lost amid fame

•        In one instance, Jackson saw doctor before speaking in public, came out "woozy"

(CNN) -- Rabbi Shmuley Boteach, the Orthodox Jewish rabbi who is known for his secular outreach and TLC show "Shalom in the Home," was friends with Michael Jackson for several years.

The two worked together on the "Heal the Kids" charitable initiative and stayed in touch until the middle of the decade, when they drifted apart, Boteach has said.

He talked with CNN's Campbell Brown about his impressions of the late entertainer, including his abilities as a father and Boteach's concerns about his emotional state. The following is an edited version of that interview.

CampbellBrown: Rabbi Shmuley, welcome to you.

Rabbi Shmuley Boteach, family and relationship counselor: Nice to see you, Campbell. Nice to be with you on the set. Thank you for having me.

Brown: Thank you for being here. I want to ask, you were so concerned by what you saw of Michael Jackson's drug use that all the way back in 2004, you told CNN you thought he'd die young.

I mean, what did you see that made you feel that way?

Boteach: Well, there was no one around to stop him. ... People ... are not going to interfere with what Michael was trying to do. And what he was trying to do was curb pain.

Michael always thought that he had ailments of the body. He always had a neck that hurt, a foot that was twisted. Really, he had an affliction of his soul. He was extremely lonely, he was extremely unhappy. He felt purposeless, he felt lethargic. And the way he dealt with that pain -- and he was especially afraid of evasion, of that perhaps his best years are behind him.

And instead of reinventing himself and entering a new phase, he decided to medicate away his pain. And no human body was going to -- would be able to sustain that kind of assault. This was inevitable, it was shocking, it's tragic. But it could have easily been averted.

Brown: You -- we've heard people talk about his use of Demerol, of OxyContin.

Did you talk to him about his drug use? Did you ever tell him you were worried?

Boteach: Are you kidding me?

Here's an example. You know, we went to give a lecture together at Oxford University. One of the main remedies I had for Michael's existence was to get serious. I said very few people in the history of the world, Michael, have achieved your level of notoriety. Do you realize what you could consecrate that to?

So, we gave a lecture together at Oxford University on a children's bill of rights. And Michael brings -- Michael -- he goes into these rooms with doctors, comes out light-headed, very woozy and it was always before a performance.

Michael -- you know, we think that he wasn't afraid of crowds but as I said, I think because he gave the public a key to his own self esteem, because he substituted love for attention, he was. This was always an issue before he went in front of crowds.

And I would say to him, "This is poison. This is killing you. You need to be razor sharp, Michael." And he knew that it was bad for him. But the next night we had dinner together and he stood up during dinner and I said, "Why don't you sit down." He said, "Because I took your advice. My back hurts but I'm not medicating it."

Brown: So, that's amazing to me. That he would get high and then he would be medicated before he would perform, essentially, in front of a crowd. Was he under the influence of drugs around his kids, also?

Boteach: Well, let me make something absolutely clear: I never saw Michael before a concert. I never saw him in a concert.

I'm speaking specifically as the years went on, I think Michael lived with a profound fear of rejection. And Michael told me once -- and this is a heartbreaking conversation between us -- "Shmuley, I promise I'm not lying to you," he said. "I'm not lying to you." He said that twice. "But everything I've done in pursuing fame, in honing my craft" to quote his words, "was an effort to be loved because I never felt loved." And he used to say that to me all the time.

And you can imagine if you're trying to get love from the crowd and you're not sure how they're going to react to you because time is going on, they [call you] "wacko-jacko," -- you've become a tabloid caricature. You live in phenomenal fear. And I think that a lot of this -- the prescription drugs -- was used to address and alleviate the anxiety. And it was just tragic to watch.

And a lot of questions need to be asked about who facilitated this. Because there are three kids who are orphaned right now and we need to know why this happened.

Brown: Did he separate that part of his life from his children? Were they aware of what was going on with him?

Boteach: Look, I am not someone who whitewashes Michael's sins. Michael was not a saint. He had great virtue and there was a part of him that unfortunately grew corrupt over time.

So, I have no agenda in saying this -- Michael was an extremely responsible father. I know there was that outrageous incident of the baby dangling from that patio but that actually was so out of character because if anything, he was overprotective of his children.

So, he was a very responsible father. He did not travel anywhere without the kids. The kids adored him, he adored them. In fact, in the last years of his life, that was probably the only inspiration he still received. He lived for these children and that's why I'm so concerned about their welfare because there's no mother in their life and now there's no father either.

Brown: You talk about the people around him and that that needs to be followed up on. Who were the good guys? I mean, was his family trying to get him help? I mean, obviously you talked to him but what could have been done?

Boteach: Well, let's be honest. If we in America want to have an honest conversation about Michael Jackson -- who the good guys are.

Look, Michael brought out some of the worst qualities in all of us -- in the media, in good people.

Very few people are around that level of attention. And to be around it ... [it]made you feel special. And you could see a lot of good people who started with Michael and little by little the corruption just grew. So even people who were good guys didn't necessarily remain that way.

If you look at the media circus, we're not even mourning the death of man anymore. We're just sort of thinking about an icon. So all of us are conflicted in this if you want to be honest about it.

So, but the good guys? I tried to be one of the good guys. Being a good guy meant if you had to risk your relationship with Michael, that you had to put your relationship on the line -- you had to look him in the eye and say, "Michael, you are killing yourself," or "Michael, you have -- there's no normality in your life," or "Michael, you have lost spiritual anchor."

I mean, Campbell, Michael was a very spiritual, religious man. He was not only a Jehovah's Witness, he was a missionary. ... He used to knock on people's doors selling Watchtowers. Then suddenly he fell out with the church. So you had this mega attention and nothing to balance and nothing to correct it. Little by little he became more egotistical.

Brown: So what did he say -- when you confronted him, when you said these things to him, how did he react to you?

Boteach: Well, for a year he listened to me and used to tell me how much he loved me and cared about me and we were very close.

I mean, I cannot begin to describe the degree of friendship that existed between us. I tried to be a Rabbi to him. But after a year -- and I believed there was a lot of progress in that year. You know, Michael came with me to synagogue. He was never going to become Jewish but he needed some sort of spiritual base. He used to come for regular Sabbath dinners at our home.

But after a year he really began to see me almost as a nuisance. I would speak to him and I could see a complete difference in body posture. He would begin to cringe. He would almost curl up, evolve into an embryonic position. He was unaccustomed to hearing any kind of criticism.

And -- but then he would get his managers to sort of try to stop me and it came to a head one day in his hotel room. We went to give out books to parents of low-income families in Newark, New Jersey.

And on the way back I could see Michael was angry at me, although he never had a temper so he wouldn't show it, but he was withdrawn. So, I said, "What's wrong?" So, his manager says to me, in front of him, "Shmuley, you want to make Michael accessible and normal. Don't you understand he's famous because he's not normal. And then I understood the full tragedy of his existence. Michael was terrified that the moment he became average that the public would forget him.

And that was the end of our relationship. I knew I could not help him and I -- there was no choice but to sever the relationship.

But at that stage -- you asked who the good guys are -- you have a choice. You can either hang on as a hang-along, or you can move on. Because the orbit of a superstar is just too great to be in there partially. It's an all or nothing sum game.

Brown: Rabbi Shmuley, I mean, there have been so many rumors with regard to this story. What's the one thing that you've heard that you want to clear up about Michael Jackson? What should the public know?

Boteach: More than anything else, I want people to understand as they read all of these very unfortunate stories about Michael. And let's face it, Michael may have -- I don't know -- but may have been guilty of very serious, serious crimes.

I want people to understand that even if it were true and I have no idea if it is or it isn't, that this was a tortured, tortured soul, who from the earliest age did not know love because he felt that he had to perform to earn love. He lived in permanent insecurity. He was one of the most tortured souls I ever came across.

After all the fame and fortune there was a part of him that we almost could not reach and I would hope that the public, in judging and assessing Michael Jackson, would do so ... knowing that that child star suffered these terrible, terrible things.

That's why all you parents out there, when you're sitting with your kids and they show you their report card and it's not an A, please don't say to them immediately, you could have done better. That's what happened with Michael. And so he always had to perform and that's what ultimately killed him.

Campbell, honestly, when they announced these concerts I thought the end was near. He was in no state to do 50-odd concerts. Not a psychological state, emotional state. Michael was burned out. He was just going to get more medication to deal with his inability to live up to his former glory of self and the outcome was going to have to be tragic.

Brown: I do want to ask you, you talk about the accusations against him, criminal accusations. Did he ever talk to you about that?

Boteach: Absolutely. He said that they were absolutely not true. He promised me that he would never be alone with a child again. So I said to him, even if they're not true, Michael, after 1993, you can never be alone with a child. And he said he understood that.

I also told him, you're not the children's messiah. Like it was his 44th birthday and he invited me and my family to spend it with him at Neverland. And I could see that he was a bit depressed. I said, what's wrong. He said, I want to help kids but I can't because of all -- everything that happened.

I said, well, the best way to help kids, Michael, is not to help them directly -- you're not a messiah -- it's to inspire their parents to prioritize them. And that's when we started working together to get parents to read to their children, have regular family meals.

So he always said to me, it wasn't true, he could never harm a child, it was all lies, et cetera. I mean, you know, I knew the child who was the second accuser. And the second accusation to be honest, I never believed. I sort of knew the family, I knew the child. I was all around in the first one and he had no business getting so close to other people's children -- that's for certain. And whether he was guilty of a crime or not, he had no right at all -- whatsoever -- to share a bed with a child that wasn't his own.

But Michael crossed lines all the time. And this is part of the problem of being a superstar when no one wants to say no to you and when you simply withdraw from anyone who does say no to you.

And Campbell, I have to make a plea to the Jackson family. I met Katherine who's a very pious, very religious woman, very devout Jehovah's Witness. And I met Joe Jackson. Michael really wanted his father's approval. And he loved his father very deeply. I know that in many interviews he spoke about anger towards his father. But when he was with me he said that he lived for his father's approval and he told this to his father on the phone when I was in the car with him.

And I really think this is the time for the family to really be at the forefront of Michael's legacy and not to cross lines of saying, God forbid, his death as an opportunity for anything aside from that mourning. Because Michael wanted his father's love more than anything else. His father meant the world to him. And I think that one of the things he lived with more than anything else -- the pain he lived with -- was this constant feeling that he never quite earned his father's affection.

Brown: And just finally, for at least now, temporarily, his children are going to be with his mother. Do you think that's the right thing? Is that what he would want? That his family would be involved in taking care of these children?

Boteach: Michael adored, adored his mother. He would always call her the saint. That was the expression he used for her. He introduced me to her on two occasions. Both times she was reading the Bible when I walked in. Michael thought the world of his mother. He wanted his children to be very close to his mother.

From what I witnessed, yes, it would seem to me that that would be his wish. That he trusted his mother implicitly, he trusted her values, he trusted her character, he was inspired of her and he spoke of her with the highest devotion and love. He never uttered even a hint of criticism against his mother.

Brown: Rabbi Shmuley Boteach, so appreciate your time.

Boteach: Thank you, Campbell. And I hope that people will remember Michael in the most charitable light