In Beyond the Sea, there is a man standing with a woman and two young boys at Capt. Scully's funeral. Since Mrs. Scully states that only the family was present at her husband's funeral, we can assume that the man is Charles, the woman is his wife, and the boys are his sons.
Even though we never see Charles, he seems to care about his family. In Christmas Carol, Bill tells his sister that Charles sent a present. Although invisible, the youngest Scully child always seems to be present.