Family Issues Support Group
Family issues is a huge range that go from minor conflict to major misbehaviour and even abuse on the part of individual members of the family occur. Whether you feel you are in a dysfunctional family or you're dealing with a very specific issue, this is the place to talk about it and find others who might be going through the same thing.
http://www.dailystrength.org/c/Physical-Emotional-Abuse/support-group
My answer to you is that the signs are there.
If the father was callous enough to say something like that in front of his children in front of other people he is probably doing much much worse behind closed doors.
Your daughter defending him and pulling away from you is a good indication as well.