Hi jonnytekno, I am not an expert but I used to teach children and I had an interest in learning about the effects of music/sound on children, so I have my own opinions that are in tune with your beliefs. I definitely think that aggressive voices/sounds will be detrimental to a young child, especially a baby.
I did a few quick searches on google and found a few interesting things that give weight to your argument:
There was a study that demonstrated that new born babies preferred the music they were exposed to from the womb - for up to a year after they are born. This means that babies, from an extremely early stage develop memory associated to sounds that they hear. (http://www.edu-cyberpg.com/literacy/whatresearchwomb.asp
Another study looked at whether or not loud noises/noise pollution in neonatal intensive care unit's affected babies and if the distress caused was related to hypoxemia (oxygen deficiency). The abstract I read said "The AAP Committee on Environmental Hazards3 has recommended that physicians and hospital personnel be alert to and eliminate unnecessary noise in nurseries." (http://pediatrics.aappublications.org/content/65/1/143.short
Whilst these two studies don't focus directly on aggressive voices, they do strongly suggest that sound does have a significant affect on babies. So if your child is being exposed to harsh, unsettling/violent, aural stimulus it will likely have an impact on him/her. I think it's possible that your baby could feel stressed out and anxious, which could cause physical side effects such as apnea, elevated blood pressure or restlessness. I also think that it's possible that the aggressive voices could effect you babies behavioral development - the human brain and psyche is so delicate, especially when so young. Children and babies especially are like little sponges that absorb information and retain imprints of the environment around them.
I hope this helps - maybe talk to your wife about the fact that there are studies out there which show that sound/what a baby hears does have a strong effect on them. Perhaps buy your wife some wireless TV headphones so that she can still watch her show - but this way you baby won't have to hear it. You can buy it cheaply for about $13 from Ebay.