I can't take my brother who has Asperger's anymore

I am at my wits end with my brother who has Asperger's Syndrome.  He keeps talking about my belated husband and is trash talking.  He keeps talking a million times about all the abuse he suffered from my mom, others and his ex-wife Debbie.  I am at the point in my life that I just don't want to hear it anymore.  I am losing my patience.  I am ready to yell at him now.  It is driving me crazy because afterwards I just want to cry.  I just can't take it anymore.  It is worse because i just lost my husband on Nov. 15, 2009.  I am at the point where I don't want my brother over to my place anymore.  I told him not talk about those things with me anymore.  We shall see what happens.  Any advice for me?