How can we stop drifting further apart?

My husband and I have a 6 week old baby girl and I know that lack of sleep due to a new baby and getting through the holidays can be very hard for anybody. It just seems like we are drifting further and further apart. We had a huge fight about where we were spending the holidays (his parents or mine). We take turns at night watching the baby so we never get to fall asleep together. I am back to work full time now and his hours were cut back so he is right now the main caregiver and I am the breadwinner and I know this bothers him but my job pays to well for me to quit and stay at home. I don't want us to drift further apart and yet don't know what to do to keep from it. HELP!!!