I think it's best to look at the camera, just as in a performance you try to make eye contact with one person at at a time instead of gazing off into the distance. Imagine a friend sitting right behind the camera lens. It may also be a good idea to look at your partner once in a while--especially if you can arrange to look at one another--and smile!--at the same time ("when we get to the second chorus..."). When I've watched myself on video, what looks the worst is when I glance down or to the side.

I would suggest watching a few musical TV performances over the next few days, and see what the performers do that makes you most comfortable as a viewer.

Aloha,
Mark