The verb "to sit" is perhaps a confusing example to use, since it can refer to two different actions: that of taking a seat (usually clarified with the addition of "down") and that of remaining seated. semi-submersible is concentrating on the second meaning (remaining seated), where the time at which "I" assumed the seat is immaterial to the sentence; the process of remaining seated was still in progress when the other action occurred. The concept of "in progress" is why the tense is called the past progressive.

As an example of the past progressive with the first meaning of "sit", consider: "I was just sitting to dinner when the intractable vermin slipped a whoopee cushion under my descending posterior." Of course, this tense has other uses, as a previous poster indicated; for instance, "He was just sitting, staring into the distance" (more emphasis on the duration of the action than with "he just sat").

The simple past ("sat") doesn't necessarily imply that the action has completed: "the mountain sat like a sepulchre upon the landscape" (and it's probably sitting there still, if the strip miners haven't gotten to it).

While the simple past and the past progressive can often be swapped, there's usually a shift of focus, however subtle--if not an overt change in meaning. The difference is usually significant enough that we make the right choice automatically, without conscious deliberation--it's only when we attempt to describe why we chose what we did that we get confused.