As I heard it (In Boy Scouts many decades ago). It started as 'Starboard & larboard' but due to mishearing shouted orders, larboard was re-named 'port'.
Remembering this, the left/right bit is easy. larboard was on the left, and has been renamed 'port'. (when facing foreward - toward the 'pointy end')