Your 'not getting it' is well expressed. Calling the referendum and declaring article 50 were passed by majority votes in the House of Commons. They were Parliamentary decisions. The fact that Labour did not5 out vote those decisions does not make them Tory. They are parliamentary.