When I first read that poem, which was when I was back at school, I found it quite confusing. It becomes clearer when you learn Logue was a pacifist and very active in CND. So it is very likely that he felt Labour was not left wing enough: hence the 'deep in my heart/I am a Conservative' couplet.

I am sure he would be far closer to Corbyn than Johnson - but still think Corbyn is not left wing enough.