The thing about a barge pole is that it is very long. It might be felt that something suspicious floating in the water could be safely touched with a very long thing, but some things shouldn't even be touched with a barge pole. Possibly a naval expression - the large rowing boats used to ferry officers out to their ships were called barges, and would have been equipped with barge poles to fend them off from crashing into ship or dock sides. Barges were also used for river transport, but the craft used on canals were known as narrow boats, not barges. So many naval expressions came into general use after the Napoleonic war, when huge numbers of men were released from the navy.

Presumably before the introduction of the barge pole, people had to say 'I wouldn't touch them with a very long thing' instead.