Mostly to avoid misfortune caused by untimely boasting or self-congratulation (OED). Now often used more widely.
Earliest quote 17th century.
Latin: absit Omen! (far be that omen from us!)
I can see where this request can spawn reams of useless speculation.

Touching a piece of the "true cross" to receive a blessing may or may not have anything to do with the saying.

Digression- Among the Irish, when one says a person is touching wood, it means that the person is close to being encoffined.