Is it enough to just not do harm? We should also do good. If I let a person next to me starve to death by not interacting with him at all, I haven't actively harmed him. But that's not enough. We have to not just not harm, but also do good. "Sins of omission" they used to call them, back when people still believed in "sin".