The lethality (as mentioned above) is from the current, not the voltage.

IF I recall correctly 20 millimps is lethal, but I could be misremembering.

If the 4 AAs discharge all at once into the pool, the water will presumably conduct the current along the path of least resistance, probably down the drain pipe or fill pipe. If you are floating in the pool, you probably won't feel it.

They have a combined total capacity of 6 volts, so to produce a current of .002 Amps, they would have to be all at once directed a long a path of 3,000 ohms, if my figures are right.
The body ranges between around 4K and 20K ohms.

A