I was mistaken, thank heavens.

The hollering man was one of the night nurses, not a patient. The patient in question had in fact never left the premises, but did the orderly thing and went to the nurses' station. And from there, the night nurse personally went down to the service entrance to, erm, question the delivery men. Then returned to the station, and reported to the patient what the delivery men said to him, the nurse.

So it was all above board, and I let my fears run away with me.
So I have apologized to the patient for suspecting him, he has accepted my apology, and I am rightfully humbled.
And actually the truckers are going to look into adjusting the delivery time. The kitchen staff were unaware that the delivery of the foodstuffs happened THAT early in the day, and it startled them to be told as much. So maybe a compromise can be reached.