But if you do that, you can't use them as morter to lay bricks.

There's an old recipe for mortar for storework, such as is used in castles and such:

Of sande, 3 partes
Of oereos well crusheed, 7 partes
wette wit the blud offe an oxenne until weel wetted and then use as you would any mortyr for stonnnes. Twil withstant the fury of ye trebuchette, ye oranger, ye minning, et ye cannonne gonne 175mm howitzerre.

This is from the Bludynez Manuscript, which is found in many public libraries throught Idaho and Maine.