The real limit won't be the compression limit of the brick, but that any openings for doors and windows will tend to create sideways forces that could eventually put part of the structure into tension. When that happens, bricks and mortar are bad news.

There are some brick churches of cathedral size in central Europe.