I think best thing here is to make a group out of the brick walls only, double click to enter the group, right click, choose "edit mesh" and adjust the number of subdivisions to a higher value (maybe also decrease the pixel threshold if needed).
Probably, you'll need a little tweaking before getting the correct result!