The solution here is to use 2 separate brick textures, one for the small wall and one for the big wall. The easy way would be to set it up and scale it appropriately for one of the walls, then duplicate it, assign it to the other wall and scale it as needed. This ensures it is identical except for the scaling.