If you want to grow the caper bush in your front yard, choose a sunny spot that drains well. It will actually thrive in poor or sandy conditions, and doesn't mind rocky ground.