It is not uncommon for your penis to shrink slightly as you age due to declines in testosterone production, but there other possible causes including obesity, prostate surgery, prostate cancer ...