Yes it is only from penile tissue. From the epithelial and stromal layers.
ncbi.nlm.nih.gov/m/pubmed/24959691/
So it is possible that increased AR expression leads to hypersensitivity of the receptor to low levels of androgens. When we take finasteride we likely increase AR expression. When we stop finasteride our bodies ramp up production of DHT and so you have lots of androgens binding to a hypersensitive receptor. It is possible that a post translational modification takes place after the receptor shutting down AR signalling and the activation of androgen responsive genes. This idea would explain why people have only a minimal response to androgens and even perhaps a negative response.
Normally yes increased androgen receptor would lead to more androgen activity. But we modulated things when we took the drug.
I also want to add forskolin and Vitamin E both affect AR too. Forskolin affects ligand binding though so affects the binding of testosterone to receptor. This is likely not a good idea.
ncbi.nlm.nih.gov/pubmed/12032296
So in summary these things affect the receptor.
Selenium,
DIM,
Milk thistle
Vitamin A
Resveratrol
Vitamin E
Forskolin (probably avoid this one)