Aha. The "mystery shrub" is just privet


I feel sort of silly not having identified it but I'm used to privet being very small, having very shiny leaves, and I thought it had spines.

Oh well. At least not I don't feel bad about cutting the ones that have invaded my backyard. They're not native and there's no shortage of them.