Neuronal communication relies on synaptic vesicles undergoing regulated exocytosis and recycling for multiple rounds of fusion. Whether all synaptic vesicles have identical protein content has been challenged, suggesting that their recycling ability may differ greatly. Botulinum neurotoxin type-A (BoNT/A) is a highly potent neurotoxin that is internalized in synaptic vesicles at motor nerve terminals and induces flaccid paralysis. Recently, BoNT/A was also shown to undergo retrograde transport, suggesting it might enter a specific pool of synaptic vesicles with a retrograde trafficking fate. Using high-resolution microscopy techniques including electron microscopy and single molecule imaging, we found that the BoNT/A binding domain is internalized within a subset of vesicles that only partially co-localize with cholera toxin B-subunit and have markedly reduced VAMP2 immunoreactivity. Synaptic vesicles loaded with pHrodo-BoNT/A-Hc exhibited a significantly reduced ability to fuse with the plasma membrane in mouse hippocampal nerve terminals when compared with pHrodo-dextran-containing synaptic vesicles and pHrodo-labeled anti-GFP nanobodies bound to VAMP2-pHluorin or vGlut-pHluorin. Similar results were also obtained at the amphibian neuromuscular junction. These results reveal that BoNT/A is internalized in a subpopulation of synaptic vesicles that are not destined to recycle, highlighting the existence of significant molecular and functional heterogeneity between synaptic vesicles.