Taxonomic identification of whale bones found during archaeological excavations is problematic due to their typically fragmented state. This difficulty limits understanding of both the past spatio-temporal distributions of whale populations and of possible early whaling activities. To overcome this challenge, we performed zooarchaeology by mass spectrometry on an unprecedented 719 archaeological and palaeontological specimens of probable whale bone from Atlantic European contexts, predominantly dating from ca 3500 BCE to the eighteenth century CE. The results show high numbers of Balaenidae (many probably North Atlantic right whale (Eubalaena glacialis)) and grey whale (Eschrichtius robustus) specimens, two taxa no longer present in the eastern North Atlantic. This discovery matches expectations regarding the past utilization of North Atlantic right whales, but was unanticipated for grey whales, which have hitherto rarely been identified in the European zooarchaeological record. Many of these specimens derive from contexts associated with mediaeval cultures frequently linked to whaling: the Basques, northern Spaniards, Normans, Flemish, Frisians, Anglo-Saxons and Scandinavians. This association raises the likelihood that early whaling impacted these taxa, contributing to their extirpation and extinction. Much lower numbers of other large cetacean taxa were identified, suggesting that what are now the most depleted whales were once those most frequently used.