Understanding the role of geography and climatic cycles in determining patterns of biodiversity is important in comparative and evolutionary biology and conservation. We studied the phylogeographic pattern and historical demography of a rock-dwelling small mammal species from southern Africa, the rock hyrax Procavia capensis. Using a multilocus coalescent approach, we assessed the influence of strong habitat dependence and fluctuating regional climates on genetic diversity. We equenced a mitochondrial gene (cytochrome b) and two nuclear introns (AP5, PRKC1) supplemented with microsatellite genotyping, in order to assess evolutionary processes over multiple temporal scales. In addition, distribution modelling was used to investigate the current and predicted distribution of the species under different climatic scenarios. Collectively, the data reveal a complex history of isolation followed by secondary contact shaping the current intraspecific diversity. The cyt b sequences confirmed the presence of two previously proposed geographically and genetically distinct lineages distributed across the southern African Great Escarpment and north-western mountain ranges. Molecular dating suggests Miocene divergence of the lineages, yet there are no discernible extrinsic barriers to gene flow. The nuclear markers reveal incomplete lineage sorting or ongoing mixing of the two lineages. Although the microsatellite data lend some support to the presence of two subpopulations, there is weak structuring within and between lineages. These data indicate the presence of gene flow from the northern into the southern parts of the southern African sub-region likely following the secondary contact. The distribution modelling predictably reveal the species’ preference for rocky areas, with stable refugia through time in the northern mountain ranges, the Great Escarpment, as well as restricted areas of the Northern Cape Province and the Cape Fold Mountains of South Africa. Different microclimatic variables appear to determine the distributional range of the species. Despite strong habitat preference, the micro-habitat offered by rocky crevices and unique life history traits likely promoted the adaptability of P. capensis, resulting in the widespread distribution and persistence of the species over a long evolutionary period. Spatio-temporal comparison of the evolutionary histories of other co-distributed species across the rocky landscapes of southern Africa will improve our understanding of the regional patterns of biodiversity and local endemism.