The ability of cells to count and remember their divisions could underlie many alterations that occur during development, aging, and disease. We tracked the cumulative divisional history of slow-cycling hematopoietic stem cells (HSCs) throughout adult life. This revealed a fraction of rarely dividing HSCs that contained all the long-term HSC (LT-HSC) activity within the aging HSC compartment. During adult life, this population asynchronously completes four traceable symmetric self-renewal divisions to expand its size before entering a state of dormancy. We show that the mechanism of expansion involves progressively lengthening periods between cell divisions, with long-term regenerative potential lost upon a fifth division. Our data also show that age-related phenotypic changes within the HSC compartment are divisional history dependent. These results suggest that HSCs accumulate discrete memory stages over their divisional history and provide evidence for the role of cellular memory in HSC aging.