One of the ways scientists use to measure that is to find unaltered rocks from the time of formation on the Earth, and to take a radiometric reading of the radioactive decay within the atomic nucleus of the elements in the rock. The oldest material found so far is a zircon crystal from the Jack Hills formation in Western Australia, which is 4.4 billion years old. Yes, it is a semi-precious stone (see pics below).
However, the most expensive rocks are not diamonds, certainly not zircons, but lunar meteorites called anorthsites, a pale crystallized rock that will fetch between US$10,000 to US$20,000 per gram. High quality diamonds only cost between US$1,000 to US$2,000 per gram. Crazy!