The long evolutionary journey that created modern humans began with a single step—or more accurately—with the ability to walk on two legs. One of our earliest-known ancestors, Sahelanthropus, began the slow transition from ape-like movement some six million years ago, but Homo sapiens wouldn’t show up for more than five million years. During that long interim, a menagerie of different human species lived, evolved and died out, intermingling and sometimes interbreeding along the way. As time went on, their bodies changed, as did their brains and their ability to think, as seen in their tools and technologies.

To understand how Homo sapiens eventually evolved from these older lineages of hominins, the group including modern humans and our closest extinct relatives and ancestors, scientists are unearthing ancient bones and stone tools, digging into our genes and recreating the changing environments that helped shape our ancestors’ world and guide their evolution.

The oldest-recovered DNA of an early human relative comes from Sima de los Huesos, the “Pit of Bones.” At the bottom of a cave in Spain’s Atapuerca Mountains scientists found thousands of teeth and bones from 28 different individuals who somehow ended up collected en masse. In 2016, scientists painstakingly teased out the partial genome from these 430,000-year-old remains to reveal that the humans in the pit are the oldest known Neanderthals, our very successful and most familiar close relatives. Scientists used the molecular clock to estimate how long it took to accumulate the differences between this oldest Neanderthal genome and that of modern humans, and the researchers suggest that a common ancestor lived sometime between 550,000 and 750,000 years ago.

Pinpoint dating isn't the strength of genetic analyses, as the 200,000-year margin of error shows. “In general, estimating ages with genetics is imprecise,” says Joshua Akey, who studies evolution of the human genome at Princeton University. “Genetics is really good at telling us qualitative things about the order of events, and relative time frames.” Before genetics, these divergence dates were estimated by the oldest fossils of various lineages scientists found. In the case of H. sapiens, known remains only date back some 300,000 years, so gene studies have located the divergence far more accurately on our evolutionary timeline than bones alone ever could.

Though our genes clearly show that modern humans, Neanderthals and Denisovans—a mysterious hominin species that left behind substantial traces in our DNA but, so far, only a handful of tooth and bone remains—do share a common ancestor, it’s not apparent who it was. Homo heidelbergensis, a species that existed from 200,000 to 700,000 years ago, is a popular candidate. It appears that the African family tree of this species leads to Homo sapiens while a European branch leads to Homo neanderthalensis and the Denisovans.

As the physical remains of actual ancient people, fossils tell us most about what they were like in life. But bones or teeth are still subject to a significant amount of interpretation. While human remains can survive after hundreds of thousands of years, scientists can’t always make sense of the wide range of morphological features they see to definitively classify the remains as Homo sapiens, or as different species of human relatives.

Fossils often boast a mixture of modern and primitive features, and those don’t evolve uniformly toward our modern anatomy. Instead, certain features seem to change in different places and times, suggesting separate clusters of anatomical evolution would have produced quite different looking people.

No scientists suggest that Homo sapiens first lived in what’s now Morocco, because so much early evidence for our species has been found in both South Africa and East Africa. But fragments of 300,000-year-old skulls, jaws, teeth and other fossils found at Jebel Irhoud, a rich site also home to advanced stone tools, are the oldest Homo sapiens remains yet found.

The 160,000-year-old skulls of two adults and a child at Herto, Ethiopia, were classified as the subspecies Homo sapiens idaltu because of slight morphological differences including larger size. But they are otherwise so similar to modern humans that some argue they aren’t a subspecies at all. A skull discovered at Ngaloba, Tanzania, also considered Homo sapiens, represents a 120,000-year-old individual with a mix of archaic traits and more modern aspects like smaller facial features and a further reduced brow.

So when did fossils finally first show fully modern humans with all representative features? It’s not an easy answer. One skull (but only one of several) from Omo Kibish looks much like a modern human at 195,000 years old, while another found in Nigeria’s Iwo Eleru cave, appears very archaic, but is only 13,000 years old. These discrepancies illustrate that the process wasn’t linear, reaching some single point after which all people were modern humans.

Our ancestors used stone tools as long as 3.3 million years ago and by 1.75 million years ago they’d adopted the Acheulean culture, a suite of chunky handaxes and other cutting implements that remained in vogue for nearly 1.5 million years. As recently as 400,000 years ago, thrusting spears used during the hunt of large prey in what is now Germany were state of the art. But they could only be used up close, an obvious and sometimes dangerous limitation.

Even as they acquired the more modern anatomy seen in living humans, the ways our ancestors lived, and the tools they created, changed as well.

Humans took a leap in tool tech with the Middle Stone Age some 300,000 years ago by making those finely crafted tools with flaked points and attaching them to handles and spear shafts to greatly improve hunting prowess. Projectile points like those Potts and colleagues dated to 298,000 to 320,000 years old in southern Kenya were an innovation that suddenly made it possible to kill all manner of elusive or dangerous prey. “It ultimately changed how these earliest sapiens interacted with their ecosystems, and with other people,” says Potts.

As with fossils, tool advancements appear in different places and times, suggesting that distinct groups of people evolved, and possibly later shared, these tool technologies. Those groups may include other humans who are not part of our own lineage.

Many genetic analyses tracing our roots back to Africa make it clear that Homo sapiens originated on that continent. But it appears that we had a tendency to wander from a much earlier era than scientists had previously suspected.

A jawbone found inside a collapsed cave on the slopes of Mount Carmel, Israel, reveals that modern humans dwelt there, alongside the Mediterranean, some 177,000 to 194,000 years ago. Not only are the jaw and teeth from Misliya Cave unambiguously similar to those seen in modern humans, they were found with sophisticated handaxes and flint tools.

Other finds in the region, including multiple individuals at Qafzeh, Israel, are dated later. They range from 100,000 to 130,000 years ago, suggesting a long presence for humans in the region. At Qafzeh, human remains were found with pieces of red ocher and ocher-stained tools in a site that has been interpreted as the oldest intentional human burial.

Among the limestone cave systems of southern China, more evidence has turned up from between 80,000 and 120,000 years ago. A 100,000-year-old jawbone, complete with a pair of teeth, from Zhirendong retains some archaic traits like a less prominent chin, but otherwise appears so modern that it may represent Homo sapiens. A cave at Daoxian yielded a surprising array of ancient teeth, barely distinguishable from our own, which suggest that Homo sapiens groups were already living very far from Africa from 80,000 to 120,000 years ago.

Even earlier migrations are possible; some believe evidence exists of humans reaching Europe as long as 210,000 years ago. While most early human finds spark some scholarly debate, few reach the level of the Apidima skull fragment, in southern Greece, which may be more than 200,000 years old and might possibly represent the earliest modern human fossil discovered outside of Africa. The site is steeped in controversy, however, with some scholars believing that the badly preserved remains look less those of our own species and more like Neanderthals, whose remains are found just a few feet away in the same cave. Others question the accuracy of the dating analysis undertaken at the site, which is tricky because the fossils have long since fallen out of the geological layers in which they were deposited.

While various groups of humans lived outside of Africa during this era, ultimately, they aren’t part of our own evolutionary story. Genetics can reveal which groups of people were our distant ancestors and which had descendants who eventually died out.

All living non-Africans, from Europeans to Australia’s aboriginal people, can trace most of their ancestry to humans who were part of a landmark migration out of Africa beginning some 50,000 to 60,000 years ago, according to numerous genetic studies published in recent years. Reconstructions of climate suggest that lower sea levels created several advantageous periods for humans to leave Africa for the Arabian Peninsula and the Middle East, including one about 55,000 years ago.

“Just by looking at DNA from present day individuals we’ve been able to infer a pretty good outline of human history,” Akey says. “A group dispersed out of Africa maybe 50 to 60 thousand years ago, and then that group traveled around the world and eventually made it to all habitable places of the world.”

While earlier African emigres to the Middle East or China may have interbred with some of the more archaic hominids still living at that time, their lineage appears to have faded out or been overwhelmed by the later migration.

For most of our history on this planet, Homo sapiens have not been the only humans. We coexisted, and as our genes make clear frequently interbred with various hominin species, including some we haven’t yet identified. But they dropped off, one by one, leaving our own species to represent all humanity. On an evolutionary timescale, some of these species vanished only recently.

On the Indonesian island of Flores, fossils evidence a curious and diminutive early human species nicknamed “hobbit.” Homo floresiensis appear to have been living until perhaps 50,000 years ago, but what happened to them is a mystery. They don’t appear to have any close relation to modern humans including the Rampasasa pygmy group, which lives in the same region today.