They said villages had been attacked by men with machetes who came from nearby hills.
Troops have now been deployed in the area and dozens of arrests are said to have been made.
Acting President Goodluck Jonathan has ordered security forces to prevent more weapons being brought into the area.
Jos has been under a military curfew since January when at least 200 people died in clashes between Christians and Muslims.
Caroline Duffield, BBC News, Lagos
Already this is being described as retaliation for the outburst of killing in January in which hundreds more people were killed.
Back then the largest losses were suffered by the Hausa Fulani community. In the village of Kuru Karama more than 100 people were killed and their bodies thrown into wells and sewers. Grave accusations were made that the local government had stoked the violence. This time it is clear that the targets were Berom Christians.
For weeks there have been rumours of retaliation in these villages and people have been living in a state of anxiety. Many families left. These killings are often painted by local politicians as a religious or sectarian conflict. In fact it is a struggle between ethnic groups for fertile land and resources in the region known as Nigeria's Middle Belt.
The latest attacks are said to have been reprisals for the January killings.
The AFP news agency reports that the villages are now calm after troops and military vehicles entered them.
An adviser to the Christian-dominated Plateau state government, Dan Manjang, told AFP: "We have been able to make 95 arrests but at the same time over 500 people have been killed in this heinous act."
Another Plateau state official, Gregory Yenlong, urged people to "remain calm and be patient as the government steps up security to protect lives and property in this state".
Many of the dead in the villages of Zot and Dogo-Nahawa are reported to be women and children.
Mark Lipdo, from the Christian charity Stefanos Foundation, said Zot had been almost wiped out.
He said: "We saw mainly those who are helpless, like small children and then the older men, who cannot run, these were the ones that were slaughtered."
A resident of Dogo-Nahawa said that the attackers had fired guns as they entered the village before dawn on Sunday in defiance of a curfew.
JOS, PLATEAU STATE
Deadly riots in 2001, 2008 and 2010
City divided into Christian and Muslim areas
Divisions accentuated by system of classifying people as indigenes and settlers
Hausa-speaking Muslims living in Jos for decades are still classified as settlers
Settlers find it difficult to stand for election
Communities divided along party lines: Christians mostly back the ruling PDP; Muslims generally supporting the opposition ANPP
"The shooting was just meant to bring people from their houses and then when people came out they started cutting them with machetes," Peter Jang told Reuters news agency.
Some witnesses said villagers were caught in fishing nets and animal traps as they tried to escape and were then hacked to death. Mud huts were also set on fire.
Mass burials took place on Sunday and scores more bodies were laid out in the streets of the three attacked villages, awaiting further burials on Monday.
Figures given for the death tolls in the ethnic clashes have varied widely, sometimes to achieve political ends or to reduce the risk of reprisals, or simply because victims are buried quickly.
Jos lies between the mainly Muslim north of Nigeria and its largely Christian south.
Analysts say the latest attack seems to be in reprisal for clashes in January, which claimed the lives of at least 200 people and displaced thousands of others.
Hundreds of people have fled from Jos in the aftermath of the fighting, the International Committee of the Red Cross says.
The clashes represent a challenge for Acting President Jonathan. He formally took over last month from President Umaru Yar'Adua, who has a heart problem.
Mr Yar'Adua returned from three months of treatment in Saudi Arabia two weeks ago but has still not been seen in public.