Sedentism itself would have been a big factor in reducing life expectancy; reduced cardiovascular exercise, increase of passed infection and illness through proximity to large groups of people in unsanitary conditions, repetitive farming methods. Things modern 1st world man doesn't worry about when it comes to diet and living.
Hopefully paleo evolves back to its source, with people travelling miles by foot to gather vegetables and stalk game.