So eating is like sex for you, amirite?
If you're suggesting that knowledge is still out in the wild, that fails my Occam's Razor test as much as the old tropes of "We can't explain it, therefore UFO aliens" or "We can't explain it, therefore God".
I could see that the braintrust behind the construction of the pyramids would be limited to a select few in favor of the Pharaoh's court, much in the same way only a select few in the world today know how to build quantum computers. That requires a more limited blast radius to extinguish in popular knowledge for sure, especially when the majority of people are illiterate. All it takes is a quantum computing conference in Tokyo and enough sarin gas on the trains.
So you're absolutely right in that some specialized knowledge is inherently less resilient than others.
The rough dates between when the first Egyptian pyramid was built (Pyramid of Djoser) and the complex at Giza was only about one century. Which is a lot of time to innovate over five generations of architects. But we're also talking about something from 4500 years ago, of which a century is a tiny window to have anything survive that long that wasn't designed for a 4500-year shelf-life.
Just think of the Pyramids as ancient Egypt's Intel Corporation of its time.