Bake the yams with skins on for 40 minutes at 350 degrees.
Remove while still a little firm in the center. Peel and slice the yams. (I also included a sweet potato or two because, honestly, I couldn't tell the difference.)
Mix the brown sugar, cashews, and ginger in a bowl and set aside
In a 10" x 6" x 2" baking dish, layer half the yams, half the peaches and half the ginger/cashew/sugar mixture. Then repeat using the remainder of each.
Pour about 1/2 cup of the reserved peach liquid over the yam mixture, dot with butter and season with a little salt and pepper.
Bake covered for 30 minutes at 350 degrees, then remove the cover and bake for another 30 minutes.