“The Importance of Being Kennedy” by Laurie Graham. This book takes a look at JFK and his siblings from the nanny’s point of view with lots of historical (and pop culture of the times) details. While it’s definitely fiction, Graham has done her homework.
“Caleb’s Crossing” by Geraldine Brooks. This book is about the first Native American to attend Harvard, but really it’s about the intersection of cultures in those early days of New England. It’s Brooks’ writing that makes this novel a good read.
“Towers in the Mist” by Elizabeth Goudge. This book might be hard to find now, but if you like Tudor England, this novel imagines Oxford in that time period, with a young Sir Philip Sydney and Sir Walter Raleigh appearing in the cast of characters.