Field was born in Brooklyn, New York City, to a family of Ashkenazi immigrants. He grew up in Lynbrook, Long Island, New York, and, being Jewish, he and his family faced antisemitism and discrimination. He played cello in the Field Family Trio, which had a weekly radio program on WGBB Freeport. He served in World War II in the 8th Air Force in England and France, as a navigator in heavy bombers, and flew 25 missions over Germany. In February 1945 he took part in a raid on Berlin with his B-17. His bomber was crippled by flak and crash-landed in the North Sea. All ten crew members made it into the plane’s life rafts, but only seven of them managed to resist till the moment they were rescued by a British air-sea boat hours later.