Carroll performed in several small roles in films under his original name until 1935, when he first used the name John Carroll in Hi, Gaucho!. He appeared in several Western films in the 1930s, including the role of Zorro in Zorro Rides Again in 1937.
Probably his best known role was as Woody Jason in the 1942 movie Flying Tigers with John Wayne.
He was also notable as a Cajun soldier, aptly nicknamed "Wolf", in the 1945 comedy A Letter for Evie.
Carroll worked steadily through the mid-1950s, but his work began to fade out in the latter half of the decade. His last role was in Ride a Pink Car in 1974.
Carroll was married twice; first to Steffi Duna (the couple had a daughter, Julianna Benito), and then to Lucille Ryman (until his death).
Carroll died of leukemia in Hollywood, California.