Gillette was born Anita Luebben in Baltimore, Maryland, the daughter of Juanita (née Wayland) and John Alfred Luebben. Raised in suburban Rossville, she graduated from Kenwood High School. Gillette studied at the Peabody Conservatory and made her Broadway debut in Gypsy in 1959. Additional Broadway credits include Carnival!, All American, Mr. President, Kelly, Jimmy, Guys and Dolls, Don't Drink the Water, Cabaret, They're Playing Our Song, Brighton Beach Memoirs, and Chapter Two, for which she was nominated for the Tony for Best Actress in a Play. She received the 1960 Theatre World Award for her performance in Russell Patterson's Sketchbook.
Gillette's first television appearance was on The Ed Sullivan Show in 1963. She joined the cast of The Edge of Night in 1967, leaving the next year. Gillette's biggest exposure on a national scale came as a celebrity guest on various New York City-based game shows, mostly those produced by Goodson-Todman and Bob Stewart. She served as a semi-regular on the syndicated What's My Line?, Match Game, and on the various Pyramid series, among others.
Gillette's roles in the 1970s included the short-lived series Me and The Chimp with Ted Bessell and Bob & Carol & Ted & Alice with a then-unknown Robert Urich and a young Jodie Foster. She also appeared in Norman Lear's All That Glitters.
The 1980s marked Gillette's transition from Broadway and television into that of a character film actress. Prior to this transition, she had sizeable television roles as Nancy Baxter on the of the national run of The Baxters, Quincy's second wife Dr. W. Emily Hanover on the last season of Quincy M.E. (having previoulsy portrayed his deceased first wife Helen Quincy in a flashback), and a role on Search for Tomorrow at the end of that series' long run, as well as the early David Chase series Almost Grown.
After the end of Search for Tomorrow in late 1986, Gillette transitioned to film with a variety of notable roles such as that of Mona in 1987's Moonstruck. Many of these roles have had her as an on-screen mother to characters played by notable actors such as Jennifer Aniston's mother in She's The One, Mary Louise Parker's mother in Boys On The Side, Bill Murray's mother in Larger Than Life, Jack Black's mother in Bob Roberts, and the mother of Bobby Cannavale's love interest in The Guru. Her return to television in 2000s short-lived Normal, Ohio had her playing the mother of John Goodman's character (coincidentally with fellow former game show regular Orson Bean as her on-screen husband).
In the 1990s, Gillette starred in two Hallmark Hall of Fame movies, The Summer of Ben Tyler with James Woods and A Christmas Memory with Patty Duke. In 2004, she appeared as Miss Mitzi, the lonely alcoholic owner of a struggling dance studio in Shall We Dance? opposite Richard Gere, Jennifer Lopez and Susan Sarandon. She recently completed filming Hiding Victoria, and has made several appearances as Grandma Betty on Fox's The War at Home.
Gillette has kept a busy schedule, with frequent TV and movies roles. Recurring characters included Joan Gamble in the TV series Normal, Ohio; Betty in The War at Home; Lily Flynn, the mother of criminalist Catherine Willows in three episodes of CSI; and the mother of Liz Lemon in two episodes of 30 Rock.