Gregory was born in Paris, France, in 1934 to Russian Jewish parents. He studied at Harvard University, where he was affiliated with Adams House.

During the 1960s and 1970s, Gregory directed a number of avant-garde productions developed through ensemble collaboration, the most famous of which was Alice in Wonderland (1970), based on Lewis Carroll's two classic Alice books. He founded his own theatrical company, The Manhattan Project, in 1968. In 1975 he directed Our Late Night, the first produced play by Wallace Shawn, which began a long working relationship between the two men.

Shortly afterward, Gregory's growing misgivings about the role of theatre in modern life, and what he felt was a trend toward fascism in the United States, led him to abruptly abandon theatre and leave the country. As described in the film My Dinner with Andre (1981), he traveled to Poland at director Jerzy Grotowski's invitation, where he developed a number of experimental theatrical events for private audiences. He spent several years in a variety of esoteric spiritual communities (such as Findhorn) developing an interest and practice in what could be called New Age beliefs.

Although Gregory left the theatre in 1975, he has returned several times to direct small productions, usually for invited audiences. These included a long-running workshop of Uncle Vanya (adapted by David Mamet), which was developed from 1990 to 1994 and featured Shawn and Julianne Moore. Though never publicly performed, it was released as the film Vanya on 42nd Street by Gregory and Louis Malle. He appeared as himself, directing the play featured within the film. Gregory also directed a radio production of Shawn's play, The Designated Mourner, in 2002.

He has had occasional film roles as a character actor, including John the Baptist in The Last Temptation of Christ and Reverend Spellgood in The Mosquito Coast, and as Dante, a restaurateur, alongside Rosanna Arquette, David Bowie, and Buck Henry in The Linguini Incident.

His best-known film performance was as the title character in My Dinner with Andre (1981), directed by Louis Malle, in which he and Wallace Shawn, playing characters based on themselves, have a long conversation over dinner. They discuss Gregory's spiritual sojourn in Europe and his doubts about the future of theatre and of Western civilization in general.

He appeared with Goldie Hawn in Protocol (1984). In 1988 he played the father in Some Girls, with Jennifer Connelly and Patrick Dempsey. In 1993, he performed in the movie Demolition Man with Sylvester Stallone.

Returning to theatre, Gregory directed Shawn's play Grasses of a Thousand Colors, which premiered at the Royal Court Theatre in London in May 2009. He next worked with Shawn on a new version of Ibsen's The Master Builder. This resulted in the film Fear of Falling (2013), directed by Jonathan Demme. The film was retitled A Master Builder at its opening in New York in June 2014.

In 2013, he directed Grasses of a Thousand Colors and The Designated Mourner, starring Shawn in a co-production between Theatre for a New Audience and The Public Theater in New York.

A 2013 documentary about Gregory's life, Andre Gregory: Before and After Dinner, was directed by his wife, Cindy Kleine. He and Kleine discussed it on the May 3, 2013, episode of Charlie Rose.