Anne Scott-James was born in London, England, to a family of writer-journalists. Her father R. A. Scott-James was a journalist, literary critic, author, and newspaper editor, and her mother was also a journalist. She was educated at St. Paul's Girls' School and Oxford University. She got a job as a secretary with British edition of Vogue in 1934, and quickly rose to become first a columnist and then beauty editor. She made and ended a brief first marriage in 1939, and at the outbreak of World War II, joined the staff of Picture Post, where she was women's editor from 1941 to 1945. She married war correspondent and broadcaster Macdonald Hastings, with whom she had two children, one of whom is the journalist and former editor of The Daily Telegraph Max Hastings.
From 1945 to 1951, Anne Scott-James was the editor of the British edition of Harper's Bazaar. She published her first novel, In the Mink, in 1952. She became woman's editor for the Sunday Express in 1953 and a columnist for the Daily Mail in 1960. After the end of her marriage to Macdonald Hastings, she married Osbert Lancaster in 1967.
In the late 1960s, she embarked on a new career as a gardening writer. Her books in this genre, including Down to Earth (1971), Sissinghurst: The Making of a Garden (1975), and The Cottage Garden (1981), are considered classics.