The story begins in seventeenth-century Boston, then a Puritan settlement. A young woman, Hester Prynne, is led from the town prison with her infant daughter, Pearl, in her arms and the scarlet letter “A” on her breast. A man in the crowd tells an elderly onlooker that Hester is being punished for adultery. Hester’s husband, a scholar much older than she is, sent her ahead to America, but he never arrived in Boston. The consensus is that he has been lost at sea. While waiting for her husband, Hester has apparently had an affair, as she has given birth to a child. She will not reveal her lover’s identity, however, and the scarlet letter, along with her public shaming, is her punishment for her sin and her secrecy. On this day Hester is led to the town scaffold and harangued by the town fathers, but she again refuses to identify her child’s father.
The elderly onlooker is Hester’s missing husband, who is now practicing medicine and calling himself Roger Chillingworth. He settles in Boston, intent on revenge. He reveals his true identity to no one but Hester, whom he has sworn to secrecy. Several years pass. Hester supports herself by working as a seamstress, and Pearl grows into a willful, impish child. Shunned by the community, they live in a small cottage on the outskirts of Boston. Community officials attempt to take Pearl away from Hester, but, with the help of Arthur Dimmesdale, a young and eloquent minister, the mother and daughter manage to stay together. Dimmesdale, however, appears to be wasting away and suffers from mysterious heart trouble, seemingly caused by psychological distress. Chillingworth attaches himself to the ailing minister and eventually moves in with him so that he can provide his patient with round-the-clock care. Chillingworth also suspects that there may be a connection between the minister’s torments and Hester’s secret, and he begins to test Dimmesdale to see what he can learn. One afternoon, while the minister sleeps, Chillingworth discovers a mark on the man’s breast (the details of which are kept from the reader), which convinces him that his suspicions are correct.
Dimmesdale’s psychological anguish deepens, and he invents new tortures for himself. In the meantime, Hester’s charitable deeds and quiet humility have earned her a reprieve from the scorn of the community. One night, when Pearl is about seven years old, she and her mother are returning home from a visit to a deathbed when they encounter Dimmesdale atop the town scaffold, trying to punish himself for his sins. Hester and Pearl join him, and the three link hands. Dimmesdale refuses Pearl’s request that he acknowledge her