English: Looking west at the North Slave Quarters at Arlington House, the Robert E. Lee Memorial, at Arlington National Cemetery.

As the house was built beginning in 1802, so were two slave quarters. The two-story brick structures were set perpendicular to the main house and formally aligned with it. They are Neoclassical in design, with pebble-dash stucco exteriors. The facades facing inward toward the yard (to the left, in this image) were plain, but the facades facing outward toward the gardens (to the right, in this image) were formally decorated in a Neoclassical style and highly ornamented.

The two-story Northern Slave Quarters was built on the edge of a ravine to the west of the house, and "banked" so that much more of the northern foundation was visible. The north face of this building features nine windows, a curving arch recessed into the wall about a foot, and four Doric pilasters under the arch. Three doors provided access to the two interior rooms of this building. Each door had a panel above it on which were painted hunting scenes. The eastern half of the first floor (visible here) was a summer kitchen, a kitchen with little insulation and good ventilation which would be used in hot summer months (rather than the indoor or "winter" kitchen) so that the main house would stay cool. The western half of the building was used as living quarters for the African slave who served as coachman, storage space for gardening tools, and a steep narrow ladder-stairs to the second floor. The windowless upper story contained three rooms which were used as slave housing. Fireplaces in the east and west ends of the structure provided heat.

Beginning in 2008, Arlington House began a major conservation and restoration effort. A modern HVAC system was installed to help prevent moisture and mold damage to the house and its contents, and conservation and restoration of its structural elements also occurred. The effort was due to end in 2012, but the August 2011 earthquake resulted in moderate structural damage to the house. The back wall separated from the mansion, requiring the rear passageway, conservatory, and North Wing to be declared off-limits. The second floor is also closed to visitors. The National Park Service says it has no idea when the house will reopen, as most NPS money is going for repairs to the Washington Monument.

Arlington House was built by George Washington Parke Custis, adopted son of George Washington, in 1803. George Hadfield, also partially designed the United States Capitol, designed the mansion. The north and south wings were completed between 1802 and 1804. but the large center section and portico were not finished until 1817.

George Washington Parke Custis died in 1857, leaving the Arlington estate and house to his eldest daughter, Mary Custis Lee -- wife of General Robert E. Lee.