No 10 Downing Street, built by George Downing between 1682 and 1684, has been the residence of British Prime Ministers (on and off but latterly fairly consistently) since it was gifted by George II to the office of the First Lord of the Treasury in 1732.
In his 1720 Survey of the Cities of London and Westminster[ref]An update of John Stow’s classic work of 1598/1603.[/ref] John Strype described the cul-de-sac as ‘a pretty open place especially at the upper end where are four or five very large and well built houses fit for persons of honour and quality; each house having a pleasant prospect into St James’s Park with a terrace walk.’ James Boswell lodged in the street in 1762 and Tobias Smollett tried to establish a surgeon’s practice there.
However, the houses in Downing Street were shoddily built, prone to sinking as they stood on soft soil, and many First Lords (or Prime Ministers as they came to be known) in the 18th century shunned No 10 as inferior to their own private homes (even Harold Wilson preferred to live elsewhere). The street gradually became run down and by the beginning of the 19th century, it was a place of crime and prostitution. Much of it was demolished in the mid 19th century, leaving only Numbers 10, 11 (customarily the Chancellor of the Exchequer’s residence), and 12 (offices for Government Whips). A fire destroyed the upper floors of Number 12 in 1879.
Perhaps because of the street’s unsavoury reputation, there do not seem to be many images of Downing Street from the early 19th century. Here is a sketch from 1899, looking towards St James’s Park.