Superdelegates exist to make it harder for the party to nominate a candidate who would have big electoral trouble and to provide a signal to voters about which candidates are respected by people who know candidates and their records well.
The long version: Through most of American history, party nominees were picked by party leaders in the states, sometimes via a state party convention and/or county and municipal conventions and sometimes by leaders. During the Progressive Era, some states adopted direct democracy mechanisms, which included caucuses and primaries.
In 1964, in the midst of the civil rights movement, the Mississippi Freedom Democratic Party organized a multiracial delegation to go the national convention to try to be seated. They had a hearing before the convention’s credentials committee
. When a black woman named Fannie Lou Hamer testified, her statement was so compelling that President Johnson quickly called a press conference so the tv would have to cut away. This ended up backfiring because tv aired her testimony at night when there were more viewers.
Negotiations in 1964 led to this delegation being offered four seats but the delegation refused the offer. Lyndon Johnson was to win in a landslide, but the convention situation made issues related to representativeness of delegates more important in coming years.
After President Johnson did poorly in the 1968 New Hampshire primary, with Sen. Eugene McCarthy doing better than expected, Johnson decided not to run for reelection. Sen. Bobby Kennedy jumped in but was assassinated after winning the California primary. Sen. George McGovern placed his name in for the nomination, but the delegates selected Vice-President Hubert Humphrey, who had not run in a single primary.
The 1968 Democratic convention was a disaster due to what a report later called a “police riot” in response to protestors in the streets. Moreover, many Democrats were upset about how the nomination turned out. Humphrey lost the popular vote narrowly, but the electoral vote much larger. Republican Richard Nixon was elected president.
In the aftermath, the Democratic Party set up a commission to discuss procedures to select convention delegates. Headed by Minneapolis Mayor Don Fraser and South Dakota Senator George McGovern, it was known as the McGovern-Fraser Commission. It limited the role of state party apparatuses and led to a major increase in the number of primaries, and it set out guidelines for the percentages of delegates by sex, race, and youth. When McGovern decided to run for president, he resigned from the commission but the rules were seen as helping him gain the nomination.
The 1972 contest started with Maine Sen. Ed Muskie, the 1968 Democratic vice-presidential nominee, as the front-runner. Efforts by Nixon’s Committee to Reelect the President engaged in dirty tricks against Muskie and he dropped out. McGovern won the nomination. Then Democrats lost 49 states.
In response to the McGovern electoral debacle, the Democratic Party again considered how delegates were chosen. Gov. Jim Hunt of NC headed another commission, and it decided to set aside delegate slots for state party chairs and vice chairs and Democratic members of Congress — the superdelegates. These individuals were supposed to add political insight to the process and hopefully prevent the epic failure of 1972.
Since then, Democratic Party rules have changed still more, with a process that’s more democratic than it was before 1972 but less directed by the electorate than it was in 1972. Superdelegates have never changed the outcome of a nomination contest.
For a great deal of information on delegate selection, with information about both major political parties, see political scientist Josh Putnam’s blog, Frontloading
This post has been edited to clarify the provisional nature of superdelegate counts before the actual national convention votes.