An American presidential election, as both Senator John McCain and Senator Barack Obama well understand, is a game of two halves.

In the first half, during the primary season that ends with the two parties' nominating conventions, the goal is to be the chosen candidate of one of the two major parties. In recent years, each has won the votes of something in the region of one half of the electorate. Only a rare successful third-party candidacy, like that of the billionaire Ross Perot in 1992, changes that picture significantly.

Also in openDemocracy on the United States election: openUSA is a new part of the openDemocracy network, publishing daily commentary and analysis of the 2008 election - both from the United States itself and around the world - and links to the best campaign coverage To access openUSA, click here

After the conventions, and more specifically after Labor Day at the beginning of September, when Americans get back from their vacations and the "general election" gets going in earnest, the two candidates strive to win the votes of a majority of the whole electorate.

So, notionally, in the first half of the campaign, the candidates are trying to get to 26%, or just over a half of a half. After the conventions, they are shooting for 51%, or more than a half.

That affects their whole strategy, including the policy proposals they put forward. In the first half of the campaign, they are preaching, as it were, to the converted. They can and do stress their party credentials. Republicans are hell-bent on showing how conservative they are, Democrats avoid the word liberal because for nearly thirty years that has a toxic stench. But they do emphasise their progressive credentials.

A flawed diagnosis

So round about mid-July in a presidential election year there is always a certain grinding of gears. Up to now, John McCain has been bent on selling himself to natural Republican voters as a reliable conservative. Before the campaign started, he had a well-earned reputation as a maverick. A conservative, of course: a strong patriot, an upholder of the traditional values and shibboleths of the Grand Old Party, he was also unpredictable. He reached out, for example, across the aisle to the Democrat, Senator Russell Feingold, in their joint campaign against the excessive role of money in American elections.

Once he was running again, however, and especially after his campaign seemed to wilt, he became a more orthodox conservative. He began to look as if he were running for George W Bush's third term. He picked up Bush's plans for "reforming" - that is, for partially privatising - the Social Security pension system, set up in 1935 by Franklin Roosevelt, and since then such a sacred icon for many older voters that it has been called the "third rail" of American politics.

Godfrey Hodgson was director of the Reuters' Foundation Programme at Oxford University, and before that the Observer's correspondent in the United States and foreign editor of the Independent. He reported the presidential elections of 1964, 1968, 1972, and 1976 for various British and American media, and was co-author (with Lewis Chester and Bruce Page) of the best-selling account of the 1968 campaign, An American Melodrama (Viking Press, 1969). Among his other books are The World Turned Right Side Up: a history of the conservative ascendancy in America (Houghton Mifflin, 1996); The Gentleman from New York: Senator Daniel Patrick Moynihan (Houghton Mifflin, 2000); More Equal Than Others: America from Nixon to the New Century (Princeton University Press, 2006), and A Great and Godly Adventure: The Pilgrims and the Myth of the First Thanksgiving (PublicAffairs, 2007) Among Godfrey Hodgson's recent articles on openDemocracy:

The United States: democracy in trouble" (30 September 2007)

"America in 2008: the next realignment?" (6 November 2007)

"Washington discovers Islamabad" (27 November 2007)

"The United States election: time for ‘change'" (10 January 2008)

"America's change election: reality or mirage?" (11 February 2008)

"'Superdelegates' and the US election" (25 February 2008)

"The lost election year" (15 May 2008)

"Barack Obama: at the crossroads of victory" (11 June 2008
He started playing down the economy's troubles, and on Iraq, he actually went further than Bush. He suggested that the United States must stay there for a hundred years if necessary.

Now that Barack Obama has won the Democratic nomination, he is accused of moving towards the centre, and it is true that he has taken a number of positions that are bound to upset those who expected him to be an orthodox liberal. Republicans gleefully accuse him of being a "flip-flopper", while those on the left of his party angrily accuse him of abandoning the principles that made them support him.

He was one of the first to criticise the Iraq war, and one of only slightly more than half of the Democrats in the Senate (twenty-nine against twenty-one) to vote against it. He not only spoke with apparent impatience of the white working class in small mid-western industrial towns who take out their political frustration by talking about God and guns. That you might expect from a Democratic progressive. But in a Father's Day speech he also criticised African American mores in biting terms. He even chided rappers for using bad language.

His new posture provoked an onslaught from David Brooks, who has replaced William Safire as the house conservative on the editorial pages of the New York Times. Obama's liberal supporters, Brooks said, expected him to be a gentle, herbivorous politician, when in fact he was ferociously ruthless. "God", Brooks wrote in humorous criticism of his friends, "Republicans are saps" (see "The Two Obamas", New York Times, 20 June 2008).

"On the one hand, there is Dr. Barack, the high-minded, Niebuhr-quoting speechifier who spent this past winter thrilling the Scarlett Johansson set and feeling the fierce urgency of now. But then on the other side, there's Fast Eddie Obama, the promise-breaking, tough-minded Chicago pol who'd throw you under the truck for votes.

"This guy is the whole Chicago package: an idealistic, lakefront liberal fronting a sharp-elbowed machine operator. He's the only politician of our lifetime who is underestimated because he's too intelligent. He speaks so calmly and polysyllabically that people fail to appreciate the Machiavellian ambition inside."

This no doubt revealed more about Brooks than about Obama. It showed that the New York Times columnist shared childish conservative stereotypes about wet, sheep-like liberals, to be contrasted with red-blooded conservatives.

It also revealed a serious misunderstanding of what sort of fellow Obama is. Anyone who has read his two books, his autobiography (Dreams from My Father) and his political essay (The Audacity of Hope), must have grasped how far the man is from being a predictable or conventional progressive (see Ryan Lizza, "Making It: How Chicago shaped Obama", New Yorker, 21 July 2008).

He is, for example, an unblushing American exceptionalist. He misses no opportunity to proclaim his patriotism, his love of America, his apparent belief that political decency was invented in the United States. No neo-conservative could do more.

He also makes plain his belief in the importance of religion. Both for practical political reasons, because so many Americans are Christians, and because his own political credo is so influenced by Christian values, he scolds his fellow progressives for undervaluing the importance of faith. He confesses to intellectual doubts about some basic Christian teaching, and admits that for him religious conviction has come, not as a blinding flash of truth, but after a long and dusty journey. Yet Jesus Christ, he says, has entered his life, and he is guided by Christian values.

Understandably, given his father's early departure from his marriage to his mother, and the brief duration of her second marriage, Obama puts heavy emphasis on the importance of the family. In this and other ways he shares many conservative values.

There is an even more important error lurking behind the failure of a David Brooks to "get" Obama. It is the assumption that political ideals and political effectiveness are somehow antithetical: that if a fellow knows how to manoeuvre through political difficulties, he must be an unprincipled, "Machiavellian" thug who would push you under a truck.

The real question

In empirical and historical terms, this does not stand up. Pick any effective reformer from the past of the United States or any other country, and you see a man or a woman who knew how to spin on a sixpence when no unbreakable principle was at stake, but who could be firm and even ruthless when such principles did need to be defended. This was true of Franklin Roosevelt and of Lyndon Johnson, as it was of David Lloyd George or Charles de Gaulle.

Idealism and political skill are inseparable. Idealism without political effectiveness is irrelevant. Political skill without ideals is pointless. The political grandmasters, progressive or conservative, have never been ineffectual dreamers. They knew how to acquire power and how to defend it.

Certainly in the American system, every candidate who aspires to be president must first secure his or her own base, then reach across to poach some of the opponent's voters. There is an old saying that you must go hunting where the ducks are, in the middle of the pond. That is what Obama is doing. It is also, so it would seem, what John McCain is so far failing to do.

True, the punt from which Obama hopes to hunt his ducks has run into choppy water. The fallout from the public disquiet over the opinions of his former pastor, the Rev Jeremiah Wright, has not quite subsided. The vulgarly expressed hostility of the Rev Jesse Jackson, the most successful African American candidate for the presidency before him, has done him no good with African American voters. He will find it hard to pull off his planned visit to the middle east without reawakening questions in the minds of both his progressive admirers and Israel's supporters.

McCain, too, will have problems negotiating the transition from the pre-convention to the post-convention phases of the campaign. The "movement conservatives" who dream of the restoration of Ronald Reagan will never quite accept McCain, however large the tax cuts he promises.

Obama's dilemma is how to appeal simultaneously to African Americans and the left on the one had, and to centrist instincts on the other. Yet McCain's dilemma is at least as hard. How can he move to the centre when his own base is so reluctant?

All along, the great question of the 2008 election has not been an artificial choice between idealism and pragmatism. It has been a question, not for the candidates, but for the voters, though both candidates have articulated it with some clarity. It is: how much change do the Americans voters want?

Republicans may plead that the Iraq adventure has been a sort of success. They can hardly deny that after five years, the Bush administration has achieved the exact opposite of what it intended. It has left the middle east dominated by Iran and its Shi'a allies in Syria, Lebanon and Iraq. For good measure, the American economy is a shambles, and America's reputation, after Guantànamo and Abu Ghraib, needs heroic therapy.

Obama argues that he will be swept into the White House by a national yearning for change, whatever exactly that means. John McCain sees no such need for change, and no such yearning. The period from mid-July to 4 November 2008 will see a political battle with more at stake than at any time since 1980 or even 1968.

There are what Donald Rumsfeld called "unknown unknowns" ahead. What if Israel is not bluffing about an attack on Iran? How deep will the world, and the United States, spiral down into recession? Will the pragmatic idealist beat the idealistic pragmatist? If McCain wins, it will be by a narrow margin. If there is to be a landslide, it will be for Obama.