It is a satirist's dream come true. John Crace looks back over a decade of poking fun at clunky plots and dodgy dialogue

I could be the only person who has never forgotten William Sutcliffe's Love Hexagon. It was the first book I ever digested and I'd like to be able to say I'd spent a lot of time selecting it. But it wasn't like that.

A few days earlier I'd been stopped in the corridor by the new editor of the Editor, the Guardian's standalone digest of the week's news (RIP), and asked if I'd like to take over a little-noticed column called the Digested Read. She wandered off before I had time to answer, but she didn't need to hang around. The ­Digested Read is a dream job for any satirist and I would have done it for almost nothing. Come to think of it, I did. But I still needed to choose a book and as I hadn't yet got the hang of ringing publishers, asking to bite the hand that feeds, I went to see the literary editor, who poked around in her cupboard for something she didn't want. So Love Hexagon it was.

I doubt it's much consolation to Sutcliffe now, but I soon realised it was a poor choice. The Digested Read works best with authors who are getting the most media attention in any given week – be they Ian McEwan, JK Rowling, Nigella Lawson or Katie Price – and since that first week, it is a principle to which I have tried to stick.

It's not infallible. Publishers tend to keep their big names for the spring and summer; in these months there's often too much choice and it can be a straight toss-up between JM Coetzee and AS Byatt. At other times of the year, particularly January, the publishing lists are thin and books squeeze in that normally wouldn't get a reading. It happened once with the brother of a well-known author, a mistake for which I've clearly never been forgiven by the victim; a year ago someone kindly directed me to his blog where he continues to regularly rubbish me seven or eight years on. Books do also just get missed. I never gave The Da Vinci Code a second thought when it came out.

Over the last 10 years, the Digested Read has changed locations several times – from the Editor to the main paper to G2 – but the format has remained the same; rewriting a book in 700 words in the style of the author. The primary goal is to entertain – something the book itself has often failed to do – but it's also intended as a (semi-) serious critique, for much of the fun is derived from clunky plot devices that don't work, pretentious stylistic tics, risible dialogue and an absence of big ideas. Literary criticism does not have to be dull to be serious.

Some people object to its cruelty. I have no defence. Satire often is cruel, especially when it's accurate. Here's the thing. I read every word of every book I digest, scribbling notes on the pages as I go along. I can't afford not to because if I get something wrong, I'm stuffed. So you could argue that I show rather more respect for the integrity of an author's work than a reviewer who gives a book the thumbs up after a skim read. And that does happen. I've read reviews of books I've ­digested and can see the critic has only read the blurb, the first few chapters and the ending. But who cares so long as it's a positive review? Certainly not the author or the publisher. You might, though, if you fork out £10 to buy it.

And many authors do seem to "get" the Digested Read. I'm continually delighted – and astonished – by the number of writers who are more generous about my work than I am about theirs and get in touch to say how much they enjoy the column. Especially when it's someone else's books. Some even email to say they've liked what I've done to their own book. That I don't understand. Publishers are also surprisingly complimentary; some authors would be surprised to discover how much their egotism gets up the noses of their editors and publicists. My favourite compliment is this from the New York Times: "The best book-related feature in any of this planet's English-language newspapers." That will go on my gravestone.

No writer has yet – and I'm not keen for a precedent to be created – emailed to tell me they hate me. It would be nice to imagine this was because they all thought I was so wonderful, but I suspect this is wishful thinking. More likely they are maintaining a dignified ­silence, or have their minds on higher matters.

Not that authors don't have their strops. Jilly Cooper moaned to the Daily Telegraph that I had given away the plot of her book. I hadn't been aware there was one; the ­ending was blindingly obvious from about page 20. One award-winning young author had a complete strop after I digested their partner's book, and threatened never to write for the Guardian again; a threat that hasn't been kept.

One last thing. Sometimes I am asked if I enjoy reading. How could I not? Do you ­really imagine the last 10 years have been an extended exercise in masochism? Especially now that I also digest a classic each week. Few books are as good as their publicity – and it's more often than not the difference between hype and reality I try to exploit – but there haven't been many that have had no redeeming qualities.

Reading is, and remains, a pleasure. As does digesting. Edith Wharton's The Age of Innocence is a great book. It's also great to satirise. The two aren't mutually exclusive. So here's to ­another 10 years digesting. If you'll have me.

A complete archive of John Crace's Digested Reads guardian.co.uk/digestedread

It has been the basis for at least five novels, most famously Robert Louis Stevenson's Kidnapped. But the newly revealed story of James Annesley is more incredible than any of the tales it inspired

As yarns go, it pretty much has it all. There's a street waif who's actually an aristocrat, heir to half a dozen titles and estates in England, Ireland and Wales. A dastardly uncle who'll stop at nothing to usurp him. A kidnapping most foul, and a decade of toil as an ­indentured servant in 18th-century America. Then, against impossible odds, a dashing return, and a quest for justice through the courts that held all society spellbound.

The extraordinary story of James Annesley has inspired at least five novels, including Sir Walter Scott's Guy Mannering and, most famously, Robert Louis Stevenson's Kidnapped, one of the best-loved adventure books of all time. Yet the true story behind a case that was in its day every bit as sensational as those of Oscar Wilde, Myra Hindley or OJ Simpson were in theirs has never fully been told – and it is, if anything, even more spectacular than the fictions spun around it.

"I think one reason why there's been so little recent interest in the Annesley saga is that many modern historians and literary critics simply have not considered it to be true," says Roger Ekirch, an award-winning American historian whose impeccably researched yet rip-roaring rendering of ­Annesley's life, Birthright, is published this month. "People were just not ­inclined to believe it. That was certainly my take, for a long time."

Ekirch and his fellows could be forgiven. The principal source of ­information on Annesley was a ­fanciful if much-reprinted volume from 1743, Memoirs of an Unfortunate Young ­Nobleman ("Return'd," the title ­continues in classic 18th-century ­plot-spoiling style, "from Thirteen years Slavery in America, where he had been sent by a Wicked Contrivance of his Cruel Uncle; A Story founded on Truth, and address'd equally to the Head and Heart").

The events related in the book ­appear so far-fetched, however, that most of those who have read it, says Ekirch, "have tended to dismiss it as merely a sentimental fiction, written during an age when overblown stories of impossible adventures were a ­popular literary genre".

But then the historian ­happened across an obscure diary by an 18th-century Somerset rector that cited, as the event that had most marked the year 1743, a trial in which a young claimant who had returned unexpectedly from abroad sued his uncle for a lost inheritance. "It rang a bell," Ekirch says. "It sent me back to the Memoirs."

And after seven years spent with trial transcripts, family documents, newspaper reports, House of Lords records and a treasure trove of nearly 400 legal depositions unearthed in Dublin and at the National Archives in Kew, it is now clear to Ekirch that those Memoirs are, essentially, true. "Annesley wasn't the author, but he was the source of the ­information," he says. "You don't have to dig far to substantiate it."

So who was James – or Jemmy – Annesley? He was born at Dunmain, County Wexford, in the spring of 1715, into Ireland's privileged, powerful and often dissolute Protestant aristocracy. Even in such company, the Annesleys were a particularly unprincipled lot, says Ekirch: "I seriously doubt whether any family could rival them in venality or violence."

But they were wealthy. Jemmy, son of Arthur, Baron Altham, and Mary, illegitimate daughter of the Duke of Buckingham, was putative heir to a family fortune that included two English peerages – one of them the prized earldom of Anglesea – and lands whose rental income alone would be worth, by the time he came to claim them, £50,000 a year: maybe £5m today.

His adventures began young. The boy was barely two when Altham threw Jemmy's mother out of ­Dunmain on a pretext. Father and son embarked on a nomadic and ­increasingly impecunious existence; at six, Jemmy was riding a small sorrel mare and sporting a scarlet silk coat with silver buttons, but the following year Altham, short of cash as long as his elder cousin, the current Earl of Anglesea, was alive, took up with a wealthy heiress "as much", says Ekirch, "out of self-­preservation as of passion".

Resented by his father's new mistress, Jemmy was beaten and eventually banished from the home. He became "a street urchin" in Dublin, says Ekirch. "For four years he worked as a shoeblack and ran errands for ­Trinity College students." Often he slept rough, before landing up, in the ­summer of 1727, at the home of a kindly butcher named John Purcell.

Enter – with suitably ­menacing drum roll – Altham's younger brother and Jemmy's uncle, Richard. He saw only two obstacles between himself and the Earl of Anglesea's lands and title: Altham and Jemmy. "In 30 years of writing history," says Ekirch, "Uncle Dick is the most sinister person I've ever encountered. His chaplain said of him later that no man was more penitent at the time of his death. Frankly, few men had more to be penitent about."

Indeed, Ekirch is now more or less sure that Richard, a serial bigamist, did Altham in. "I've become progressively convinced he poisoned his brother," the historian says. "He had the ­motive. The symptoms Altham displayed strongly suggest poisoning. And from later court documents we know that Richard visited the butcher Purcell just three weeks before Altham's death – plainly to find out whether Jemmy was ever likely to claim his title. The butcher told him he hoped Jemmy would be reunited with his father: the last thing Richard wanted to hear."

Altham, in any event, died on 15 November 1727. Richard was at the funeral, as – in tattered breeches and a filthy coat – was a distraught Jemmy, still only 12. Soon after, strange men ­began hanging around Purcell's yard. The butcher saw off one lot with his cudgel. But the following April, Jemmy was seized in Ormond Market, accused of "stealing a silver spoon", and led by Uncle Dick to George's Quay and a waiting longboat. He was rowed out to a ship (called, almost unbearably, the James), kidnapped and America-bound.

Wicked Uncle Dick had to wait 10 years before the redoubtable Earl of Anglesea finally expired. Nor did he enjoy the fruits of his plotting for long: after 12 miserable years as an indentured servant in the backwoods of Delaware, Jemmy regained his freedom in 1740. Now 25, he found passage on a merchant ship bound for London via Jamaica, and – war with Spain having broken out in the Caribbean – enlisted as an able seaman on arrival at Port Royal. There he also made his true identity known and, in one of this story's many stranger-than-fiction moments, was instantly recognised by several fellow sailors, including one who had been at school with him.

The news burst like a bomb in London and Dublin. Amid the back numbers of the London Daily Post, Ekirch found a breathless report dating from 12 February 1741, announcing that in Jamaica had been found a recently recruited seaman, "the only son of the late Lord Altham, who was heir to the title and estate of the Earl of Anglesea".

In London by September of that year, James could now embark on the battle to reclaim his birthright. ­Before it could even begin, however, he found himself accused of murder in a sensational trial at the Old Bailey that was manipulated from start to finish by his scheming uncle, who confided to a friend that if Annesley hanged he "should be easy in his titles and estates".

Safely acquitted by 1742, James had assembled enough witnesses in Ireland to bring a test case against his uncle. First he would need a pretext to prove his identity and stake his claim. A tenant for 1,800 acres of disputed land in County Meath was installed by James and, as expected, instantly evicted by Richard's agents. Dirty Dick was by now fighting mean: James faced two clear attempts on his life before the trial of the century came to court in November 1743.

Press and public interest on both sides of the Irish Sea was immense. At stake, after all, were five peerages, and the largest estate ever to be contested in a court of law. A string of witnesses swore Annesley was who he said he was, and that his story was true; his kidnappers made a full confession. But many more witnesses, often in Uncle Dick's pay, perjured themselves ­shamelessly, declaring James the ­bastard son of his wetnurse, the memorably named Juggy Landy.

"It was extraordinary," says Ekirch. "It shocked me, reading the documents. Seldom, if ever, can so many people have lied so brazenly and with such ­apparent conviction in a court of law." Finally, at the end of what was at the time the longest trial ever heard in the British Isles, the jury found for the tenant, thus confirming Annesley's identity. Even that, though, wasn't the end. James, whose funds were limited, could now sue in Dublin and London to recover his full birthright – but Richard played every delaying tactic in the book.

The affair dragged on for 15 long years. In April 1759, James was reduced to petitioning for his case to be heard as a pauper. Before it could be, on 5 January 1760, he died, to be followed a year later by his nemesis Uncle Dick, and a year after that by Annesley's only son. The press, says Ekirch, went overboard: Annesley, a "most remarkable and unfortunate man" who had "engrossed the attentions of three kingdoms more than any private man ever did", had surely died "of a broken heart", "truly a victim of the avarice, inhumanity and injustice of others".

The saga was finally concluded only in the 1770s, when, in a final flurry of lawsuits, Richard's bigamy (­"irregular and immoral way of life", it was called) was at last exposed. There was, the House of Lords' Committee of Privileges announced, no legitimate heir to his ill-gotten titles: the earldom of Anglesea was extinct. This was not the predictable, anti-climactic ending that Ekirch, when he set out on the story, says he most feared, but "a bittersweet one, full of poetic justice. It truly bears out that old French adage: Revenge is a dish best served cold".

There is no doubt, says Ekirch, that Stevenson's Kidnapped, published in 1886, was inspired by the Annesley story. "The setting is Scotland, and David Balfour never makes it to America," he says. "But it's the usurpation of an orphan's inheritance by a wicked uncle who conspires to send his nephew to the colonies as a servant. You couldn't get a much better dovetail than that. And we know for a fact that Stevenson read about the case." A number of other 19th-century novels, such as Charles Reade's The Wandering Heir, echo James's life even more closely.

No wonder. Here, says Ekirch, "was a real life drama that arguably no ­novelist could imagine, and if they did, it would be so incredible that even as fiction no one could possibly take it seriously."

The historian's one regret is that so little of that story – apart from the testimony he gave at his murder trial – survives in Jemmy's own words. He left no diaries, few papers. The key details of James Annesley's life, nonetheless, are now known beyond reasonable doubt, and it remains "a quite extraordinary saga of betrayal and loss, but also of survival, resilience and redemption," Ekirch says. "This is not just a story about 18th-century England and Ireland, but about the iniquities and virtues of human nature."

Birthright: the True Story that Inspired Kidnapped is published by Norton on February 25, price £17.99.

Read more about Robert Louis Stevenson.guardian.co.uk/books/robert-louis-stevenson

From Huck Finn to Holden Caulfield and Humbert Humbert, the novelist provides an entirely trustworthy guide to some of literature's slipperiest characters

Henry Sutton was born in Norfolk in 1963. After training as a journalist he worked for a number of national newspapers and magazines. He is the author of five previous novels, including Gorleston, Flying and Kids' Stuff, and a collection of short stories, Thong Nation. He also teaches creative writing at UEA and lives in Norwich with his family. His new novel, Get Me Out of Here, is published by Harvill Secker.

Buy Henry Sutton books at the Guardian bookshop

"Something strange happened to unreliable narrators in the mid-20th century: they became a little more reliably unreliable, and a lot nastier. In the late-19th century they tended to be untrustworthy either because they were hiding something about themselves or had failed to recognise the truth, generally because of some kind of psychological weakness. However, as modernism shifted into post-modernism and we all became that much more cynical, most narrators were expected to be complicated. Unreliability became inextricably linked with malevolence – not to mention duplicity, delusion, even derangement. Of course, as the parameters stretched, unreliable narrators also became a lot more fun, with humour often countering the blackness. The challenge was to make tricksy first-person characters both intriguing and entertaining."

1. Lolita by Vladimir Nabokov (1955)

Never straight with himself, let alone the ladies and gentlemen of the jury to whom he is ultimately addressing his words, Humbert Humbert arrived halfway through the 20th century, intent on justifying his appalling crime. Nabokov's syntactical genius is the one true triumph.

2. The Turn Of The Screw by Henry James (1898)

Is it a ghost story, or the tragic tale of a young woman undergoing a breakdown? Believing her two young charges are communing with the spirits of her two dead predecessors, the prim governess of Bly House becomes increasingly panic-stricken and erratic, until she's left with a dead boy in her arms.

3. The Heart Of Darkness by Joseph Conrad (1902)

Right at the start we're told that Marlow likes to spin yarns. However, his tale of journeying up the Congo, in search first of ivory, and then the infamous Kurtz, is one of the most powerful stories in literature. Whether his story is strictly faithful becomes irrelevant, as Marlow ends up highlighting the moral corruption at the heart of all humans.

4. Money by Martin Amis (1984)

John Self is one of literature's most repulsively addictive narrators. The book might be subtitled "A Suicide Note", but it is in fact a love story, with Self dreaming up ever more extravagant ways to shed his wedge while pursuing entirely corruptible Selina Street, among others. The fact that Self might never have actually existed, revealed towards the end, is Amis's sly take on the death of the self.

5. American Psycho by Bret Easton Ellis (1991)

Patrick Bateman makes John Self look even more out of shape, when it comes to commenting on the big brands and applying his murderous hands to the unsuspecting and the vulnerable. Yet Ellis's great comment on consumerism and the death of high culture could just be a mirror to our own deluded thoughts, and Bateman nothing more than a sickly funny fantasist.

6. The Killer Inside Me by Jim Thompson (1952)

It was Jim Thompson, not James M Cain, who put the hard into hard-boiled, the noir into roman noir. He was also one of the first crime writers to take us into the heads of seriously twisted killers, if not out-and-out psychopaths. Deputy Sheriff Lou Ford is regarded as a pillar of the small Texan community he serves. Yet he's in possession of a secret he doesn't even admit to himself. When the bodies start to appear, the net slowly tightens.

7. The Catcher in the Rye by JD Salinger (1951)

Classic unreliability when first published in the early 1950s which now looks almost tamely reliable. Of course young Holden Caulfield is anything but clear about what his short, privileged life has already led him to believe – he's a teenager. Naturally everything's phoney, except his beloved sister Phoebe. Though even she is abandoned as Holden loses his fragile grasp on reality.

8. The End Of Alice by AM Homes (1996)

Narrated in the first person by a hyper-intelligent paedophile, and from the third person perspective of a 19-year-old girl with an unhealthy fixation on a much younger boy, Homes's homage to Nabokov didn't just question the nature of desire, but that of literary taste and acceptability. A brutally brave and truly experimental novel that, over here, fell very foul of the Daily Mail.

9. We Need to Talk About Kevin by Lionel Shriver (2003)

Shriver's Orange Prize-winning novel is a postmodern masterclass in unreliability, as the principal theme of nature versus nurture trickles through the slow revelations of exactly what Kevin has done. Told in a series of letters by Kevin's mother, Eva, to her estranged husband, Franklin, the reader is never quite sure of whether it was Eva or Kevin who exhibited the most disturbing behaviour. Franklin, meanwhile, is guilty of chronic denial.

10. The Adventures of Huckleberry Finn by Mark Twain (1884)

In his search of freedom, as he floats down the Mississippi, Tom Sawyer's best friend "Huck" Finn finds himself travelling out of his rational mind. First published in 1884, Twain himself described his controversial masterpiece, as "... a book of mine where a sound heart and a deformed conscience come into collision and conscience suffers defeat".

An anti-Stalinist author who died in obscurity in 1951 may be the greatest Russian writer of the last century, his English translator Robert Chandler explains to Daniel Kalder

Stalin called him scum. Sholokhov, Gorky, Pasternak, and Bulgakov all thought he was the bee's knees. But when Andrei Platonov died in poverty, misery and obscurity in 1951, no one would have predicted that within half a century he would be a contender for the title as Russia's greatest 20th-century prose stylist. Indeed, his English translator Robert Chandler thinks Platonov's novel The Foundation Pit is so astonishingly good he translated it twice. Set against a backdrop of industrialisation and collectivisation, The Foundation Pit is fantastical yet realistic, funny yet tragic, profoundly moving and yet disturbing. Daniel Kalder caught up with Chandler to talk about why more people should be reading Platonov.

Why did you translate Platonov's Foundation Pit twice?

No other work of literature means so much to me. I translated it together with Geoffrey Smith in 1994 for the Harvill Press, and again in 2009, together with my wife Elizabeth and the American scholar Olga Meerson, for NYRB Classics. There were two reasons for retranslating it. First, the original text was never published in Platonov's lifetime, and the first posthumous publications – on which our Harvill translation was based – were severely bowdlerised. One crucial three-page passage, for example, is entirely missing.

Second, Platonov is hard to translate: in the early 1990s we were working in the dark. During the last 15 years, however, I have regularly attended Platonov seminars and conferences in Moscow and Petersburg. One indication of how deeply many Russian writers and critics admire him is the extent of their generosity to his translators; I now have a long list of people I can turn to for help. Above all, I have the good fortune to have my wife, who shares my love of Platonov, and the brilliant American scholar, Olga Meerson, as my closest collaborators. Olga was brought up in the Soviet Union; she has a fine ear, knows a great deal about Russian Orthodoxy, and has written an excellent book on Platonov. She has deepened my understanding of almost every sentence.

You've argued that Russians will eventually come to recognise Platonov as their greatest prose writer. Given that he's up against titans such as Gogol, Tolstoy and Chekhov this is quite a claim.

Well, it probably sounds less startling to Russians than it does to English and Americans. I've met a huge number of Russian writers and critics who look on Platonov as their greatest prose writer of the last century. In my personal judgment, it was confirmed for me during the last stages of my work on Russian Short Stories from Pushkin to Buida, an anthology of short stories I compiled for Penguin Classics. I worked on this for several years, did most of the translations myself and revised them many times. I read through the proofs with enjoyment – I was still happy with the choices I had made – but there were only two writers whom I was still able to read with real wonder: Pushkin and Platonov. Even at this late stage I was still able to find new and surprising perceptions in Pushkin's The Queen of Spades and Platonov's The Return. This didn't happen with any other writers. 
 
Readers who encounter Platonov for the first time are often struck by his surreality: in the Foundation Pit, for example, a bear staggers through a village denouncing kulaks [supposedly wealthy peasants]. But you've said that almost everything he writes is drawn from reality.

Platonov's stories work on many levels. When I first read his account of the kulaks being sent off down the river on a raft, I thought of it simply as weird. Then I realised that it's one of many examples of Platonov's way of literally realising a metaphor or political cliché; the official directive is to "liquidate" the peasants – and this unfamiliar word is interpreted as meaning that they must be got rid of by means of water.

Many years later I found out that this scene is also entirely realistic. The Siberian Viktor Astafiev wrote in his memoir: "In spring 1932 all the dispossessed kulaks were collected together, placed on rafts and floated off to Krasnoyarsk, and from there to Igarka. When they started loading the rafts, the whole village gathered together. Everyone wept; it was their own kith and kin who were leaving. One person was carrying mittens, another a bread roll, another a lump of sugar." Any educated Russian reading these lines today would at once imagine that they were written by Platonov.

As for the bear, he's drawn from many sources. He is the generally helpful but somewhat dangerous bear of Russian folk tales; he is a representative of the proletariat – strong but inarticulate. As a hammer in a forge, he is linked both to Stalin, whose name means "man of steel" and to Molotov, whose name means "hammerer". He is the tame bear often employed by a village sorcerer. Platonov's bear "denounces" kulaks by stopping outside a hut and roaring; in the late 1920s an ethnographer working in the province of Kaluga recorded the belief that "a clean home, outside which a bear stops of his own accord, not going in but refusing to budge – that home is an unhappy home". And one of Platonov's brothers has written that there really was a tame bear who worked in a local blacksmith's.

Platonov started off as a committed communist, but was appalled by collectivisation and the excesses of Stalinism. Uniquely – unlike others who adopted an oppositional stance, or wrote critiques for the desk drawer – he tried to negotiate a space within Soviet culture in which he could write honestly about what was going on. Is it fair to say that he failed?

I don't think so. Some of the stories he managed to publish – The River Potudan, The Third Son and The Return – are as great, in their more compact and classical way, as the novels he was unable to publish. The Return was viciously criticised, but it was published in a journal with a huge circulation and may well have been read by hundreds of thousands of people. And there is no knowing how important Platonov's example was to younger writers. Vasily Grossman, for example, was a close friend. They met frequently during Platonov's last years and read their work out loud to each other. Grossman gave the main speech at Platonov's funeral. His last stories are very Platonov-like. And Platonov's very last work – the moving, witty versions of Russian folk tales he composed after the war – was included, without acknowledgment, in millions of school textbooks. Platonov was not widely known, but he was widely read. Here again he is in a similar position to Grossman, whose words are carved in granite, in huge letters, on the Stalingrad war memorial, without acknowledgment of his authorship.

Platonov's language is often extremely intimate yet also strange: alienated and alienating. Is he exceptionally difficult to translate? And does he sound more "normal" in the original than in translation?

He is certainly difficult to translate. On the other hand, I've sometimes been surprised by how much of him evidently survives even in a poor translation. I've met people who have been deeply moved after first encountering him in a very poor translation indeed. As for your second question, you need to ask someone who is entirely bilingual and not involved in the work. All I can say myself is that all languages have norms that can be infringed, and that we do our best to infringe English norms just as Platonov infringes Russian norms. It is for you and other readers to judge how much we have succeeded!
 
Sometimes I think you have a secret plan to steer readers away from familiar authors such as Chekhov towards more angular, difficult work such as Platonov, thus reshaping perceptions of 20th-century Russian literature.
 
Well, I'd put it at least a little differently! I love Chekhov's stories as much as anyone, and would especially love to translate The Steppe and A Boring Story. But then Chekhov isn't so very easy or smooth either, though many of his complexities and contradictions are smoothed over in translation. What's certainly true is that I think we have a distorted view of Soviet literature. For many decades it was impossible for a Soviet writer to achieve fame in the west except through a major international scandal. This is what happened with both Pasternak and Solzhenitsyn. Both are important writers, but they are not greater writers than Grossman, Platonov and Shalamov.

Things are changing, however. Grossman is far better known in the west now than he was 10 years ago. Platonov is at least beginning to be noticed – Penelope Fitzgerald and John Berger are two of the English writers who have been quickest to realise his genius. And there is a chance that the Yale University Press will soon be commissioning a complete translation of Shalamov's Kolyma Tales. One more point: we have found it easier in the west to learn to appreciate the 20th-century writers who wrote from outside the Soviet experience. Bulgakov reached adulthood long before the revolution. He was never taken in by it; he looks down on everything Soviet. Grossman, Platonov and Shalamov, however, belong to a generation 10 to 20 years younger. All of them, at least for a while and to some degree, shared the hopes of the revolution. They write from inside the Soviet experience. This perhaps gives them a greater depth and complexity; their work contains no ready-made answers.

• Robert Chandler's new co-translation (in collaboration with his wife and Olga Meerson) of The Foundation Pit will be published in the UK by Vintage Classics later this year.   
 

Afterthoughts of a Worm Hunter and Collectible Spoons of the Third Reich lead the shortlist for this year's Diagram prize for the oddest book title

Afterthoughts of a Worm Hunter, David Crompton's musings on his career as a parasitologist, is emerging as early favourite to take the Bookseller magazine's prize for the oddest title of the year after landing a place on the shortlist this morning.

Crompton's competition for the annual Diagram award includes Collectible Spoons of the Third Reich, featuring spoons that "were once in the hands of the German history makers of the Third Reich era", children's book What Kind of Bean is this Chihuahua? and Governing Lethal Behavior in Autonomous Robots.

"Two other wormy tomes have made previous Diagram shortlists," said the prize's custodian Horace Bent. "New Guinea Tapeworms and Jewish Grandmothers made the 81 shortlist, while Earthworms of Ontario missed out to Reusing Old Graves in 95. Crompton's Worms could wriggle a win."

The shortlist is completed by The Changing World of Inflammatory Bowel Disease and Crocheting Adventures with Hyperbolic Planes, which informs its readers how to "crochet models of the hyperbolic plane, pseudosphere, and catenoid/helicoids", as well as exploring geometry and its historical connections with art, architecture, navigation and motion, and the history of crochet. It is the bestselling book on this year's shortlist, having sold 34 copies in the UK and 588 copies in the US, compared to zero for Afterthoughts of a Worm Hunter, according to the Bookseller.

"Selecting a shortlist proved a Herculean task, as numerous books carried titles that furrowed the brow – not least I Stopped Sucking My Thumb ... Why Can't You Stop Drinking?" said Bent. "However, the vast number of submissions has, in my humble opinion, created one of the most competitive shortlists in the history of the prize." The winner will be decided by public vote at www.thebookseller.com, and announced on 26 March.

The first ever winner of the prize, which was established in 1978, was Proceedings of the Second International Workshop on Nude Mice. Other triumphant titles include Versailles: The View From Sweden, The Theory of Lengthwise Rolling and How to Avoid Huge Ships.

Mystery donor presents 18th-century seducer's 3,700 pages of memoirs to French national library

For centuries they exerted the same knee-trembling pull on collectors and curators as their rakishly charming author had on the women of 18th-century Europe. But the international battle to pull off the ultimate literary conquest ended in Paris today as the French national library announced it had acquired the original manuscripts of Giacomo Casanova's memoirs.

In what is believed to be the most expensive manuscript sale ever, a mystery donor purchased the 3,700 yellowing pages on behalf of the Bibliothèque Nationale de France (BNF) for a price which has not been made been public but is believed to be in excess of €5m (£4.4m). The papers, transferred to the BNF on Monday in 13 protective boxes, are the uncensored, uncorrected basis of what went on to become the Venetian lothario's legendary Histoire de Ma Vie (Story of My Life).

The manuscripts, over which Casanova slaved in the years before his death in 1798, have been seen by a mere handful of experts, having been kept under lock and key for most of the past two centuries and considerably altered to form the versions widely available in print.

But they could soon be accessible to the general public. The BNF plans to digitalise them as part of its online library, and to display them in an exhibition next year.

"This is the most significant purchase the BNF has ever made in terms of monetary value," said Bruno Racine, the library's director, who has worked for the past two years to push through the deal in the utmost secrecy.

"Casanova's memoirs have become universally known but they have been censored … and even changed. These manuscripts are the authentic and definitive original."

Racine would not give any details about the identity of the anonymous sponsor that came forward with the €5.25m requested. According to French press reports, the donor was a business which had volunteered the funding when other sponsors pulled out due to the recession.

The sale was first mentioned to the BNF in the autumn of 2007, when Racine received a phone call from France's ambassador to Berlin informing him that a "mysterious emissary" wanted to talk to the library about the manuscripts. The contact turned out to be a representative of the prominent German publisher Brockhaus in whose possession the pages had been since 1821, when Casanova's family sold them.

Hubertus Brockhaus told the German news agency DPA he was "delighted" to transfer the work – written entirely in the author's second language of French – to the BNF.

"I wanted to make [the manuscript] accessible to the public," he said.

The tightly handwritten papers are justifiably famous for their gripping tales of seduction and skulduggery. But parts of the story of how they have come, over 200 years after their author died, to finally be shown to the public are themselves fitting for a text Racine describes as "mythical".

The pages survived the second world war bombing of Leipzig, during which the Brockhaus offices were destroyed. Having found the work miraculously saved on the floor of his cellar, Frederic-Arnold Brockhaus arranged for it to be smuggled out of the city in a US military truck in 1945.

Due to the nature of the content, which details conquests of over 100 women, possibly several men and at least one nun, the originals fell victim to a series of alterations by over-zealous editors and it was not until 1960 that a full version was printed in French.

Words of love

"One morning she came to my bed bringing me a pair of white stockings she had knitted herself; after dressing my hair, she told me that she had to try them on me to see if she had made any mistakes … The Doctor had gone to say his mass. Putting on the stockings, she said that my thighs were dirty and at once began washing them without asking my leave … Bettina carried her zeal for cleanliness too far, and her curiosity aroused a voluptuous feeling in me which did not cease until it could become greater. Thus calmed, it occurred to me that I had committed a crime and that I should ask her forgiveness …

"The company of this angel made me suffer the pains of hell. Though constantly tempted to deluge her face with kisses … I scrupulously avoided taking her hands; for me to have given her a single kiss would have blown my edifice sky-high, for I felt that I had become as inflammable as straw. When she left I was astonished that I had won the victory; but, my appetite for laurels being insatiable, I could scarcely wait for the next morning to come so that I might renew the sweet and perilous battle. It is shallow desires which make a young man bold; strong desires confound him."

Jan Petersen's Our Street, a million seller in its time, to reappear in publisher's print-on-demand series

A manuscript which was smuggled out of Nazi Germany in a cake is being brought back into print by Faber & Faber.

Jan Petersen's Our Street tells the true story of left-wing resistance in the fascist Germany of the 1930s. Set on Wallstrasse, in the Charlottenburg district of Berlin during the period from just before Hitler became chancellor to the early days of Nazi government, the book – a fictionalised version of events to avoid reprisals for those still working in the underground in Germany – tells of the violence exacted on the street's inhabitants in revenge for the killing of a stormtrooper. It opens by printing the names of 18 victims, "The Charlottenburg Death List".

Petersen, a communist whose name was twice put on the Gestapo's black list, finished the manuscript in 1934 and made two copies, sending one to Hamburg where it was to be taken to England by a German soldier but was eventually thrown into the sea to avoid last-minute detection. Friends tried to smuggle a second to Czechoslovakia, but months went by with no word, so Petersen decided to take the third – and final – copy to Prague himself. He baked the manuscript into two cakes and, dressed in skiing clothes to give the impression he was going on holiday, he smuggled it past the SS guards.

"Well, you know what women are, don't you? I told my wife I was only going away for three days, but she would go and bake me two whopping big cakes," Petersen told the customs guards. "It'll take me a week to eat one. Just look at the size of them." And they let him through.

An English translation of Our Street was published in 1938 by Gollancz's Left Book Club, and Faber is now bringing it back into print through its print-on-demand Faber Finds imprint. "For me the most striking thing about the book is its courage, both in its writing and its publishing," said editor John Seaton.

Petersen's daughter Bienchen Ohly said the story of her father baking the book into a cake was "absolutely true". "My father was very sporty so was convincing in the role of hiker. He was also very charming and could win anyone over to his point of view," she said. She called the book "a very important historical document with very strong humanitarian values".

"Despite all the tragedies and loss of life and horrors, it has a very optimistic core belief in the goodness in man," she said. "I have always thought it a unique description from an insider's perspective of resistance efforts in Germany. Too many people think the whole German population was in agreement with Hitler, which is not true at all. Also, so many young people today have no concept of the hardships and bravery shown by many people who were prepared to risk their own lives to prevent the rise of fascism – not just in Germany, but worldwide."

Petersen emigrated to Switzerland, France and then England during the late 1930s and was deprived of his German citizenship, but returned to East Germany after the war where he won a number of literary prizes. He would have been "extremely pleased" that Our Street was being republished, Ohly said, and that it would be reaching "a whole new young generation who perhaps do not sometimes have the same moral values of his generation".

"When the book was published in his lifetime it sold more than a million copies worldwide," she said. "I think he would hope to have a wide new readership of people who would make life better for others as a result of reading his book."

Terence O'Neill, whose brother's killing provided a plot for Agatha Christie, has won a publishing deal for a memoir about the events

The brother of the boy whose brutal death inspired Agatha Christie to write The Mousetrap has landed a book deal to tell his own story.

The plot of Christie's theatre play – now, in its 59th year, the longest-running in the world – sees police detective Sergeant Trotter arrive at a guest house on skis to inform the snowed-in residents that a killer may be on their way. A woman who abused children in a nearby house – one of whom died before the courts removed the other children – has recently been murdered, and a note left on her body implied further deaths related to the child abuse would follow.

Terence O'Neill was 10 years old when he testified in court against his foster parents Reginald and Esther Gough, who were accused of the manslaughter of his brother Dennis. The pair had been through many foster homes before they ended up on the Goughs' Shropshire farm in 1945, where they were beaten regularly until one morning, Terence couldn't wake his brother up. A post-mortem examination found that Dennis had been starved for months and weighed only four stone.

The public outcry over the brothers' treatment led to the first provisions to protect other vulnerable children from neglect and cruelty, and provided the basis for Christie's radio play Three Blind Mice, which later became the theatre play The Mousetrap.

Last year, O'Neill – who is now in his 70s – decided to tell the story of his experiences himself, posting his writing on HarperCollins's online slush pile Authonomy, which gives unpublished writers the opportunity to show their work to an online community of readers. His writing was spotted by HarperCollins editors Carole Tonkinson and Kate Latham, and is set for publication as Someone to Love Us next month.

"We're very excited to be publishing it," said Laura Summers at HarperCollins. "We have published a few novels picked up on Authonomy which have gone on to be bestsellers, and we hope the same thing will happen here. Terry the author is so thrilled it's being published."

Geoffrey Hill and Anne Stevenson are among the names being suggested, as hunt for a successor to Ruth Padel begins

The names of eminent poets including Geoffrey Hill and Anne Stevenson are being suggested as potential candidates for the Oxford professor of poetry post, as nominations open today to find a successor to last year's controversial winner Ruth Padel.

Padel was elected in May by 297 votes to Indian poet and critic Arvind Mehrotra's 129, but resigned less than two weeks later after admitting passing on material to journalists alerting them to claims of sexual harassment which had been made against her rival for the post, the Nobel laureate Derek Walcott. Walcott had earlier pulled out of the race.

The scandal prompted Oxford University to change the voting system for the election, which had previously only allowed Oxford graduates to vote in person at the university on a single day. Now, graduates will be able to vote online, as well as to cast their vote in person over a period of time.

The university opened nominations today to find a new candidate for the 300-year-old position, seen as the most prestigious in poetry behind that of the poet laureate. Aspiring professors of poetry must be nominated by at least 12 Oxford graduates by 5 May. If there is more than one contender, then graduates will be able to vote for their favourite.

Former poet laureate Andrew Motion, whose name was put forward as a possible candidate, ruled himself out of the running, as did poet, critic and author Blake Morrison. "Geoffrey Hill would get my vote, if he can be persuaded to run," said Motion. Morrison agreed. "I've not written enough poetry in recent years to remotely consider it [but] Geoffrey Hill, Andrew Motion and Lachlan MacKinnon are all possible – and all would be good," he said.

Stevenson – winner of the Lannan prize for a lifetime's achievement in poetry – is also seen as a strong candidate, although the poet said she felt that "the whole notion of campaigning like a politician seems to me only to cheapen the post, as recent events have sadly shown".

"I have always (probably naively) assumed that the professorship of poetry at Oxford was an honour that a poet was asked to accept," said Stevenson. "If still so, having practised the craft of poetry for more than half a century, I would be pleased to be asked, and I would certainly accept the post, grateful for an opportunity to illustrate and affirm the literary values to which I have given my creative life. I don't, though, cheer myself with great expectations."

Other names being considered by the poetry establishment include the Scottish poet and critic Douglas Dunn, and the Pulitzer prize-winning American poet Jorie Graham. Broadcaster and writer Clive James said in Standpoint magazine last July that he "would rather throw himself off a cliff" than take the job, although he admitted later in the article that he did find the role "just about the most attractive cup of its kind in existence". "The botched election might have made it a poisoned chalice, but what a chalice it is," James wrote.

Mehrotra, who missed out to Padel in the last election, said late last year that he was unlikely to run for a second time. "Really, running for the professorship once seems like enough," he told the Guardian at the time. "There was a point to be made – which is that occasionally, say once every five years, Oxford ought to look to points east and south for its poetry professor – and I think it has. There's also the question of the Indian media. Alas, these things cannot be done quietly. The last time the media here got so excited about the whole affair – I mean, it's Oxford, and then it's the first Indian running for the job – that every journalist and her grandpa wanted a quote. In the end, the cacophony became too much. A lot of people I meet though think all the noise was about some big prize I'd won and even now congratulate me on it."

Mehrotra is in the company of a host of distinguished unsuccessful candidates for the position, including CS Lewis, FR Leavis, Robert Lowell and Stephen Spender. The professorship, established in 1707 and coming with a stipend of £7,000, has been held in the past by poets including Matthew Arnold, WH Auden, Robert Graves, Paul Muldoon and Seamus Heaney. Literary critic Christopher Ricks held the post for five years, stepping down in 2009.

"We want someone who's going to do what Christopher did - to enthuse everyone about poetry, not just literary students but the wider public," said Dr Seamus Perry, deputy chair of Oxford's English faculty board. Although there are no official nominations as yet, Perry said this could be a "very distinguished poet" such as Geoffrey Hill ("considered by many to be the greatest poet in England and also a very fine critic"), the current poet laureate Carol Ann Duffy ("there's nothing to stop her running"), JH Prynne, John Fuller, Jorie Graham, Simon Armitage or Alice Oswald. But it could also be a critic, he said, or even a novelist such as AS Byatt, who "has written wonderfully about Browning".

Perry hoped last year's events wouldn't put any candidates off from running. "It was all a bit of a kerfuffle and a regret, but I don't think it actually reflects on the post at all," he said. "I think it emerged pretty much unscathed, and in a funny kind of way it drew attention to its distinction."

If more than one candidate comes forward for the position, the winner will be announced once voting is completed on 18 June, to be in post for the new academic year in autumn 2010.

Friend of Willy the Wizard author Adrian Jacobs says addition of JK Rowling to suit raises possibility of multi-jurisdiction action

Publishers could face legal action worldwide over claims that JK Rowling stole ideas for Harry Potter from a British author's book called The Adventures of Willy the Wizard.

The estate of the late Adrian Jacobs yesterday added Rowling as a defendant in a case originally filed in June against Bloomsbury Publishing, Potter's UK publisher, for alleged copyright infringement.

Max Markson, a PR executive representing the estate, told the Guardian the addition of Rowling's name to the action opened up the possibility of multi-jurisdiction action.

"We believe that she [Rowling] personally plagiarised the Willy the Wizard book. All of Willy the Wizard is in the Goblet of Fire. We now have a case which is not just against Bloomsbury."

Markson, who was a friend of Jacobs, said Rowling was added to the lawsuit after it was learned that the statute of limitations to sue her had not run out as previously thought. She is named in the suit under her married name of Joanne Kathleen Murray.

"I estimate it's a billion-dollar case," Markson said. "That'll be the decision of the courts, obviously."

Rowling denies the claims. "I am saddened that yet another claim has been made that I have taken material from another source to write Harry. The fact is I had never heard of the author or the book before the first accusation by those connected to the author's estate in 2004; I have certainly never read the book," she said in a statement.

"The claims that are made are not only unfounded but absurd and I am disappointed that I, and my UK publisher Bloomsbury, are put in a position to have to defend ourselves. We will be applying to the court immediately for a ruling that the claim is without merit and should therefore be dismissed without delay."

The suit claims Rowling's book Harry Potter and the Goblet of Fire copied substantial parts of Jacobs' 36-page book The Adventures of Willy the Wizard – No 1 Livid Land. The plagiarism claims stem from both Willy and Harry being required to solve a task as part of a contest, which they achieve in a bathroom assisted by clues from helpers.

Jacobs' estate also claims that many other ideas from Willy the Wizard were copied into the Potter books. Goblet of Fire was the fourth book in Rowling's series and was published in July 2000. No 1 Livid Land was published in 1987.

According to Markson, Jacobs had sought the services of the literary agent Christopher Little, who later became Rowling's agent.

Jacobs was a solicitor and accountant who lost heavily in the 1987 stock market crash. He suffered a stroke soon after and was bankrupted for a second time in 1991. He died in a London hospice in 1997, Markson said.

"Willy The Wizard is a very insubstantial booklet running to 36 pages which had very limited distribution. The central character of Willy The Wizard is not a young wizard and the book does not revolve around a wizard school," Bloomsbury said last year.

"The claim was unable to identify any text in the Harry Potter books which was said to copy Willy the Wizard."

Markson said the plagiarism allegation concerned the story plot rather than the words. The Jacobs estate is seeking legal advice on whether the Harry Potter films and soon-to-be-opened Harry Potter theme park breach copyright law.

In 2007 Rowling and Warner Bros, which made the Potter films, sued the publishers of the Harry Potter Lexicon, an encyclopaedia of the series. A shortened and modified version of the lexicon was published last year.

Human rights court rules that censorship of 1907 erotic novel The Eleven Thousand Rods 'hindered public access to a work belonging to the European literary heritage'

Turkey violated freedom of expression laws and prevented access to Europe's literary heritage when it banned Guillaume Apollinaire's classic French erotic novel The Eleven Thousand Rods, the European court of human rights ruled yesterday.

The court found in favour of Turkish publisher Rahmi Akdaş, who complained to it after he was convicted under the Turkish criminal code "for publishing obscene or immoral material liable to arouse and exploit sexual desire among the population" when he released a Turkish translation of Les onze milles verges (The Eleven Thousand Rods) in 1999. The book details the erotic adventures of the debauched Romanian aristocrat Mony Vibescu and his fellow sybarites, containing graphic scenes of intercourse, sadomasochism, paedophilia, necrophilia, coprophilia and vampirism. It was banned in France until 1970 and Apollinaire himself never claimed authorship, fearing prosecution under France's public obscenity statute.

Akdaş had argued that the book was fiction, that it used techniques such as exaggeration and metaphor, that it contained no violent overtones "and that the humorous and exaggerated nature of the text was more likely to extinguish sexual desire", but the Turkish courts ordered the destruction of all copies of the book and fined the publisher approximately €1,100. An appeals court later quashed the destruction order, but upheld the conviction.

Akdaş subsequently complained to the European Court of Human Rights, saying the ruling violated Article 10 (freedom of expression) of the European Convention on Human Rights. The Strasbourg-based court ruled yesterday that although states can interfere to protect morals, Turkey was wrong to do so in this case as more than a century had elapsed since Les onze milles verges was published. The erotic novel had also been released in many different languages in a number of countries, and had gained literary acclaim, it said, so its ban and Akdaş's conviction "hindered public access to a work belonging to the European literary heritage".

"The heavy fine imposed and the seizure of copies of the book had not been proportionate to the legitimate aim pursued and had thus not been necessary in a democratic society, within the meaning of Article 10. There had therefore been a violation of that provision," the ruling said.

But bestselling fantasy author insists that he is not finished yet, and urges impatient fans to 'calm down'

Bestselling epic fantasy author George RR Martin has sent his fans into a frenzy after announcing that he has written 1,261 pages of the long-awaited fifth volume in his Song of Ice and Fire series.

It is five years since Martin published the fourth novel in the epic series, set in a medieval-esque world where a wall of ice protects the land of Westeros, and the wait for A Dance with Dragons has prompted extraordinary levels of anticipation from readers. Some have even created a poster mimicking the Conservative party's new campaign, which shows David Cameron promising "vote for us and we'll ensure A Dance with Dragons is released in 2010". "I seem to have become a Tory campaign issue," wrote Martin in a message to his publisher.

Fans' vocal impatience for the next instalment reached such a pitch last year that Martin issued an angry statement to stem a rising tide of anger. "Some of you are angry about the miniatures, the swords, the resin busts, the games. You don't want me 'wasting time' on those, or talking about them here. Some of you are angry that I watch football during the fall," the author wrote. "Some of you don't want me attending conventions, teaching workshops, touring and doing promo ... After all, as some of you like to point out in your emails, I am 60 years old and fat, and you don't want me to 'pull a Robert Jordan' on you and deny you your book. OK, I've got the message. You don't want me doing anything except A Song of Ice and Fire. Ever. (Well, maybe it's OK if I take a leak once in a while?)"

Now, the author described as "the American Tolkien" by Time magazine has poked his head above the parapet again, telling fans that A Dance with Dragons has now become the second longest volume in his series, that he's been knocking off chapters and having "good, productive" days of writing. Huge excitement ensued from readers, but Martin attempted to douse the flames. "Jeez, guys. Calm down," he said on his blog. "This is why I hate to do updates. I say I have good day, and immediately I have 100 people deciding this means that Dance is finished. I'm not the oracle at Delphi ... When I finish Dance, you'll know it. I will write something like this: 'I have finished A Dance with Dragons.' You won't need to parse any hints."

But he failed to calm some readers down. "Where's the book, Mr Martin, where's the book? :) Here in Russia we are waiting, and waiting, and waiting," wrote one of the more eager fans.

Martin is not the only author to come under pressure from avid readers – and the fantasy genre appears to provoke particularly insistent readers. In his memoir On Writing, Stephen King related how he had been contacted by a sick old woman desperate to find out how his Dark Tower fantasy series ended before she died. Might King share his ending with her? The author said he had to tell her that he himself didn't know what was going to happen in the series, which eventually spanned seven books. The Robert Jordan referred to by Martin, meanwhile, is the author of the epic Wheel of Time fantasy series. He died after writing 11 Wheel of Time books, with the series currently being finished by author Brandon Sanderson.

Edmund White on a memoir that captures all the elements that made New York in the 1970s so exciting

Patti Smith has a mythic imagination. As a young, desperately poor poet from southern New Jersey, she headed to New York to seek her fortune, nothing in her purse. Her mother had assumed she would follow her into waitressing. But Patti, though practical and a survivor, had her sights set not on slinging hash but on searching for immortality and beauty and magic. She already recognised a divine succession of poets – Blake, Baudelaire, Rimbaud, Genet and the Beats – and she wanted to join them. She was creative and liked to write, read and draw. Eventually, she became the renaissance woman of the punks, a great rock singer and composer – but before that she had to fashion her look, her personality and her verse.

And survive. She had no real friends when she arrived in New York, just a few names, and no job prospects. But it was July 1967, she was not yet 21, and other drifters and hippies helped her find food and shelter. Eventually, she got a job working in a bookshop, she met Robert Mapplethorpe, who was the same age and just as poor, and they took a Brooklyn apartment together. They each collected little ­talismanic objects and set great store by the way they dressed; both had an ­innate and highly original sense of ­personal style. And he was fiercely ­ambitious and coveted artistic success.

In her careful, sometimes painful self-sculpting, Smith had found an inspired and equally determined collaborator in Mapplethorpe. As she says in this memoir, which is so full of memorable sentences: "We were both praying for Robert's soul, he to sell it and I to save it." (Robert's theme song was the Stones' "Sympathy for the Devil".)

Patti and Robert were both born in 1946 and both were raised by poor parents, she in Germantown, Pennsylvania and then New Jersey, he by a Catholic family on Long Island. Like all ­lovers, they told endless ­stories to each other about their childhoods: "We used to laugh at our small selves, saying that I was a bad girl ­trying to be good and that he was a good boy trying to be bad." They both succeeded. As a child he'd been a ­mama's boy and had made necklaces for his mother, but later, as an adult, he identified himself in the public mind through his photographs with pain and blood and exotic sexual ­practices, and even with something as seemingly transgressive (but actually innocent) as pictures of child ­nudity. She had held factory jobs in New Jersey, where the other workers accused her of being a communist ­because she was reading a bilingual edition of Rimbaud's Illuminations. She'd given birth out of wedlock, as we used to say, to a child she'd had to put up for adoption. Later, when she lived with Mapplethorpe in Brooklyn, she turned herself into a ­disciplined poet and breadwinner. For a long spell she supported the skinny, charismatic Mapplethorpe, who at the time was making "altars" of found objects somewhat in the manner of the American surrealist Joseph Cornell. He discovered photography only later, but once he settled on it as a career he was tenacious and highly tactical in plotting his rise in the world.

She obviously has a great gift for ­appreciation, though in her case that should not imply a lack of discrimination. It seems that from the very beginning she was alert to influences that would help her to explore and to firm up her peculiar sensibility, which was at once edgy and lyrical, both demotic and hieratic. She was more relaxed about their ability to survive; Robert was much more ­anxious about money. She was primarily interested in sniffing out people with talent, not as a careerist but ­always out of respect for their artistry. Mapplethorpe had his eye on the main chance.

In those days, before the internet and Google, it was difficult for ­working people to put their hands on books and information. All these years later, Smith still remembers the few art books she possessed and that she would consult again and again, just as she remembers their few records and books of verse. And she recalls in vivid detail her first encounters with William Burroughs and Allen Ginsberg, saints in her pantheon of great artists. For her, many sites in New York were ­sacred: "It was exciting just to stand in front of the hallowed ground of Birdland that had been blessed by John Coltrane, or the Five Spot on St Mark's Place where Billie Holiday used to sing, where Eric Dolphy and Ornette Coleman opened the field of jazz like human can openers." Robert's idols were visual artists, though very cerebral ones – ­Duchamp and Warhol. Patti indulged in long introspective bouts, but she learned from Robert just to get on with it and forge ahead in her work – a trajectory that for her was always God-centred, doing drawings "that magnified His motion".

While I was working on my biography of Jean Genet in the late 1980s and early 1990s in Paris, I would receive phone calls from Patti in Detroit. I'd never met her and was introduced to her only many years later in front of the Metropolitan Opera in New York, but she was calling me to encourage me to persevere and to finish this ­onerous seven-year task. She knew how ­devoted I had been to Mapplethorpe, with whom I'd collaborated on journalistic stories about Truman Capote and William Burroughs (I also wrote an ­essay for one of his first ­gallery shows, in Amsterdam). She told me that she and Robert (who had recently died of Aids) used to read Genet out loud to each other when they lived together. When my bio­graphy finally came out, Patti was staging her big comeback with a free concert in New York's Central Park; she told her audience that they must all go out to buy my Genet book.

This genuine devotion to her private artistic saints and to her old friends characterises the entire book. It is her own Lives of the Saints, and it is thoroughly imbued with faith in her own artistic mission.

Her love affair with Mapplethorpe, to be sure, had its painful moments, ­especially as they were both discovering that he was gay. Although they had been sexually intimate for several years, he began to pick up extra money as a rent boy. Jim Carroll, a friend who went on to be a punk musician and the author of the autobiographical The Basketball Diaries, was also hustling, in his case to support his heroin habit. When Mapplethorpe asked him how he could be certain he wasn't gay, ­Carroll said he'd never done it without being paid – which was not the case with Mapplethorpe. Before long, Robert had a handsome young lover and eventually a much older and even more handsome lover, Sam Wagstaff, a rich art collector who launched his career.

What Patti found even more difficult to accept than Robert's homosexuality was his social ascent. She could understand his love for men, but in order for her to spend time with his new, rich friends she would have had to change her ways.

Just Kids should interest any reader who wants to know how an artistic ­career can be launched. Smith gave a carefully staged and prepared poetry reading at St Mark's in New York that won her lots of attention and publication – and even the offer of a record contract. She began to work as a music journalist for Crawdaddy! and Rolling Stone. She begged the editor of Rolling Stone to let her write a piece on Lotte Lenya, Kurt Weill's wife, muse and ­favourite singer; when Patti handed in her article, the editor said "that although I talked like a truck driver, I had written an elegant piece". She had an affair with Carroll and with Sam Shepard, with whom she wrote a play.

Her transition to musician seems, in this account, to have been disconcertingly easy. She bought a guitar and soon knew how to play it. She turned some of her poems into songs. She put together a band – and before long she was a megastar touring the world. Mapplethorpe produced a portrait of her that undoubtedly helped to ­cement her image; with her gift for phrase-making, Patti writes: "Robert was ­concerned with how to make the ­photograph, and I with how to be the photograph." Suddenly, Robert was showing photos in galleries attended by "a ­perfect New York City mix of leather boys, drag queens, socialites, rock and roll kids and art collectors".

Like that art opening, this book brings together all the elements that made New York so exciting in the 1970s – the danger and poverty, the artistic seriousness and optimism, the sense that one was still connected to a whole history of great artists in the past. This was a small community that was carefully observed by the media; it also flourished at the moment when New York was becoming the cultural capital of the western world.

Edmund White's latest book is City Boy: My Life in New York During the 1960s and '70s.

This superb account of last year's crash-landing in New York lays bare the truth behind the pilot and his plane, writes Geoff Dyer

Since this is a book about quick ­reactions, it seems appropriate to ask: after Captain Chesley Sullenberger ditched his Airbus A320 in the icy waters of the Hudson river on 15 January 2009, how long did it take William Langewiesche to decide that the events of that day would be the subject of his next book?

His last, The Atomic Bazaar, delved into the murky trade in secondhand nuclear materials. Before that, The ­Outlaw Sea covered ship-breaking in Bangladesh, the sinking of a passenger ferry and contemporary piracy (an instance of a reporter being a little too ahead of the curve and getting to a story before it became news). These were subjects that interested and came gradually to absorb him. But Langewiesche, I wager, knew within seconds that the story of Airbus A320 had his name ­written all over it.

His father was a pilot who, in 1944, published Stick and Rudder, a benchmark book on flight and navigation. Not surprisingly, William grew up feeling at home in the sky; he trained and worked as a pilot and, in turn, assembled a book of his own aviation pieces. First published in 1998, Inside the Sky combines meditations on his adventures as a pilot and meticulously investigated reports on air crashes. (The essays in that book form the core of an updated and expanded collection, Aloft, to be published as a Penguin Modern ­Classic later this month.) So Langewiesche was uniquely qualified to write about the Hudson ditching. Beyond the realm of sympathetic expertise, however, was a subtler recognition of the subject's broader themes.

Sully's successful crash-landing – not a single life lost – was a symbolic reversal of the catastrophe of 9/11. In ­American Ground (2003), Langewiesche had tracked what he called "the unbuilding of the World Trade Center", the long operation to recover bodies and clear up the devastation at Ground Zero. Although full of praise for the people "without previous rank who discovered balance and ability within themselves" during the excavation, he was, nevertheless, rigorously unsentimental. At a time when the moral authority of the New York City Fire Department was unimpeachable, Langewiesche was pilloried for unearthing incontestable evidence – the cab of a wrecked fire truck, for example, "filled with dozens of new pairs of jeans from the Gap"– of looting by some firemen.

Demonstrating a similar willingness to confront "a complex and emotional subject", he is just as adamant about rescuing "the Miracle on the Hudson" from the realm of instant myth. The plane lost both engines to a low-altitude strike by Canada geese minutes after take-off from New York's ­LaGuardia airport. The crew responded ­perfectly, especially Sullenberger, who ­"ruthlessly shed distractions, including his own fear of death" as he urged the stricken plane through what remained of the Manhattan sky and on to the ribbon of water. Such grace under pressure is in keeping with our ideal of the pilot, as famously expressed by Saint-Exupéry's flights of Nietzschean lyricism, or, more recently, by the pragmatic drawl of Chuck Yeager – "I've tried A! – I've tried B! – I've tried C!..." – in Tom Wolfe's The Right Stuff. Part of the allure of the Hudson ditching, Langewiesche points out, is that it tempted us to forget that the days of the pilot locked in a lone struggle with ­destiny are pretty much gone.

As he repeatedly insists, it's not just the passengers crammed into economy who are fed up with flying, resigned to the "terrible this-is-my-life monotony of the job"; the pilots themselves are part of "a once-proud profession" in "inexorable decline". This is partly the drearily familiar saga of demoralisation in the wake of deregulation, wage cuts, longer shifts and so on, but it's also a result of the way that, as more and more control has been ceded to computers, pilots have come to resemble frontline passengers, only in a state of permanent, highly knowledgable alertness. The Hudson landing appeared to restore the lofty agency of the pilot to its rightful place, when, in fact – Langewiesche insists – Sully, for all his skill, had the good fortune to be flying a plane of "radical semi-robotic European design". This is the meaning of "flying by wire": the aircraft's built-in safety systems are designed always to overrule and correct the pilot, never vice-versa. Not a bad idea since, in the past dozen or so years, "far more passengers have been killed because of pilots than because of airplane failures".

A pathological example of the former – the extraordinary story is recounted in Aloft – is the Egyptian pilot who, in 1999, declared: "I rely on God" while directing his plane into the sea. Had he been flying an ­Airbus, he would not have been able to do that (and so, paradoxically, his faith might have been justified). A shriller note of fictive resignation is sounded in Don DeLillo's White Noise when a plane goes into a steep dive. Relieved to hear the captain's voice over the intercom, passengers are treated instead to a shrieking announcement of impending doom: "We're a silver gleaming death machine!"

Sully and his co-pilot, by contrast, kept their cool, kept flying – that's the thing about airborne problems, "you cannot just pull over and stop" – and started weighing up their options. He negotiated with air traffic control, decided the Hudson was his best bet, found time to warn ("This is the captain. Brace for impact"), and concentrated on easing the maimed aircraft into the concrete-hard water. Which he did, impeccably. But even if he hadn't, the plane would have overruled him, and did, in fact, iron out a few wrinkles in the glide by "constantly adjusting the control surfaces to provide an extraordinarily stable platform to fly". Metaphorically speaking, Sully won the spelling bee – but was wired up to Microsoft spell check just in case. In Langewiesche's words, the plane "participated actively in the survival of the passengers".

Much of Fly by Wire is rigorously technical but Langewiesche shares with Len Deighton the ability to imbue aeronautical specs and data with a narrative propulsion of their own. Unlike Sully, he is under no compulsion to shed distractions. On the contrary. There is a mass of information about the nesting and migratory habits of the much-maligned Canada geese who did for the plane and themselves (he even consults an expert who speculates on what might have been going through the minds of the geese as the giant Airbus roared into their patch of sky); on the long history of bird strikes (some of which have occurred at a mind-blowing 37,000ft – how do they breathe up there?); and the demographic make-up of the passengers onboard. At times such unwieldy cargo makes this short book handle a little awkwardly but for most of the ride the method established in Aloft – a writing-by-wire combination of interviews, black-box data, cockpit voice recordings and reporterly nous – winds the reader into a state of white-knuckle absorption. It's a tribute to Langewiesche's narrative command that, although the outcome is known, the penultimate section of the book, as the plane approaches the slim hope of the river, vies in brilliance with James Salter's account of two fighter pilots in Korea nursing their fuel-less jets back to base in his Korean war novel, The Hunters.

The focus, inevitably, is on pilot, crew and air traffic controllers but the passengers and, later, a passing ferry play their part, too. The plane is about to hit the water. A passenger near the back shouts: "Exit row people get ready!" Once the plane is in the water there's a bit of a devil-take-the-hindmost stampede but the standard of behaviour remains surprisingly high. Sully is the last person out of his aircraft. It's a wonderful story expertly told, and the ending is not just happy but uplifting: almost everyone involved comes out of it not just safely but extremely well. Except the geese. Their day totally sucked.

Helon Habila is moved by Chinua Achebe's collection of essays

Many years ago, Chinua Achebe and other writers were invited to a symposium to commemorate one millennium of the city of Dublin; the theme for their presentations was "Literature as celebration". Most of the participants, Achebe writes, couldn't easily make the link between literature and celebration. Achebe, on the other hand, found a ready parallel in his Igbo culture's ­ritual of mbari, which he describes as "a celebration, through art, of the world and of the life lived in it".

In a way this collection of essays could be viewed as a celebration of Achebe's world, and the almost 80 years he has lived in it. As his new collection shows, this world is large and all-encompassing – his essays range from the political to the historical to the personal, yet they are all projected through an intimate, biographical lens, thus making each a milestone on his long journey on this earth. They were written at different times, the earliest in 1988, the latest in 2009. They range from the author's childhood in Ogidi village in south-eastern Nigeria, to his education in Ibadan, to fame as a writer, to exile and family. And though most of them cover territories the author's followers are familiar with from other collections, such as Home and Exile, or Hopes and Impediments, it is a mark of Achebe's genius as a narrator that one could hear him many times on the same subject and never grow bored – a reminder that in the art of the storyteller, it is not content alone that matters, it is also the performance, the presentation and the passion.

In the title essay, which serves as a sort of introduction to the collection, Achebe makes it clear that this is not a scholarly work, explaining that he missed his chance to be a scholar when, 40 years ago, "Trinity College, Cambridge turned down my application to study there after I took my first degree at the new University College, Ibadan." At first this may appear to be a lowering of the readers' expectation out of modesty – but then, later, he tells us: "I am not a very modest person." And it becomes clear as one reads that Achebe is simply educating the reader on how to read these essays.

This may not be a scholarly work, but what it lacks in scholasticism, it more than makes up for in wisdom and passion, as well as those rare and often overlooked attributes of great literature, clarity and consistency of vision. And for that, we cannot thank Trinity College, Cambridge enough. But of course the essays are not only wise, detached and reflective, but also combative. In "Politics and Politicians of Language in African Literature", Achebe calls out his erstwhile sparring partner, Ngugi wa Thiong'o, for playing politics with Africa's history by over-simplifying the language question: "The difference between Ngugi and myself on the issue of indigenous or European languages for African writers is that while Ngugi now believes it is either/or, I have always thought it was both." In the same essay we are taken to Makerere University in 1962, to the first conference of African writers, and we are casually shown such participants as Langston Hughes and James Baldwin from America, Wole Soyinka, Christopher Okigbo and Léopold Senghor, all under the same roof. There was also the elusive Obi Wali, who was the first to declare in his famous essay "The Dead end of African Literature?" that African experience can never be captured in European languages; but then, Achebe writes, Obi Wali, "having made his famous intervention, like a politician, simply dropped out of sight".

Achebe's world view can best be described as tragic realist: at its best it is optimistic that though the world may not be perfect, yet we have a duty to rise to our best, and to do that, we have to acknowledge others, to realise we can never be human if we deny ­others humanity. Or, in the words of the Bantu wisdom which he quotes many times in this collection: "A human is human because of other humans." He acknowledges his colonial legacy, how it has moulded and changed his world, in good and bad ways, but his greatest quarrel with colonialism was its inability to see the colonised, to accept them as human; its willingness to falsify and deny others' history in order to justify its mission.

At its bleakest the tragic realist ­vision can grow dim, even bitter, as in the essay "Travelling White" when, in 1960, he finds himself on a travelling fellowship in segregated east Africa, and is asked to go to the back of the bus, which he refuses to do; or in essays such as "Stanley Diamond", where he reflects on the lost ideal that was Biafra. What is in evidence throughout, though, is Achebe's consistency of ­vision and the thoughtfulness he brings to every subject.

Helon Habila's Measuring Time is published by Penguin.

Veronica Horwell on the pioneers who made a global business out of female self-presentation

Madeleine Marsh has compiled a modest guide for collectors of the accoutrements of appearances, published by a geek company that also does A Historical Dictionary of Railways and An Illustrated History of Anchors. She knows what her readers will want: dates, anecdotes, very specific pictures for identification. Yet of these three accounts of the way we looked, look and want to look, hers is the only one that truly connects with the transformational possibilities of artifice, ostensibly also the basis for Carol Dyhouse's work on a century of female self-presentation in Britain and for Geoffrey Jones's tome, which adds up to $330bn annual world expenditure on cosmetics and toiletries.

Jones notes in Beauty Imagined that Ernest Wertheimer, an Alsatian Jew, went into partnership in Paris in 1898 with a food trader who had bought a radical stage make-up firm called Bourjois, just as overt face paint was beginning to transfer from theatrical to daily use and perfumes were expanding from discreet essences to complex, partially synthetic mixes. Wertheimer understood perfume (his company later manufactured and owned Gabrielle Chanel's No 5), and was so inspired a businessman that he loaned Galeries Lafayette the money to rebuild its department store the more prominently to feature Bourjois products. Thirty years later, the Wertheimer family had set up a US Bourjois company, too, and commissioned Ernest Beaux, who had created No 5, to blend a scent for the American middle market. It smelt, alas too transiently, of violets, rose and lilac; Bourjois decanted the amber liquid into what became a famous art deco object, a cobalt blue bottle with silver label, the Wertheimer racing colours. They called it Evening in Paris, because Bourjois already sponsored a US radio show by that name, to promote its rouge. The Wertheimer saga went on and on (they came to own all Chanel, not merely the perfumes; Karl Lagerfeld is on their payroll), but Bourjois faded and Evening in Paris was deleted in the late 1960s.

Thirty years of glamour, though: how did they do that? Jones doesn't ­inquire, but all is revealed on page 239 of Marsh's Compacts and Cosmetics with a gorgeous portrait of a 1947 "mint and sealed" Evening in Paris powder box. The lid is deep midnight blue, with metallic stars and tiny ­figures – café table, artist and model, horse-drawn carriage. The mutual ­romance between Americans and Paris, from Edith Wharton to Gene Kelly, is encapsulated in that box. Not in its contents – whatever pigmented blend of talc, kaolin and cornstarch was the industry standard – but its forever unfulfillable promise of an elsewhere beyond ordinary ­existence.

Dyhouse has carefully researched whatever it was that admitted an Englishwoman to that "elsewhere" as it was currently fantasised: Phul-Nana and Shem el-Nessim perfumes that suggested an exotic orient, the 1930s farms that raised silver foxes for stoles to shimmer against platinum hair, and the mink farms that superseded them after the second world war. She has a sharp page on Lady Norah Docker, a thrice-married Wag precursor, defiant in diamonds and a Daimler limo customised in crocodile, in her best chapter. Here, she parses possible glamour roles in the 50s – the princess, real or Hollywood's Audrey Hepburn or Grace Kelly; the ballerina or mannequin; the bad girl, jammy about the mouth; the ­sophisticate clanking with costume jewellery cast by demobilised tank manufacturers.

Dyhouse has a real feeling for that era, because the artifice was so deliberate, a reward for effort and experience (Docker was 50ish at the nadir of her notoriety); the young were unable to afford the grooming or carry off the stylisation of self that is necessary for glamour. Then it all changed, in part since the adult desire for White Mink perfume and novelty compacts was near-saturated, and advertising-led market growth had to expand downwards, among the young and the paypackets of the less elegant classes.

After the revolution – successive revolts really, the triumph of youth, of the "natural", of ceaseless identity projection through photography and the related cult of celebrity – Dyhouse moves into summarising the feminist and post-feminist literature on glamour. She does it scrupulously, but her book slips away: chapter nine of Madame Bovary is much shrewder about false aspiration through glamour than most of the academic arguments Dyhouse glosses, because Flaubert knew exactly the gaps in a life that could be filled by dreaming about that better elsewhere.

Nothing slips away in Jones's book, except beauty, which is not actually his business since his task is to narrate the history of the lotions and potions industry from early Eau de Cologne (a disputed brand name, and as likely to be drunk or used as a rub-down as dabbed behind the ears) to Coty's Chinese launch of its Calvin Klein fragrance, renamed Euphoria Blossom for the Sinomarket.

It's an enormous assembly of inter­linked stories, many of them very personal, at least at their start. The American epics usually begin with no-nonsense suppliers of bulk goods, such as Harley Procter and Norris Gamble, who steered their family chandler's business in Cincinnati away from dwindling sales in candles into palm-and-olive-oil soap. The European romances go back to a charismatic individual who supplied a personal service: Eugène Schueller of what became L'Oréal and Hans Schwarzkopf were chemists who devised hair dyes and the first commercial shampoo; Max Factor (Faktorowicz) was apprenticed as a Berlin wigmaker, then painted the visages of the Russian Imperial Opera Company before he slathered his creamy greasepaint over the theatrical district of Los Angeles just as the ­movies arrived.

All the great cosmetic monsters ­rampage through the book, including that mad man Charles Revson, for whose Revlon brand the most daring ads were imagined, from Fire and Ice in 1952 ("Do sables excite you, even on other women?") to the Cosmo-reader striding out sprayed with Charlie fragrance in 1973. And there are quiet successes, too, such as Avon, which personally cherished 40% of the African-American market in the US, calling on their homes and disdaining to spend more than 2.7% in advertising even as Revson expensively sponsored the rigged $64,000 Dollar Question ­television game show.

The drama never lets up. Companies merge and fall victim to sequestration, nationalisation, war, hostile takeovers (Max Factor was relocated to Stamford, Connecticut under the control of Playtex bras), and absurd acquisitions (Pond's bought a spaghetti-sauce ­manufacturer). They lapse, are revived, and, in the case of Colgate-Palmolive, introduce the use of toothpaste to the Philippines. The opening of the first duty-free shop, at Shannon airport in 1946, changed perfume sales irrevocably. Hindustan Lever's Fair & Lovely skin lightening cream dominates the huge Indian market and sells in 40 Asian, African and Middle Eastern countries. There are now three million direct saleswomen of cosmetics in ­Russia.

I would have been dazed by Jones's billions and corporate holdings were not so many of his products familiar – Nivea cream, Mum deodorant, Dove soap – although I was glad to have Marsh's book to hand for reference when it came to untangling the past, or grasping, say, the 80s connection between Margaret Thatcher, the Dynasty bitches and Sally Hansen's Hard-As-Nails talon paint.

Marsh answers the simple questions, too. Maybelline mascara? Coal dust and Vaseline, moistened with spit, dates from 1915. Were there products to combat the second world war US stocking shortage? Elizabeth Arden's Velva leg paint, so sexily suggested in a sketched ad as to make modern fake tan seem crude. Why did Barbara Hulanicki's decadent packaging help push her Biba emporium into going bust? Because girls made themselves up with the testers at the counter but never purchased a thing. As for the pull of that Evening in Paris image, I just bought a new Bourjois rouge on the web, with a modern Parisian scene atop its box. A lovely elsewhere. Also on sale in Korea and Vietnam.

Steven Poole tunes in to an engaging analysis of the sound of music

How do we hear music? As the science writer Philip Ball's book shows thrillingly, listening to Lady Gaga is a hugely complex mental activity. It is already strange that we hear a single note played on the violin or piano as a single note, rather than a chord of harmonic overtones. Add harmony, polyphony and rhythm, and the brain has to recruit considerable processing resources, as recent research in cognitive science shows, to understand even the simplest ditty. Apart from the very small proportion of the population who are authentically tone-deaf, it is inpermissible to say "I am not musical." "There are cultures in the world where to say 'I'm not musical' would be meaningless," Ball writes, "akin to saying 'I'm not alive'."

So far, so refreshing, for chapters about the harmonic series, tuning, scales, why bells sound weird and so on. But Ball is concerned not only to explain how we hear music, but then to judge music itself according to how it fits with brain scans and psychological experiment. A tang of scientism intrudes: now science reveals the truth about music, and musicians of the past can be retrospectively praised for conforming to our models, or scolded for not fitting in. Thus, 19th-century composers are given gold stars for making note-choices that statistical analysis now quantifies as more or less "original", while clever Bartók and Grieg somehow intuited that toying with listeners' expectations through cadences and modulations would one day be found interesting in the lab.

On the other hand, serialism is said to have made "excessive demands" on the listener's cognitive capacities, and large-scale formal structures "don't have much to do with the way we perceive music, unless we have studied the composition in advance". Ball offers no argument here, however, as to why studying the composition in advance should result in a somehow less authentic perceptual experience; and the claim that such large-scale structures have no effect on the unprepared listener is itself not proven.

Mind you, to study the composition in advance would require familiarity with musical notation, on the subject of which Ball offers a rather ludicrously negative interlude. "Notated compositions can't evolve: their alteration comes to be seen as sacrilegious, and they stand at risk of ossifying." The idea that anything and everything should "evolve" is fatuous. Would you enjoy your favourite novel more if the text changed every time you read it? Ball continues: "In pop and jazz music, notation has always been superfluous," which is simply historically false. Duke Ellington could not have composed and performed what he did without notation; and before the radio and phonograph, notation was how popular music was distributed, as well as being the only technology for communicating and preserving the great classical masterpieces. (Ball's own book is, as it should be, littered with notated examples.)

This odd little anti-notation rant is symptomatic of a sort of fashionable cultural anti-authoritarianism throughout, according to which the figure of the composer is to be devalorised, the better to emphasise the importance of other contributors. "It is performance that breathes fire into the formulas," Ball says, condescendingly reducing the score of Mozart's 40th symphony or Berg's Lyric Suite to a collection of more-or-less-clever arithmetical transformations.

Perhaps such superstitions about the supremacy of performance and improvisation explain why Ball offers no analysis of one of the most irreducibly dictatorial genres: modern electronic dance music. Examples from old-school Belgian techno or contemporary progressive trance would have been highly germane to his discussions of how music exploits and defeats rhythmic and melodic expectations, and might have fruitfully extended his brief musings on how timbre (the sonic "character" of a sound) could be important to musical cognition: modern dance producers are nothing if not scientists of frequency distribution and timbral shift.

That said, Ball is to be applauded for the range and aptness of the musical examples he does choose – from Albinoni to Led Zeppelin, Bach to The Sound of Music – as well as for his attention to musics outside western traditions. (I was happy to learn of the Sirionó Indians of Bolivia, among whom "each member of a tribe has a 'signature' tune that forms the basis of all he sings".) There are deft analyses of the limitations of attributing "emotion" to music, or considering it as a "language" (Lévi-Strauss: if music is a language, it is an "untranslatable" one); and the book is impressively ­engaging for one so dense with detail and argument. He describes the pop-music fadeout, rather beautifully, as leaving the listener with a feeling of "contented yearning", and characterises Chopin's music wittily as "riddled with dissonance".

In the modern pop-cognitive-­psychology genre, the reader is usually well advised to imbibe gratefully the reported data while taking the overarching "lessons" with a very large pinch of salt, and that is the way to read Ball's flawed but fascinating work. It will be the rare music-lover who does not come away without having learned many interesting things; but aesthetics cannot be replaced wholesale by bean-counting analysis. Despite the sometimes strenuous pleading, music remains largely unexplained.

'I'm not turning into Kingsley. I'm already Kingsley'

The following correction was printed in the Guardian's Corrections and clarifications column, Tuesday 16 February 2010

"In truth this is easily the most unusual thing about me," wrote Martin Amis in the column below: "I am the only hereditary novelist in the anglophone literary corpus." A reader points to at least one other, Anthony Trollope, following from his mother Frances.

I was born in Clapham in 1922. My literary career kicked off in 1956 when, as a resident of Swansea, South Wales, I published my first novel, Lucky Jim. This was followed by That Uncertain Feeling and Take a Girl Like You, among others; but my really productive period began in 1973, when I published both The Riverside Villas Murder and The Rachel Papers. 1978 saw the appearance of Jake's Thing and Success; in 1984 it was Stanley and the Women and Money; in 1991 it was The Russian Girl and Time's Arrow. This last was shortlisted for the Booker prize; but I had already been a winner with The Old Devils in 1986. I am, incidentally, the only writer to have received the Somerset Maugham award twice – the first time for my first first novel, the second time for my second first novel.

That period, alas, came to an end in 1995. Since then, though, I have been far from sluggardly. This year, for instance, at the age of 88, I publish my 37th work of fiction, The Pregnant Widow, and next year will see another novel, State of England – my 67th book, which nicely sets the scene for my 90th birthday. I have written five volumes' worth of journalism; I have taught at Princeton, Cambridge and Manchester. May I quote Anthony ­Burgess? "Wedged as we are between two eternities of idleness, there is no excuse for being idle now." I have been married four times (two of my wives are novelists), and I have eight children and seven grandchildren – so far. Oh, and I almost forgot to mention my Collected Poems (1979).

❦

The creature described above is of course imaginary. But such a phantasm, such a monster of longevity and industriousness, seems to exist in the minds, or in the anxiety dreams, of a tiny stratum: British – no, English – feature-writers who occasionally address themselves to literary affairs. Incidentally, this is what they're groping to express when they say I'm "turning into Kingsley". They should relax: I'm already Kingsley. In truth, this is easily the most unusual thing about me: I am the only hereditary novelist in the ­anglophone literary corpus. Thus I am the workaholic and hypermanic, and by now very elderly, Prince Charles of English letters. I have overstayed my welcome. I have been about the place for much too long.

About 90% of the coverage has passed me by, but some new tendencies are clear enough. What's different, this time round, is that the writer, or this writer, gets blamed for all the slanders he incites in the press. Some quite serious commentators (DJ Taylor, for one) have said that I'm controversial-on-purpose whenever I have a book coming out. Haven't they noticed that the papers pick up on my remarks whether I have a book coming out or not? And how can you be ­controversial- on-purpose without ceasing to care what you say? The Telegraph, on its front page, offers the following: "Martin Amis: 'Women have too much power for their own good'." This is the equivalent of "Rowan Williams: 'Christianity is a vulgar fraud'." I suppose the Telegraph was trying to make me sound "provocative". Well, they messed that up too. I don't sound ­provocative. I sound like a much-feared pub bore in Hove.

And yet experienced journalists will look me in the eye and solemnly ask, "Why do you do it?" They are not asking me why I say things in public (which is an increasingly pertinent question). They are asking me why I deliberately stir up the newspapers. How can they have such a slender understanding of their own trade? Getting taken up (and recklessly distorted) in the newspapers is not something I do. It's something the news- ­ papers do. The only person in England who can manipulate the fourth estate is, appropriately, Katie Price. But there I go again. No, the vow of silence looks more and more attractive. That would be a story too, but it would only be a story once. Wouldn't it?

❦

To return briefly to the longevity theme – and all the stuff about street-corner suicide parlours, and the "silver tsunami" (which is the demogaphers' shorthand for what has been described as "the most profound population shift in history"). The press reacted to my remarks with righteous dismay; but I saw no recent headlines saying "Terry Pratchet is mad", by way of commentary on his resonant statement about euthanasia. In addition, it turns out that 75% of Britons (but none of the political parties) agree with him and agree with me. Thus the euthanasia question, eerily, is the reverse image of capital punishment at the time of its abolition. The people wanted judicial killing, but the government, highmindedly and quite rightly in this case, said no.

Of course, Sir Terry's dignified ­remarks were taken from a public ­lecture; mine were a mishmash of half-quotes from a satirical novel. For the interested, the passage reads (I am ­referring to Europe's distorted age structures): "Hoi polloi: the many. And, oh, we will be many (he meant the generation less and less affectionately known as the Baby Boomers). And we will be hated, too. Governance, for at least a generation, he read, will be a matter of transferring wealth from the young to the old. And they won't like that, the young. They won't like the silver tsunami, with the old hogging the social services and stinking up the clinics and the hospitals, like an ­inundation of monstrous immigrants. There will be age wars, and chrono­logical cleansing . . ."

Then, too, Sir Terry has Alzheimer's – a condition made yet more tragic by the liveliness of the mind it here afflicts (I am thinking also of Iris Murdoch and Saul Bellow). And Sir Terry is older than me. Or is he? Well, yes and no. I am 88 – but I am also 24. Look at the photographs. A 60-year-old grandfather, I am still the "bad boy" (not even the bad man) of English letters. Who could possibly "manipulate" ­perceptions as chaotic as these?

Three of the publisher's most successful authors sweep Alison Flood off her feet for an impassioned journey through the tempestuous challenges of writing romance

The Wealthy Greek's Contract Wife. The Prince's Chambermaid. The Italian Billionaire's Secretary Mistress. Mentioning the titles of Penny Jordan and Sharon Kendrick's latest novels for Mills & Boon draws embarrassed chuckles from both of them.

"Titles are contentious, I tell you," says Kendrick. "[Mills & Boon] want the title to reflect exactly what's in the book" – the subtext being that the authors might prefer something a little more subtle. "I never bother about the title," agrees Jordan. "When I buy books I buy by author. But Harlequin must know how to run their business."

In Kendrick's and Jordan's cases, they clearly do. Jordan is the acknowledged queen of Mills & Boon. She's been writing for the publisher since 1981, has produced more than 170 novels and sold more than 70m books around the world. Kendrick, meanwhile, has just delivered her 75th book. That's 75 heroes, 75 heroines, 75 all-consuming love affairs and an estimated 150 sweaty sex scenes – Mills & Boon couples usually do it at least twice in the course of their 55,000-word romances. How, exactly, do these authors keep it up?

"It is very difficult to have a new take on an old story, and romance is an old story – it's been there forever. It has to ring true to the reader but at the same time you have to write in a way that keeps them turning pages," says Jordan, who churns out 5,000 words a day, writing four Mills & Boon novels a year, as well as two sagas for HarperCollins as Annie Groves. "You know you've got to grab their attention by the end of the first page." In fact, in her romance A Bride for His Majesty's Pleasure, the scene is set by the end of the first paragraph: "'And if I refuse to marry you?' Although she did her best not to allow her feelings to show, she was conscious of the fact that her voice trembled slightly. Max looked at her. 'I think you know the answer to your own question.'" The reader knows what they'll be getting – ruthless ruler, virgin bride – right from the start.

Jordan always begins, she says, with the issue the characters have to overcome in order to be together. "Romance is romance. For me a lot of the fun of writing comes from the problems I give the characters. They have to deal with them in order to feel confident with the relationships they have," she says. "I start with the central conflict, with the problem, then I build characters who will enable the problem to work from the readers' point of view. In the book I've just finished, neither the hero nor the heroine want commitment. He's a bit of a playboy, she's quite withdrawn. It goes back to them both feeling abandoned by their parents."

Maisey Yates, who landed her first contract with Mills & Boon in December, agrees. A "stay-at-home mom" in southern Oregon, Yates produces around 2,000 words a day. At the beginning, she went out to write in a coffee shop when her husband came home, but now she knows what she's doing, she'll write at home with the kids. Five books a year, she thinks, "is doable for me". "Usually I'll get a vague idea of a conflict, then I'll start to think of a character," she says. "Once I've got my first character, and it can be the hero or the heroine, I try to figure out their issues. Then I think about who could come along and mess things up." In The Virgin Acquisition, which will be published in August, Yates's heroine is trying to win back her father's company from the hero. In another of her novels, yet to be published, a career woman who wants to be a mother goes to a sperm bank, and mistakenly ends up with the hero's sperm – it was meant to be a sample for his wife, but she's passed away.

Kendrick, who writes four romances a year, admits to getting ideas "all over the place", even through reading the Daily Mail. "Let's be honest: you have to have some kind of vehicle, and that's the real challenge. Everyone knows the hero and heroine are going to end up married so really the only reason to read them, like all good books, is a compelling story." She insists that, in order to write with integrity, "You have to believe." If people approach them cynically, or try to write tongue-in-cheek, it doesn't work.

She explodes the myth that Mills & Boon writers are provided with templates for their stories. "The structured plan is rubbish. We are allowed as much artistic freedom as will work," she says. "Obviously there are things that work and don't work. The plotline where the hero is trying to build a factory and the heroine is trying to save a rare toad is not a very sexy premise. And you wouldn't want a short fat balding hero – women know too many men like that. Mills & Boon is about escapism and fantasy. It drives me mad when people say 'don't you think you're deceiving women?' I don't think we're completely thick." Although Jordan is clear that she doesn't "get given a tip sheet" for her books, she acknowledges that "every genre has its own little rules". "They're not written down, but if you diverge from reader expectations they won't read your second book," she says.

Once the conflict is in place, the writers look to identify their heroine. While Kendrick admits that "It doesn't matter how you describe her, you'll always have a dead-ringer for Angelina Jolie minus the tattoos on the front cover," her heroines, she says, "are not always beautiful, and like most women are plagued by insecurities. I'm not very good at writing high-powered career women. It could be because I haven't had a high-powered career myself. But if she's a barrister or a newspaper editor, it wouldn't really be feasible – I want her to be spending time with the hero. She tends to have to be flexible. And if she's a chambermaid, if she's sacked it's not the end of the world."

Jordan isn't so sure. "I'm always interested in giving them interesting careers", she says. "There was a fad for Cinderella-type heroines. I've tried them, but it doesn't fit me so well. I have had them with money problems, but with careers prior to money problems. I want them to assert themselves when necessary." Yates agrees, saying she likes to go for "feisty career women". However, Jordan, who's been writing romances for 33 years, usually makes her heroine either a virgin, or inexperienced. "I think of it as a shorthand for me," she says. "It's always by choice. When my heroine meets the hero, she wants to go to bed with him. For the reader, that's the mark of the effect he has on her. Because you've only got so many pages, it would be very difficult for me to create a heroine who's had lots of partners and immediately knew there was something different about this one."

Then comes the hero. Sheikhs are popular, Jordan and Kendrick say, as are Italian billionaires, Greek tycoons and princes. The sheikh, Kendrick says, "represents the ultimate female fantasy – dark, autocratic, completely powerful, outrageously chauvinistic". However, she says, "he often isn't predatory, as he doesn't need to be. In the 70s and 80s the Mills & Boon hero was putting it about, then with the advent of Aids we had to make slowly sliding on a condom part of the love play."

For Jordan, the hero also has to have a charitable side. "He's obviously got to be sexy and high powered because they go together. And they always like them to be well off. But for me he has to have some interest in charity, to do something for the good," she says. "Often when my heroines discover that, their animosity is melted. I don't like a hero without a softer side. He's often damaged by something that's happened in his life, often to do with money. He will be more outrageous to the heroine, and harder on her. He realises he is beginning to feel, he has to resolve that conflict."

And Yates, who at 23 is the publisher's youngest author, says she likes "to play with the conventions a bit. He's still an alpha male, but he's maybe a little more willing to talk about things at times". Her characters are usually pure imagination, but sometimes, she says, she'll "grab a picture, usually of a model, not someone well known" as a template. "I'm really picky about my heroes though, they're a little more perfect in my head," she adds. Kendrick doesn't "do the picture thing. Others put up pictures of actors or models with awful overdeveloped six packs. [But] imagination is much better."

Once the two central players, and their issues, are in place, then, of course, comes the sex. "It is very, very difficult to write about sex," admits Jordan. "You think, did I say that before? I don't have a set of actions, one to five, but there are only so many variations. I try to make it unique for each set of characters, but obviously I must go over the same territory. Straight sex is straight sex. It's more really trying to capture the emotional intensity."

Over the years, Jordan says, more sex has crept into her books. "There is more now, and it's more detailed," she admits. "But I've always wanted my heroines to enjoy sex. Perhaps in the earlier books they were more reluctant to admit they enjoyed it. Now it's a battle within them – they're enjoying sex with someone they might be falling in love with, but they don't like." She never, she adds, writes abusive sex.

Kendrick insists that, no matter how many times you've written one, it's important not to be blasé about sex scenes. "That might imply I'm complacent and that does not make a good bedfellow," she says. "Some writers will go through and leave gaps. I won't do that. It's all the flow of the story. I have to write sex knowing how they're feeling. I have to be her, and imagine him – by that time I will be in love with the hero so it's not that difficult to write." It's a similar process for Yates. "Maybe I was embarrassed afterwards, reading them, but at the time, because I'd spent so long building up all the tension, those scenes came the fastest."

All three authors are adamant that this is a great way to make a living – although Jordan is a little shocked to discover she's written quite so many. "Have I? I probably have. I've been writing since just after my 30th birthday and I'm 63 now. Should somebody my age still be writing romances? Am I still on trend with things? I don't know. I still love writing them. It's the readers' decision," she says with rather touching concern – the readers are still buying her books in their thousands. "The best way," she muses, "to describe the difference between now and then is to say, in the words of Mrs Patrick Campbell, that it's like 'the deep, deep peace of the double bed after the hurly-burly of the chaise longue'."

What Jordan loved – what she still loves – when writing, is "learning about the character, what prevents them from reaching happiness. There comes a point when the character becomes real. It still delights me," she says. "At the end of the day everything I write is about relationships. I'm never going to be a great big famous writer because I don't write great big famous-making scenes, more the nitty gritty of everyday life. And that's what I really enjoy."

Doubleday, £18.99

Chrissie had thought it astonishing that neither she nor her three daughters had cried in the hospital when the doctor shook his head. Richie's big northern heart had stopped beating and broken theirs. "Don't worry, it's perfectly ­understandable," said Chrissie's friend, Joanna. "There are four stages of grief: shock, denial, anger and acceptance."

"That's very convenient," said Amy, the youngest daughter. "Because there are four of us. We could have one stage each. Unless you think that's a bit too obviously mechanistic."

"Not at all. That will be fine," Joanna smiled, soothingly. "In which case, I'll be shock," cried Chrissie. "I'll never get over Richie's death." "Bagsy denial," said Tamsin, "I'm going to nip over to see my dreary boyfriend." "Thanks for nothing, you bitch," Dilly yelled. "I guess I'm anger, then."

"Which makes me acceptance," Amy sighed. "So I'll tell Dad's first wife, Margaret, and his son Scott, so they can come down from Newcastle for the funeral."

"I'm not having that cow down here," Chrissie exclaimed in shock. "She's not his first wife, she's his only wife. Your Dad never married me and even though he never gave me any cause to be jealous in the 23 blissful years we were together, I've always been insecure about it."

"I didn't hear that," said Tamsin. "What a bastard. So we're all bastards," Dilly shouted. "I think we're all over-reacting a little," Amy sighed. "I'll call them."

Oop north, Margaret put the phone down. "Your Dad's died, son," she said, fingering the wedding ring she had never removed because deep down she had never quite got over her divorce and been able to move on in a healing way. "That's bad," Scott replied, staring moodily into the middle distance over the Tyne and wondering if his father leaving him at 14 was the reason he'd never been able to find a fulfilling ­relationship with a woman.

"I'm afraid I have some bad news," said Mr Leverton, the lawyer. "Richie left the Steinway piano and the royalties on all his songs that were published ­before you met him to his first family."

"La la," shrieked Chrissie, "he should have left everything to me." "I didn't hear that," said Tamsin. "Don't give those northern bastards a penny," Dilly snarled.

"It seems perfectly reasonable to me," Amy sighed. "After all, we get everything else." "But we're broke," Chrissie sobbed hysterically. "We're down to our last £500K. We're going to have to move out of Hampstead and I'll have to get a job."

Oop north, Scott shrugged his ­shoulders with northern stoicism. Much as he'd have liked to play the Steinway, he'd understood that his ­father's second family needed time to deal with their grief. Much like his mother. But it was good that he and Amy had begun to talk on the phone.

"We're broke," she had told him. "So I'm giving up my A-levels to become a prostitute." "That's not a good idea," he had replied. "All right, I won't then."

"It's time we all moved on a little," Joanna cooed. "How about we arrange to deliver the piano to Scott." "La la," shrieked Chrissie. "I didn't hear that," said Tamsin. "Thieving northern ­bastards," snarled Dilly. "I'll get on with it," Amy sighed.

Scott's dumpy northern fingers ­tinkled the ivories contentedly. How wonderful it was to have the piano and how wonderful that Amy had come to visit him in Newcastle now she had got seven As in her A-levels. "Gosh," Amy gasped. "Newcastle isn't the third world, after all. I'm going to go to ­university here."

"I've got a job as a receptionist and I've rented a penthouse flat in Highgate from an LA film director," Chrissie smiled. "I heard that," Tamsin said. "Peace and love," Dilly grinned. "Isn't it amazing how we've all managed to come to terms with our grief in different ways?" said Amy, as Scott nodded with northern knowingness.

Margaret thought it was time she too moved on. "I'm ready to let go," she said, tossing her wedding ring into the Tyne with a splash that became a tidal wave as readers hurled this book in after it.

Digested read, digested: Lucky Richie.

Beneath the still surface of the poet's life lay a fiercely passionate nature and a closely guarded secret, argues her lastest biographer

Emily Dickinson was a great poet whose life has remained a mystery. The time has come to dispel the myth of a quaint and helpless creature, disappointed in love, who gave up on life. I think she was unafraid of her own passions and talent; that her brother's sexual betrayal and subsequent family feud had a profound effect on the Dickinson legend that has come down to us; and perhaps most significantly, I believe that Emily had an illness – a secret that explains much.

It was Emily herself who helped to devise the blueprint for her legend, starting at the age of 23 when she declined an invitation from a friend: "I'm so old-fashioned, Darling, that all your friends would stare." In place of the tart young woman she was, she adopted this retiring posture. Born in 1830 into the leading family of Amherst, a college town in Massachusetts, she never left what she always called "my father's house". Townsfolk spoke of her as "the Myth".

On the face of it, the life of this New England poet seems uneventful and largely invisible, but there's a forceful, even overwhelming character belied by her still surface. She called it a "still – Volcano – Life", and that volcano rumbles beneath the domestic surface of her poetry and a thousand letters. Stillness was not a retreat from life (as legend would have it) but her form of control. Far from the helplessness she played up at times, she was uncompromising; until the explosion in her family, she lived on her own terms.

Her widely spaced eyes were too keen for the passivity admired in women of her time. It's the sensitive face of a person who (as her brother put it) "saw things directly and just as they were". At 17, as a student at Mount Holyoke in 1848 (the same year that the women's movement took a stand at Seneca Falls), she refused to bend to the founder of her college, the formidable Mary Lyon. At this time Massachusetts was the scene of a religious revival opposed to the inroads of science. Emily, who had chosen mostly science courses, makes her ­allegiance clear:

"Faith" is a fine invention

When Gentlemen can see –

But Microscopes are prudent

In an Emergency.

When Miss Lyon pressed her students to be "saved", nearly all succumbed. Emily did not. On 16 May, she owned, "I have neglected the one thing needful when all were obtaining it." It seemed that other girls desired only to be good. "How I wish I could say that with sincerity, but I fear I never can." When Miss Lyon consigned her to the lowest of three categories – the saved, the hopeful and a remnant of about 30 no-hopers – she still held out.

During a creative burst in the early 1860s, she invited a Boston man of letters to be her mentor, but could not take his advice to regularise her verse. Helpful Mr Higginson, a supporter of women, who thought he was corresponding with an apologetic, self-effacing spinster, was puzzled to find himself "drained" of "nerve-power" after his first visit to her in 1870. He was unable to describe the creature he found beyond a few surface facts: she had smooth bands of red hair and no good features; she had been deferential and exquisitely clean in her white piqué dress and blue crocheted shawl; and after an initial hesitation, she had proved surprisingly articulate. She had said a lot of strange things, from which Higginson deduced an "abnormal" life.

There was an increasing divide between people she wished to know and those she didn't. Her clarity could not endure social talk instead of truth; piety instead of "The Soul's Superior instants". Her directness would have been disconcerting if she did not "simulate" conventionality, and this was "stinging work". But a more threatening challenge, deeper below the surface, fired the volcanoes and earthquakes in her poems – an event, as she put it, that "Struck – my ticking – through –".

Something in her life has so far remained sealed. The poems tease the reader about "it" and her almost overwhelming temptation to "tell". I want to open up the possibility of an unsentimental answer. If true, it would explain the conditions of her life: her seclusion and refusal to marry. Once we know what "it" is, it will be obvious why "it" was buried and why its lava jolts out from time to time through the crater of her "buckled lips".

During the poetic spurt of her early 30s, Dickinson transforms sickness into a story of promise:

My loss, by sickness – Was it Loss?

Or that Etherial Gain –

One earns by measuring the Grave –

Then – measuring the Sun –

Sickness is always there, shielded by cover stories: in youth, a cough is mentioned; in her mid-30s, trouble with her eyes. Neither came to much. In her poems, sickness can be violent: she speaks of "Convulsion" or "Throe". There's a mechanism breaking down, a body dropping. It "will not stir for Doctors". "I felt a Funeral, in my Brain", she says, and "I dropped down, and down". Allowing for the poet's resolve to tell it "slant", through metaphor, are we not looking at epilepsy?

In its full-blown form, known as grand mal, a slight swerve in a pathway of the brain prompts a seizure. As Dickinson puts it, "The Brain within its Groove / Runs evenly", but then a "Splinter swerve" makes it hard to put the current back. Such force has this altered current that it would be easier to divert the course of a flood, when "Floods have slit the Hills / And scooped a Turnpike for Themselves".

Since the falling sickness, as epilepsy used to be known, had shaming associations with "hysteria", masturbation, syphilis and impairment of the intellect leading to "epileptic insanity", it was unnameable, particularly when it struck a woman. In the case of men secrecy was less strict, and fame in a few – Caesar, Muhammad, Dostoevsky – overrode the stigma, but a woman had to bury herself in a lifelong silence. If this guess is right, it's remarkable that Dickinson developed a voice from within that silence, one with a volcanic power to bide its time.

Prescriptions (one from an eminent physician, others in the records of an Amherst drugstore) show that Dickinson's medications tally with contemporary treatments for epilepsy. The condition, which has a genetic component, appeared in two other members of the Dickinson family. One was Cousin Zebina, a lifelong invalid, immured at home across the road, whose bitten tongue in the course of a "fit" is noted by Emily in her first surviving letter at the age of 11. "I fit for them," she announced in a poem of c1866. Then her nephew, Ned Dickinson, turned out to be afflicted. He was the son of Emily's brother Austin and his wife Susan Dickinson, who lived next door. To the family's dismay Ned, aged 15, had an epileptic fit in 1877. Horrendous attacks continued, about eight a year, recorded in his father's diary.

We can't know whether Emily Dickinson suffered as her nephew did. There are many forms of epilepsy, and the mild petit mal does not involve convulsions. The mildest manifestations are absences. A schoolmate remembered that Emily dropped crockery. Plates and cups seemed to slide out of her hands and lay in pieces on the floor. The story was designed to bring out her eccentricity for, it was said, she hid the fragments in the fireplace behind a fireboard, forgetting they were bound to be discovered in winter. This memory is more important than the schoolmate realised, because it suggests absences, either accompanying the condition or the condition itself.

Her violent images, the "spasmodic" rhythms Higginson deplored, and the sheer volume of her output show that she coped inventively with gunshots from the brain into the body. She turned an explosive sickness into well-aimed art: scenes with "Revolver" and "Gun". Contained in her own domestic order, protected by her father and sister, Dickinson saved herself from the anarchy of her condition and put it to use.

The mystery the poet was not to "tell" continues to this day to be encased in claims put out by opposed camps who fought for possession of her greatness. These camps go back to the feud. It began with adultery between Emily's brother Austin, in his 50s, and a newcomer to Amherst, a young faculty wife of 27, Mabel Loomis Todd. After the poet's death, the feud came to focus on Emily as her fame grew: who was to own her unpublished papers? Who had the right to claim her?

Both camps proceeded to wrap the poet in legends that stress her pathos: where Dickinson legend built up a bereft Emily in a dimity apron turning away the one and only man she loved, Todd legend built up a pitiful Emily "hurt" by her "cruel" sister-in-law, Susan Dickinson. How can we crack through the sad-sweet picture to find what Dickinson called the red "Fire rocks" below?

One way is to go back to acts of ­adultery that changed utterly those who were to be the first keepers of her papers. The advantage of approaching the poet through the feud is the entrée it provides to emotional currents in the family. Assignations – sometimes "with a witness" – are on record, recounted precisely as to time and place in the lovers' corroborating diaries. The impact of adultery on the family is plain – and not so plain, for the riddles in the poet's notes to her brother's mistress must be solved if we are to understand where she stood.

A recurring fact during the first years of the affair is crucial to the poet's position. Because it was difficult to keep adultery secret from the tattle of a small town, the safest place was the irreproachable home of the Dickinson sisters. There, the lovers would occupy the library or the dining-room (with its black horsehair sofa) for two to three hours. The door would be shut, blocking the poet's access to her second writing table in one room or to her ­conservatory via the other.

Austin Dickinson blew apart his family when he rejected his wife, Susan, who had long been the poet's keenest reader. Who had they been before this happened, and why, earlier, did Dickinson speak of a "Bomb" in her bosom? The Bomb may refer to periodic explosions in the brain, but emotionally both Austin and Emily had an eruptive vein, which Emily channelled into poetry. Her letters show that she cultivated adulterous emotions, if only in fantasy, for an unnamed "Master". How did this affect her response to her brother's sudden outbreak into active adultery?

In September 1881, David Todd and his wife, Mabel, had arrived in Amherst from Washington. She was a dressy urban beauty bent on maintaining standards in what appeared to her a negligible "village" full of retired clergymen and elderly academics. Mrs Todd, extending an immaculate white glove, her smile sliding up one cheek, was invited everywhere and was in a position to choose whom to favour. In Amherst, the Dickinsons were like royalty: Mrs Todd was taken with "regal", "magnificent" Austin Dickinson and his wife's dark poise, set off by a scarlet India shawl, when they called on her. Behind Austin's back, Amherst children mocked his auburn hair, arrayed like a fan above his head, and his sniffy walk, tapping his cane as he went.

At first, all the Dickinsons (bar Emily, who kept to her room) warmed to Mrs Todd's accomplishments: her solos soared above the church choir, she painted flowers to professional standard and published stories in magazines. She soon won the friendship of the bookish Susan Dickinson, before it became apparent that she was flirting with Susan's son, 20-year-old Ned, who fell painfully in love. This happened just before his father became a rival. Austin's love for Mabel Todd was to last for the rest of his life.

The result was what came to be known as "the War between the Houses". Austin turned against his children when they sided with their distraught mother. New evidence reveals that, far from withdrawing from the feud, Emily Dickinson took a stand. Unlike her sister Lavinia, who sided with the lovers, she refused to oblige her brother by signing over a plot of Dickinson land to his mistress. In August 1885 the poet wrote to her nephew Ned, confirming her resistance. "Dear Boy," she starts her letter assuring him he would find "no treason". "You never will, My Ned." This letter ends: "And ever be sure of me, Lad – Fondly, Aunt Emily.

When she died, Mabel got her land. Three weeks after the funeral the deed was signed and the Todds' house rose on the Dickinson meadow – a venue for future assignations.

This might have been a routine story of a femme fatale were it not for the presence of mysterious genius. As the feud sharpened its focus on the poet, it would be seen how Mabel had quickened to the poems of Emily Dickinson and how willing Mabel would be to undertake years of toil with difficult manuscripts. She was to show herself ready in other ways, one of only three people during the poet's lifetime to recognise Dickinson's genius. The name of Mabel Loomis Todd will always be associated with the poet.

Mabel appears to act out a familiar plot – the seduction of a man in power – but what differs here is the presence of another and grander form of power, that of a poet who selects her own society, then shuts the door. To Mabel Todd, with her discerning taste, that shut door, and the elect intelligence behind it, offered an irresistible challenge. So, on 10 September 1882, accompanied by Austin, Mrs Todd knocked on the Homestead door, and had herself admitted to the parlour where she sang to Lavinia and Austin. As she did so, Mabel imagined the poet listening in her fastness upstairs, captivated, as the trained voice trilled through the house.

Over the years to come Mabel was to re-enact this scene, fantasising a bond with the invisible poet. She would ­insist on this bond yet although she was in and out of the Homestead, she never once laid eyes on Emily ­Dickinson. On this initial occasion, the poet sent in a glass of homemade cordial together with a poem, which Mabel told herself had been composed spontaneously as a tribute to so pleasing a guest. Then, within 24 hours, on 11 September, there was a declaration of love for Austin – the "Rubicon" where he abandoned marital fidelity at the gate of his home before the pair entered to play a game of whist with the unsuspecting Sue.

Mabel's entry into the Homestead looks politely innocuous beside this initiation of adultery, but it was to present a parallel and more lasting threat to family peace. In time, ­Mabel would take possession of a large cache of Emily Dickinson's papers, and ­market them in her own terms, so that the strange nature of the poet would be ­obscured as a victim of Susan Dickinson. So it was that an eruptive poet sending out her "bolts", "Queen" of her own existence, would be subject to a false plot acted out in the unstoppable momentum of Todd's takeover.

A new and prolonged phase in the war between the houses began with the poet's death in 1886 and her sister's discovery of a lifetime's poems in her chest of drawers. Within a short time, Austin persuaded Lavinia to hand over the papers to his mistress. Yet Austin must have been aware that in his own home, his estranged wife treasured a separate collection – poems Emily had given her over the years. Fuelled by adultery, antagonism between Susan Dickinson and Mabel Todd mounted over possession of the poet, with the success of Todd's four editions of Dickinson (two co-edited with Higginson, two put out on her own) during the 1890s followed by the poet's growing stature in the course of the 20th century. Insistent legend continued to wrap her in the image of the modest, old-fashioned spinster. But the bold voice of the poems can't be categorised: "I'm Nobody," she says, "– who are you?" It's a voice we can't ignore, confrontational, even invasive, defying façades with a question about our nature.

The feud fed into a succession of increasingly public conflicts, starting with a court case in 1898 when Lavinia Dickinson changed sides and took a stand of her own against the Todds' further claim to Dickinson land. At the heart of the trial is Mabel Todd's assertion that this strip of land was due to her as compensation for her years of toil in bringing a great poet before the public. Poems (1890) had sold 11,000 copies in its first year. Her defence turned on her undoubted feat in transcribing, dating and editing piles upon piles of unpublished manuscripts.

Hatred did not die with the deaths of the first generation. The daughters of the feud, Susan's daughter Martha Dickinson and Mabel's daughter Millicent Todd, did battle through adversarial books during the first half of the 20th century. At its height in the 1950s, the feud turned into a conflict over the sale of the Dickinson papers.

The Dickinson camp appeared to win that round. But before Millicent Todd died in 1968, she set up a posthumous campaign that could not fail. Her plan was to co-opt a writer of impeccable credentials for a book she had in mind. To this end she appointed Yale professor Richard B Sewall as her literary executor, granting him exclusive rights to the Todd papers. Her partisan agenda was clear: this executor was to "set the whole network of Dickinson tensions in proper perspective". So it came about that Sewall perpetuated the Todd positions in a two-volume biography of Emily Dickinson that has remained standard for the last 36 years.

Mabel Todd's persuasive grace in presenting her point of view was reinforced by the educated rigour of her daughter's voice on tape as she took Sewall through the legal history of the feud, bristling with facts and dates. These she laid out in the orderly manner of a scholar. To the unwary her testimony would appear objective and informed, and yet in every instance the Todds turn out to be the victims of Susan Dickinson and her fearsome daughter. To hear the tapes is to understand their impact on a biographer. Sewall felt "haunted" by Austin's statement that he went to his wedding as to his execution. Only no one can know what Austin said: the image of execution was transmitted by a mistress determined to oust his wife, and not only in the usual manner, but in various ways to obliterate Sue's centrality in the poet's life.

A biographer tempted by exclusive access to an archive of such eloquence is bound to be influenced, and though Sewall relayed what he found in a cautious manner, he passed on the trove of Todd untruths: that Emily Dickinson had favoured Mabel; that the poet's withdrawal into seclusion had been the result of a family split preceding Mabel's appearance; and that Austin (contrary to evidence in the trial) had "deeded" to the Todds a second strip of land. The biographer even outdoes the Todds when he suggests that Dickinson's "failure" to publish was a result of a family quarrel.

Legends of this kind spread to theatre and fiction. In 1976 an award-­winning play The Belle of Amherst reinvigorated the sad-sweet image: a "shy", "chaste", "frightened" poet hardly knows what she says, so keeps busy with baking. The playwright called it an "enterprise of simple beauty", backed by "audiences who have taken our 'Belle' to their hearts". In a novel of 2006 a spiteful Sue ends up "hating" Emily. In a novel of 2007 Sue becomes a death-dealing Lucrezia Borgia. She awaits her victims in the hall of her house, a vamp in décolleté black velvet waving her fan. Can evil go further? It can. Sue "could make mincemeat pie of the Dickinson sisters and eat it for Christmas dinner".

So the pathos has persisted even though Dickinson's words reveal a woman who was fun: a lover who joked; a mystic who mocked heaven. This woman was not like us: to know her is to encounter aspects of a nature more developed than our own. Her ­poems turn on the communicative power of the unstated between two people attuned to it. So, the question of contacts is crucial: for whom is she writing? Who is being trained in her unique mode of communication? Who provokes her to further communication? "Be Sue – while I am Emily – ", she commanded the friend of her youth who became her sister-in-law, "Be next – what you have ever been – Infinity".

An initiation in infinitude was the gift Dickinson offered to the few she admitted to intimacy. Sewall's assumption that men changed her has dated. It was she who operated on others for the brief periods they could bear it. She created certain people in the same way as she created her poems, many enclosed in letters as extensions of them. She half-found, half-invented a receptive reader in Sue to whom she sent 276 poems – more than twice the number sent to anyone else. In a similar way she created a deathless love for the person whom she called "Master".

Biographers have sought meaning behind the bearded and married "Master", who appears in three mysterious letters from spring 1858 to the summer of 1861. Evidence remains thin, and biographers have taken their pick from an array of unlikely candidates. These letters race from one literary drama to another, including Jane Eyre's encounter with her married "Master" and deathless love in Emily Brontë – in 1858 Dickinson had acquired a copy of an 1857 edition of Wuthering Heights – and it seems likely that the "Master" letters were as much exercises in composition as letters addressing a particular person. The most popular candidate originated in hearsay that the love of Dickinson's life had been the married Rev Charles Wadsworth, whom she met during a visit to Philadelphia in 1855 and then, supposedly, renounced. (Lugubrious, beardless, with stringy locks, Wadsworth sent Miss "Dickenson" a dull pastoral letter about her sufferings – without a clue what those sufferings were.)

In her late 40s and 50s, a new drama began when she turned to fierce Judge Lord of the Massachusetts supreme court. But though she thought of his touch at night, interrupted her writing to anticipate his weekly letter and played up to the comic character he assigned as "Emily Jumbo", she would not marry him. Epileptics in her time were not supposed to marry, and some American states passed laws against it. Drafts of her love letters have survived: they are witty, confident, open (not coded like letters to "Master"), and within the limits of her unrelenting control over her existence, abandoned – hardly the way 19th-century ladies were supposed to behave.

Dickinson found love, spiritual quickening and immortality, all on her own terms. One model remained: Wuthering Heights. Yet unlike the ­anarchic lovers of the Heights, Dickinson was a moral being, a product of upright New England: she grasped the potential destructiveness – to her sanity, for a start – of the "Bomb" in her bosom; and she witnessed the eruption of the feud – during her lifetime, another secret within the family. She refers repeatedly to a secret "Existence" – primarily her poetry – that must be seen in terms of New England individualism, the Emersonian ethos of self-reliance that in its fullest bloom eludes label. It's more awkward and less lovable than English eccentricity – dangerous, in fact, as Dickinson owned when she said, "My Life had stood – a Loaded Gun –".

From the right to life, to the right to die: an extract from Michael Scammell's biography of one of the most influential figures of the 20th century

'I don't believe in humanity, I believe in the individual'

During his long life Arthur Koestler investigated a multitude of political movements, religions and scientific disciplines, from Zionism to Catholicism and even Buddhism, from anti-fascism to communism and anti-communism, from astronomy and evolution to neurobiology and parapsychology. His literary and political odyssey spawned more than 30 books, among them six novels, four autobiographies, four scientific treatises, four volumes of essays, three non-fiction investigations, and innumerable newspaper articles. Among all the different causes he espoused, the one for which he is perhaps best remembered is his campaign against the death penalty in Britain. His opposition to capital punishment was of long standing, stemming from the time when he was imprisoned in Seville after being captured while reporting undercover on the Spanish civil war. Koestler spent months under threat of death and witnessed the executions of many fellow prisoners, which deeply affected him. By the mid-50s Koestler's opposition to the death penalty was beginning to be shared by many others in Britain after some notorious miscarriages of justice.

In 1953 a mentally retarded youth called Derek Bentley was executed for a murder in which he had been a barely witting accomplice. In 1955 Timothy Evans was wrongfully hanged on the evidence of a neighbour who turned out to be the multiple murderer being sought. Most controversial of all was the death sentence imposed on Ruth Ellis, a young mother who had shot her abusive and unfaithful lover in a fit of jealousy.

After an article on the Ellis case by Raymond Chandler and a follow-up letter by the publisher Victor Gollancz appeared in London's Evening Standard in July 1955, Koestler impulsively rang Gollancz to propose a national campaign. Gollancz enlisted Canon John Collins, then the head of Christian Action, and in August the freshly minted National Campaign for the Abolition of Capital Punishment (NCACP) held its first committee meeting. Koestler offered to write a book making the case for abolition, and he struck a remarkable deal with the crusading editor of the Observer, David Astor. Astor agreed to put several members of his staff at Koestler's disposal to carry out the research, and Koestler agreed to let his book be serialised by the Observer in weekly instalments.

Cynthia Jeffries, who was to become his third wife, had worked for him as a secretary when he lived in Paris, and had just arrived in England to spend a vacation with her family. Koestler shamelessly asked if she would work for him instead. Cynthia, who had been addressing her letters to Koestler from America as "your loving ex-slavey", didn't hesitate. After four and a half days with her mother and sister at the seaside, she was back in London and ready to begin.

To judge from her lyrical account of that summer, it was a fateful decision. Koestler was possessed by his subject with a fierce passion he hadn't experienced for years, and Cynthia entered into his mania so completely that they bonded in a wholly new way. Koestler began to refer to their project as a folie à deux and to Cynthia as his "junior partner". Cynthia was enthralled. "I sat in his study, curled up in the armchair beside the fireplace. Every bit of space was covered with books. We were living in a world of gallows and gibbets 'creaking and groaning with the bodies of criminals'. He had warned me that parts of the book would be stomach-turning. Sometimes he turned pale when dealing with the physiological facts about hanging and looked to see whether I could bear it."

Koestler worked frantically through the month of August and into September 1955. Whenever they broke for lunch at the pub or went for a drink, ­­Koest­ler would engage the publican and his customers in a discussion about capital punishment. The publicans were die-hard hangers to a man (as were most of their customers), and Koestler honed his dialectical skills by debating with them before returning home at night to dictate ever new arguments to Cynthia. On some nights, according to Cynthia, he would continue dictating even in his sleep. There was a heatwave that summer – a rare luxury in England – but they worked straight through it, and Cynthia postponed her return to the United States for 10 days to help Koestler finish.

For Cynthia it was a "magical" summer. She left for the US feeling "strangely elated" and wrote from New York to say she would love to come back and work for him again whenever possible. Continuing without her, Koestler finished his book, Reflections on Hanging, in October and dispatched it to Gollancz. After Christmas, Koestler's Reflections on Hanging was serialised in five lengthy instalments in the Observer, scoring a huge success with readers and not incidentally raising the sales of the Observer by a considerable amount (Astor's gamble had paid off). He argued that Britain was barbarous and out of step with of the rest of the world in clinging to capital punishment, and that the death penalty was immoral, ineffective, and bad law. He also set out to arouse readers' emotions against it by painting a vivid picture of its injustices and the sufferings of its victims. The tone was set in a preface in which Koestler recalled his memories of prisoners being executed in Seville.

"These three months left me with a vested interest in capital punishment— rather like 'half-hanged Smith', who was cut down after 15 minutes and lived on. Each time a man's or a woman's neck is broken in this peaceful country, memory starts to fester like a badly healed wound. I shall never achieve real peace of mind until hanging is abolished." Capital punishment was not just a problem of statistics and expediency but of "morality and feeling". Koestler had many more arguments against capital punishment up his sleeve, but two that have an undeniably contemporary ring were the difference in treatment for rich and poor and the prevalence of error in identifying the criminals. Money governed a great deal in capital cases, wrote Koestler, ranging from the type of legal advice the accused could get to the testimony of expert witnesses to the quality of the defence lawyers. As for error, there were far too many cases where the wrong person had gone to the gallows, and each fresh execution of an innocent victim was an additional stain on the conscience of society.

The serialisation of Koestler's book made a stirring impression on British public opinion, but its very success led to a seriocomic rift with Gollancz. He and Koestler were much alike – energetic, egotistical, ambitious, and domineering – and Gollancz was mortified when he returned from a business trip to America to find Koestler the toast of the town and top dog in the NCACP committee. "Don't people realise that I'm much more famous than Arthur Koestler?" he said to Peggy Duff, the committee's secretary and treasurer. When the House of Commons finally held its long-awaited debate on capital punishment, it voted 293–262 in favour of abolition, and Koestler, Gollancz, Gardiner and Duff were present in the visitors' gallery. The jubilant Gollancz announced victory and an end to the NCACP's meetings, but Koestler opposed him on the grounds that the Commons had voted for abolition before and never done anything about it. Once more he was prescient.

Prime Minister Anthony Eden decided to leave the issue of abolition to a private member's bill, which was not necessarily a kiss of death but severely weakened its prospects of passing. Gollancz and Koestler both offered to resign from the NCACP committee, and Koestler finally did so, saying he would continue to "write and work for abolition" but not attend meetings. True to his promise, Koestler persuaded Astor to let him write an occasional column for the Observer under the name of "Vigil", and his last column, published as a pamphlet entitled Patterns of Murder, was sent to every member of Parliament.

Koestler's frontal assault on the judges, and especially on Lord Chief Justice Goddard, hadn't escaped notice in the House of Lords, where the judges sat as members. Lord Mancroft, under-secretary of state for home affairs, accused Koestler of misquoting a confidential Home Office instruction to prison governors, and Lord Hailsham said that Koestler and the Observer should be investigated by the Press Council. Koestler did some research and found that the phrases he was alleged to have omitted had never been published – until, that is, Lord Mancroft revealed them in the Lords. He demanded, and got, an apology from the two peers, but it was so grudging that Koestler himself referred the matter to the Press Council, which censured the noble lords instead.

In the summer of 1956 the conservative House of Lords (egged on by the judges in its midst) threw out the private member's bill to abolish capital punishment, and it took another 10 years to get the death penalty suspended in Britain. Outright abolition came only in 1970, but it was generally agreed that Koestler's book, and the campaign he started, were hugely influential in altering the climate of opinion and making it possible. When the initial campaign was over, David Astor wrote to Koestler, "I don't want to sound sententious, but want to say all the same that I believe your 'hanging' journalism has contributed something of real value to this country and this newspaper. It is the episode that most deserves to be recorded in the history of this paper since I've been here. I'm very proud of being associated with what you've done."

Koestler also felt pleased with himself. "People say: 'He used to want to save humanity. Now a dozen souls are enough,'" he told a French interviewer some time later. "It's true. To snatch even a single man from the gallows is very gratifying. It's the whole point of my life's path. I don't believe that the end justifies the means… I don't believe any more in humanity. I believe in the individual."

Given his immensely strong views on the individual's rights over his own life – and death – it is no surprise that Koestler was drawn to the idea of suicide throughout his life. His first brush with it came in 1934 when he was upset by the rejection of an early novel called The Adventures of Comrade Dickybird and His Friends in the Emigration. But it was only a half-hearted attempt at suicide. The scene he describes (an open gas tap, a book falling onto his nose) is suspiciously reminiscent of a failed suicide attempt in the fictional Comrade Dickybird but, according to Koest­ler, it was a rare instance of life imitating art. He considered suicide again six years later when fleeing the Nazis after the German invasion of France. Having been on the run for some weeks, Koestler ran into several Paris comrades seeking escape in Marseille in early August. Among them was Walter Benjamin, who asked him if he had "anything to take" in case of disaster. Benjamin showed him 64 tablets of morphine that he had carried with him since his 1933 expulsion from Germany. Koestler had nothing, and Benjamin gave him half the tablets. Benjamin was on his way to Spain, and a few days later committed suicide after crossing the border. The Spanish police had arrested him and threatened him with deportation back to France, but by the time the police came for him he was dead.

Koestler learned of Benjamin's death on the last day of September, having finally reached Lisbon. He decided to follow Benjamin's example. Being Koestler, he jotted down some notes describing how he took a first dose of four tablets just before 1am and began to monitor his pulse and heartbeat. "No fear, slight (manic) agitation. Curiosity. Before I swallowed the pills I accidentally saw myself in the mirror and had a moment of panic. Now nothing of the kind, not even self-pity. It's hardly natural, I still can't believe that this whole thing is irreversible."

By the end of the first hour he had taken 20 pills, still without the desired result. A bit later he tried to imagine his own death mask and the "unappetising shindig" that would be raised in the Pension Leirense, where he was staying, when they found him. At 10 minutes past two he decided to swallow the rest of the pills, making 31 in all, and to lie down. At half past two, he wrote a last note before going to bed. "It's funny, I don't know whether tomorrow morning I'll wake up dead." But the next morning he did wake up, and vomited up the last of the pills. Either they were too old or he had made a mistake in taking them gradually instead of all at once. He never did discover which.

In 1969, 14 years before his eventual death, Koestler joined the British Euthanasia Society and supported a bill in Parliament to legalise assisted suicide. For a while he had contemplated writing a pamphlet on the subject but never got around to it and instead wrote a preface for the society's Guide to Self-Deliverance. In it he repeated the distinction he had made in an interview between the fear of being dead and fear of the process of dying. Mystics and believers had their faith, and if agnostics could be assured of "an easy way of dying", they too would be much less afraid of being dead.

Diagnosed with Parkinson's disease in 1976 and three years later with chronic lymphocytic leukaemia, Koestler was re­conciled to thoughts of his own impending death but was troubled about Cynthia. A scary car accident in 1979 had prompted him to wonder what would happen to her when he died, for she was, as he told Celia Paget, the twin sister of his second wife, Mamaine, kontaktlos [without contacts]. Celia said she'd be delighted to have Cynthia to live with her, though she didn't expect Cynthia would agree. Celia said she'd often thought about Cynthia's life if Koestler died first and the idea of it was "almost too awful to contemplate". Koestler had been the centre of her life for so long, and Celia saw, like Koestler, that Cynthia had "hardly any friends of her own".

Finally, in June 1982, Koestler sat down to write a careful suicide note. He made it clear that the decision to kill himself was his own, that he had legally collected and stored the necessary drugs over a period of time, and that he would take the drugs "without the knowledge or aid of any other person," a phrase intended to spare Cynthia any unpleasant legal consequences. He knew it was a gamble, adding that if it failed he didn't wish to be kept alive by artificial means, nor removed to a hospital. He said he wished his friends to know that he was leaving their company "in a peaceful frame of mind, with some timid hopes for a depersonalised after-life beyond the confines of space, time, and matter, and beyond the limits of our comprehension."

The most difficult part about making his decision, he wrote, was knowing the pain it was bound to inflict on his "few surviving friends", and "above all my wife, Cynthia. It is to her that I owe the relative peace and happiness that I enjoyed in the last period of my life – and never before."

Koestler and Cynthia moved back to Montpelier Square later that autumn and continued to have close friends over for the occasional dinner or drinks and Scrabble, but Cynthia had to bathe and dress Koestler every day, and it was increasingly hard for him to get through an entire evening. Soon after Koestler's 77th birthday in September 1982, Cynthia typed two new paragraphs at the bottom of his handwritten suicide note. The first read: "Since the above was written in June, 1982, my wife decided that after 34 years of working together she could not face life after my death," and was signed with the initials A.K. (in handwriting much shakier than in the body of the note itself). The second paragraph read: "I fear both death and the act of dying that lies ahead of us. However, I cannot live without Arthur, despite certain inner resources. Double suicide has never appealed to me; but now Arthur's incurable diseases have reached a stage where there is nothing else to do." It was signed with Cynthia's full name.

Koestler briefly rallied in the first weeks of 1983, but it was only a temporary reprieve, and after falling down a number of times he began to spend most of the day in bed, with a nurse to look after him. On 27 February, Cynthia rang their friends the Hungarian-born humourist George Mikes and Lady "Pinkie" Beckett to cancel their planned Scrabble games, and called another close friend, the literary agent Pat Kavanagh, to cancel dinner at a Hungarian restaurant. On Monday the 28th she cancelled the newspapers, and on Tuesday took her beloved dog, Davy, to be put down. She told the cleaner, Amelia, that she had taken him to stay with friends. Amelia left without seeing Koestler that day, and Cynthia arranged for her to return two days later, on Thursday 3 March.

That evening, with Cynthia's help, Koestler got fully dressed, put on a jacket and tie, and dictated two letters for Cynthia to type, one to his doctor and the other to his solicitor. Then he and Cynthia entered their sitting room and sat down facing each other, he in his favourite leather armchair, she on the couch, and poured themselves their usual drink before dinner. Arthur's was his favourite brandy, Cynthia's was scotch. On a small table between them was a bottle of wine, a large bottle of Tuinal sleeping tablets and a jar of honey. They sipped their drinks and steadily chewed on the Tuinal tablets coated with honey until one by one they slipped into a coma and passed away.

Koestler had planned the suicide down to the last detail, and had done his homework well. The barbiturates in the Tuinal were quick-acting, and their effect was speeded up by the alcohol. Koestler and Cynthia must have been dead within hours. Koestler had chosen a method that would leave the bodies (as he told Mikes one day) looking "dead but not disgusting". He had a lifelong horror of physical disintegration. "When a man fights his biological condition, he fights an enemy, and the body is an enemy; the smell of the body, everything about myself then becomes loathesome, infected, putrid, and intimacy becomes loathsome."

Thanks to the arrangement with Amelia, the bodies remained undiscovered for another whole day and a night.

The funeral was held at Mortlake Crematorium in south London on 11 March, 1983. Between 20 and 30 people attended, mainly friends and professional colleagues. Koestler had no near relatives, except for his unacknowledged daughter, Cristina. Cynthia had just one, her sister, Pamela Merewether. Two plain coffins with handwritten labels were brought out and placed side by side in the mortuary chapel. There was no religious service, just a reading by the undertaker of five passages from Koestler's Bricks to Babel. Their friend Julian Barnes was surprised by the undertaker's eloquence (not knowing the undertaker was an amateur actor), and watched as he solemnly placed the book atop Koestler's coffin. The small congregation filed out and there was a moment of silence before the coffins slid through the curtains to the oven. The ashes were later taken to their country house at Denston, near Cambridge, and scattered around a garden bench where Koestler and Cynthia used to sit together.

The above is an extract from Koestler: the Indispensable Intellectual by Michael Scammell, published by Faber, £25.

My purchasing habits are out of control, so I've decided to renounce the charms of Amazon and concentrate on stuff I've already bought

Inspired by Bibi van der Zee's powers of self-denial, I've decided to get all Puritan on my literary ass. I'm not talking about giving up books for a week: that would be weird; I'd have to talk to people. No, what I'm going to do is put an end to buying the little blighters.

We all know how it is these days. The faultline between desire and action has faded to a smudge. I'm not even sure I still bother going to Amazon. It's as if some dastardly sales whizz has infiltrated my brain, hooking my dopaminergic neurones straight up to PayPal. I read about a book. Mmmm... interesting, I think. And two days later it's sitting by my bed.

It last happened last week. I was enjoying the Kapuściński controversy and before I'd even bothered to finish the article I'd spent eight quid. The worst thing, is I'm really excited about my latest acquisition: Kapuściński's book on Haile Selassie. I recently read The Soccer War and loved it. But when will I get to it? What about Travels with Herodotus, which has been patiently awaiting my courtesy? And The Shah of Shahs, which I bought in Foyles in November? And these are merely my Kapuściński whims.

The oniomania has got to stop. I hereby impose a six-month moratorium on book-buying. (I was thinking of a year but I couldn't quite face it.) And now for the fun bit: there's a box in my room filled with unread purchases and I am systematically going to give them my attention. These represent only a fraction of the total (I'm currently "between" abodes, and most of my library is doing time in a depot in Norwood.) So this short list will have to do for now ...

• The Enchanter by Vladimir Nabokov. Reflexively bought on Amazon while reading about The Orginal of Laura several weeks ago. The pre-Lolita "throb". Another pederast and russet-haired colt. I wonder what will happen.

• The Secret History by Donna Tartt. My friends all told me to read it. My girlfriend told me to read it. I was passing a bookshop and I bought it. Then I didn't read it.

• The Book of Disquiet by Fernando Pessoa. I spent most of last year living in Lisbon, a gleefully self-referential literary city, strewn with homages to the dapper lusophone ventriloquist. Every day, I walked past the café at which he wrote. Every day, his book glowered at me from on high in our apartment. I think I might have to read it in a pub.

• David Copperfield by Charles Dickens. Also bought in preparation for my year abroad. I am going away, I said to myself one afternoon while browsing in Bloomsbury. I will have time to read big novels like this. It's a wonder I haven't read it sooner. It didn't even make my suitcase.

• A Jew Must Die by Jacques Chessex. Bought after a talk at a bookshop in north-west London a fortnight ago, mostly on account of the cheery title. It did win the Prix Goncourt though.

• Dangerous Liaisons by Choderlos de Laclos. An ex-girlfriend bade me read it, romantically enough, when we were strolling through Versailles. I think I may even have bought it later that day at Shakespeare & Co. Then I saw the John Malkovich film. And Cruel Intentions. And then I forgot about it.

• Walden by Henry David Thoreau. I have no idea how or when or why this ended up in my box. I suspect it might annoy me. But I've been carting it from place to place for years, so I think I should probably give it a go.

• For Whom the Bell Tolls by Ernest Hemingway. I have never read any Ernest Hemingway, I remarked to myself, alarmed, at a bookshop in Brixton a couple of years ago. How can I never have read any Ernest Hemingway? I have still never read any Ernest Hemingway.

• Operation Shylock by Philip Roth. I bought this during a mad spat of impulse buying while researching for a book. That's the problem with research: the more you read, the more you realize you haven't read. And then the more you buy.

• Guns, Germs and Steel by Jared Diamond. I always mean to read more science books. In fact, I always mean to read less fiction and more fact. The thing about fact is, even if it's boring it tends to make the people who read it less so. And Jared Diamond isn't boring. And any book that claims to answer the most "obvious", "important" and "difficult" questions about the whole of human history gets my vote.

NB: In light of the subject matter, and the numerous links that appear on this page, I would be interested to know if anyone has impulse bought any books during the course of this article.

Every writer of reportage ought to learn from the Kapuscinski controversy. Creative non-fiction is a slippery slope

Had he lived a few years longer, Ryszard Kapuscinski might well have won the Nobel prize for literature. Although these things are shrouded in Vatican-like secrecy, I bet that he was on the Swedish Academy's rolling shortlist. Journalists in many countries would then have hailed him as the first "non-fiction" writer to win it since Winston Churchill in 1953. Now a huge row has broken out in his native Poland over a new book which suggests that his non-fiction was not so non-fictional, after all. This row has already blown round the world, because Kapuscinski's name is a global byword for a certain kind of literary-political reportage.

I have just read the book, which is called (in Polish) Kapuscinski Non-Fiction. Its author is the journalist Artur Domoslawski, to whom Kapuscinski had been model, mentor and friend, and it has been criticised on several grounds. These include his handling of the travelling writer's allegedly numerous love affairs, which I do find insensitive, and of his communist past and occasional contacts with the secret police, which I think Domoslawski handles well.

More broadly, the book is condemned as being a denunciation of a former mentor. Kapuscinski's widow calls it "patricide". This is not how I see it. I find that the author tries to be fair, allowing many different voices to speak. He captures the Ryszard I knew, starting with a brilliant evocation of his warm, nut-brown, disarming smile. Literally disarming in Ryszard's case, because that almost pantomime-humble smile got him through many a dangerous confrontation with armed men, in Africa and elsewhere. But this book is the protracted cry of a worried and even a disappointed disciple – one who, in his nearly three-year journey of investigation, found things that deeply disturbed him.

The heart of the matter, for Domoslawski, me, and probably the wider world, is the frontier-crossing between fact and fiction. Some of us have been worrying about this for years. In 2001, to mark the 100th anniversary of the Nobel prize for literature, the Swedish Academy held a symposium on Witness Literature, delicately indicating that prizeworthy Literature, with a capital L, was not confined to fiction and poetry.

I gave a talk (now reprinted in my book Facts Are Subversive) in which I observed that "with Kapuscinski, we keep crossing from the Kenya of fact to the Tanzania of fiction, and back again, but the transition is nowhere explicitly signalled". In the same year, the anthropologist and writer John Ryle wrote a coruscating review essay in the Times Literary Supplement, documenting numerous inaccuracies, exaggerations and mythifications in Kapuscinski's writing on Africa. He argued that most of them tended towards what Ryle called the "tropical baroque", in which everything becomes more exotic, wild, savage, extreme and, dare we say, oriental. Now Domoslawski retraces some of the master's footsteps, to Addis Ababa, for instance, where Kapuscinski researched his famous book on the fall of Haile Selassie, The Emperor, or to Santa Cruz, Bolivia. He finds Kapuscinski's own witnesses complaining of inaccurate and fabulated material. There are numerous examples.

What Kapuscinski did is really no longer in doubt. The question is what we make of it. One school is represented by the American writer Lawrence Weschler, whom Domoslawski quotes as saying: "What does it matter which shelf we put The Emperor and Shah of Shahs on: fiction or non-fiction? They will always be terrific books." A schoolfriend of Kapuscinski says The Emperor is "the best Polish novel of the 20th century". And of course those books were also about Poland. They were read by Polish readers partly as allegories of their own situation, and they might have been blocked by the communist regime's censors had they not been firmly presented as non-fiction about far-off reactionary places.

A second school, which one might call "Ryszard's handwringing defenders", is well represented by Neal Ascherson, himself the author of superb reportage from Poland and elsewhere. Kapuscinski was a great storyteller, not a liar, he writes on the Guardian books blog, and there is an important difference between the news reporting and the books. But then he makes this, to me, very surprising statement: "Almost all journalists, except for a handful of saints, do on occasion sharpen up quotes or slightly shift around times and places to heighten effect. Perhaps they should not, but they – we – do." Really, Neal? And how much, pray, is "slightly"? And how far may one go in "sharpening up"? In the rest of his blog, however, Ascherson goes on to worry that Kapuscinski did not make it clear enough to the reader what he was doing.

The third school, to which I belong, says that even if there is not – as Ascherson vividly puts it – a "floodlit wire frontier", there is nonetheless a vitally important line, or frontier zone, that writers of non-fiction should strive never to cross. If we do cross it, we should put a different label on the resulting product. Domoslawski names one reason for this: simple fairness to readers. Readers need to know what they are getting. After all, at least some of the excitement of reading a writer like Kapuscinski comes from believing these things actually happened. He was there. He saw it with his own eyes. He nearly died getting the story. The rhetoric of his own writing often beats that drum.

The second reason goes deeper. There are, it seems to me, few more responsible callings for a human being armed with a pen than that of being a veracious witness to great and grave events. In introducing that 2001 Nobel symposium on Witness Literature, the then secretary of the Swedish Academy, Horace Engdahl, suggested that "truth is initially nothing but that which a credible witness certifies". This may not work as a universal philosophical rule, but it certainly applies to what writer witnesses do, especially when they stand alone amid tragedy or triumph. To bear witness to genocide, war, revolution and human courage amid inhumanity is – forgive the pathos – a sacred trust.

Yes, in our selection of facts, images and quotations, in our characterisation of the real people we write about, writers of reportage do work in many ways like novelists. But in recognition of that responsibility to history, as well as the "non-fiction" promise we make to our readers, we must stick to the facts as best we can find them. We must not change the order of events even "slightly", nor "sharpen up" anything that appears between quotation marks. We all make mistakes. No one sees the whole picture, or can be truly objective. Everyone has a point of view. But if I say I saw that, then I saw that. It was not in a different street, at a different time, or told me by someone else over a drink at the hotel bar.

I see two ways forward. One, humorously suggested by Domoslawski himself in an post-publication interview, is that in bookshops there should be a shelf between fiction and non-fiction, with a new category marked simply "Kapuscinski". The other is to learn from Kapuscinski's marvellous work, but also from his transgressions – and hence to bear truer witness.

Timothy Garton Ash will be talking with Jon Snow at the Frontline Club in London on 16 March

As Disney rebrands Rapunzel as Tangled, we imagine what other children's stories and fairy tales could be made more appealing to boys

Disney is taking no chances. Book publishers have long since realised that anything that sounds too obviously girly is a complete no-no for the unfairer sex – hence JK Rowling's books weren't published under the name of Joanne Rowling. Hollywood has taken rather longer to make the connection. But after less-than-spectacular US box-office receipts for The Princess and the Frog, the studio has decided to rebrand its forthcoming cartoon in an effort to win the little chaps back. So Rapunzel has become Tangled – complete with an all-action male swashbuckling hero. It's worth a go, I suppose. Here are some other titles boys might like to see.

Malice in Wonderland

Freddy Krueger has a day out in Alton Towers and picks off a coachload of schoolchildren one-by-one in a gore schlock-horror fest before a grinning Cheshire cartoon cat and his trusty dormouse lieutenant come to the rescue.

Red Riding in Da Hood

A young Che Guevara pimps his BMX bike and heads off to the Bronx to take out a gang of neo-fascist hyenas who have been terrorising the local community of multicultural zebras.

You Beauty and the Beast

It's the last minute of extra time in the World Cup final, the score is 0-0 and the game is heading for penalties, when Wayne Rooney starts his run in his own half. He beats one German Hofmeister bear, then another, and another, before curling the ball into the left-hand corner.

GI Snow and the Seven Dwarfs

Matt Damon flies south to Colombia where he rounds up his cute band of seven undercover chihuahuas – Sneezy, Dopey, Edgy, Wired, Wasted, Psychotic and Sleepless – and destroys the world's largest cocaine factory.

A revolutionary new system for achieving results in both personal and professional settings is now available from author JD Meier. Meier's new book "Getting Results the Agile Way" details his Agile Results™ system -- the smarter, faster, easier way to great results. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/2010/03/prweb3666474.htm

Carly Spindel plans to embark on a six-week program to find the perfect man. She will be testing out a course developed by her mother, Janis Spindel, America’s top matchmaker with two decades of experience. Then, she plans to challenge single women around the world to try the course for themselves. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/Dating_Advice/Matchmaker/prweb3667364.htm

In her groundbreaking book, Wander Woman: How High- Achieving Women Find Contentment and Direction (Berrett-Koehler Publishers, June 2010, paperback), Marcia Reynolds, Psy.D., draws on solid research and a wealth of clinical and personal experience to identify an emerging identity among high-achieving women. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/Wander_Woman/Marcia_Reynolds/prweb3703034.htm

Unique opportunity for an investor to acquire a senior, non-performing, whole mortgage loan collateralized by sizeable buildings (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/2010/03/prweb3706494.htm

Boost your children's self confidence by teaching them people skills and developing their character. Rob Heller's new book, "BoostKids: Teaching Kids Life's Most Important Lessons," is an excellent resource for parents looking to build their child's self confidence using BoostKids techniques. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/buildingchildconfidence/kidssocialskills/prweb3707804.htm

Inspirational writer and speaker Shawn Anderson (http://www.ShawnAnderson.com) introduces his latest fictional book, Amicus 101: A Story About the Pursuit of Purpose and Overcoming Life’s Chaos (http://shawnanderson.com/products). The motivational book highlights the teachings of Amicus, one of the world’s most unique and inspiring success coaches. Amicus helps his fictional student, Jay Garfield, overcome personal life chaos by sharing 21 empowering and life-transforming success lessons. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/motivational_book/self-help_books/prweb3708844.htm

Black Christian News: www.BCNN1.com / Black Christian Book Company: www.BlackCBC.com (Based upon Amazon.com rankings, BarnesandNoble.com rankings, and BlackCBC.com rankings.) (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/2010/03/prweb3702814.htm

Petey the parrot is absolutely distraught. He's an emotional mess. He's losing his feathers because of Jezebel, the charming jungle parrot that caught his eye on the vacation-paradise island of Mongo Tongo. Petey is on the mainland, living with his owner Mrs. Chairbottom; Jezebel had to stay behind on Mongo Tongo – and love-struck Petey is beside himself. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/2010/03/prweb3698624.htm

"Life Stories – Healing and Hope in the Wake of Loss," is Ted Hopkins' personal story and the stories of five other families of how they found life, love, and a future in the midst of painful tragedy and loss. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/2010/03/prweb3699374.htm

New Author David MacDerment liberates readers from the confusion culture imposes. (PRWeb Mar 10, 2010)

Read the full story at http://www.prweb.com/releases/2010/03/prweb3699594.htm