Archive July 2013
No noir story will match
the ones told by Big Data. In the future, noir stories will emerge from Big Data
only it won’t be fiction. Authors of crime fiction, noir, hardboiled, or
otherwise, are like monks writing manuscripts before the printing press. Our end
will be as noir as their end. Here’s the story of how that will come
I’ve thought of writing as
a way to discover and explore vanishing points, light fading to the void of
total darkness. That is the point where we can no longer predict what will
happen next. It is a brick wall. A blank. We stop at the door to the future and
are resigned it will never open.
Data: A Revolution, the authors Viktor Mayer-Schoenberger and Kenneth Cukier
have opened that door a crack. But don’t buy this book. You don’t seriously want
to know what is inside our near future in the Data-Time-Space
Towards the end of this
provocative book, the authors sum up: “The ground beneath our feet is shifting.
Old certainties are being questioned. Big data requires a fresh discussion of
the nature of decision-making, destiny, justice.”
That is only the beginning
of the transformation that will happen in our life time. It is already
happening, it’s started to come into the open. The huge weight and force of Big
Data and the hunger of power to own it, share it, distribute it, and exploit it.
We are in the middle of that big data war. Government officials and big business
owners are in their bunkers figuring out what to do next. No one has explained
clearly what is at stake, the options, or the current state of play. Big
Data, A Revolution attempts to provide context and meaning in an era where
data is no longer scarce or expensive, but readily available and infinitely
valuable in making predictions about future outcomes.
attitudes, and mental states will be predicted with an advanced probability
software and hundreds of millions equations—and that raises a number of
It is happening now as you
read this essay. You are the composite of your data; your choices, likes,
purchases, friends, emotional connections, and routine have been datafied. This
data of your past can’t be erased, deleted or changed; it will follow you
wherever you go into the future. The days of starting over are finished.
You can never go missing or disappear completely as you pull behind
yourself a history that is your digital DNA.
Your mental thumbprint is
now in the system and attached to this blog. It stopped there. Who else who has
ever read this blog is an association? That data is stored in the system.
Websites, blogs are hovered for information, and this how Big Data continues to
grow four time faster than America’s G.N.P. There is a probability that
your digital presence here means that you may share certain habits, buying
traits, or be connected to some free thinking troublemakers who also visit this
You can no longer control,
handle, supervise or understands the scale and scope of your data or the Big
Data. But we have seen nothing yet. Big Data is set to grow exponentially. Some
of that will be extremely useful in understanding and dealing with important
problems like climate change, curing diseases, or advancing entire domains such
as physics, chemistry, and mathematics. The assumption is that our understanding
of the world, describing it, predicting it is a limitation on quantification of
To fully exploit the
potential of big data we need to appreciate the scale and scope of the power
that comes from collecting, storing, distributing, selling and analyzing the
range of correlations that emerge when N=All. We will also pay a substantial
price. Big Data is not ours without some long-standing beliefs, habits,
attitudes and customs being changed. The next stage of development are data.
They are being built from masses of data as you read this essay. Real economic,
social and political owner will reside inside them.
Since the thirteenth
century, we have searched for answers about the world and behavior that are
precise and exact, and we seek out causation between events, people, and things.
Our quest is to know if what we believe about the world is true or false, right
or wrong, good or bad—we bring our moral and emotional sense of being in the
world in the cross-hairs when we address the implications of Big
Big data works not off
exactness; it is premises that reality is messy and the data can provide a
probability of what will emerge in the future. Big data promises a set of
predicted outcomes according to a scale of probability based on what will likely
happen. In turn, we give up the mission to understand why something has happened
or may happen. The ‘why’ question is one that asks about causes to explain what
is the nature of the world. Big Data leaves causation to the side because it is
not helpful. The messiness of reality renders inquires about causation and
precision less reliable. These ideas spring from an the old way of thinking when
sense had to be made out of limited information and data. Causation and
precision are relics of data scarcity and can be largely ignored as correlation
is sufficient in the world of Big Data. Limited or Little Data required us to
formulate a theory about what we’d expect the Little Data to prove, and then we
used the Limited Data to test as to whether it had proved or disproved the
theory. Think of climate change and theory of CO2 concentrations as the cause.
That’s the old way of using Limited Data modeling.
Randomness in large big
data gives a probability analysis that is more useful and predictive than a
targeted, sample size of data. Sampling of data, the default measurement of the
world, has become or will very soon become obsolete. Those conducting the data
gathering in the past lacked the tools (processing speed, storage facilities,
etc) to collect big data and the tools (software and algorithms) to analyze such
vast quantities of data. They opted for precision, sampling, and theory testing.
This old paradigm goes out the window with big data in many cases. With the full
dataset offered by big data, researchers can explore many more angles and
perspectives whether it is predicting the next bird flu outbreak or match fixing
in sumo wrestling matches in Japan.
Big data has the capacity
to scale entire populations of a city, region or country. Now when all telephone
calls, emails, Internet searches, Twitter mentions and retweets, and Facebook
‘likes’ are captured and stored, this isn’t a sampling; it is the whole
enchilada. “[W]e can accept some messiness in return for scale. ‘Sometimes two
plus to two can equal 3.9, and that is good enough.’”
We already have an example
of the limits of our capacity when tested against advanced algorithms. There are
chess algorithms that are used once the computer has six or fewer pieces left on
the board and allows the computer to processes the probability for
every possible move (N=all). The Big Data authors conclude,
“No human will ever be able to outplay the system.”
We have created a big data
system that is much better at making predictions about outcomes than we can make
using our native brain power. We humans have dropped down in the league ranking
of the best, fastest brain processing capacity in the world. In coming up with a
translation program, Google didn’t test a billion words, they used a trillion.
Its services cover 60 languages and are more accurate than other systems. It
won’t be long until computer translations, like playing chess, will perform
vastly better than any human being.
Big Data also
demonstrates the transition in thinking between viewing the reality of the world
as not only messy but one in which predictions of what will happen rest on
correlations that emerge from big data. Amazon has recommendations for you. Each
time you visit Amazon they remember your digital history and present you will
the kind of books that from your prior purchases indicate you are a ‘reader of
interest’. One-third of Amazon’s business is from buyers like you who
click on and buy the recommended purchase. For Netflix the percentage of online
rentals that come from a recommendation is seventy-five percent of all the
Amazon and Netflix offer
two good examples of how using probability tools can increase the revenue of a
company. There is no certainty that you will buy the recommended book on Amazon
or rent the recommended film online from Netflix, but you can see the
probability makes the effort pay off in rich rewards for both
Big Data can’t tell Amazon
why you buy a particular title. Indeed it is not interested in the why
question; it is focused on what you are likely to buy given your past purchases
and searches through their catalogue of books. The data opens up links that are
also useful. A secondary use of the same big data may show that California
international crime fiction readers are more probable to book a ticket to
Beijing, Tokyo, Hong Kong or Bangkok, and targeting them with discounted fares
may increase sales. The big deal about Big Data is that it has the potential for
multi-uses, and many of those uses only become apparent much later. That’s one
reason why storing data for long periods is in the interest of business and
governments, and they will fight to keep this option; they want indefinite
storage as they can’t predict what future technical and social dynamics might
arise and they want all of the cards, old and new, on the table.
We were born into an
information poor world. Our beliefs, political and social structures, our
science and education were created out of a small sampling of the information
about the world. We’ve spent our life making decisions, forming opinions and
making judgments based on limited data giving us precise, exact answers as to
the state of the world and each other. We are wired to look for causation. In
the big data world we are told this is delusion. There is no math that can
easily show causal links; but correlations are easily translated into
Big Data, the
book, looks at the risk of big data as it presents a real “risk [of] falling
victim to a dictatorship of data.” While Amazon uses algorithms to recommend
books, lawn mowers, watches, and clothes to you, there is the potential for
repression if the gathering, storage, use and distribution is left to be carried
out in secret. We don’t know the limits that push back against the collection
and use of Big Data. In a generation, people will look back and see our time as
the tipping point when we lost privacy. The big data world will continue to
strip away the possibility of privacy. Privacy existed because of the messiness
of information, it’s limited nature and the expense and difficulty of collecting
information about the world. You once had the power to divulge personal
information. In the average day, you willingly and largely unknowingly disclose
pieces of data about yourself—your likes, dislikes, activities, friends,
purchases, health, schooling, and plans. We’ve uploaded our life onto the common
Big Data network, a small fragment at a time, and by doing so we are forfeiting
our own privacy. Privacy as we know it will vanish.
Crime and punishment will
change as will opinions about proof beyond a reasonable doubt, and presumption
of innocence. If big data can show a correlation between a person’s big
data/information file that he has, say, a 79% chance of committing rape or
murder within the next three years, will the state make a decision that a
‘probable perpetrator’ should be removed from society in order to protect
society? The state would hold this person not because he’s committed a crime but
the prediction is high that he will commit the rape or murder in the future.
Many people may feel that with a high probability that the state should
intervene and prevent the harm from happening.
The Big Data
authors find that “the very idea of penalizing based on propensities is
nauseating.” The future causes a sense of vertigo. It doesn’t share our values,
our thinking, or account for difference between potential actions and the real
thing. The authors fall back on the premise that it isn’t the problem of big
data but the way we will use the predictions. The irony is the book is a call to
loosen our fixation on causation and theories, and to learn to embrace messiness
and predictability. When push comes to shove on preventive detentions, the
authors retreat back into the world of causation and find decisions based on
predictions ‘nauseating’. My view is once we jettison causation in the big data
world, the use of predictions won’t be easily caged inside Amazon and Netflix’s
world of recommendations. The data will get bigger, the prediction more
accurate, and once that happens ‘assigning’ guilt based on a person’s
particular act will appear as another example of medieval thinking.
An important takeaway from
Big Data is, “In the era of big data, however, when much of data’s
value is in secondary uses that may have been unimagined when the data was
collected, such a mechanism to ensure privacy is no longer suitable.” The debate
we will soon have is what is the continuing role of human agency in deciding
individual responsibility for actions. Another part of that debate will be
whether the decisions of big data will ultimately be made by machines. Humans
will likely never fully understand or control the moves any more than an
international grand master of chess in a game against Big Blue. Time moves on as
does the debate; and the tools continue to improve, faster processors, larger
memory capacity, better algorithms, and we wake up one day to find that
“rational thought and free choice” are no longer part of a world that we
The data story doesn’t end
with Big Data. There is no endgame as has always been the case with new
technologies. Each innovation seems so incredible that we can’t imagine an
improvement Remember the Beta cassettes? Our current technologies for Big
Data will look like Beta cassettes in 5 to 10 years. Probably much sooner. As
the period of change has accelerated from centuries to decades to years and
looks ready to upend existing technologies in months. This period is a prelude
to a much bigger transition in humanity’s quest to understand the world, and our
place in it. We have gone “from compass and sextant to telescope and radar to
today’s GPS.” Compared to the promise of what lies in our immediate future, our
existing technologies to harness Big Data will be judged by future generations
as closer to finger painting a horse on a cave wall.
Buy Big Data and
give it to someone you want to give a freight load of sleepless nights. My
predictions about scale and scope of big data, what will replace it, and how we
will change our values and attitudes as a result, are beyond what we now know.
It seems that all bets are off that this transition will be easy or smooth.
Adjust to the fact that others will have infinitely greater information about
you than you can ever imagine. You have become datafied. You can’t shake free,
you can’t hide, you can’t go missing, and you can’t even hold your own
The founder of Amazon has
bought The Washington Post. Will the owner use the newspaper to suggest
recommendations to politicians and others as to what policies, regulations and
laws are the ones they should adopt? Will somewhere between one-third and
seventy-five percent of The Washington Post click on and download those
recommendations into their memory? The sale of the Washington Post is
not just another sale of a newspaper to someone who is very rich, it is the sale
of the newspaper to one of the founders of the new paradigm of gathering and
distributing information. It is as if the owner of printing press bought a
failing monastery and scribes writing manuscripts. You know that change is
You’d be a fool to bet
against the odds that one morning you we wake up to the fact that you live
inside a data panopticon and there is anyway out. Not heard of
panopticon? Get use to seeing more reference to that word. It is the prevailing
metaphor of our time.
Psychology, economic, law
and mathematics have interesting perspectives on the dynamics between two or
more people who must decide to co-operate or betray the other person to minimize
Here’s an example of how
the Prisoners Dilemma works. Two suspects, Larry and Carl are arrested after a
warehouse break in. The circumstantial evidence indicates they were the guilty
party. Circumstantial evidence may be insufficient convict, and if both of the
suspects co-operate and say nothing to the police, they will both walk free.
Experienced criminals know the, but not all suspects are experienced and they
are anxious and afraid and the good cop/bad cop can do wonders to convince one
to defect and incriminate the other person.
Larry is told that if he
co-operates by testifying against his partner, Carl, then Larry will walk out
free and Carl will get three years. They also tell Larry that Carl has been
offered the same deal, so don’t wait too long or it will be you serving the
three year stretch while Carl is out spending the proceeds of the warehouse
Does Larry trust Carl
enough for him to call the bluff? Or does Larry think that Carl is weak, selfish
and likely to crack, thinking that Larry will take the deal and screw
Both are better off
co-operating. Game Theory is based on the premise that you are better off
betraying your partner and escaping the penalty you’d receive if you let him
betray you first.
The Prisoner Dilemma is a
dilemma for a good reason—it demonstrates the relationship between the duality
of our mental processing. We are at once both rational and irrational actors. At
any given moment, the scale tips toward one or the other of these two
If both people are totally
rationale, they co-operate in that way they are both better off. As we know, the
irrational mind is filled with anxiety, fear or worry that the other person
won’t act in a rational way.
Some clever academics
(economists of course) decided to test the Prisoners’
Dilemma on real life prisoners. The payoffs were in coffee and
cigarettes to the prisoners. The women prisoners who participated in the
experiment were housed at Lower Saxony’s primary women’s prison. The results
were compared with a Prisoners’ Dilemma experiment with students.
The researchers thought
the prisoners would be more cynical, hardcore and less likely to co-operate. The
result surprised them. The results were in three categories: simultaneous game,
pair basis, and sequential game.
In the simultaneous game,
the women prisoners co-operated 56% of the time while the students came in
second at 37% in cooperation. In the pair basis category, the actual prisoners
had the best outcome and co-operated 30%, compared to just 13% among the
students. For sequential games, way more students co-operated (63%).
The telling test is the
simultaneous game, which is based on blind trust. The suspects have no precedent
to go by. The conclusion reached in the experiment is the actual behavior of
people fails to correspond with the prediction made by the Nash Equilibrium—that
says it is rational to defect, though it has been noted that Nash (The
Beautiful Mind was the film based on his life) was paranoid at the time he
came up with the Nash Equilibrium.
There are criticisms of
the experiment. First, the actual prisoners after the game ends must go back to
a prison environment and if they’ve betrayed another even in a game that might
offer nasty blow-back once the experiment was over and the prisoners returned to
the prison population. Also, those who come from a crime sub-culture have the
ethos of co-operating against the ‘system’ or the ‘cops’ and close ranks when
outsiders ask them to betray one of their own.
Another commentator has
suggested that the test subjects were women and that women are more likely to
co-operate with each other than men. Others have come to the opposite
conclusion—men are more co-operative with each other than women.
Other factors might be at
play. Cultural attitudes about co-operation are important in Asia. Could it be
the outcome of the Prisoners Dilemma turns, at least in part, on underlying
cultural attitudes? This expands the inquiry into the ethnicity, culture,
language, gender and class of the prisoners and of the interrogator. One should
not assume that all three parties will share the same set of cultural
Beyond culture is the
environment of the experiment. In other words, the setting of the interrogation
is another factor that has potential importance in the outcome. Suspects held at
a police station are in a different situation than suspects held inside military
prisons or safe houses where water-boarding, torture or other enhanced
interrogation methods are employed.
Would two Japanese
criminals be more likely co-operate if the interrogator was an English, Canadian
or American cop? Or if one of the criminals was Chinese and the other Thai, and
the Americans interrogated the two men about Golden Triangle activities, would
they co-operate or defect? Would it matter if the interrogator was a woman of
Norwegian ancestry and the suspects Asian men? If the suspects are a mother and
daughter, does this relationship make it more or less likely one will defect?
Generational difference between the suspects may be another factor that
influences the suspects’ decision.
The point is how we go
about how two prisoners placed in different rooms and under great stress reach a
consensus as to the best course of action is clouded by criminal mentality,
cultural norms, gender, prior relationship (and ongoing relationship) between
the parties (and their families).
How we calculate our
self-interest is rooted in what our culture teaches us about the self, the
individual, and the community.
|When Godot is an Assassin and You Don’t Have to Wait
The 2013 Thai Most Wanted
Hitmen list has 100 names. The 2011 list had only 75 names. That’s a 25%
productivity and employment increase in two years. If this were the economy,
people would be in the streets celebrating. This list is not Thai companies on
the stock exchange but a list of Thai hired killers who are in a bullish
Like the Booker Award, the
2013 list is a long one. We’ll get to the short list and the machinery to choose
the winner a bit later. No literary award I am aware of has ever announced a
long list with a name of 100 authors. In the real world, down those mean streets
walk not writers taking notes for a great crime novel but hired killers the
police would like to catch. And there are at least 100 of them, which works out
about 5 or 6 hitmen for each author on a typical crime fiction award
Authors must choose their
hitmen carefully. It seems there are difficulties in apprehending the Most
Wanted Hitmen—they are even more careful than most authors. After all they have
a lot more at stake, and more to lose.
Thailand law enforcement
challenges aren’t unique (though what country exists where the citizens in huge
numbers don’t believe this?). The police in every country face the same set of
problems—suppressing crime and capturing criminals who refuse to be suppressed.
Techniques of crime suppression and catching the bad guys are glimpses into the
culture of the legal justice system and the social system.
The Thai police have used
Most Wanted list and have made what translates as ‘criminal suspect calendars’, which feature a photo of the
bad guys (or bad women). Maybe the photographs were old, blurry, with bad
lightning and horrible angle—the usual things people say about my photos. In any
event these calendars (we’re not told where they were displayed) failed to bring
phone calls from the public with information that they just saw what looked like
#73 eating som tum at a food stall on Sukhumvit Road. The police phone
didn’t ring. Or if it did, the caller wasn’t reporting the location of a wanted
Faced with the bold
facts—can’t suppress them, can’t catch them—the police decided on a new campaign
to hunt down the gunmen for hire in Thailand. Social hierarchy is the lifeblood
of Thai society—and the building blocks are the Lego like tropes of family
names, titles, rank, private schools, and private clubs. A Thai can step back in
any social scene and immediately experience another person’s place on the
pyramid grid as though they had a sonar system that picks up frequencies that
foreigners simply don’t perceive.
Why not rank hitmen? That
seems like a logical extension to the normal way people perceive themselves and
others—they are either above or below you. This genius for ad hoc hierarchy
making as a blueprint for hitmen pyramid is far more impressive than anything
you’ll ever find in Egypt. If you are raised and educated in seeing social
relations as pyramids, why not adapt that idea to how you design your Most
Here’s how the new Most
Wanted Hitmen List will work—according to the Thai police.
Level one is for the top
gun. The Professional. A Level 1 hitman has proved himself capable, reliable,
with many successful assignments on his resume. The assassins on this list are
not limited to those wanted under an arrest warrant. Apparently just because
you’ve committed an assassination doesn’t automatically mean you will have an
arrest warrant issued. The example given by the authorities is the hitmen
who has just been released from prison having served time for his last job.
Apparently the concept of double jeopardy gives way to preventive action. Once
you’ve done your time for a hit, you are a Level 1 guy would is wanted by the
The Hired Gunman Pro who
is always wanted by the police, arrest warrant or not, is at the top of the
hierarchy. It is important to emphasize this point so no one is confused or
walks away from a citizen’s arrest of such a hitman who might argue there is no
outstanding warrant. Get the guy. Bring him in. If you’re working at Level 1,
the police want you even if there’s no paperwork other than the list. The
privilege of the top rank is to be always wanted.
There’s always some new
guy breaking into the game. Same as in sports. One day you are kicking in goals,
and the next day you’re on the bench because some new kid can kick the ball
better and farther than you. These are the semi-pros looking for the chance to
play in the PGA-level hitmen’s league. They are still building a resume showing
their wins. The police warrant these are the most dangerous players—young,
hungry, trigger-happy and as resume obsessed as a student trying to get accepted
for a Harvard MBA program. The police statement was silent as to the necessity
of any outstanding arrest warrant before such a person goes on at Level 2. It
might be that the arrest warrant exclusion is for only Level 1—give them a bit
of hierarchy pride. As it is unclear, no doubt it could lead to arguments, and,
no need to remind you, these people are heavily armed, that is never a good
thing in Thailand.
Level 1 and Level 2 are
your pro or semi-pro freelance, free agent players. They take assignments from
anyone with the cash and the desire to see someone dead. The Level 3 hitmen are
a different breed. They fit the mode of the in-house lawyers. They work for an
influential figure or the mafia. Yes, in Thailand there is apparently quite a
distinction between the two categories worth an essay on its own. The
third level players raise an interesting policing issue. Why not check with the
godfather, “Seen #43 recently?”
“No, he’s been on the sick
list,” godfather. “No, he’s been transferred to sales and is attending a seminar
“Well, if you see him,
give us a call.”
“You’ll be the first to
Level 3 is the place where
no one ever seems to find any evidence. It all disappears down that Alice in the
Wonderland rabbit hole without leaving a tiny, bitty trace. The gunman signs on
for the usual company benefits, and enters the workplace where whatever evidence
he leaves behind will magically disappear, and he draws a regular salary. The
police admit Level 3 is a toughest nut to crack.
We are at the bottom of
the pyramid on a dark night. In a sand storm. In the desert looking for whom?
These guys are not yet qualified to be hitmen. No, they’ve not earned their
stripes. The most you can say for them is they’ve murdered people in a conflict.
That’s not what professional killers do, who have no emotional connection with
the victim or conflict. The police want to put a lid on the possibility that
these hot-headed, hot-blooded killers who get into lethal fights and arguments,
don’t suddenly become cool under fire, chilled water running through their veins
and climb up to either Level 2 or 3. The greater fear is a lateral entry into a
Level 3 position with a godfather.
Supposedly 30% of the
Level 4 killers have contacts with the Level 3 players and bosses. This assumes
that bosses at Level 3 given a choice would take a level 2 or Level 4 guy.
In a pinch, a Level 4 guy might be given a chance to see if he can kill someone
he doesn’t hate without first punching him out. As a general rule, it’s horses
for courses in the play book for most godfathers.
The Thai police, despite
the limitations of the list, have an Ace up their sleeve. Thais are highly
sociable. They are hard to separate from their parents, friends and relatives.
The police have figured there is no level of assassin, which can sustain
isolation. The loneliness of being on the run is too much for the Thai hitman
who will sooner or later head to his parent’s house, his favorite mia
noi’s room, and the hangout where he drinks and sings karaoke with his
friends. The idea is the police will look for clues among the hitman’s relatives
and close associates.
No discussion of hitmen
can be separated from the price ticket for their services. The no frills, basic
level hit of an ordinary person starts at Baht 50,000 (or roughly US $1800).
Most of the hits at the low end of the market are the result of love affairs
that implode like a star that blows up. Only in this case, the black hole is
between the eyes. If the target is a ‘somebody’ in one of the other social
hierarchies, the price can shoot up.
How have the Thai police
been doing in catching the professional killer included on the 2013 Most Wanted
List? Six months into 2013 they’ve arrested four, and two have died. There is no
report on what level these 6 hitmen came from. The main takeaway is that your
chances of being arrested for being an assassin for hire is only slightly higher
than dying of old age. The next time someone mentions the word ‘noir’ in terms
of crime novels, you can ask them, “And what is your view on how the Most Wanted
Hitmen List for 2013 fits into the definition of noir?” To answer that question
would require a multi-volume series and given a dozen books, I’d only be
sweeping the sand from one side of the path leading to the base of the pyramid
only to watch it blow back the next day.
|When the Cuckoo Calls Your Name: A lesson in success for writers
Most of the time we humans are predictable in our reaction to the success of others. Anger, jealous, envy, hatred and self-doubt spill out like pennies in a clay piggy bank hurled against a brick wall. Another person’s success is felt like a punch in the face.
In the entertainment business, the gag reflect is in full swing.
Our hackles rise reading articles with openings like this:
Robert Downey Jr.
Robert Downey Jr. claims top earning spot with $75 million last year thanks to his role in “Iron Man.”
How many actors who are waiting tables in New York, Los Angeles, London and Paris dreaming of their big break would like to make one percent of that amount? The chances are they won’t have commercial success. They will never experience a year or a career like Robert Downey Jr. But that is hardly Robert Downey Jr.’s fault. Nothing in the universe was set to make his rise to fame and fortune inevitable. It could have been another actor. It could have been you.
Writers face the same problem. A handful of authors make the lion share of money from writing. James Patterson, Dan Brown, J.K. Rowling, John Gresham, Stephen King are some of the familiar names guaranteed to deforest mountains in British Columbia, to sell container loads of books, to dominating bestseller list, book review coverage, and public perception of how to measure a writer’s success.
It is the .001% of authors who are profiled in the major press, and the press never fails to mention the money they earn, the number of rooms in their house, private planes, boats; how they are cocooned inside a wall of well-paid staff. The 99.999% of writers scramble with other jobs to cover the cost of their rent, food, and transportation cost. Outside of a few lions, the rest of the animals roaming the literary savannah survive on near starvation rations.
Like Robert Downey Jr., the James Pattersons and J.K. Rowlings hit the big time. They were in the right place, at the right time, and not one of them, their agent or publisher would ever have predicted the scale of such success.
The idea of scaling hasn’t been discussed in the saga of Rowling’s The Cuckoo’s Calling. For those who haven’t followed the disclosure of Rowling’s novel published under another name, he’s a brief summary.
When J.K. Rowling sought to go undercover and write a crime novel titled The Cuckoo’s Calling under the pen name Robert Galbraith she discovered what most non-famous writer already know. It is tough finding a publisher, and having found a publisher, it is even more difficult for a really good crime novel to break out and acquired a Harry Potter-sized audience.
A couple of points worth noting, from everything I’ve read about J.K. Rowling, she is a decent, kind, sincere and genuine person. She doesn’t need to prove anything as J.K. Rowling. She has a brand. She knows that and like any author she must have in the back of her mind a doubt she’d like removed. That doubt is whether a novel written without the brand attached would find a publisher. The Cuckoo’s Calling had been rejected by a number of publishers. Rowling’s own publisher and editor decided to publish it under the pen name.
They created a fictional bio for Robert Galbraith and sent it out for review. Indeed the book received a good reception among critics (The Cuckoo’s Calling had good reviews). But the sales told a different story. Given the publishing world has something called a returns right—meaning bookstores buy the books but have a right to return unsold copies for a credit—the sales of The Cuckoo’s Calling range from 500 to 1500 copies.
A don at Hertford College, Oxford named Peter Millican created a software programe that could compare the text of one book with the text of books by famous writers. Professor Millican told the BBC, “I was testing things like word length, sentence length, paragraph length, frequency of particular words and the pattern of punctuation,” he explained. He concluded the probability was high that Rowling was the author of The Cuckoo’s Calling.
A book that had small sales under the name Robert Galbraith was now on the bestseller list. The limited hardback edition of the Robert Galbraith books is now going for up to two thousand pound sterling. The failed attempt to experiment with publishing outside of the brand name J.K. Rowling, has given a good insight in the concept of scaling.
When you aren’t famous and you write a book, you are no different from any other person with a product or service that is untested in the marketplace. Markets come in various shapes, forms and sizes. The market for your novel might be for yourself, family and friends. When that market is saturated, you’ve had your success. The problem is that most of us think the market for what we write has a larger market. You might be the star of your community theatre but your heart is set on Broadway and Hollywood. The same for an author who has a community theatre-sized audience for his or her book believes that he or she is one review away from a New York deal.
How do you know if the book you’ve written will ‘scale’ from an audience of a couple of hundred, or a couple of thousand, to millions around the world? The answer is you don’t know. No agent or publisher knows either. The same with films even with established stars, no one is sure whether the movie will scale and capture a huge market or flop like a fish in the bottom of a boat.
Inexperienced authors judge themselves by the standards of established authors. When their book doesn’t have J.K. Rowling success, they feel like they are a failure. Status in the entertainment world—film, painting, photography and books—is bestowed by measuring commercial success. And commercial success is what we call a work of art that scales much like the Big Bang from a pinpoint to an entire universe in a nanosecond.
Most books are fragile in the marketplace. They never ‘bang’; they whimper and die and are assigned to a potter’s literary grave. In retrospect, we can say the book didn’t scale because the subject was too narrow, the writing not artful enough, the characterization weak, the story derivative and a hundred other reasons that support the decision of the marketplace. None of this is to be taken seriously. Anymore than an analysis as to why someone believes the stock market dropped 5% in one day, or an earthquake hit China.
Those authors whose books scale across the literary universe are not necessarily some rare literary genius. There are hundreds of writers who have published books as good as or better than the one people line up by the thousands at midnight to buy. J.K. Rowling was on welfare, working out of coffee shops. She had no special connection in the literary world. No doubt she can write, but with Harry Potter she won the literary lottery, and most likely, like most lottery winners was as bewildered and surprised as anyone else.
Authors without broad brand recognition doom themselves by using the J.K. Rowling measure of success. Her lesson with The Cuckoo’s Calling published under another name is that the talent of a writer, any writer, is only one part of the complex network of gears grinding below the surface of life. Once in awhile the great machine produces a book that explodes, gathering millions of onlookers, both readers, occasional readers and non-readers. The author’s life jumps from the book review pages and lands on vastly larger stage of the news and social columns. The author becomes newsworthy, her houses, cars, boats, her likes and dislikes, what she eats for breakfast, her charities and hobbies, and her lectures and travels. A celebrity is born and like any new star shines bright.
How or why this mysterious event happens to anyone particular author is difficult to explain. But this has happened before and will happen again. When the audience for a book scales on the order of magnitude of the Big Bang, nothing can ever be the same again for that author. Whatever he or she writes thereafter will enter the public consciousness. Attempts to hide behind another name will likely fail. That new star in the literary sky just doesn’t twinkle, it dominants the literary sky and most of asteroids in the vicinity disappear from sight.
If you are a writer, you won’t allow bitterness and regret to color your opinion of the success enjoyed by authors such as J.K. Rowling. You will make a decision not to expend emotional energy over what you can’t possibly control. You will also understand that the essential feature of any author’s life isn’t whether the book scales to reach the mountaintop of the richest, but whether the author has gone into the world and climbed mountains. Be the writer who has put experience of life above striving for status.
Be the writer with an inexhaustible curiosity, a hunger for knowledge, and a humility that goes hand in hand with a wisdom that the world each day has something new to teach. Be the writer who disconnects from the Internet, cell phones and TV, and goes out into unfamiliar neighborhoods and observes the lives of people you wouldn’t otherwise meet. Be the writer who is the student and not the professor. Be the writer who is a child and not a parent. Be the writer who withholds making a quick judgment.
Be the writer who gets out of the apartment or house and enters a courtroom, a classroom, a prison, or a hospital and who watches the flow of people passing through these public places. The people in these places have lives worth understanding, and they will share their secrets, dreams, desire, disappointments and pain. Many of them are inside these places which cause them stress, duress, and anxiety. Here you will find courage, desperation, corruption, hatred, love, hope, depression, the elements that define who we are and the nature of our troubled times.
If you want to embark on a path as a writer, enter the flow of lives around you. Leave your comfort zone. Be the writer who explores cultures, religions and languages to discover the forces that shape our differences in perception, understanding, and emotional reactions.
After this exploration, whether your book scales to the higher elevations of J.K. Rowling’s commercial success, it won’t matter. You will have scaled to the top of your personal intellectual and emotional mountaintop, planted your flag and looked out on life in a way that few ever will. That, my friend, is success.
|4th Year Anniversary of International Crime Authors Reality Check
On the 15th
July 2009 a small group of writers joined together to write weekly essays for
this blog—International Crime Authors Reality Check. We were and remain
novelists who write essays once a week. In those essays we test notions of
‘reality’ in the context of social and political issues of the day. In these
essays, we have patrolled the borderline between good and evil, right and wrong,
facts and opinion.
Crime fiction has helped
shape our world of ideas about social justice, the way actual legal systems
function in other countries, and the way modern technology continues to change
the nature of criminal investigations and indeed the nature of crime.
Non-fiction is usually thought to be about truth and mirror reality. But often
it is fiction that comes closer to the mark in describing truth and reality.
That irony isn’t lost on the bloggers who write for you every week.
I’ve logged 214 essays
since 15th July 2009, and my fellow bloggers have more than pulled
their share of the weight. It takes a special breed of crime writer to
consistently produce essays each week. We have a number of distinguished alumni
who have written for the blog. It is understandable that other commitments
require authors to bow out of the weekly essay routine. There are only so many
hours in the day.
Our bloggers who currently
write each week are: Barbara Nadel (Turkey), Quentin Bates (Iceland), Jarad
Henry (Australia), and myself (Thailand). My writing colleagues essays have
often been a detailed examinations of the writing game, politics, social and
cultural developments, and insights into the world of police
Other crime fiction
writers who made a significant contribution through their essays during the last
four years include: Colin Cotterill (Laos/Thailand), Matt Rees (Middle-East),
Margie Orford (South Africa), Jim Thompson (Finland), and John Lantigua (South
and Central America). I thank each of them for sharing their insight and
applying their talent to the difficult art of an essay.
All of us feel that our
essays allow us to give something back to the readers of our novels—a glimpse of
the intellectual concerns and interests that can be developed independent of
plot and character. We don’t write behind a pay wall. Our essays are our way of
giving back to readers what we hope will be of value.
If you have enjoyed our
essays, the best way of expressing your appreciation is to buy and read one of
our novels, or send it along as a gift to a family member, colleague or friend.
On the right hand side is a scroll with a cover of our most recent
To our readers, thank you
for your support and we hope to publish more essays from the world of crime
fiction writers your way for sometime into the future.