Creating an Open Society

The way our long-term explanations, beliefs, and intentions take shape is largely determined by which ideas consistently manage to “fit” circumstances and sustain themselves by staying relevant, making new connections, and generating more uses.

It’s as if our ideas are literally “attracted” to the kinds of causal roles they’re qualified for. Our ideas seem to constantly move into opportunities to connect in more relevant ways, just as we do. When we observe things happening in the world, specific ideas step forward to nominate themselves for attribution, and we act or express ourselves (or continue to think) in some way shaped by the idea.

Frequently, thoughts and actions are affected by chance variations. Remember the provision that sometimes people do things because they must be doing something, if only because everything has temporal qualities. This is often how discoveries and creative works come to be made. In fact, the definitions of discovery and creation essentially require that they are unexpected.

This analogy between creativity and the evolutionary process has been studied and made explicit by psychologists for well over a century. Research indicates that creatively successful people owe their success primarily to a willingness to try more things than most people do. By virtue of generating more ideas and trying more combinations — and producing many failures in the process — they eventually produce more successes as well.¹

Shortly after the publication of Darwin’s Origins of Species, the notion of natural selection was almost immediately taken up by philosophers like Herbert Spencer to enlighten or explain cognitive processes. William James did the same; in fact he pointed out that Spencer’s theory wasn’t Darwinian enough. Historian Robert J. Richards summarizes James’s correction of Spencer, telling us James believed that “the novel ideas produced by men of genius — and ourselves on occasion — were not due to direct adaptations,” but rather “sprang up in the mind as spontaneous mental variations…”²

Of course, it isn’t enough just to generate wild and random ideas. Creative geniuses spend years (usually about ten) gaining enough “domain-relevant knowledge and skills,” to be able to recognize and capitalize on their novel ideas. As Louis Pasteur famously said, “chance favours the prepared mind.”

In that time, creative people develop what Howard Gruber called a “network of enterprise” — not merely a handful of discrete projects, but a dynamic system of related interests and activities. Networks of enterprise enable creators to adapt and keep working when unexpected barriers and opportunities emerge, as “an insight in one project may reawaken a dormant enterprise. A new skill mastered in the interests of one enterprise may suddenly become relevant elsewhere in the tangle.”³

It’s no coincidence that the Web is essentially a large-scale model of the same kinds of processes. Tim Berners-Lee’s idea for the World Wide Web was deliberately modeled after the human mind’s ability to generate new, meaningful associations.⁴

On a practical level he wanted to design a better system for finding relevant expertise within a large, complex institution. On a more ambitious level he was re-imagining how people collaborate and create.

Creativity occurs in different ways in different realms — often in different silos. We’re all familiar with these barriers within and between organizations, but on an even deeper level there are certain assumptions and conventions preventing us from seeing the connections between art, science, commerce, civics, etc.

My sense is that even these general categories will have to be re-conceived. The future I imagine now is one in which all of these aspects of our society have become closely integrated, almost unified in some aspects, while being re-differentiated across different axes. It isn’t just a few ideas here and there that are being re-organized; our most basic assumptions about what it means to be an artist, researcher, entrepreneur, and activist are variously converging and diverging into new categories.

Keep in mind that the ongoing process of conceiving and investigating new possibilities is purposeful in itself. And remember that the key to creativity is freedom and availability to form new associations. This imaginative, speculative approach will generate a lot of mistakes — but without the mistakes we won’t generate any creative insights or opportunities.

Also remember that none of us have to do any of this brainstorming ourselves (though I happen to enjoy it). Creativity is increasingly a social process, facilitated by markets and organizations, in which innovations aren’t generated by individual geniuses but by organizations and networks working together.

Big corporations can be slow and stodgy, but in general, commercial enterprise has an astonishing ability to adapt and grow.

Look at Silicon Valley; it’s a kind of “entangled bank” of entrepreneurial activity, with researchers, founders, employees, and investors constantly forming new, often spontaneous associations complementing each other.

Every year, thousands of entrepreneurial startups around the world — think of them as “chance variations” — are born, becoming either “selected” to thrive or reabsorbed back into the chaos from which new associations may emerge. A lot of startups fail, but the ones that succeed keep investors in business, circulating capital back to the entrepreneurs.

As with any kind of creativity, it’s impossible to know how well any particular idea will work out until it’s tried. The more ideas you try, the more likely you are to have one that succeeds. In this way, success doesn’t occur despite failure, it occurs because of it — as long as you’re actually learning, cultivating some kind of mastery in the process, and continually re-investing towards the future, and not failing to the point of catastrophe.

On a smaller level some of the most successful companies recreate this evolutionary process within themselves, encouraging employees to experiment, play, and try new ideas knowing that eventually a few will turn out successfully. Google famously keeps services in beta for years after “launching early and launching often”⁵ so they can get information on how to improve as quickly as possible.

Paradoxically, it’s this redundancy and unpredictable dynamism that makes systems sustainable over the long term. It’s the constant experimentation, failure, and iteration that keeps things evolving. If the system became too homogeneous and stable it would become inbred and vulnerable.⁶

The challenge for people who genuinely want to understand these processes is to appreciate that successful and apparently stable organizations, practices, and ideas, rather than having some kind of inherent merit that makes them exceptions to the rule of failure, are essentially just experiments that were fortunate to work for their time and place — experiments that haven’t failed yet.

Look at the history of any great company and notice how much their success relied on serendipity: a chance introduction that forms a partnership, a spontaneous insight leading to innovation, a couple of key hires that created unique team chemistry, a few gut decisions that luckily turned out well — aided by bad decisions by key competitors, etc.

That isn’t to diminish the importance of hard work and persistence. The other side of the coin is that when people believe they can’t fail, they’re less likely to. Appreciating our creative and entrepreneurial enterprises as experiments means accepting on one hand that most people who try something genuinely creative, believing they can’t fail, end up failing, but on the other hand, virtually nobody who anticipates failure ends up succeeding.

But too often, desire for success turns into a willingness to cut corners, embellish results, manipulate appearances, and game the system. Those behaviours undermine the process.

The whole process relies on failure. People have to be willing to accept failure and admit to mistakes, or the process won’t work. If we artificially hide information to deny failures — whether it’s done in the name of optimism or is simply a manifestation of greediness — then the process becomes toxic and corroded, errors fester unnoticed, and catastrophe-making chain reactions are set in motion.

As long as we live in a world in which there are barriers and profound disincentives to saying, “sorry, I don’t understand,” or “I screwed up,” we’ll keep losing our grip on our economic and social challenges.

This brings me back to the refrain I’ve used throughout this book: the process of creation and discovery is itself the answer. When it’s ok to love learning for its own sake — rather than assuming education is merely something that goes on a résumé — then we don’t have to worry about how to motivate people to solve unexpected problems, because solving unexpected problems is intrinsically rewarding.

But to be able to solve emerging problems and create new opportunities effectively, we need access to all of the pertinent information. The process doesn’t work if a few organizations artificially maintain their own relevance and control by erecting barriers and withholding information.

My ideal is summed up simply by Karl Popper’s description of an open society as a society that “sets free the critical powers of man.”⁷ Popper’s ideas and criticisms were an attempt to infuse the whole public sphere with the spirit of science — not as something that presumes to find out what the truth will always be (which would be an excuse from critical responsibilities — which is perhaps precisely why that attitude is so compelling to some) but rather as a process of constantly testing and refining our assumptions.

More importantly, science means allowing others to test and refine our assumptions as well. This is where my belief in open government comes from: not some vague, principled need to let everyone express their opinion, but rather a process of submitting decisions to scrutiny from more perspectives, to reveal where subjective biases and false assumptions might be affecting the process, to ensure the whole system doesn’t become vulnerable to the intrinsic fallibilities and weaknesses of a few individuals.

Popper distinguished that to be scientific about social issues does not mean deriving ideas from a supposedly solid foundation of scientific evidence, but rather appreciating all of our predictions and plans (and the assumptions they’re based on) as hypotheses to be falsified or provisionally corroborated by objective results.

This notion — the importance of appreciating fallibility — was outlined earlier by Charles Sanders Peirce,⁸ and is a integral to pragmatist philosophy, which will be taken up in a later chapter. This attitude is an essential element of my thinking, prediction-making, and planning. For example, if I turn out to be wrong about what I’ve written about the future of things (e.g. newspapers, government, education), I can trace my thinking back to these ideas so I can find and correct my mistakes. It’s a kind of “conceptual accounting.”

More importantly, you can trace my thinking back to get a better sense of where I’m coming from. And if you’ve kept an account of your thinking, I can follow that back too. If we have a disagreement we can compare notes and look for the inconsistencies, then try to “inter-subjectively” identify the best ideas. Subsequently, we shouldn’t come away from that conversation with the feeling that one person won and the other lost, we can come away feeling that we’re both engaged in an ongoing process of learning.

It’s important to appreciate that when approaching challenges in this manner, decisions are never final. Whenever we “solve” a problem, or make any kind of decision, we shouldn’t just say, “There’s that problem solved” and forget about it. I try to treat every idea as a prototype or experiment that needs to be followed-up on and assessed. We need to actively watch the results to see how the solution is performing — and not be surprised or defensive when it performs poorly — and make adjustments accordingly.

Again, it comes down to the love of learning. If you genuinely want to learn, then this approach will come naturally (with some supplemental discipline). If you don’t genuinely want to learn, you’ll be tempted to use persuasion and positive thinking to deflect people’s attention (including your own) away from mistakes and weaknesses.

This is not just idealism on my part. In just the past few years we’ve seen a real, pronounced opening-up of the decision-making processes in business and government. Companies aren’t just soliciting feedback from customers; by using the Internet they’re able to find conversations where and when they occur, in real time, and because these digital comments are already documented and measurable, they immediately generate usable analytics that can affect a company’s decisions with relatively little interpretation required.

Feedback is even richer and more useful when the product or service itself is digitized and networked. Companies can get an immediate sense of what users like and don’t like. So much more development is now done after an application is released.

Twitter is an especially prominent example. After launching as a minimalist service, users began to develop specialized practices and semantic conventions (for example, using @ to address messages to specific individuals and # to tag messages with topical or event-specific keywords) which were later coded directly into the service. Other developers have also been free to make complementary applications for updating, reading, and analysing Twitter — to such a degree that many people rarely even visit the website itself.

The company and its founders couldn’t have possibly thought of all of these features and functions. Even if they could, they still wouldn’t have had enough resources to develop them — certainly not when you consider how much time and energy went into developing all of the unsuccessful applications, without which it might not have been possible to recognize the few truly useful ones.

Again we come back to the evolutionary characteristics of these social and technological “ecosystems.” There is virtually no risk for individual users to play around with using @ and # in their tweets, whereas if Twitter had assumed too much certainty about how people would eventually use the service, they might have created barriers to those developments and the service might have been a failure as a whole.

If that had been the case, the loss would not just have been Twitter itself; we would have lost the opportunity for all of the innovative and interesting uses it generated — i.e. the way it has opened up media and marketing backchannels where information is shared and these kinds of ideas are discussed. This is ultimately what it’s all about. Applications are just tools; the truly profound innovations are going to be social and institutional ways to facilitate greater autonomy and relevance — a phase that’s only just beginning.

Just as these generative networks are open to relatively low risk experiments, they’re also open to more objective observation and evaluation. And the resulting criticisms and conversations themselves are open to criticism. In turn, that process affects individual reputations — and these are open to objective criticism as well: we can look at people’s past statements, we can look at exactly who supports them, etc, and we assess how they fit with the rest of our connections.

Here once again, our experience with the Web is our way to understand even greater possibilities. Because we can observe how software platforms evolve and develop, we can get a better sense of how this process can be used to improve our organizations and institutions themselves — all the way up to the notion of what Tim O’Reilly calls “government as a platform.”⁹

It’s time to seriously re-conceive what democracy can be. Elections and other democratic traditions developed to fit social circumstances that have since evolved. Not only do we have better tools for many tasks, we have different tasks. Our challenges are also more complex — requiring expertise so specialized it’s tough even to find the appropriate expert.

By conceiving government as a kind of ongoing, open source project, people with the appropriate skills and knowledge for specific needs are able to self-select and contribute more effectively — whether by joining an actual project, or independently developing an application that taps into open data. In such a system, innovative solutions and opportunities are more likely to be discovered. Because of the relatively low risks on experimentation, it occurs within a larger, diverse ecosystem of collaborators and critics who constantly filter and absorb failure while naturally propagating the most effective practices and ideas.

Understanding open government requires a significant shift in mindset, starting towards the notion that government’s first responsibility is to improve itself — not in an idealistic, “rebuild it from the ground-up” sort of way, but simply in a way that approaches every project as a learning experience, every policy as a hypothesis that we expect to re-evaluate as we move forward, which improves our understanding and helps us answer the next set of questions.

To make that turn, we need to think and frame things in terms of collaboration rather than merely deliberation.¹⁰ That means making things, prototyping and iterating, seeing what works and using real observations to ground ongoing decisions.¹¹ This will require a considerable shift in how we think about “leadership,” for example. Leaders need to think like builders rather than merely bosses¹² — learning through the decision-making process as a kind of design process.¹³

We still need criticism and dialog, but dialog needs to refer to what we’re actually doing — things we’re trying, experiments that will generate objective, observable results — rather than merely arguing and trying to persuade each other with increasingly costly rhetoric.

Obviously on most issues such an approach isn’t feasible — and hardly even conceivable — yet. But there are plenty of challenges we can start addressing this way, specifically improvements to our information and communications infrastructures. By approaching those challenges with much larger possibilities in mind, our experience through them will provide the verification or falsification we need to outline a larger strategy — if such a strategy is feasible at all. We can’t know for sure until we experiment.

Above all, it requires a growth mindset.¹⁴ We need to accept that everything is a work in progress, nothing is perfect, and this fact is not something to lament; it’s something to celebrate. What’s life for if not to aspire and work towards something better? This is what “openness” means in the fullest sense; not just transparent, but open-ended — open to the future, and generative¹⁵ like the process of evolution itself.

An open society needs to engage people’s interests, competencies, and passions. Sustainable systems of participation have to provide opportunities for flow and conceptual consumption, to receive feedback, to socialize, to recognize we actually have an effect on the world — specifically, to recognize our effects as uniquely and autonomously our own, which fit with our personal identities and stories, not effects that are indistinguishable and generic — to place ourselves in relation to others, to learn, and to feel that our lives ultimately mean something bigger and more enduring than what we’re given.

As I’ve argued throughout this book, such a process isn’t merely a means to building a better society, such a process is itself the better society we want. Collaborative, dynamic, learning-oriented processes are intrinsically gratifying. A society in which such processes flourish — and enable each of us to, in our own distinct and relevant ways — is already a better society.

1Dean Keith Simonton, Origins of Genius (1999).

2Richards, Darwin and the Emergence of Evolutionary Theories of Mind and Behavior (1989): pg. 427.

3Howard Gruber, “An Evolving Systems Approach to Creative Work” in Creative People at Work, edited by Wallace & Gruber (1992): p. 20.

4Tim Berners-Lee, Weaving the Web (1999).

5Quoting Marissa Mayer, Google’s VP of Search Product and User Experience on Charlie Rose’s TV show (March 5, 2009). She was paraphrasing “release early, release often,” a notion that developed with the open source Linux project, as described by Eric Raymond in The Cathedral and the Bazaar (2000): available online at http://www.catb.org/~esr/writings/homesteading/cathedral-bazaar/ar01s04.html

6See Nassim Taleb’s Black Swan (2007) and Clayton Christensen’s Innovator’s Dilemma (1997) on unexpected events and disruptive technology respectively.

7Karl Popper, The Open Society and Its Enemies (1962, Princeton 1971): V. 1, p. 1.

8See “The Fixation of Belief” (1877), Collected Papers: V. 8; p. 223 – 247. Also “The First Rule of Logic” (1898) in the same volume: see especially p. 408 – 410.

9“Government As a Platform” in Open Government, Lathrop & Ruma, eds. (2010): p. 11 – 39.

10See Beth Simone Noveck, Wiki Government (2009) or “The Single Point of Failure” in Open Government (2010): p. 60.

11See Tim Brown’s Change By Design (2009) for an elaboration on how this iterative “design thinking” process works.

12Umair Haque, “The Builders’ Manifesto” at HBR Blogs (December 18, 2009): http://blogs.hbr.org/haque/2009/12/the_builders_manifesto.html

13IDEO’s Colin Raney & Ryan Jacoby, “Decisions by Design: Stop Deciding, Start Designing,” in Rotman Magazine (Winter 2010).

14Carol Dweck, Mindset: The New Psychology of Success (2007).

15On “generative technology” and enterprise, the book to read is Jonathan Zittrain’s Future of the Internet–And How to Stop It (2008).