Information Curation: 1 dot 1

Connecting the Dots

kusama3_bodyOn an uncharacteristically warm Summer evening in 2012 I made my way into the Tate Modern as everyone else was making their way out. It was part of my work to understand the curatorial process and its relevance to information management through one of the Tate’s infrequent but excellent curator talks. This one, from Frances Morris, concerned the recent and enormously popular Kusama exhibition.

 

The notion that curation is an emerging skill in dealing with information is not a new one. It is covered by Jeff Jarvis in his blog post ‘Death of the Curator. Long Live the Curator’ where Jarvis applies them to the field of journalism. It is also the subject of Steven Rosenbaum’s excellent book ‘Curation Nation’ which examines the meme more broadly.

 

Abundance

Japanese artist Yayoi Kusama is prolific. Her work span the many decades of her life, first in rural Japan then New York in the 60’s and in contemporary Tokyo today. It is enormously varied. Her signature style of repeating dot patterns, whilst the most famous, represents only a small part of a vast and sprawling body of work. It is the perfect artistic allegory for information overload. Kusama has too much art for any one exhibition in the same way that information professionals in the age of Big Data have too much information for any one decision.

 

Morris, I figured, must have wrestled with Kusama’s prodigious nature. The problem is not one of assembling a coherent and factual account. Instead, it is one of separating out that which is relevant and that which is extraneous. It is a process of  building a series of working hypotheses and building a story that is a reality, that is a ‘truth’.

 

Analysis and Curation

Like many managers, Morris had a vague sense of the story she wanted to tell but the final story could only be told through material facts, works or ‘data’.  At first, she considered, selected, dissected and parsed as much as possible. Over time Morris selected works through more detailed  research. She travelled extensively spending time with Kusama herself in a psychiatric institution which has (voluntarily) been Kusama’s home since 1977. She also visited locations important to Kusama including her family home and museums in Matsumoto, Chiba and Wellington, New Zealand where others had curated and exhibited her work. This parallels the analytical process. One of  starting with very few, if any assumptions, and embarking a journey of discovery. Over time, through an examination of historical and contemporary data points, the story begins to unfold.

 

In the Next Post (1 dot 2)

Already we can see that curating is a process of research and selection. It has strong parallel’s with early stages of information analysis. In the next post we will look at filtering, relevance and how the curatorial process helps us understand which comes first … data or information.

Advertisements

Decision Making problems are not new, in fact they are centuries old

Not Frank BuytendijkFrank Buytendijk delivered a great keynote at 8am in Las Vegas at the TDWI conference in February 2012. He avoided the technicalities of data architectures, the rigours of  data modelling and the disciplines of agile methods.

 

Instead, over breakfast, he dipped into the world of philosophy and asked us to consider the centuries old problems of what is true? what is real? and what is good?

 

Referring to Plato, Thales and Machiavelli Buytendijk lead us through some fundamentals about decision making.

What is True?

Firstly decisions are not just about the data. Do we decide to pay for parking because we calculate the cost of a ticket against the cost of a fine but factored by the risk of getting a fine? Or do we do it because we think it is the ‘right’ thing to do, the ‘civic’ thing to do?

 

What is Real?

So often, even with all the dashboards, scorecards, reports and charts, senior executives don’t seem to know what’s going on. Like in Plato’s Cave, the shadows on the wall are not reality, they are representations of reality. How much could really be told by listening to our customers directly rather than waiting for analysis much later?

 

What is Good?

Predictive analytics can provide great information that allow micro-segmentation. For example it could help an insurance company to identify those most likely to claim on their insurance policy for back and neck strain based on their on-line behaviours. Increasing their premiums might protect the business from additional costs but  the insurance business model is about distributing the risk not identifying it perfectly. Taken to it’s conclusion then there is no need for insurance, we all pay for the cost of our health care as and when it happens. However, if the insurance company used this information to promote lifestyle changes for this group then ethics and business models are aligned.

 

What’s it all about?

Buytendijk’s quirky, thought provoking start to the TDWI conference tells us that in IT, we  are wrestling with problems that preoccupied philosophers centuries ago. It also tells us though that in IT we can think too much and reflect too little.

The Social Triangle: Business, Brand and Analytics

Social TriangleMention social and we immediately think about the dizzying number of people using Facebook and, as businesses, how we reach them as customers or prospective customers.

 

This is only part of the story though.  Today, my own business, a provider of information software and services, will not find it’s customers on Facebook however hard we look.  However, this doesn’t mean that social isn’t relevant to us. That would be a limiting and ‘traditional’ view of Social as a Brand only which is a single point on the Social Triangle.

 

Michael Brito, SVP of Edelman Digital, commented in a recent article on Brainyard distinguished between the Social Business and the Social Brand.  The Social Brand, he argues, is a company, product, or individual that uses social technologies to communicate with social customers, their partners and constituencies, or the public. The Social Business, on the other hand, is one that has integrated and operationalized social media within job functions internally. The third point on the triangle is Analytics, the practical use of information to make decisions.

 

The aspiration is that both Brand and Business are for engagement not just broadcasting and that Analytics is used as actionable information. Let me offer an example.

 

I recently tried to book a London hotel room for my Son because he had a very early train journey on the Eurostar. I wanted to pay so that it was one less thing for him to be concerned about at 4am. I made an advanced reservation and several days, calls, emails and faxes (yes faxes) later and the hotel chain could still not confirm this part of the arrangement. Whilst I don’t do this often, I resorted to tweeting a #fail.

 

What happened next was pure Social Brand. A number of other hotel chains messaged me to offer me deals in London Hotels. Indeed, they still do. It left an overriding impression that everyone listened but no one heard.

 

A Social Business with a Social Brand using Social Analytics would have behaved completely differently. The tweet would have appeared in a dashboard and tagged as negative sentiment and that this related to dissatisfaction with the booking process.  Social Analytics would have been able to identify that I was a frequent traveller with children in university and that I was highly likely to use UK hotels over the coming 12 months. Social analytics would also have been able to identify the level of influence I have with others in this socio and demographic group (not as high as I think)

 

The information would have been shared around the organisation not just Marketing and it would have been shared efficiently using social tools not email.   A customer services representative may have tried to resolve the specific for me but the general issue would have found it’s way to a manager responsible for the booking process after which a decision will be made to  either fix this in their booking systems to attract other ‘surrogate bookers’ or to continue to deal with it as exception or even to do nothing. Next time a frustrated parent booking arrived everyone would know how to handle it or what the policy was because the whole dialogue would have been captured and tagged in a searchable activity stream. The marketing team might even build a new campaign that focused on how they understand their customers better and the ease of parental bookings.

 

A Social Brand engages in meaningful dialogue with it’s customers, a Social Business engages a motivated workforce to fix problems or to exploit new opportunities. Finally Social Analytics keep the whole process informed with timely and relevant information so that the focus is on the right customers and products and that effective, insightful and informed decisions are made.

Big Data Analytics: Size is not important

There was a time when databases came in desktop, departmental and  enterprise sizes.  There was nothing larger than ‘enterprise’ and very few enterprises needed databases that scaled to what was the largest imaginable unit of data, the terabyte. They even named a database after it.

We now live in the world of the networked enterprise. Last year, according to the IDC, the digital universe totalled 1.2 zettabytes of data. And we are only at the beginning of the explosion which is set to grow by as much as 40 times by 2020. Massive data sets are being generated by web logs, social networks, connected devices and RFID tags. This is even before we connect our fridges (and we will) to the internet. Data volumes are growing at such a click that we needed a new term, Big Data (I know) to describe it.

What is meant by ‘big’ is highly subjective but the term is loosely used to  describe volumes of data that can not be dealt with by a conventional RDBMS running on conventional hardware. That is to say, alternative approaches to software, hardware or data architectures (Hadoop, map reduce, columnar, distributed data processing etc) are required.

Big Data is not just more of the same though.  Big Data is fundamentally different. It’s new and new data can present new opportunities. According to  the Mckinsey Global Institute the use of big data is a key way for leading companies to outperform their peers. Leading retailers, like Tesco, are already using big data to take even more market share.

This is because Big Data represents a fundamental shift from capturing transactions for analysis to capturing interactions. The source of todays analytic applications are customer purchases, product returns and  supplier purchase orders whilst Big Data captures every customer click and conversation. It can capture each and every interaction. This represents an extraordinary opportunity to capture, analyse and understand what customers really think about products and services or how they are responding to a marketing campaign as the campaign is running.

Deriving analytics from big data, from content, unstructured data and natural language conversations requires a  new approach. In spite of the name though, it’s less about the size and more about the structure (or absence of structure) and level at which organisations can now understand their businesses and their customers.

Informed Decisions are Fairer Decisions

I have spent a fair amount of blogspace this year discussing how good business decisions need more than information. That the evolution of Business Intelligence tools need to extend beyond crosstabs, charts, scorecards and dashboards to collect and share social intelligence. However, this does not mean that decisions should be made without information, it means that information is the absolute but mandatory minimum for a good decisions. More than that, I would argue that we have an obligation to our customers and our workforce to base our decisions about them on good, solid data.

Informed and Fair

Take consumer credit for example. In the 1920’s and 1930’s it is unlikely that you or I would get credit. Credit was awarded to businessmen. And I mean men. The decision to offer credit would be based on criteria that we would find objectionable today like race and gender. Curiously, according to Larry E Rosenberger, John Nash and Ann Graham authors of ‘The Deciding Factor’, the decision would also be based on factors that included ‘punctuality’ and highly subjective assessments of ‘honesty’. Few of us could argue that a system which assesses an individuals eligibility for credit based their previous repayment behaviour, their income and their employment history not only represents good business but fair business too. Sure, it’s not perfect. In the 1980’s I found it difficult to get a mortgage in spite of being a well paid independent IT specialist with as predictable an income as any of my peers that were ‘permanently’ employed. At the time, I might have argued that ‘it was a pain’ but a little additional dialogue and process and it was sorted. Compare this with the brick walls of the 1920’s built around the subjective prejudice of a few controlling individuals and I would conclude that we have made a step, even a leap forwards.

Informed and Innovative

Informed decisions can be the basis of innovation too. Take, for example, the Swedish Company Klarna. Klarna make it possible to shop on-line and pay only after you have received your goods. They are providing a service which means that consumers can shop on-line but can see and feel their goods before they pay for them. In order to do this, they pay the store and take on the credit risk and they can only do this by efficiently analysing mountains of data that assess creditworthiness.

Informed but Rounded

Organisational decisions do need more than hard data. They need to be openly debated, controlled and they need to be informed by tacit, hard-to-communicate knowledge as well as analytics. However, information is the first step in taking organisations closer to unbiased, objective and therefore fairer decisions.

Who makes the decision anyway?

You’re Fired

I know that anyone that watches ‘The Apprentice’ is not doing so for an insight into how a modern business is run but hearing the words ‘You’re Fired’ frequently bellowed through an office door couldn’t be further from my own experience. It represents a clichéd and caricatured view of management that I last saw to ‘comedic’ effect in Terry and June  a 70’s BBC Sitcom. I am sure it is a style that exists but hopefully in a diminishing minority of organisations that haven’t found a way to deal with the bullying and haranguing of greying and dysfunctional dinosaurs.

A New  Generation of Decision Makers

I was born in the 60’s which you have probably already worked out given the reference to Terry Scott and June Whitfield. I don’t recall being consulted by my parents on family decisions too often. Loving and supportive as they were, they were part of a generation that didn’t ask what kind of party we wanted, what cut of jeans we preferred or which destination we preferred for a day out.

Of course their choices were far narrower but this was the generation of parents that pre-dated Parenting magazine let alone Parenting.com.

Compare this with the generation entering the workforce today. Most have been involved in choices that affect them, carefully consulted in family decisions. Some, including those like Montessori educated Google founders, Larry Page and Sergey Brin have taken their progressive education and created progressive and hugely successful organisational cultures.

Waning Autonomy in Decision Making

The connection with Alan Sugar’s pantomime boss and the future of BI is this. The purpose of BI is to make better decisions. Those decisions, two decades ago, used to be made by one person (and in the main it was a man) Increasingly those decisions are made by teams, peer groups, special interest groups and the staff that are impacted by it.

Waxing Collaboration in Decision Making

The drivers for the need for increasing collaboration in decision making are largely cultural. This includes the small matter of a whole generation entering the workforce that expect to be consulted and who are sociologically predisposed to sharing responsibility for the outcomes of those decisions. This means growing engagement in successful outcomes in organisations from a much broader group.

Until very recently, this was just too difficult to do. The cultural implications aside, how do you poll groups, get their input, collate views, share opinions and establish any kind of consensus without committee’s, sub-committees and employee councils? How to you distribute the information, the hard numbers, that are needed to make a decision that aligns the needs of the business, it’s stakeholders with the needs of those that participate in the business as employees and partners?

Social business tools and their convergence with BI are an enabler. They have made collaboration in decision making possible.

The opportunity is an engaged and an informed workforce that can positively participate in the decision making process. Even if the individual did not support the outcome, they will know that their voice was heard.

No, You’re Fired

If an engaged workforce sounds ‘fluffy’ then ask yourself what’s your organisations largest cost? It isn’t usually paperclips. Estimates vary but some suggest that knowledge workers will account for 80% of the cost in the US labour force in 2012.

If 80% of costs were in a machine would we be content on it idling, running at 25% capacity which some HR studies suggest is the current average level of engagement? I doubt it.

So the manager of yore, jealously keeping information to themselves so that they can exercise power and control and ultimately make autonomous decisions without offering or taking counsel… You’re Fired.

BI and Poor Decision making

Good Decision/Bad Decision

This has been something of a preoccupation for me of late. We spend much of our time debating the technologies. We invest valuable time in deciding if we should we go with mega-vendors (IBM, Oracle, SAP) or a challenger? We agonise over should it be cloud or on-premises, mart or warehouse, dimensional or relational? And it is all, frankly academic if the businesses is not making good decisions.

There is no shortage of material that try and make sense of why good people and great businesses make monumentally bad decisions. In the book ‘Thing Again:Why Good Leaders Make Bad Decisions’ by Sydney Finkelstein, Jo Whitehead and Andrew Cambell the focus is on the strategic decisions that have dramatic and highly visible consequences for the organisation.

Good People in Great Organisations Can Make Poor Decisions

An example is one of the UK’s premier retailers Boots which enjoys one of the largest footfalls in the UK. Established in the 19th century, it is now a subsidiary of £20billion Alliance Boots. In September 1998, the Chief Executive, Steve Russell excitedly announced a range of healthcare offerings including dentistry, chiropody and laser hair removal. Five years later, the initiative had lost in the region of £100m and Boots needed to break open the piggy bank and look down the back of the sofa for another £50m just to close down the operation and convert that premier retail space back to being … retail. It almost goes without saying that the changes were implemented by a new CEO, Richard Baker.

Apparently, one of the chief reasons for making the move into Healthcare services was  that a slowdown in the Beauty business ‘had been detected’. However a spokesman was later quoted in the Telegraph as saying that ‘they recognised that these areas are still growing strongly’.

Let’s stop there for a second. Spotting trends in sales and revenue by product category is probably marketing and business 101. And even the most rudimentary business intelligence solution should be trending sales over time. Yet the trend in sales in a key category for Boots was diagnosed as slowdown and only a few months later as growth. Of course, the slowdown may have been a short-term blip but the point of trending is to smooth these out for the purpose of longer-term planning. And, the error in trending might be more understandable had it not been for the fact that the later growth was characterised as ‘strong’.

Of course, I am not on the board of Boots and I have an advantage shared with all those analysts and commentator that put the boot (or should that be Boots) into Mr Russell … hindsight. Indeed, it’s a testimony to the strength of Boots as a high street giant that they can make major booboo’s and still go on to survive and thrive.

The Problem with Decisions …

And organisations are complex systems of individuals and interactions. Large organisations are very complex. This is why organisational decision making doesn’t always stand up to the scrutiny of us as individuals who retrospectively try and apply the logic of rational decision making to such mistakes.

There are a number of problems associated with individuals making decisions. Individuals have bias, self-interest, pre-conceptions. There are also a number of problems with organisational decisions. Groups have to manage conflict, disagreement and there are dynamics that can produce undesirable outcomes like Groupthink.

Today BI’s only Contribution is a Report, Chart or Dashboard

So if we accept that the purpose of Business Intelligence is to help organisations make better decisions (surely there is no debate here?) then Business Intelligence applications have to be more than reports, dashboards and charts.

They need to make decisions easier to collaborate around, they need to link decisions directly to the information that is required to make them. Furthermore decisions need to be open, transparent, accountable not just for the regulators but so that the whole organisation can buy into them.