Information Curation: 1 dot 1

Connecting the Dots

kusama3_bodyOn an uncharacteristically warm Summer evening in 2012 I made my way into the Tate Modern as everyone else was making their way out. It was part of my work to understand the curatorial process and its relevance to information management through one of the Tate’s infrequent but excellent curator talks. This one, from Frances Morris, concerned the recent and enormously popular Kusama exhibition.

 

The notion that curation is an emerging skill in dealing with information is not a new one. It is covered by Jeff Jarvis in his blog post ‘Death of the Curator. Long Live the Curator’ where Jarvis applies them to the field of journalism. It is also the subject of Steven Rosenbaum’s excellent book ‘Curation Nation’ which examines the meme more broadly.

 

Abundance

Japanese artist Yayoi Kusama is prolific. Her work span the many decades of her life, first in rural Japan then New York in the 60’s and in contemporary Tokyo today. It is enormously varied. Her signature style of repeating dot patterns, whilst the most famous, represents only a small part of a vast and sprawling body of work. It is the perfect artistic allegory for information overload. Kusama has too much art for any one exhibition in the same way that information professionals in the age of Big Data have too much information for any one decision.

 

Morris, I figured, must have wrestled with Kusama’s prodigious nature. The problem is not one of assembling a coherent and factual account. Instead, it is one of separating out that which is relevant and that which is extraneous. It is a process of  building a series of working hypotheses and building a story that is a reality, that is a ‘truth’.

 

Analysis and Curation

Like many managers, Morris had a vague sense of the story she wanted to tell but the final story could only be told through material facts, works or ‘data’.  At first, she considered, selected, dissected and parsed as much as possible. Over time Morris selected works through more detailed  research. She travelled extensively spending time with Kusama herself in a psychiatric institution which has (voluntarily) been Kusama’s home since 1977. She also visited locations important to Kusama including her family home and museums in Matsumoto, Chiba and Wellington, New Zealand where others had curated and exhibited her work. This parallels the analytical process. One of  starting with very few, if any assumptions, and embarking a journey of discovery. Over time, through an examination of historical and contemporary data points, the story begins to unfold.

 

In the Next Post (1 dot 2)

Already we can see that curating is a process of research and selection. It has strong parallel’s with early stages of information analysis. In the next post we will look at filtering, relevance and how the curatorial process helps us understand which comes first … data or information.

Social Analytics … At a Glance

First off, let me stipulate that I absolutely support the notion that technologists of a certain age (let’s go with over 40) should regularly evaluate what they need to ‘unlearn’ in order to make way for new thinking.

However, some older techniques really do stand the test of time when attempting to understand new concepts. Take Social Analytics. It’s a hot topic and information specialists are trying to get their heads around what it it and what the business benefits are.

To help me understand, I started researching in the usual way. Books, white papers, articles and opinion. But as I did so, I drew up a ‘Dimension Map’ and a list of questions. Two simple devices that are as useful today in clarifying information requirements and their usefulness as they were err, a few years ago.

The first, a dimension map, has as it’s columns, the dimensions of measurement. So a revenue dimension map would typically have columns for product, customer etc. The rows are hierarchical levels so time (and most things are measured over time) might be years, months and days. The final column is a list of metrics. These are the measurements that can be analysed by the dimensions so a revenue dimension map would include sales value, sales qty. Easy, right. The social media dimension map below is very much a work in progress but I trust you find it useful as a ‘Social Analytics at a Glance’ diagram from which you can expand your thinking in the way that I intend to.

The question list is self explanatory but is a really simple and illustrative way to remind us that the purpose of information is to make decisions by answering business questions.

Social Analytics Dimension Map

Questions;

  1. Who does or does not like me|my product|my campaign|my brand?
  2. Who influences my customers?
  3. Do influencers like me|my product|my campaign|my brand?
  4. Are my customers talking more about me|my product|my campaign|my brand? than my competitors? i.e what is my share of voice?
  5. What are my customers saying about my competitor? i.e. what are the competitive opportunities or threats?

Social Gestures and Social Business Intelligence

Social Gestures

If, in the middle of a conversation,  I put my index finger to my lips you would instinctively lower your voice. If I held my hand out flat towards your face whilst you were talking you would stop mid-flow. These social gestures are powerful forms of human communication. They are efficient communication short-cuts.

Cheque Please

One of the most universally useful social gestures is pinching the thumb and forefinger together as if holding a pen followed by a short wave in the air as if writing. Used in restaurants around the world it invariably results in receiving the bill even if your previous attempts at the local language resulted in you getting the wrong dish which you ate whilst wishing you had swallowed your pride rather than what ultimately arrived and pointed clumsily at the menu when ordering. Human social gestures are shaped by culture, history, collective memory and there are are a growing amount of equivalent online social gestures.

The Power of Online Social Gestures

Online Social Gestures are specific and granular interactions supported by social tools. In a single gesture or click it is possible to ‘like’ a document ‘rate’ it’s value or ‘share’ it’s content. Online Gestures in social tools are powerful for two reasons.

Firstly, they simplify interactions. Let’s take another universal human gesture, lifting a single thumb in the air to signify our approval. A single click on a thumbs up icon communicates the same message as adding the comment ‘I like this’ but it’s easier. It’s a single click rather than two clicks and nine key strokes. The difference in ease of use is marginal but it removes a barrier, albeit a low one, to participating in the social discourse.

The second reason though is more significant. In adding a ‘like’ button we have identified a common interaction within the universe of interactions that we can isolate, capture, analyse and make further use of. For example, we can capture and count how individuals rate a document. The availability of that document in searches and lists can be influenced by the rating. The more readers positively rate a document, the more others will become aware of it and the wider and audience for good content as decided by the ‘crowd’.

Decision Making Gestures

There are specific Social Gestures for Social Business Intelligence and the interactions associated with decision making in an organisation. Today, the social interactions are very simple but we should expect these to grow as vendor offerings grow in their levels of sophistication. The following represent social gestures for Business Intelligence. Some exist today, some are inspired by the behaviour of decision makers as we see it so are effectively our suggestions to Social BI vendors;

  • Explain. Open a question to explain the real ‘meaning’ of a BI report is identified. This is likely to be a gesture initiated by a Senior Manager
  • Resolve. Explore the possible solutions to the issue now identified as a result of the investigation that has taken place.
  • Decide. Agree and announce the most suitable solution to the issue
  • Approve. Seek approval or challenge from those involved in the decision making discourse
  • Action. Execute the solution identified in the resolution discourse.

Capturing Decision Making Gestures

If we simplify and capture these gestures we can start to understand the significance of separating, capturing and analysing each of them;

  • Explain. Senior managers can easily delegate the task of discovery to domain experts and analysts until there is consensus on the meaning of a BI report. For example
  • Resolve. All considered solutions along with the discourse and debate can be tied back to the issue identified in the BI report
  • Decide. The chosen solution can be clearly identified along with all those that were considered
  • Approve. Those involved in the decision making process can agree, disagree or register a challenge to the decision either as part of a consensus driven approach or simply ‘for the record’
  • Action. The activity decided upon to resolve the issue can be tied back to the decision and it’s outcome as a success or failure recorded so that it can be considered as an option for similar future decisions

So online social gestures, as they relate to decision making are the starting point form which we can begin to understand the decisions in our organisations, how they came about and what they tell us about improving future decisions.

Who makes the decision anyway?

You’re Fired

I know that anyone that watches ‘The Apprentice’ is not doing so for an insight into how a modern business is run but hearing the words ‘You’re Fired’ frequently bellowed through an office door couldn’t be further from my own experience. It represents a clichéd and caricatured view of management that I last saw to ‘comedic’ effect in Terry and June  a 70’s BBC Sitcom. I am sure it is a style that exists but hopefully in a diminishing minority of organisations that haven’t found a way to deal with the bullying and haranguing of greying and dysfunctional dinosaurs.

A New  Generation of Decision Makers

I was born in the 60’s which you have probably already worked out given the reference to Terry Scott and June Whitfield. I don’t recall being consulted by my parents on family decisions too often. Loving and supportive as they were, they were part of a generation that didn’t ask what kind of party we wanted, what cut of jeans we preferred or which destination we preferred for a day out.

Of course their choices were far narrower but this was the generation of parents that pre-dated Parenting magazine let alone Parenting.com.

Compare this with the generation entering the workforce today. Most have been involved in choices that affect them, carefully consulted in family decisions. Some, including those like Montessori educated Google founders, Larry Page and Sergey Brin have taken their progressive education and created progressive and hugely successful organisational cultures.

Waning Autonomy in Decision Making

The connection with Alan Sugar’s pantomime boss and the future of BI is this. The purpose of BI is to make better decisions. Those decisions, two decades ago, used to be made by one person (and in the main it was a man) Increasingly those decisions are made by teams, peer groups, special interest groups and the staff that are impacted by it.

Waxing Collaboration in Decision Making

The drivers for the need for increasing collaboration in decision making are largely cultural. This includes the small matter of a whole generation entering the workforce that expect to be consulted and who are sociologically predisposed to sharing responsibility for the outcomes of those decisions. This means growing engagement in successful outcomes in organisations from a much broader group.

Until very recently, this was just too difficult to do. The cultural implications aside, how do you poll groups, get their input, collate views, share opinions and establish any kind of consensus without committee’s, sub-committees and employee councils? How to you distribute the information, the hard numbers, that are needed to make a decision that aligns the needs of the business, it’s stakeholders with the needs of those that participate in the business as employees and partners?

Social business tools and their convergence with BI are an enabler. They have made collaboration in decision making possible.

The opportunity is an engaged and an informed workforce that can positively participate in the decision making process. Even if the individual did not support the outcome, they will know that their voice was heard.

No, You’re Fired

If an engaged workforce sounds ‘fluffy’ then ask yourself what’s your organisations largest cost? It isn’t usually paperclips. Estimates vary but some suggest that knowledge workers will account for 80% of the cost in the US labour force in 2012.

If 80% of costs were in a machine would we be content on it idling, running at 25% capacity which some HR studies suggest is the current average level of engagement? I doubt it.

So the manager of yore, jealously keeping information to themselves so that they can exercise power and control and ultimately make autonomous decisions without offering or taking counsel… You’re Fired.

If only BI was as efficient as Facebook

BI, Facebook and Decision Loops

I  was at an analysts briefing event with IBM last week who were sharing their thinking on Social Business and what I believe is the inspired and innovative pairing of Connections Collaboration and Cognos Business Intelligence. IBM’s Social Business Leader for Northern Europe, Jon Mell shared a slide that compared the number of operations it takes to share a photo and gather feedback with friends on facebook and  the number of operations it takes to do the same on email.

This set my mind racing. If there are efficiency gains on something simple like sharing and getting feedback on a photo, imagine the productivity gains on sharing critical business information through Business Intelligence reports.

Why do I say this? Because sharing a photo is typically a single ‘sharing loop’ process. Someone publishes the photo, others contribute with their clever and witty observations. Done.

A single loop … Count ‘em … One. (A quote from Muppet Treasure Island, btw)

The out-dated view of BI is that it is shared this way too. That it’s published and the job is done. This just doesn’t hold true any more and I am not sure it ever did. BI requires many sharing or decision loops. Ten, distinct loops to be precise but some of these are repeated which means there can often more decision loops than a bowl full of fruit loops (an all too infrequent guilty pleasure of mine)

  1. Meaning Loop. Gain and assign agreement on the meaning of the information
  2. Implication Loop. Decide if the implication is neutral or if there is a problem or opportunity
  3. Investigation Loop. If there is an issue then it will be rare that the one piece of business intelligence will provide the full story. This loops is about investigating the problem or opportunity is in more detail.
  4. Solution Loop. Determine possible solutions to exploit the problem or resolve the problem
  5. Decision Loop. To decide on the best possible solution
  6. Action Loop. Once the solution is determined it will be broken down into tasks and assigned to individuals to be actioned.
  7. Progress Loop. Providing feedback on the progress of the solution
  8. Monitoring Loop. To determine if the solution has been successful or if the group need to return to refine the tasks or redo some loops.
  9. Conclusion Loop. Closure. Establish agreement that that there are no further actions and that the problem or opportunity is resolved.
  10. Celebration Loop. Acknowledge the support and contributions of those involved

That’s ten loops which means that if sharing a photograph on Facebook is more efficient than sharing it in the office using email, the productivity benefits of doing ‘real’ business are tenfold.

There are those that are sceptical about Facebook styled social platforms in the office because they may waste time. I understand this, I really do. However, the opposite is true. Organisations need social platforms, particularly for collaborative decision making. Without them, they are wasting time.

BI and Poor Decision making

Good Decision/Bad Decision

This has been something of a preoccupation for me of late. We spend much of our time debating the technologies. We invest valuable time in deciding if we should we go with mega-vendors (IBM, Oracle, SAP) or a challenger? We agonise over should it be cloud or on-premises, mart or warehouse, dimensional or relational? And it is all, frankly academic if the businesses is not making good decisions.

There is no shortage of material that try and make sense of why good people and great businesses make monumentally bad decisions. In the book ‘Thing Again:Why Good Leaders Make Bad Decisions’ by Sydney Finkelstein, Jo Whitehead and Andrew Cambell the focus is on the strategic decisions that have dramatic and highly visible consequences for the organisation.

Good People in Great Organisations Can Make Poor Decisions

An example is one of the UK’s premier retailers Boots which enjoys one of the largest footfalls in the UK. Established in the 19th century, it is now a subsidiary of £20billion Alliance Boots. In September 1998, the Chief Executive, Steve Russell excitedly announced a range of healthcare offerings including dentistry, chiropody and laser hair removal. Five years later, the initiative had lost in the region of £100m and Boots needed to break open the piggy bank and look down the back of the sofa for another £50m just to close down the operation and convert that premier retail space back to being … retail. It almost goes without saying that the changes were implemented by a new CEO, Richard Baker.

Apparently, one of the chief reasons for making the move into Healthcare services was  that a slowdown in the Beauty business ‘had been detected’. However a spokesman was later quoted in the Telegraph as saying that ‘they recognised that these areas are still growing strongly’.

Let’s stop there for a second. Spotting trends in sales and revenue by product category is probably marketing and business 101. And even the most rudimentary business intelligence solution should be trending sales over time. Yet the trend in sales in a key category for Boots was diagnosed as slowdown and only a few months later as growth. Of course, the slowdown may have been a short-term blip but the point of trending is to smooth these out for the purpose of longer-term planning. And, the error in trending might be more understandable had it not been for the fact that the later growth was characterised as ‘strong’.

Of course, I am not on the board of Boots and I have an advantage shared with all those analysts and commentator that put the boot (or should that be Boots) into Mr Russell … hindsight. Indeed, it’s a testimony to the strength of Boots as a high street giant that they can make major booboo’s and still go on to survive and thrive.

The Problem with Decisions …

And organisations are complex systems of individuals and interactions. Large organisations are very complex. This is why organisational decision making doesn’t always stand up to the scrutiny of us as individuals who retrospectively try and apply the logic of rational decision making to such mistakes.

There are a number of problems associated with individuals making decisions. Individuals have bias, self-interest, pre-conceptions. There are also a number of problems with organisational decisions. Groups have to manage conflict, disagreement and there are dynamics that can produce undesirable outcomes like Groupthink.

Today BI’s only Contribution is a Report, Chart or Dashboard

So if we accept that the purpose of Business Intelligence is to help organisations make better decisions (surely there is no debate here?) then Business Intelligence applications have to be more than reports, dashboards and charts.

They need to make decisions easier to collaborate around, they need to link decisions directly to the information that is required to make them. Furthermore decisions need to be open, transparent, accountable not just for the regulators but so that the whole organisation can buy into them.

Decision Making Black Holes

 

A Funny Thing Happens at the Forum

Meetings are one of the most common decision making ‘forums’ we are all regularly involved in. In fact one in five company meetings we take is to make a decision. As a way of making decisions though, they can be problematic. Once the meeting has concluded, the connection between information shared, decisions made and actions taken can be weak even lost. It’s as if the meeting itself were a decision making black hole.

Some Decisions are More Equal Than Others

Some decision making meetings are impromptu for making a timely, tactical decision quickly. Others are regular, formal and arranged around the ‘drum beat’ or ‘cadence’ of a business to make more strategic decisions. The more strategic the decisions and longer term the impact the less frequent the forum so a Senior or Executive Management Team may only meet quarterly for a business review (QBR)

How a QBR ‘Rolls’

A typical QBR will see Senior Managers sharing results in PowerPoint, possibly with financial results in spread-sheets which I would hope have at least been extracted from a Business Intelligence application.

If the SMT are reasonably well organised, they will summarise their conclusions and actions in meeting minutes. The meeting minutes will be typed up by an assistant in a word document and then distributed in email.

Throughout, they will all have been keeping individual notes so will walk out with these in their daybooks. The most senior manager in the room might not do this particularly if it’s their assistant who’s taking the minutes.

Later, actions from daybooks and minutes are likely transferred to individuals to-do lists and all follow-up will be conducted in email and phone calls.

An Implosion of Information, Conclusion and Decision

So let’s recap. Critical decisions about how resources are going to be allocated will be discussed in a ‘QBR’ and yet the artefacts of this critical decision making forum are scattered into Word documents, excel spread-sheets, emails and outlook tasks. Tiny fragments of the discussion, information, conclusion, decisions and activities implode around the organisation. To be frank, the team are now only going to make progress because the forum was recent and can be relatively easily recalled.

Of course, once time or people move on so does the corporate memory of the decision. Conversations begin with ‘what did we agree to do about that cost over-run?’ or ‘why did we say we were ok with the revenue performance in Q1?’

Executive Attention Deficit Syndrome

Many executives complain of a syndrome that feels like ADS. This is because the more senior the manager the more things they will probably have to deal with at an increasingly superficial level. A functional head will probably spend no more than 15 minutes on any one thing. To productively make decisions they will need to be able to have the background, status and related information to hand so that they can deal with it quickly and move on to the next thing. Decision making black holes contribute to this feeling of EADS.

CDM and Corporate Memory

Corporate Decision Making platforms will be successful when they connect;

  • Decisions
  • Information on which the decision was made
  • Insight derived from the information
  • Actions taken on the decision
  • Results of the actions

This means total recall of corporate decisions good and bad so that, over time, decisions can be recalled, evaluated, re-used or improved. A far cry from current decision making forums which whilst functional are inherently flawed, fragmented and are not improving the timeliness and quality of decisions in our organisations.

BI Requirements Should not just be Gathered

There are many resources remonstrating with the IT community on the importance of gathering requirements. Failing to gather requirements, they warn, will lead to a poor solution delivered late and over budget. This is largely inarguable.

However, I would warn that simply ‘gathering’ requirement is as big a risk. Fred Brooks, author of ‘The Mytical Man Month’ once said that ‘the hardest part of building a software system is deciding what to build’. And deciding what to build is a two way process rather than the act of listening, nodding and documenting that we all too often see in Business Intelligence projects.

From time to time, I hear someone cry foul on this assertion. They argue that it seems like the tail is wagging the dog or that the business cannot compromise on the requirement. I usually point out that simply building what the user asked for doesn’t happen in any other field of engineering. Architects advise on the cost of materials when planning a major new office building, City officials take advice on the best possible location for a bridge and environmental consultants are actively engaged in deciding exactly if and what should be built in any major civil engineering project.

And this is exactly how we should approach business analytic requirements. As a two-way exploration of what is required, possible solutions and the implications of each. Incidentally, this is particularly difficult to do if business users are asked to gather and document their own requirements without input from their implementation team.

An example of why this is important is rooted in the fact that many BI technologies (including IBM Cognos) are tools not programming languages. They have been built around a model to increase productivity. That is, if you understand and work with the assumptions behind the model reports, dashboards and other BI application objects can be built very quickly. Bend the model and development times increase. Attempting to work completely around the model may result in greatly reduced productivity and therefore vastly increased development time.

So be wary of treating ‘gathering’ and ‘analysis’ as distinct and separate steps. Instead, the process should be an iterative collaboration between users and engineers. Requirements should be understood but so should the implications from a systems perspective. The resulting solution will almost undoubtedly be a better fit and it will significantly increase the chance of it being delivered on time, at the right cost and with an increased understanding between those that need the systems and those that build them.

"We fail more often because we solve the wrong problem than because we get the wrong solution to the right problem", Russell Ackoff, 1974

Death of the BI Generalist

I have fond memories of the 90s. I spent a fair amount of it being a ‘Cognos Consultant’ travelling the length and breadth of the UK being an expert in all things Cognos. There was a great sense of independence and autonomy. It was possible to pitch up at a new Client with 2 CDs (One for PowerPlay, one for Impromptu) and install the software on a handful of PCs on Monday morning and by Friday I would have interviewed for requirements, built a metadata model, cubes and reports and would be huddled around a desktop with the users putting the final touches to what I dared to call a BI application.

It was the day of the renaissance BI Consultant, the Polymath. We all felt like experts from the time of we opened the jewel case to the point the user signed off their brand new Business Intelligence application.

As much as I enjoyed this, I don’t yearn after these as simpler times. The solutions we built were great and delivered real and incremental value. They didn’t, however, have the breadth, depth and coverage as the best implementations today. These are deployed over the web to hundreds, sometimes thousands of users covering multiple business domains. Modern performance management solutions have bought together Sales, Marketing, Operations, Finance and the Senior Management Team together in ways that were less achievable perhaps even impossible ten years ago.

There is a cost though. The range of skills required to deploy a modern BI (or Business Analytics) Application is broad. Take the individual that installs the software. What used to require a working knowledge of Windows and an installation manual now also needs an understanding of web servers, operating systems, virtualisation, application servers and security all in multiple flavours. It may also require systems management conversations to configure for fail-over and load-balancing in a way that wasn’t required before because it wasn’t possible. And whilst the internet means we are all effectively sharing a single network it stands to reason that it is important to understand all the different ways of communicating and processing on this single, global network.

So far, we have only unwrapped the cellophane but things get really interesting when we get our hands on data. The possibilities for delivering value from the vast array of corporate data assets as improved dramatically but this means that they need to be integrated too. This required an in-depth understanding of relational and dimensional modelling techniques, databases sql and at least one ETL tool. Whilst there are some tools that simplify this process with tools and (usually) OLAP cubes, they are typically departmental in scale. Modern solutions may also required a mix of on-premises and cloud data and a mix of  structured, and unstructured which increases the richness of the solution but also the number of things a developer needs to keep in their head.

So where does this leave us? If you have a team of homogeneous ‘BI Developers’ then their skills may be too general. I am working with one client in an advisory capacity at the moment and they have one consultant that has implemented everything. He has been very successful too. However, their business, though, is of a size where he can wrap his arms around the requirement, the infrastructure and the data. He also has exceptional aptitude and is an outstanding consultant throwing herculean effort at their projects. For the rest of us mortals we need to divvy up the responsibilities.

Many of my clients tend to do this across technical/systems, data and application. It’s not the only way but works really well. The IS team pick up the install and maintenance of the application server environment, a data team create a single reporting database/data warehouse/marts and a BI team maintain metadata and the reporting application in all it’s variants of reports, analysis, dashboards, scorecards etc.

There are interesting implications for the modern, complex out-sourcing and off-shoring organisation. Another of my clients based in the City out-source their databases including data warehouse development. Because there is a cost each time they want to add a new data item for the purpose of BI they tended to try to resolve new requirements in metadata or the reporting application. Inevitably the quality of their solution deteriorated over time because they were not always extending or fixing in the right place. Eventually, the sticking plasters gave out and they were forced to back to first principles which had implications of cost and re-work.

The more common issue is one of a shared design. Changes to data impact ETL, metadata, reports and the application. A new requirement might need to be changed in one or all of these places. Whilst usually a number of small and straightforward changes, they do need to applied consistently. To co-ordinate activity when the solution is expanded or enhanced there needs to be a common data model. In our experience this needs to be two models. One, a logical (and relational) model that represents the business data in a perfect and integrated world. Secondly a dimensional model that represents the data as the business see it in their reporting application through reports and metadata. These are surprisingly rare in our experience and usually not because they take a long time to draw (they don’t) Our suspicion is that what makes these time consuming is the need for consensus. However, if there is no consensus then there is a risk that the solution is already flawed.

So, a BI/Analytics solution cannot be built single-handedly. It requires a range of skills and it’s difficult to be an expert in all of them. The generalist then is fading away and being replaced by a team and a shared design who can deploy solutions with greater reach and richness than when we could ever have believed a few short years ago.

Single version of the truth, philosophy or reality?

Assuming you want the truth and you can handle it then you will have heard this a lot. The purpose of our new (BI/Analytics/Data Warehouse) project is to deliver ‘a single version of the truth’. In a project we are engaged with right now the expression is one version of reality or 1VOR. For UK boomers that will almost undoubtedly bring to mind a steam engine but I digress.

I have to admit, I find the term jarring whenever I hear it because it implies something simple and  attainable through a single system which is rarely the reality.

In fact it’s rarely attained causing some of our community to ponder on it’s viability or even if it exists. Robin Bloor’s ‘Is there a single version of the Truth’ and  Beyond a single version of the truth in the Obsessive Compulsive Data Quality blog are great examples.

Much, on this subject, has been written by data quality practitioners and speaks to master data management and the desire, for example, for a single and consistent view of a customer. Banks often don’t understand customers, they understand accounts and if the number of (err, for example Hotel Chocolat) home shopping brochures I receive is anything to go by then many retailers don’t get it either. Personally I want my bank and my chocolatier to know when I am interacting with them. I’m a name, not a number, particularly when it comes to chocolate.

This problem is also characterised by the tired and exasperated tone of a Senior Manager asking for (and sometimes begging for) a single version of the truth. This is usually because they had a ‘number’ (probably revenue) and went to speak to one of their Department Head about it (probably because it was unexpectedly low) and rather than spending time on understanding what the number means or what the business should do, they spent 45 minutes comparing the Senior Managers ‘number’ with the Department Heads ‘number’. In trying to reconcile them, they also find some more ‘numbers’ too. It probably passed the time nicely. Make this a monthly meeting or a QBR involving a number of department heads and the 45 minutes will stretch into hours without any real insight from which decisions might have been made.

This is partly about provenance. Ideally it came from a single system of record (Finance, HR) or corporate BI but it most likely came from a spreadsheet or even worse a presentation with a spreadsheet embedded in it.

It’s also about purity (or the addition of impurities, at least) It might have started pure but the department head or an analyst that works in their support and admin team calculated the number based on an extract from the finance system and possibly some other spreadsheets. The numbers were probably adjusted because of some departmental nuance. For example, if it’s a Sales Team, the Sales Manager might include all the sales for a rep that joined part way through the year whilst Finance left the revenue with the previous team.

It will be no comfort (or surprise) to our Senior Manager that it is also a Master Data Management problem too. Revenue by product can only make sense if everyone in the organisation can agree the brands, categories and products that classify the things that are sold. Superficially this sounds simple but even this week I have spoken with a global business that is launching a major initiative, involving hundreds of man hours to resolve just this issue.

It’s also about terminology. We sacrifice precision in language for efficiency. In most organistions we dance dangerously around synonyms and homonyms because it mostly doesn’t catch us out. Net revenue … net of what? And whilst we are on the subject … revenue. Revenue as it was ordered, as it was delivered, as it was invoiced and as it is recognised according to GAAP rules in the finance system. By the way does your number include credit notes? And this is a SIMPLE example. Costs are often centralised, allocated or shared in some way and all dependent on a set of rules that only a handful of people in the finance team really understand.

Finally, it’s about perspective. Departments in an organisation often talk about the same things but mean subtly different things because they have different perspectives. The sales team mean ordered revenue because once someone has signed hard (three copies) their job is done whilst the SMT are probably concerned about the revenue that they share with the markets in their statutory accounts.

So is a single version of the truth philisophy? Can it really be achieved? The answer is probably that there are multiple versions of the truth but they are, in many organisations, all wrong. Many organisations are looking at different things with differing perspectives and they are ALL inaccurate.

A high performing organisations should be trying to unpick these knots, in priority order, one at a time. Eventually they will be able to look at multiple versions of the truth and understand their business from multiple perspectives. Indeed the differences between the truth’s will probably tell them something they didn’t know from what they used to call ‘the single version of the truth’.