What Socrates Taught Us About Data Analysis

The Unexamined Life is Not Worth Living

If Socrates been a modern businessman, I’m sure he would have been an early adopter of Microsoft Power BI. He lived and died according to his adage: the unexamined life is not worth living. It is thanks to Socrates that we appreciate the huge power behind questions, and the work we do in data analysis is the stronger for it. Despite his famous frugality, I’ve no doubt that Socrates would have adored the visualizations.

Hemlock Pie Chart, anyone?

In today’s digital economy, the unexamined business is worth very little. Data analysis has become crucial in improving the way we work. However, data analysis isn’t a sausage machine: you don’t put data in one end and get strategy from the other. Data analysis, like life, benefits from questioning, Socratic or otherwise:

• What are we trying to achieve?
• What data do we have related to this?
• Where did the data come from?
• How accurate is the data?
• What evidence do we have to support this?
• Why is this happening?
• What does this mean?
• What do we already know about this?
• What are we assuming?
• How do we know this?
• What is causing this?
• What are the possible alternative causes?

Admittedly, not all of those questions might have been asked by Socrates, but the principle is the same - asking the right data analysis questions can be as valuable as the answers. Whilst numbers rarely lie, our interpretation of the numbers always benefits from critical questioning. The most successful companies on the planet use data analytics to test their assumptions, and methodically analyse data to get better answers. Data analysis has been behind the rise of Amazon, Nissan, American Express to name but a high-profile few.

Microsoft Power BI supports a wide range of data sources, analysis techniques, and visualizations, but all are most effective when used in conjunction with questions and hypotheses. And the results it presents are most useful when questioned carefully and critically. Undeniably, Microsoft Power BI is a powerful piece of software that’s getting more powerful every month.

At Anatec Software, we have always believed that our questions are more powerful than our answers, and the longevity of our solutions has proved us right. Every company is different, and every problem unique, so whilst the technologies we use might be the same, that’s where the similarity ends. We never try to fit problems to a one-solution-fits-all approach. Instead, we question thoughtfully, and listen hard to understand the challenges you face and the opportunities you’ve created. Only then can work with you to find the right solution.

We work with Microsoft SQL Server, Microsoft Power BI and Microsoft Excel to produce business intelligence solutions to suit all situations and budgets. We’ve got a lot of experience in helping businesses make sense of their data, and use their data to make better decisions.

So, if you’ve got more questions than answers, and a pile of data that could yield some insights, get in touch to find out how we can help. There’s no obligation, only thought-provoking questions and friendly advice.

Why Microsoft Power BI is a Big Deal

To understand why Microsoft Power BI is a big deal, you have to got back a bit. 153 years back, to be precise.

You would no doubt agree that businesses fail or flourish depending on how quickly they can adapt to new information. Whether it’s a competitive move, or a change in consumer tastes, ignoring intelligence is at best damaging, and at worst fatal. Over 150 years ago Richard Devens used the term business intelligence to describe profiting from timely decision making based on intelligence, or data. Back in 1865, acting on intelligence before the competition was unusual enough for comment, and is arguably still noteworthy today. But while we still use the term business intelligence, we actually mean something quite different.

Today businesses run on software systems - from banking to sales leads, invoicing to estimating. Business intelligence (BI) is also digital - instead of relying on the relatively haphazard activity of noticing what’s going on, we now methodically and systematically process and transform data. BI systems require a budget and project managers to turn disparate data into consolidated, useful information. While just about all businesses have more data than they can handle (estimates suggest that only 0.5% of data is analysed) not all have a business intelligence system. It seems that some things never change.

The modern term business intelligence, refers to a set of technologies that facilitate the analysis of data generated within the business. Data warehouses take data from the different systems, and bring them together into one useable form. If you’ve ever tried to figure out which system holds the correct version of the data you want, you will know this isn’t as easy as it sounds. Reporting systems then provide access to the data warehouse, giving decision makers good, clean, and timely information.

If the data in our business systems were easy to access and match up, we wouldn’t need BI systems. And if the data were always good quality, up to date, and available when we needed it, we wouldn't need BI systems. The reality is that for many businesses, data is siloed, available only to the department who owns it. Data is neither correct nor up to date, because people are busy and pulled in many different directions. The result is that people make decisions based on incomplete evidence, or carry out instructions in the belief that someone, somewhere has better data than they do.

For large businesses, the expense of implementing a business intelligence system is worthwhile; they hold a lot of data, and there are a lot of people who need access to it. For smaller businesses, there are spreadsheets. So prevalent are spreadsheets, it is often said they are the world’s most popular business intelligence tool. But they do have their limitations, and whilst it’s true that small and medium sized businesses have less data than the multi-nationals, they still have a lot of data, and they still need to be competitive. The playing field is anything but level.

Which brings us up to the present, because a new kid has arrived on the block. A powerful business intelligence system called Microsoft Power BI. Related to the spreadsheet in that it has taken some of the more powerful technologies from Excel, Microsoft has created a true business intelligence system that is capable of handling large volumes of data, producing real insights, and includes visualization capabilities that once belonged only in eye-wateringly expensive BI tools. Power BI is a truly democratizing piece of software, and a big deal for all businesses.

If you'd like to know more about what Microsoft Power BI can do for your business, follow the link to our web site and get in touch.

How to Design a Business Dashboard

8 Simple (but not easy) Steps

An effective dashboard can help you achieve your goals faster, and more efficiently. But not all dashboards work as intended, and some are actively unhelpful. So how do you get the design right? And how do you decide what should be included, and what should be omitted? Dashboards originated in cars, and the car is a great place to look for inspiration.

When driving, the objective is to get from A to B safely, efficiently, and without unnecessary expense. So a car’s dashboard communicates the essentials: speed, fuel, and problems with the engine - in that order. Sure, there’s more information such as whether safety belts are on, and doors are closed, but when you are driving the focus is on the essentials.

Varying weather conditions, hazards on the road, and the stress of navigating in unfamiliar places mean that car dashboards have to be clear and uncomplicated. This is not the place for clever, interactive graphics. Car dashboards have data that is large enough to be seen, and simple enough to be understood. But as with many things that seem simple, their elegance hides some clear thinking.

So here’s eight simple, but not necessarily easy, lessons from car dashboards:

1. Clarify the goal. A different goal will result in a different dashboard.
2. Identify the biggest risks to reaching the goal.
3. Identify the single more important metric to help you reach the goal (or avoid a mistake).
4. Identify other important metrics that will mitigate risks, or communicate progress.
5. Remove unnecessary metrics.
6. Communicate information simply and clearly, giving prominence to the most important metric.
7. Be disciplined in designing the dashboard around these decisions. If the design comes out differently, back up to check each decision.
8. Do the first things first; don’t start designing graphics before understanding the goal and important metrics.

Business goals are more varied than driving goals, and so don’t benefit from the "one size fits all" dashboard design. And as with driving, the potential to get distracted is constant, making the dashboard a vital tool. Business dashboards should be as well thought out, and effective, as dashboards in cars. After all, none of us want to run out of petrol before we reach the goal.

Effective Performance Measures

Creating an effective performance measure can have a positive, and sometimes dramatic effect. Good measures can:

• Motivate and inspire people to work towards an important target.
• Focus attention on what’s important.
• Identify problems.

In short, a well-designed performance measure can improve your ability to manage a business, project, or task.

A badly designed measure, on the other hand, can:

• Lead to harmful behaviours (also known as unintended consequences).
• Have no effect on reaching targets.
• Give the illusion of progress where no real progress exists. Wheels spin, but forward momentum is non-existent.

In short, a bad measure can cause damage in subtle and invisible ways.

Designing effective performance measures is important; really important. I recently came across a star rating that at first sight had all the hallmarks of a well-designed measure, but on closer inspection was anything but.

It is a rating for teachers – designed to help potential students to choose a new teacher. The star rating is from one to five, and calculated from existing students’ evaluations. After each lesson students are asked to rate their teacher on how well prepared the teacher was for the lesson, how punctual the teacher was, and how happy the student was with the lesson. The calculated result is displayed as a number of stars next to the teacher’s name.

What could be better? This is feedback from the people who are best placed to tell new students what the teacher is like. Well, maybe …

The difficulty is twofold: firstly, the close relationship between teacher and student, and secondly the subjective nature of the rating.

The close relationship means that if the student gives a poor rating, it might hurt the teacher’s feelings, or lead to the student having to explain what was wrong with the lesson. Students are likely to only give bad ratings when a number of lessons have been disappointing, and they have already decided to find another teacher. The close relationship means that only rarely, and in extreme conditions, would students give a poor rating.

The subjective nature of the rating is also problematic. The student might feel good at the end of the lesson, but have actually learned very little. Yes, the teacher was on time, yes, the teacher new their material, but how does a student rate how much they retained? That’s more difficult, and also relies on the student doing their homework and concentrating during the lesson.

As a result, a large number of teachers have a five-star rating, making the rating meaningless. Worse than that, poor teachers believe themselves to be “five-star teachers”, with no incentive to improve.

Yet the star rating system is widely used; often with the same in-built flaws. But could a better measure be designed?

There’s an old saying that actions speak louder than words. So measures that are designed around student loyalty might give better insight into effective teaching, such as measures designed around:

• The number of active students.
• Repeat bookings.
• The length of time a student has worked with a teacher.
• Students who are working towards an exam, and their exam marks.

These are tougher measures, but might give students a better idea of the quality of teaching.

This is just one example of a measure that looks good on a dashboard, but in reality doesn’t mean a great deal. Unless, of course, your objective is to sell the services of a lot of five-star teachers …

How to Improve Data Quality

Is he 98 or 298? It all depends on the data type ....

Designing a new database is exciting; new ways of looking at data are being created. But there is often frustration mixed in with the excitement as there is a certain amount of groundwork to get the project off the ground.

This includes figuring out what information needs to be stored. It is a process of going through the existing business processes, envisioning the new system, and deciding what is or is not needed. All now with an eye towards GDPR.

And making the right decisions about your data will help make the new system a success.

Having decided what data needs to be stored, there are more questions. What range of values do you expect to hold? Will you always know the value when a new record is entered? Could the value ever be negative? What is the largest value the file might hold? It is at this point that the frustration invariably turns to annoyance. "It doesn’t matter", comes back the reply. "We will think about that later" is another favourite. "After all, we’re not short of disk space, so what’s the problem?"

Yet each one of those unmade decisions is an opportunity to improve the quality of the data being held. And potentially the possibility of a bug. These decisions are not about running out of disk space, but about making sure the data you hold is the data you intended. The data type for each field is the most basic and most valuable constraint in a database. It is what separates relational databases from other ways of storing data, such as spreadsheets.

The data that can be entered into a particular field is constrained by a number of things:

• The data type. This ensures that, for example, a date value cannot be entered into a field designed to hold an integer.
• Primary and foreign keys. As well as providing relationships between tables, they also check data as it is being entered. A value entered into a foreign key field must match the value in another table’s primary key.
• NULL or NOT NULL. Fields that allow NULLs risk introducing hard-to-debug issues when data is added to the system. It is often necessary to allow NULLs, because the information may not always be available when records are added. However, fields that allow NULL can produce query results that are not the expected results.
• Allowable ranges. Whilst some data types automatically limit the range that can be added, others do not. For example, a SQL Server tinyint cannot take values greater than 255, or values that are negative. This might make a tinyint a good data type for a field holding age values. A SQL Server smallint, on the other hand, may be negative or positive, and allows values between -32,768 and +32,767, making is a poor choice for an age field, as incorrect data could easily be entered.  

There are very many more examples of how data types and constraints help keep data quality high. Whilst I fully admit that the age example is not ideal (it would be far better to hold date of birth rather than age), it is easily understood and illustrates the point.

Of course, things can be changed in a database system. You can add constraints later, or change constraints you got wrong. But everything comes at a price. And sometimes that price is reliability because every change in a system has consequences - sometimes unexpected ones.

So if you are getting impatient with never ending questions about your data, think of it as an investment in keeping the quality of your data high for years to come.