Analytics in an age of disruption

By Richard Boire

Disruption seems to be the only constant in our vastly changing world. Increased digital interconnection and tremendous capabilities in processing huge volumes of data at ever-increasing velocities are now the norm for many businesses. These increased technological capabilities have resulted in the emergence of artificial intelligence (AI) and the resulting paradigm shift towards even more automation.

The emergence of AI
Before delving more deeply into this area of AI, let us be clear what AI is not. In some conferences and articles, you will hear that AI encompasses any type of predictive models/machine learning algorithms. This is a fallacy. AI, in reality, is much more specific in that true artificial intelligence is about deep learning, which is the use of the mathematics of neural nets where you have an input layer, hidden layers and an output layer.

Complex algorithms using different optimization approaches have been developed in delivering solutions which can, and have, demonstrated great performance in the last few years. But what has caused this tremendous upsurge in AI interest given the research has been around for decades?

Like many processes, AI needs fuel, and in this case data, where it requires huge volumes. Technology has always been the limiting barrier in being able to process huge volumes. However, this is no longer the case as companies can now more fully leverage big data technology and its parallel data processing approach.

With companies now able to consume these ever-increasing quantities, the improvements caused by AI have been enormous in certain sectors. For instance, in the area of image recognition, accuracy rates have improved from rates of 45%-50% in the 1990s to well over 95% in the current era. These significant strides have also been manifested in the area of text recognition.

Even prior to the great breakthroughs achieved by AI, increased automation of many tasks and routines has been the norm in many industries, which have reduced the need for human intervention and ultimately the need to pay someone. AI has simply accelerated automation to a new level as companies seek opportunities to improve their business processes in serving customers with fewer employees. However, the emerging social impacts of these changes is really the subject of another study or book.

Data Scientist versus the Business Analyst
But in the world of analytics, what has been the impact? Yet before we explore this impact in more detail, let’s examine more closely the roles of the analytics practitioners. In today’s environment, we have essentially two levels of analysts:

The business analyst; and The data scientist.

The business analyst is typically the “face” and the key contact with the business unit. He or she is responsible in terms of presentations to the business unit and its key stakeholders, which is essentially the delivery of the solution. Storytelling is a pre-requisite skill for this individual. Meanwhile, the data scientist is a more technical person who is well-versed in the area of programming and coding alongside deep mathematical skills or at least the ability to interpret the output.

Both practitioners work closely together. The data scientist provides the technical output, such as predictive analytics solutions or the creation of an analytical file for reporting and visualization. Meanwhile, the business analyst in effect works with the technical output in order to present the solution in an understandable manner to the stakeholders of the relevant business unit.

In today’s environment, the hiring trends of many organization exemplifies these divergent skills. For the business analyst, the technical skills are the flexibility and nimbleness in being able to work with the many typical office applications, such as MS Excel and PowerPoint alongside existing data visualization tools such as Tableau, as well as emerging new software in this area. But the more important skill for them within an analytics project is arguably the much softer one of communication and storytelling.

Are technical skills enough?
For the data scientist, deep technical skills in computer programming and mathematics are the pre-requisite skills and in fact this is amplified by the number of Ph.Ds and master’s degrees recipients who are being considered for these positions. Certainly, knowledge of the techniques and its output are mission-critical for any data scientist.

But does the arcane knowledge of how an algorithm is mathematically calculated through a series of equations really necessary for every data science exercise? Are these deep technical skills fulfilling the real business needs of many organizations?

Certainly, the Googles and Facebooks of the world will always need these deep technical skills given their extensive research needs in their never-ending quest for new products and services. But for many organizations, it is more about the practical application of these technologies and how they will impact the business.

Specialized skillsets, such as extensive knowledge in how the mathematics work behind a convolutional neural net, are not really required. Rather it is how the output of a convolutional neural net can be used to classify images, thereby providing better information in such areas as claim processing and health care diagnostics. Another good example would be the use of a recurrent neural net in being able to build time-series type forecasting models. A much more generalist approach is really the required skillset in this area in applying the right technical outcome to a business need.

But what are these more “generalist” skillsets? The initial demand and arguably the most important one is the ability to identify the right problem or business challenge. Alongside this capability is the ability to create the right data environment in being able to derive a solution. With the data framework being created, the data scientist then needs to determine the appropriate approach and tools in developing the solution. At the same time, he or she needs to determine how this solution will be actioned and more importantly how it will be measured.

Keep in mind these practical demands still need to be complemented by technical skills. Without them, the data scientist lacks the knowledge in whether an advanced analytics solution is appropriate for a given business problem or challenge. For example, what does the output of a decision tree using random forests mean versus the output of a deep learning model? More importantly, how do we assess which approach is better in delivering a better solution?

These key demands of both a technical and general nature are not new. Businesses prior to the big data digital explosion have always struggled in trying to utilize both types of skillsets in resolving their business challenges.

The limiting barrier, though, was always time, as many of the tasks and activities of the data scientist were still very manual. Intensive programming whether through software like Python, R and SAS was always required in order to create the analytical file which, in most cases, consumes 85% or more of the data scientist’s time within an overall project. Once these files were created, the data scientist would then run a series of routines or load modules in either generating a report or producing a predictive model. Again, programming was required in order to run these routines against the data. All these tasks were very time-consuming despite the fact that a high level of technical knowledge was required for their execution.

Increased automation of analytics tasks
With the big data explosion, demands for data science skills have accelerated at logarithmic rates. Vendors have now emerged on the scene in an attempt to automate these time-consuming tasks and at the same time empower more people in both the development as well as execution of solutions. Automation has occurred in creating the analytical file where the practitioner does not need programming expertise, yet still requires a deep knowledge of how data works in order to create the right data framework for analysis.

Meanwhile, the actual development of predictive models has created companies where their end deliverable is the ability to “manufacture” many models quickly. This is much like what happened to the auto industry when Henry Ford transformed what was a much-customized approach to a more mass-automated approach. This “factory” type approach to predictive modelling still requires the data scientist to understand the mechanics and implications in order to more optimally leverage these tools. The phrase “A fool with a tool is still a fool” is still very appropriate in this scenario.

Finally, there is automation occurring in reporting and visualization, which is perhaps the most significant area and is exemplified by the large number of vendors attempting to meet this demand. The reason for its significance is that without this capability, all the hard work in creating the analytical file and developing predictive models is meaningless if the work is not actioned. Communication and storytelling are the keys to a solution being actioned within a business and reporting/visualization tools greatly facilitate this kind of capability.

How will the data scientist and business analyst evolve?
The essence of data science and its many activities have not changed. However, there is now more of a realignment towards those tasks and activities that require the deeper intellectual activities of the data scientist in creating business solutions. These skills translate into the softer skills of utilizing the “creative” side of their brain in order to design solutions that are tailored to the specific business needs.

At the same time, these “creative” elements are also manifested in enhanced “communication and storytelling” capabilities

. In effect, the data science role is now executing many of the tasks of the typical business analyst.

So, what, in turn, will become of the business analyst? As discussed above, the analytics tools now empower business analysts to run advanced analytics routines without any programming skills.

One can surmise that the data science and business analysts’ skills are essentially converging into what may be referred to as a hybrid. The need to both of them to apply deep analytical skills creatively is the key to developing solutions across a myriad of different business problems. As many people in the industry have mentioned, the ability to combine both art and science is really what data science is all about and really represents the hallmark of the hybrid. The hybrid is both a creative artist and engineer.

Growing demand for hybrid staff
The demand for these hybrid workers has always existed even prior to the digital explosion. But in today’s increasingly automated analytics environment, demand for hybrid staff will continue to accelerate.

The future of data science has never been brighter, even with increased automation as businesses increasingly seek more of these creative artists/engineers or hybrids. Exciting times lie ahead for the data science hybrid. In the next article, I will highlight examples and business cases of what this hybrid role might look like.

Richard Boire is currently president of Boire Analytics, an organization that is a leader in data analytics with over 30 years in applied analytics solutions across virtually all industry disciplines. He can be reached at boire@boireanalytics.com or for more information, go to: www.boireanalytics.com.