ISSN 0798 1015

Vol. 41 (Issue 12) Year 2020. Page 14

Assessing digital maturity of organizations

Evaluación de la madurez digital de las organizaciones

VARNAVSKIY, Andrew V. 1; VOLKOVA, Elena S. 2; GRUZINA, Yuliya M. 3; BURYAKOVA, Anastasiya O. 4; KLIMOVA, Elizaveta A 5

Received: 16/12/2019 • Approved: 14/03/2020 • Published 09/04/2020

Contents

1. Introduction

2. Literature review

3. Methodology

4. Results

5. Conclusions

Bibliographic references

1. Introduction

Digitalization of the economy provides unlimited opportunities to   improve the efficiency of the operating activities of organizations. However, the evolution of human capital at the modern stage of development remains one of the main problems slowing down this process. The creation and usage of an effective model for assessing the level of development of digital leadership of organizations in leading science and technology sectors and fundamental research will allow to denote not only the current situation of organizations regarding the introduction of modern technologies and the application of innovative approaches to educational and scientific activities, but also to identify “digital” reserves of growth.

Both domestic and foreign experts addressed the issues of leadership in the knowledge economy. Digital leadership - is a concept that applies to CEOs, the companies themselves, and even the whole states. Thus, the need to form a special type of leadership for high-tech enterprises arises due to the corresponding management tasks: increasing labor productivity, involving personnel in knowledge-intensive processes operating in the "virtual" and real dimensions (Gumerova, Shaimieva, 2018). A leader in the digital age should meet the following requirements: be the initiator of projects that could be a source of income for the organization; be in constant search for the quality of products or services, without increasing the costs of the organization; have the necessary IT competence; motivate employees to generate new ideas to improve the quality of the organization (Chesnokova, 2019).

Galimova M.P. in the article «Readiness of Russian enterprises to digital transformation: organizational drivers and barriers» (Galimova, 2019) suggests that modern enterprises should have up-to-date information about the level of their digital development relative to other companies. That is, the focus is no longer on the assessment of an individual, but on the whole organization. The author proposes indicators of assessing the readiness of companies for digital transformation in the context of the requirements of the “Digital Economy of the Russian Federation” program’s roadmap, which defines five basic directions. In general, the methodology is aimed at evaluating organizations in completely different areas.

However, today it is vital to use not only general methodologies, the results of which may witness the level of digitalization of an individual state, but also methods that are oriented to concrete industries. In our opinion, the assessment of digital leadership in organizations of the scientific and technological sector could become a kind of indicator of the potential of certain territories to carry out digital transformation, an indicator of the presence of digital drivers. Accordingly, such technique should contain approaches to assessing human capital, the condition of infrastructure, the possibility of generating cash flows from digitalization, the availability of domestic demand and etc. The formation of such methodology is also due to the need for a flexible tool for analyzing the redistribution of resources between state scientific organizations (Galimova, 2019).

A review of the literature allowed to conclude that the use of structural, hierarchical models is perspective. So, in the work «The monitoring of processes and the assessment of researches activities» Glukhova E. A. brings the appropriate model for compliance with quality indicators and the requirements of continuous improvement. There are three main levels: the direction of knowledge generation (knowledge creation, reproduction and exchange of knowledge, diffusion of knowledge), process indicators (publication activity, academic activity, communicative activity, patent activity), output indicators (number of publications in international journals and recommended journals, citation, etc.). Authors of this article fully agree with the idea that the success of the company in the market is not determined by what it selects projects under its management system, and the fact that it fits her ongoing projects (Glukhova, 2015). That is, the scientific research monitoring model should take into account the so-called internal factors. This provision is reflected in our methodology. However, the main difference between the model proposed by the authors of this article and the model of Glukhova E. A. will be the use of not only quantitative indicators, but also qualitative ones, by means of which it becomes possible to assess such internal factors as digital culture, innovation, the quality of implemented projects by digitalization departments, etc.

Thus, the purpose of this research consists in forming a model for assessing the level of development of organization’s digital leadership in the scientific and technical sectors, taking into account relevant international and Russian experience. The main requirement is objectivity. The consideration of many disparate factors, ranging from the presence of CDO and the number issued by the patent and ending with the efficiency of interaction of divisions, allows to create a system of evaluation, which will grant the leading positions to the companies with a high level of digital development. Our proposed model is based on mathematical modeling and allows us to identify the reserves of digital growth of organizations for each of the considered areas of evaluation.

Existing methods of assessing the level of digital leadership are diverse enough. So, if we turn to the consideration of foreign experience, first of all we should pay attention to the Oxford economic group.  The group includes companies that provide sector-specific forecasting and modeling expertise and has conducted research on the application and assessment of digital leadership in organizations. According to Oxford study, only 3% of the total number of companies in the world have completed transformation projects in the field of digitalization and have a high level of digital leadership development (Oxford Economics, 2017). The leading companies in these 3% are more focused on changing technology, empowering customers over the next two years. However, there are four key characteristics that really set them apart from the rest of the study participants (Phillips, Pulliam & Ray, 2015):

1. The true desire to transform activity;

2. The client is the center of attention;

3. The use of new generation technologies;

4. Focus on talented employees.

International consulting company IDC, in turn, has developed a six-step structure, which is allows to assess the level of development of digital leadership. According to IDC, the parameter system for evaluation should look like this (Boulton, 2019):

1. World view. Leaders must develop a culture of innovation and digital progress, focused on creating “digital” revenue streams, and identify new alternatives based on modern technology, with a clear approach to risk management.

2. Integration. To prevent adverse situations in the company, associated with system deficiencies, it is vital to introduce the company’s corporate architecture of a digital platform, which will actively upgrade the corporate infrastructure and systems to support the digital organization. SAP research confirms IDC's priorities in this measurement: leaders have created a solid base for the most efficient use of next-generation technologies, and, at present, such companies are far ahead of companies that don’t use such integration in the field of large data arrays and analytics.

3. Customer focus. To achieve the required level of digital leadership, companies should have an external customer focus for the entire IT organization. The SAP study also confirms the importance of this area: after investing in increased networking and customer cooperation, 46% of the 100 leading leaders reported that this investment had a significant influence on the result of digital work.

4. Possibility of introducing innovations. Digital transformation requires organizations to look at the massifs of information in a completely different way: it is important to identify new ways to monetize them and reinvent business models to create new revenue streams. For this, companies need employees who will implement innovative data platforms that can efficiently integrate them as needed, regardless of where they are stored.

5. Creation of profitable digital opportunities. Building a digital platform architecture aimed at modernizing enterprise applications is also necessary to achieve digital leadership. IDC researches have proven that this KPI is unique to CDOs only, while the CIO and IT department of the company must provide the right digital platform architecture to support effective digital opportunities.

6. Talent management. Market improvement forces leaders to find ways to attract new digital talents - digital developers, database researchers, experts in the field of UX (User Experience) and UI (User Interface) and design consultants. These employees in the organization will help to quickly and efficiently turn digital data experience into profitable digital opportunities.

Also, specialists from Deloitte – an international network in the field of consulting and auditing, created a model of digital maturity that can assess the level of development of digital leadership in organizations (Deloitte, 2018). The necessity to create such a model lies in the possible benefits that it can bring to companies: the transition to digitalization is able to optimize costs, increase the speed of work and, thereby, increase the efficiency of activities, as well as create favorable conditions for scientific and technological progress.

The digital maturity model is an effective tool that provides clear guidance throughout the conversion process. The five main measurements are divided into 28 subgroups, which, in turn, are divided into 179 separate criteria by which digital maturity is evaluated (Evans, 2017): «customer», «strategy», «technology», «operations», «corporate culture».

The digital measurer can be used at any stage of organization’s transformation. Its implementation is necessary in order to help identify fault and errors in the work, establish key areas for transformation and build a clear plan of changes. This model doesn’t replace a comprehensive analysis of the organization’s activities, but it can serve as a guide to action and a tool that will be used throughout the process.

The considered model is constant, but the level, which is determined as satisfactory for recognizing the digital maturity of the organization in each industry depends on three elements: selected business strategy, business model and operating model. Any organization should take these components into account while having a DMM test, as the digital maturity model provides an overview of all the capabilities of the organization and its employees, helping to prioritize in the key areas and increase their efficiency in accordance with existing digital ambitions. Among the organizations that have successfully tested this model, there are: American software company Amdocs, the British largest mobile operator Vodafone, business consulting organization BearingPoint, and Chinese telecommunications corporation Huawei.

British consulting company Digital Leadership Ltd, which helps organizations find a way to improve their level of digital leadership in the short and long term, has also developed its digital maturity model (Digital Leadership Ltd, 2019). Fifteen organizational competencies, required for implementing digital technologies in organizational strategy, are assessed at a level from 1 to 5 to get a digital assessment of an organization's maturity:

Level 1 - the use of digital technologies at a basic level or their absence;

Level 2 - an advanced level of using digital opportunities;

Level 3 - digital leadership as a channel for the development and promotion of an organization;

Level 4 - integration of digital innovations with current tasks;

Level 5 – the organization’s transition to digital maturity.

Each competency highlighted by Digital Leadership Ltd has a great influence on the level of development of digital leadership of the organization: attitude to digitalization of processes, communication plan, audience, innovation, reporting, company data, technological infrastructure, budget, digital capacity, employability, training and development, management, staff experience, project management, service. Assessment of the work on the described competencies provides an opportunity for the organization to comprehensively analyze the current state and increase the efficiency and effectiveness of its activities.

The study examined approaches to assessing the level of digital leadership development of various organizations: Barcelona school of technology, international consulting company IDC, British consulting company Digital Leadership Ltd, Deloitte and others. The most promising models were recognized as having a multi-stage evaluation structure, in which both quantitative indicators are evaluated and the method of expert evaluations is used. 19 criteria were recognized as fundamental: "Strategy", "Corporate culture", "Innovation", "Reporting", "Technological infrastructure", "Budget", "Digital capacity", "Management", "staff Experience", "Research and development", "Patent activity" and others. These criteria are reflected in the model developed by the authors of this work to assess the level of development of digital leadership.

The analytic hierarchy process (described by T.L. Saaty) can become the basis for assessing digital maturity of the organization. This method involves the study of a set of criteria. After that, the evaluated criteria are compared to form hierarchies (Saaty, 1993). Creating an assessment model causes the difficulties associated with a multicomponent interdependent system consisting of completely different parameters.

Differentiation of criteria into key areas has several advantages:

1. Selected areas reflect more detailed information about the structure of the system. Detailing is ensured by lower hierarchy levels. Factor analysis affects the general goals of a higher level hierarchy. Elements that have a global character can be represented at the higher levels of the hierarchy, others that specifically characterize the problem can be developed in greater depth;

2. Pairwise comparisons in combination with the hierarchical structure are so useful in deriving measurement.  The modular approach reflects the results more efficiently than evaluating the criteria individually;

3. Hierarchies are flexible and resistant to change. Flexibility means that adding new elements to the hierarchy will not destroy the integrity of the entire direction. Stability means that small changes in the areas structure will cause a small effect on the common result.

In particular, the method of assessing the development of the level of digital leadership is described as follows. The criteria of one area (lower levels of the hierarchy) is compared by the strength of their influence on the general result. Pairwise comparisons are obtained by a survey of experts or statistically. Delineated preferences are definite, not probabilistic, so that preferences are independent from other factors, which are not included in the study. As a result, a matrix of pairwise comparisons is formed (fig. 2). Characteristic vector, corresponding to the maximum eigenvalue of matrix of pair-wise comparison, is a priority vector. This vector provides an ordering of identified priority areas. The maximum eigenvalue of matrix of pairwise comparisons determines the measure of consistency.

Consistency is necessary to shape the model. Consistency is closely related to the transitive property (if A is greater than B, and B is greater than C, then A is greater than C). Violation of this rule will mean a lack of proportionality between the compared criteria. As a result, incorrect evaluation results will be obtained. It is also necessary to take into account the actual degree of preference, which is called numerical or principal consistency (if A is 3 times more than B, and B is 5 times more, then A should be 15 times more than B.). 

The approach to assessing digital maturity of organizations not only shows the consistency or inconsistency of the individual elements, but also gives the estimate of the deviation from the full consistency of the entire areas. The assessments express in terms of the eigenvalue, which allow to get the coordinates of the eigenvector

The priority scale must first be defined. As an example, consider three speculative criteria - A, B, C. Judgments about the level of significance of indicators are made by an expert group, which determines the degree of influence of the criterion on the eigenvector of the whole area. The comparison results are recorded in the matrix (Table 1).

Table 1
Matrix of pair-wise comparison 3х3

Denote the following values for comparison:

- if A and B have the same significance level, we put in the matrix "1";

- if A is slightly more important than B, enter into the matrix "2";

- if A is more important than B, we enter in the matrix "3";

- if A is significantly important than B, enter into the matrix "4";

- if A is absolutely and obviously more important than B, we put in the matrix "5".

The process of assessing the significance of the criteria is graphically shown in Figure 1. A scale of 1: 5 will be the most convenient to use. Three scale values ("1", "3" and "5") are the main parameters in assessing the relative importance, but "2" and "4" are compromise, intermediate options (Tikhomirova, Sidorenko, 2012).

Comparing an element to itself, equal significance is formed. Accordingly, at the intersection of the row with its column (A, A) use parameter "1". Accordingly, the main diagonal of the matrix always consists of "1". Also, a reverse comparison is used to find compromises. If A is more important than B in the matrix in cell (A, B) enter "3". At the intersection, in cell (B, A) there will be an inverse value equal to 1/3.

The consistency of the matrix of pairwise comparisons A(aij) is achieved in accordance with the condition aij x ajk = aik (associativeness of comparison).

It is known that the consistency of a positive antisymmetric matrix is equivalent to the equality of the maximum eigenvalue and the degree of matrix, i.e., λmax = n. To determine consistency, the deviation is estimated. Saaty defined the consistency index (CI) as follows:

λmax - maximum principal eigenvalue,
n -  the degree of matrix.

In order to determine the level of consistency of the resulting matrix, it is necessary to compare the obtained value of the consistency index with its maximum feasible value, calculated for each matrix separately.

The eigenvector standardized by normalization can be interpreted as the degree of importance of each alternative. In this situation, the comparison matrix satisfies the transitivity property for all pairwise comparisons.

The mathematical model is based on an approach involving the analysis of a number of criteria distributed in 5 key areas (Table 2). The selected areas, in our opinion, are able to most accurately characterize the activities of Russian organizations and determine their level of digital development. Five key areas were formulated (including 16 attributes):

1. «Educational activity»;

2. «Scientific and academic activity»;

3. «Digital operations maintenance»;

4. «Corporate culture and staff experience»;

5. «Control and reporting».

The experience of Deloitte, Digital Leadership Ltd, Bloomberg was used in creating a rating. However, the criteria / attributes were modified in accordance with the characteristics of the activities of Russian educational organizations.

Table 2
Areas structure of the model
of assessing digital maturity

Each of the areas has both quantitative and qualitative assessments. Quantitative assessments are comparative indicators that in some cases will take on a value from 0 to 1 (for example, a fraction of modern (digital) areas of study in the total number of programs), in others it will not be limited (revenue from digital research activities per department). For qualitative characteristics, the expert assessment method is used. It is assumed that experts will assign values to criteria from 0 to 1 (for example, with an interval of 0.2), depending on the situation in each particular organization.

The main objective of the model is to evaluate the highest levels of the hierarchy, taking into account the interaction of the lower levels of the hierarchy. The advantages of this model are that the hierarchical representation of the system can be used to describe how changes in priorities at higher levels affect the priorities of elements of lower levels. Hierarchies provide more detailed information about the structure and functions of the system at lower levels and provide an overview of actors and their goals at higher levels. At the same time, models of this type are stable and flexible. Flexibility is a basic property that must be fulfilled during the creation of this type of a model. The results of pairwise estimates and the coordinates of the vectors may change.

Consider one of the possible options for creating a model that was obtained based on the author’s vision of the characteristics of a digital leader in the Russian higher education sector:

1. Educational activity

Table 3
The criteria «Educational activity»

The criterion equation formulation algorithm:

1) Find the maximum eigenvalue for the resulting matrix. All calculations were made using the Wolfram Alpha system.

lmax » 3,07351,

corresponding eigenvalue lmax » 3,07351: (0,190786; 0,43679; 1)

2) Calculate in accordance with the formula:

3) Сompare the consistency index with the maximum feasible value. Based on the conclusions made in the article by A. Tikhomirov and E. V. Sidorenko the result can be considered acceptable, because for a 3x3 matrix and higher degree matrices, the maximum feasible value should not exceed 0.127 (Tikhomirova, Sidorenko, 2012).

4) Calculate the matrix vector: n1 » (0,190786; 0,43679; 1)

5) Normalized vector: 0,190786 + 0,43679 + 1 = 1,627576

6) Thus, Index «Educational activity» takes the corresponding form:

We will apply this algorithm to other areas of assessment as well.

2. Scientific and academic activity

Table 4
«Scientific and academic activity»
criterion matrix

3. Digital operations maintenance

Table 5
«Digital operations maintenance»
criterion matrix

Thus «Digital operations maintenance» Index looks like:

4. Corporate culture and staff experience

Table 6
«Corporate culture and staff
experience» criterion matrix

Respectively «Corporate culture and staff experience» Index takes form of:

X4 = 0,110658a41 + 0,402159a42 + 0,487182a43

a41 = The presence of CDO, department of digitization and corresponding staff;

a42 = Employees’ digitalization qualifications;

a43 = Innovation (implementation of employee suggestions on improving operational processes).

5. Control and reporting

Table 7
The criterion «Control
and reporting» matrix

Thus «Control and reporting» Index looks like:

Thus, general model of assessing digital maturity of organizations in the science and technology looks like:

Table 8
Digital maturity
criterion matrix

Thus, «Digital maturity» Index equation will look like:

The proposed method of calculating the model of assessing the level of development of digital leadership is based on the "external" and "internal" aspects of organizational performance in the leading scientific and technical sectors and fundamental research. We allocate to "external" areas educational and scientific activities, and, respectively, the efficiency of the organization's departments in introducing the technologies to optimize activities - to the "internal". The main advantage of the proposed model is the different factors consideration, including CDO, a number of patents, an interaction of units and others. Therefore, it becomes possible to create a rating system in which leading positions be occupied by having digital potential organizations that “correctly” filled out the documentation). The model allows to identify the reserves of organizations digital growth for each evaluation criterion. As a result, the methodology can be used not only by private legal entities, but also by government bodies responsible for the relevant educational activities.

At the same time, each of the directions has both quantitative and qualitative indicators (results of expert assessment). The obtained evaluation model is based on hierarchy analysis method of T. Saaty - using the lower level of the hierarchy, the formed criteria of one direction are compared in pairs by the strength of their influence on the key direction as a whole. Expressed preferences are definite, not probabilistic. As a result, a matrix of pairwise comparisons is formed. The level of digital leadership development is characterized by an aggregated indicator calculated on the basis of a normalized matrix vector. The higher the index of this criterion - the higher is the place of the organization in the ranking of digitalization.

Boulton, C. (2019). 10 digital transformation success stories. Retrieved from https://www.cio.com/article/3149977/digital-transformation-examples.html

Chesnokova, J.A. (2019). Leadership development in organization quality management system. Bulletin of the Saratov State Socio-Economic University, 1 (75), 89-91. Retrieved from https://cyberleninka.ru/article/n/razvitie-liderstva-v-sisteme-menedzhmenta-kachestva-organizatsii.html

Deloitte. Digital Maturity Model Achieving digital maturity to drive growth. Retrieved from https://www2.deloitte.com/content/dam/Deloitte/global/Documents/Technology-Media-Telecommunications/deloitte-digital-maturity-model.pdf

Digital Leadership Ltd. Digital maturity framework 2019. Retrieved from https://digitalmaturity.org/

Evans, Nicholas D. (2017). Assessing your organization’s digital transformation maturity. Retrieved from  https://www.cio.com/article/3213194/assessing-your-organization-s-digital-transformation-maturity.html

Galimova, M.P. (2019). Readiness of Russian enterprises for digital transformation: organizational drivers and barriers. Bulletin of USTU. Science, education, economics. Series: Economics, 1 (27), 27-37. Retrieved from https://cyberleninka.ru/article/n/gotovnost-rossiyskih-predpriyatiy-k-tsifrovoy-transformatsii-organizatsionnye-drayvery-i-bariery

Glukhova, E.A. (2012). Monitoring processes and evaluating the effectiveness of scientific activities. Healthcare of the Russian Federation, 5, 27-37. Retrieved from https://cyberleninka.ru/article/n/monitoring-protsessov-i-otsenka-rezultativnosti-nauchnoy-deyatelnosti

Gumerova, G.I., Shaimieva, E.S. (2018). Intelligent leadership in the digital economy (based on the concept of leadership in the knowledge economy). Russia: trends and development prospects, 13-1, 486-490. Retrieved from https://cyberleninka.ru/article/n/intellektualnoe-liderstvo-v-tsifrovoy-ekonomike-na-osnove-kontseptsii-liderstva-v-ekonomike-znaniy

Oxford Economics. SAP Digital Transformation Executive Study: 4 Ways Leaders Set Themselves Apart. An SAP Center for Business Insight study with research and analysis support from Oxford Economics. Retrieved from https://www.sap.com/registration/protected/default-overlay.html?pdf-asset=9ec2900c-c67c-0010-82c7-eda71af511fa&page=1#page=1

Pulliam, P., Phillips Jack J., Ray R. (2015). Measuring the Success of Leadership Development: A Step-by-step Guide for Measuring Impact and Calculating ROI. Association for Talent Development. Retrieved from  https://www.td.org/insights/measuring-the-success-of-leadership-development

Saaty, T. (1993). Making decisions. Hierarchy Analysis Method. Retrieved from  https://kbi.ucoz.ru/_ld/0/13_--_-__-1993.pdf

Starodubov, V.I., Kurakova, N.G., Tsvetkova, L.A., Markusova, V.A. (2011). Russian medical science in the mirror of international and domestic citation. Health Manager, 1, 6-20.

Tikhomirova A.N., Sidorenko E.V. (2012). Modification of T. Saati's hierarchy analysis method for calculating criteria weights when evaluating innovative projects. Modern problems of science and education, 2, 261-261.

1. Blockchain Laboratory of the Institute of Digital economy development, Financial University, Moscow, Russia, AVVarnavskiy@fa.ru   

2. Department of data analysis, decision making and financial technologies, Financial University under the Government of the Russian Federation, Moscow, Russian Federation, EVolkova@fa.ru

3. Financial University under the Government of the Russian Federation, Moscow, Russian Federation, YMGruzina@fa.ru

4. Blochchain Laboratory of the Institute of Digital economy development, Financial University, Moscow, Russia, AOBuryakova@fa.ru

5. Financial University, Faculty of Public Administration and Financial Control, Moscow, Russia, elizaveta.klimova@bk.ru