Process Mining - a Buzz Word?
Pretext
In any case, it sounds like digital transformation and is therefore indispensable in our post-modern world of work and projects. Very often, process mining is also mentioned in connection with Big Data.
But is that necessarily the case? Just because process mining is very often mentioned in connection with Big Data, a lot of people, especially smaller and medium-sized companies, already shy away from it.
We have a long experience with the working method, which is called Process Mining. Let's take a closer look at this topic and demystify these "brave new words".
process mining with a B2B startup, São Paulo, Brazil
Imagine the following three projects:
In your subsidiary, the potential for outsourcing is to be worked out in order to define a real basis for decision-making, far beyond pure contribution margin accounting and classic make-or-buy decisions.
In your transformation project for the realignment of your business model the real daily lived process world should be made visible (keywords: real event-based process events and real performance and productivity indicators).
You want to buy a company. Traditional due diligence is not enough for you and you set up a pre-merger M&A project that dives deep into the processes.
In all three cases it becomes clear that, as always in daily real life, there is a discrepancy between the Blue Print Process Model, the Process Execution Environment and the defining Event Logs. Thus, one can already guess that process mining is part of process management, which is based on data and events.
Process mining is a project technique for problems where no integrated automated solution and analysis tools are available. Project experience and contextual thinking are the key factors to successfully apply process mining. So it is about finding out what is really going on in the process, based on the real data created in the process during the process execution. It must also be clear that in almost all cases this real data will be found in very heterogeneous data repositories.
Definition: Process Mining
Contrary to popular belief, the last industrial revolution did not take place between the 18th and 19th centuries, but is younger than one might think, and was made possible by the application of integrated systems in factories and the incorporation of data management. It is in this context that the concept of process mining was developed.
If we want to give a brief description, process mining is a technique whose main goal is to discover real processes, then monitor and improve them. For this purpose, data knowledge is extracted from the data available in the enterprise software.
Looking further ahead, process mining provides companies with insights and defines measures that combine machine learning and data mining, guaranteeing more certainty in decision-making to implement new and more efficient processes in companies.
But watch out!
There is also a lot of manual work involved in many cases. This is really tedious, not sexy - but in almost all cases it is worth it.
Process Mining is a methodology that is used as a complement to Business Process Management (BPM) because of its positive results. You can also say that Process Mining is part of Business Process Management (BPM), based on data and events, and therefore it must actually be an absolutely necessary preliminary work of Business Process Management (BPM).
Process Mining Model applied by iManagementBrazil Ltda., São Paulo, Brazil; we have enlarged experience using this tool since more than seven years now.
A special focus is of course on the following two operational points:
process visualization
Here we basically use trace variants for the discovery of a process model. As a result, we visualize and model the process as it runs in reality. In very many cases this often has to be done by hand.
process extension
If new activities are identified during the execution of real processes, these additional activities can be added to an existing process model.
This can already lead to a first reordering of activities in the process model (e.g. outsourcing potential). Here we can also already identify a kind of life cycle of business processes. During the analysis work, the time stamp of the event log is used to identify whether and how a process changed evolutionarily or eruptively over time. In quite a few cases, this already leads to a redefinition of the currently implemented process, or to test runs for future processes.
Of course, constant process improvement is also on the project radar. This consideration is a repetitive task to improve existing blueprint process models.
What are the reasons that often make process improvement necessary at this very moment?
change of IT systems, which leads to a finer data capture and improves the further process mining and
introduction of new ways of working within an organization for daily process execution as well as for LoFi model testing purposes.
Process Mining - three pillars
Discovery
Process of identifying the company's real process model from a set of records extracted directly from the system; this work is very often done manually and by hand and can take a lot of time. Here we quickly get an idea of whether our client has a strong or weak process blueprint. Looking at our project experience, we have to conclude that at least ⅔, if not up to 4/5 of all clients have a very weak blueprint.
Conformance
Different process models are compared to identify differences and similarities in order to diagnose potential problems and inefficiencies; here the experience from a wide variety of projects in different industrial sectors is very helpful.
Improvement
It is when the different process models are adapted with the aim of extending and optimizing the procedures, allowing the creation of complete processes or new, more efficient phases.
Workflow
To build a process mining robust, information systems are used to record everything that was done during the execution of a task, from information about the person responsible, what materials and resources were consumed, what was done, the time spent and many other factors related to the activities.
The technique aims to discover, monitor, analyze and optimize real processes by extracting the events present in the enterprise systems.
In other words, process mining aims to make event information compatible with process models in order to use them. The following four premises are constantly applied:
Verify process compliance,
detect deviations,
predict and avoid delays, and
redesign processes.
Based on the registered data, it is possible to identify the actual behavior of processes and detect and diagnose errors based on consistent information, which ensures much more reliable decision making for implementing new and more efficient processes. The registered data generates the so-called event logs.
workflow detection and documentation is an essential part of process mining
What is an event log?
An event log is defined by a set of events that occurred during the execution of a process.
Each event is a specific activity that occurred at a specific point in time of the process and can be assigned to a unique case, i.e. a process instance.
Thus, each process event has a defined timestamp. This also shows that the investigation of the events is always based on the timestamps, which also define the order.
What are the advantages of Process Mining?
Process Mining offers a wide range of advantages, which not only unfold their effectiveness in the course of the project, but also have a lasting effect in the client's company. We have noticed this again and again when we visited our former clients after a certain period of time, sometimes even one or two years after the completion of the project.
The most important benefits from our point of view are:
Process optimization:
Process mining gives our client more confidence in decision making and make it possible to act directly on the root cause of problems, since one will have concrete information on how processes work in practice; thus, with accurate and soundly verified information, the company will be much more productive and profitable in the development of its solutions; the knowledge remains in the company after the project is completed and becomes part of the management compass.
Saving resources and reducing operating costs:
Identifying and eliminating deviations, bottlenecks and inefficient processes that need to be or can be analyzed or automated. Waste due to errors or inconsistencies in processes is avoided, which also immediately increases productivity.
Increase production quality and continuously improve processes:
With process mining, we analyze your processes thoroughly and accurately, determine the origin of any problem, and translate the findings into solutions. This results in a natural consequence of working with precise real-time information. Production quality is not only expressed in the resulting product quality, but also in the dominance of production and process data.
Future perspective:
The decision-making basis for investments in technological infrastructure becomes much more robust and leads to a sound correlation of data for better predictability, scenario development and evaluation, budgeting and monthly controlling. In all our projects we can unmistakably see that the correlation of market data, suppliers and even end users is fundamentally increasing.
In this context, it is particularly important for us as project and interim managers in the client's company to work out the balance between cause and effect of the data for the management in order to understand the company's position in the market. In doing so, a derivative project dynamic is created whereby process mining helps to identify the best ways for our client to grow and consolidate in the segment in which they operate.
From our own project history, I would like to recommend three really emblematic business cases in this regard.
The first case is a transformation process of a local coffee shop franchise.
The second business case is about a crisis management of a restaurant and catering chain in São Paulo.
The third business case highlights a machinery and equipment manufacturer in the oil and gas industry in Brazil. The company recognized the symptoms of the crisis early enough and decided to act.
All three business cases exemplify how necessary and sustainable a deep dive into the processes and the real true event logs can be with the help of process mining.
What is the connection between process mining and Industry 4.0?
Especially in the third business case presented above, the so-called Industry 4.0 becomes visible on the horizon. Automation, the search for data and supplier integration, and the conscious use of the new “gold data" is the essence of Industry 4.0, and the buzz word "Big Data" quickly comes up.
But watch out!
We are not talking about the Big Data world of consumer companies here, but about industrial manufacturing.
We should therefore not be intimidated by the word. Big Data is constantly being redefined in the context of the client and the industrial sector. One should not rely here on the definition of Big Data often propagated by large management consultancies, but contextualize the idea and apply it to the very own case.
The foundations for Industry 4.0 are automation and data processing, as information is collected in large quantities (Big Data), so it can show clearer directions without requiring our clients' managers to take many risks.
Process mining can thus be related to Industry 4.0, as it is the technique that allows the transformation of collected data into useful information, so that each process of the production line is improved.
However, it should be clear at this point that there is no need at all to push the application of process mining further into the Big Data environment or even Industry 4.0.
data collection within industrial processes is the new gold standard; ist does not matter, if you are a small or mid-sized company
What is the difference between process mining and business intelligence (BI)?
The fact is that Process Mining and BI have some similar characteristics. The managers of our client companies always have the largest Business Intelligence, especially if they have been active in the company or in the industrial sector for a long time. Thus, it is also clear that our application of process mining can never be done stand alone. It is always an integrated cooperation together with our clients.
If, dear reader, a consultant offers you that he will come to your company, dominate the process mining and only needs an office space where he will analyze your processes in peace and quiet, you should become suspicious in the first place. The reaction is simple and clear: this promise has no value!
Process Mining is one of the strongest integrative project techniques with the client!
integrated project workflow - our clients and iMB
But what are now examples of similarity between Process Mining and Business Intelligence? Here are some examples from our experience:
Both have the ability to split and combine data to support correlation of information and breakdown to specific levels and scenarios.
Both tend to convey information in a visual way through a variety of graphical means.
Both are analytical tools that are able to create a picture of reality based on information from the day-to-day operations of the company.
Both are methods which can be quickly transferred to machine learning and will most certainly be helpful in the application of artificial intelligence in the near future.
However, despite no small number of similarities, process mining and BI differ fundamentally in two respects. In process mining, the information in the predefined model takes the form of events, which generally represent the following different data:
Activity name and case identification (key); the presence of this data enables the process mining process to identify the case (key).
The presence of this data enables the process mining play-in functions, i.e., functions related to the process of mapping the cases, discovering the flow, detecting bottlenecks, rework and inefficiencies, among others, as well as replay functions. This information is also responsible for feeding process models that enable compliance analysis - a function not present in BI solutions.
BI, on the other hand, is more suitable for scenarios where the input data is not event-driven. For example, time series data is not suitable for process mining analysis unless it is used to enrich other information sets that have already been studied or it is possible to convert it into events in a process mining platform.
So, we can say that process mining is more like business intelligence with advanced features. Both solutions represent different resources, but in many cases they can be combined so that organizations can count on analytics and KPIs from different perspectives.
It should also be clear that the project application of Process Mining serves to educate and train the client's managers who have accumulated the solid Business Intelligence.
The main difference between BI and process mining is the level of analysis and the results they produce, which provide different types of insights.
BI aims to point out problems in a process, while process mining identifies the root cause of inefficiencies and time and cost drivers.
In other words, business intelligence is very good at naming the problem, but not at finding the root cause of process deviations from the blueprint.
The transparency that process mining creates for entire processes answers even more important questions, such as root causes, bottlenecks, rework, and deviations.
How to apply process mining?
One of the fundamental questions that become clear as a result of the research is whether the client has a strong or weak process blueprint. From our projects we have to conclude that at least 2/3 of all clients tend to have a weaker blueprint.
The application of Process Mining is simple and practical, but very often very time consuming and not infrequently also with high manual effort. The starting point is to transform event data stored in enterprise information systems (such as ERP, CRM, and BPMS) into an event log to gain insight into business processes.
Then, automated end-to-end process visualizations are created that provide accurate and insightful information about the required improvements according to the process events. This can be created using a wide variety of mind mapping applications as well as simple ones.
One of the most frequent questions, especially from our clients, is directed to the quality dimension. Here the general question is: How can we determine whether a resulting model is good and solid?
First of all, we have to ensure and prove in a comprehensible way that different algorithms for process discovery are available and have been identified by us.
These algorithms build on the process parameters that can produce different models. At this point, it strongly depends on our experience gained in various projects in different industries. However, experience alone is not enough. Equally important is the abstraction and contextualization of the found process parameters.
Criteria for quality dimensions of the model
Modeling is a basic tool in project management, reorganizations and transformational project missions. Here it is always possible to quickly overshoot the target or to reduce the necessary complexity too far. Therefore, it is always important to use low-fi models and to test them quickly. This is then done based on the notion of underfitting and overfitting model simulations.
Underfitting versus overfitting - 03 basic quality dimensions
These quality dimensions are also the root in the creation of LoFi models, which we have been using intensively in transformation projects for several years.
At this point, the reader is also recommended to read the following link:
Fitness: does the model cover the traces from the event log? We must strive to ensure that every trace from the event log is reproduced within the model. Also, we must demonstrably ensure how many times an event occurs in a record that cannot be executed according to the developed model. This then has two implications: either it is a complexity reduction with no impact on the function of the model, or the model we have developed is not yet close enough to reality.
Accuracy: does the model allow to introduce traces that are not in the event log? It is not always easy to determine how many traces of the model are not part of the event log, since models often allow an infinite number of traces. Here, it is very much up to us as project and interim managers to get the context right and to sensitively weight all parameters. This can only be done as a team.
Simplicity: does the model represent the event log as simple as possible? A qualification should be comprehensible in many ways. Comprehensibility usually refers to the number of model elements and the model structure.
Why do we need the quality dimensions in process mining?
Already in the start phase of the project at the client this question is enormously important. Only in this way we are able to select a suitable discovery algorithm together with the client's team. Later, the quality dimension is important to determine whether the discovered model is a good representation of the event log. Thus, the quality dimension is a concurrent project milestone and part of project controlling.
The suitability can be quantified by using each trace from the event log in the model. The simplicity refers only to the model and does not consider the event log, which remains unanalyzed.
Verification of conformity
The primary focus here is to compare the event log with the Blueprint process model. We examine where the actual process execution deviates from the plan or model.
In the application, we thus identify patterns of non-compliance (e.g. missing/ omitted/wrong activity sequence, unexpected activities).
Why is conformance testing important?
It is an ongoing project controlling step with many different scenarios:
Uncovering problems and quality improvement potential in the process.
Receiving feedback on how well the process conforms to expectations and/or the intended process.
Compliance with laws and regulations.
Comparison of event logs and model traces allows us to identify relevant violation patterns.
Feedback from the comparison can be used to analyze individual traces as well as the entire log.
The comparison also facilitates the quantification of compliance.
Performing the comparison automatically can be expensive, so verification in qualified team meetings is often very efficient.
Résumé
Do you remember the three projects mentioned at the beginning?
In your subsidiary, the potential for outsourcing should be worked out in order to define a real basis for decision-making, far beyond the pure contribution margin calculation and classic make-or-buy decision.
This was an automotive supplier with an extremely differentiated production portfolio. The company produced small parts, which were invoiced on the basis of weight deliveries, up to large parts with long process times. The machinery used for production was also extremely complex. From old depreciated mechanical machines, to highly automated processes.
In order to identify outsourcing potential in production, purchasing and administration, process mining had to be carried out in all product groups to make the real processes transparent. This was the only way to create a basis for streamlining processes prior to possible outsourcing, and also to create a basis of trust so as not to overburden potential outsourcing partners and to provide them with robust process data right from the start. Only in this way could a good negotiation take place and a new process network be established.
In your transformation project to realign your business model, the real daily lived process world should be made visible (keywords: real event-based process events and real performance and productivity indicators).
In this case it was the transformation of a coffee shop franchise into an additional digital model. In order to develop and test appropriate LoFi models at all, process mining had to take place beforehand. It was also immediately clear that the physical models could not simply be transformed into digital models. Only through abstraction and contextualization was a robust digital model found and implemented.
You want to buy a company. Traditional due diligence is not enough for you and you set up a premerger M&A project diving deep into the processes.
In this third case it was the purchase of an engineering company for structural mechanical and plant engineering in extraction technology for the oil and gas industry.
Basically, services were offered in engineering development as well as site management.
In order to be able to sustainably evaluate the charged cost rates and cost structures in the projects, the usual due diligence was not sufficient.
With the help of process mining, it was possible to visualize the real hours worked and also services charged. The results were then used by the auditors in the due diligence process to better determine the profitability of the individual projects and thus the value of the company.
We have a broad experience in the application of process mining. We regularly apply this structured tool in reorganization and transformation projects, or complementary in projects of auditors.
