The Death of Process Mining?

In last month’s editorial, Paul Harmon mentioned process mining as a tool to assist process analysts and gave a plug for the first “International Conference on Process Mining.” So last week, for three sweltering days in Aachen, I took the chance to catch up with what’s new and what’s planned in the Process Mining space. The first thing to say was that the organizers did a great job, the first conference on any subject is risky, so to have over 500 people attending shows a high level of interest. Although primarily an academic conference, they did add an industry day, where users had the chance to share their experiences of applying process mining, and we had the benefit of Marc Kerremans from Gartner sharing his insights based on Gartner client inquiries and his vendor and market research. What the audience did not really get was much of a vendor perspective. Many vendors chose to sponsor and exhibit, but their views were only available to the audience via some extensive panel discussions. I think this was a shame and caused some distortion and that delegates could have gained from learning more about the commercial realities of process mining. Instead, the academics had the chance to share their views and thoughts on vendors, but the alternative vendor view on academic research was not so easily heard.

In sharing my insights, learnings, and observations, I have chosen for the most part not to name speakers or organizations, as the opinions I am sharing are mine based on what I heard or how I interpreted what I heard. This, of course, may not be either what the speaker meant or what they intended, so where possible, I have chosen to generalize and extrapolate my own views based on the inputs I heard and my own lens of analysis and experience. You may have been there, listened to the same things and chose to view them differently, which of course is fine, we each apply our own filters and have our personal objectives.

So, I need to explain why I chose to headline this article with the “Death of Process Mining” when the first conference has 500 attendees, many sponsors and is a market predicted to grow to more than $1bn over the next three years or so. Firstly, just because a market is seen to be rising does not mean that it is a survivable market in the long term. As Marc Kerremans pointed out, we saw the same thing occurring with Business Rules a few years ago. Many small vendors, with a few growing up, while in parallel, much research work was going on. In the end, many vendors were acquired, and the functionality subsumed into adjacent products and markets, e.g., BPMS. Marc Kerremans suggested the same might be right for Process Mining. I want to say that based on what I saw and heard I think that it is very likely, and people may look back in 5 years and wonder how he was able to make such an accurate prediction at a time when process mining seemed so fashionable.

My take is that when academics suggest that vendors are doing it wrong, when they suggest that all vendors need to do similar things and that vendors are not as pure in their implementations as they need to be, then they are actually doing more to drive vendors away from rather than towards a common view. We all know that in any given market the notion of selling undifferentiated products is a sure way to struggle and fail, so vendors have to find means of differentiating if they wish to recover development costs and generate revenues. Being right with a perfect product is not a sure way to survive in any market.

Two of the main areas that I was fascinated to hear research presented on were in the areas of Data Quality and Dealing with Large Data-sets. We all know from experience that running any form of analytics is tough if the data quality is not good enough. But, to learn that based on research the quality of the analysis produced by some process mining attempts is not as good as it could be and that data cleansing and gap filling had to be improved to get better is the same. For me, this highlights that while it might be interesting to research what can be done with better data, or how to attempt to backfill weak data, from a business user perspective any analysis has to produce useable, valuable insights based on the data they have, not the data they would like to have. Making too much noise to the broader market and about limitations based on data quality are more likely to stall potential buyers or limit buyers to those willing to experiment for unknown returns.

Concerning large data-sets, we heard more than one user organization say that what they most craved from the process mining community, whether vendors or academics, was the ability to deal with massive data-sets. It seems this may be an area where vendors might what to aim for differentiation, but as yet all have yet to deal adequately with the issue. At the same time, none of the research projects that I heard presented appeared to focus on easy, smarter, or faster ways of dealing effectively with large data-sets. I am not suggesting that work is not happening, just that I did not see or hear it.

Another telling comment came in one of the user case study sessions. As the speakers started their presentations, they shared their own observation on the previous day’s proceedings and some of the discussions that had gone before “I was a little bit worried yesterday, it felt as though I was at a conference discussing something completely different, and I questioned whether I was actually doing process mining at all!” Such comments highlight to me that there is nothing wrong with the research that is going on, or what developers are creating, the problem occurs when the two worlds meet. In business, it is not about science or adhering to the latest research ideas. It is about focusing on solving everyday real-world problems in a timely and cost-effective manner. Business users don’t have the luxury of investing time and money without some degree of certainty over the outcomes.

I think process mining is a great technique, and the tools do a great job in helping analyze processes, however, I think sometimes it is forgotten that what is being analyzed are only system process at best, transaction chains at worst and that they are not complete business processes and may never even have been designed as such. In all the debate about whether we should use Six Sigma etc. or process mining it seems we are losing sight of the fact that all we are learning from pure process mining is how to ensure conformance, compliance or about variations in paths through systems. Process Mining does not address manual aspects of the process, is not yet fully able to address cross system, cross-functional, or cross-organization processes. I agree with those who argue that the fact-based approach process mining delivers a better way of analyzing system behavior or the way people use a system, but I also say that feelings, the human understanding of the complete view and purpose of a process, is just as important.

It is this belief in both facts and feelings that leads me to suggest process mining is not likely to survive as a stand-alone market for very long. I don’t believe that in the long-term businesses will require separate capability. Instead, I think that process mining will be subsumed into other solutions and markets. As to whether that will be alongside different process and analysis and design capabilities such as vendors like QPR, Signavio and Software AG already offer, whether the over-financed RPA vendors will acquire the technology as the front end for selling RPA solutions, or whether it will be subsumed into the more general analytics market we can’t be sure. In all probability it will likely disappear into all three, leaving pure play process mining as a small niche market. That small market will probably be healthy for small vendors who are maybe more passionate about the technologies than revenues and not need to see massive sales to make the business worthwhile.

The integration of process mining with other technologies is also essential in helping to grow the market toward that $1bn projection. To date, most of the research on process mining has undertaken in mainland Europe, and in Australia, very little is occurring in the UK or indeed in the USA. It is also noticeable that for the most part, commercial revenues have followed the research areas, with the UK and the USA being a long way behind in terms of readiness to adopt. To achieve those growth levels, these two markets need to come on stream and quickly. However, we are unlikely to see them grow without a move away from a purist approach. At the moment it seems as though spending on process mining requires product investment, time and consulting investment, but often for an unknown return – and spend before you know the value will limit growth on markets like the USA. When process mining is bundled as a capability within a broader solution, the route to value may be easier, giving time for the ROI to work its way through.

Lastly, in the BPM world, we talked about the need to help organizations move away from inward-looking efficiency approaches to more outward-looking effectiveness driven mind-sets. Overused examples but keeping them simple, FedEx could never have been created by analyzing what other logistics companies at the time did, Uber and Lyft would never have been born by mimicking an existing taxi booking system, Dollar Shave, SouthWest Airlines, Amazon… I could go on and on. These businesses were created by starting not from analyzing what is, but asking what could be… Now if process mining could uncover patterns across journeys and help identify new possible business opportunities, or connect disconnected systems that enabled me to bring new products to market faster or identify ways of delivering better customer experience, then I am all in, and I think many of those who may have resisted so far would be too.

I have not seen what Celonis say they are doing in the area of Customer Journey Analysis, but I know from their award-winning approaches from last year that Signavio see great potential in the application of process mining to Customer Journeys, especially when linked to the underlying processes. For me, this is what I will be watching and waiting for, something that delivers the information I can’t easily access, in a way that makes it accessible and enables me to take actions I may not have realized I needed too.

PDF Version

Mark McGregor

A former Research Director at leading IT industry analysis firm Gartner, Mark has an extensive background in enterprise architecture, business process management and change management, having held executive positions with a number of technology companies. Since retiring from Gartner he now works as an independent consultant with clients such as Changepoint, Erwin, Mega, Planview, Signavio and LeanIX. Mark has authored or co-authored four books on business and process management, including “Thrive! How to Succeed in the Age of the Customer” and “In Search of BPM Excellence” and “People Centric Process Management. Widely respected for his knowledge and views on business change, he is the creator of “Next Practice” and has variously been described as a ”BPM Guru”, a “Thought Leader” and a “Master of Mindset”. Mark is passionate about the people aspects of change, he has spent much of the last fifteen years travelling the world, learning, teaching and researching the cultural aspects of change and how executives perceive business and process improvement In this capacity he has literally taught hundreds of people and been fortunate to interview and interact with many CEO’s. Mark holds certifications in Six Sigma, PRINCE2, Sales, Neuro-Linguistic Programming and Hypnosis! Mark suggests that it is the variety of his studies, which provide the depth he offers to his clients, in his words “It is the difference that makes the difference”. Connect with Mark via LinkedIn:


Leave a Reply

Your email address will not be published. Required fields are marked *