Unearthing the Tribal Knowledge through Process Discovery

Tribal knowledge is the informal skills and practices employees use to get things done. It's often undocumented and can lead to inefficiencies, compliance risks, and hinder automation efforts. Process discovery, a type of AI technology, can observe how people work and uncover these hidden processes. This "unearthing" of tribal knowledge can improve transparency, knowledge transfer, and overall business operations.

In this post, we will address the concept of unearthing tribal knowledge through process discovery. Companies try to codify processes and practices in the form of standard operating procedures, run books, policy documents, and flow charts.  All of these are worthy endeavors but oftentimes fail to capture the institutional memory rooted in people’s brain – the tribal knowledge.

What is Tribal Knowledge?

Tribal knowledge means the unwritten information and practices that are a part of every enterprise and those that drive operations without ever being explicitly codified. Tribal knowledge is manifest in almost every process and typically, it is in the heads of a few of the employees who have been there and done that for a while.

The amount of tribal knowledge in a company is directly proportional to the years of existence, the growth, and the underlying complexity.   We, at Skan.AI, refer to this phenomenon as the “Invisible Enterprise.”  It may surprise you, but informal and anecdotal evidence suggests that this could be as high as 50% of work happens in companies.

Examples of Hidden Institutional Knowledge:

• At home, you may know a particular way to jiggle the wire to turn on the lamp. It is not a part of the instruction manual.
• In a company with old applications and systems, sometimes only one or two people, in a company of thousands of employees, who know how to navigate the code and fix bugs without bringing the entire system down.
• Or a group of employees, who execute a process a specific way, because that is always the way they have done it, and the method works.

There are several reasons why tribal knowledge accumulates over time. It could be because of the natural course of the evolutions and revolutions a company undergoes through the lifecycle, and others could be a bit more sinister.

For example, a process that was done a particular way in a specific location continues to follow the same course, despite new technologies and new ways of doing things. As long as things are getting done, the practice is let to continue, and that becomes inherent in a small group of people.

On the other hand, some employees may hoard the knowledge with the intent of becoming indispensable and safeguarding their job and raising their relative importance.

And in other cases, some practices and operations abound despite the fact they are not standard policy. But people do it anyway – under the radar. This behavior could be a colossal compliance risk.

Companies try to capture tribal knowledge and hidden patterns and pathways by mapping the process flows. Then they go on and consider one of the variants as the official “golden” process and also codifying acceptable norms into standard operating procedures.

Where these endeavors fall short is when the business analysts are unable to tap into the hidden reservoir of knowledge – either by not reaching the groups that are privy to these process patterns or the inability or unwillingness of the denizens of tribal knowledge to share the practices.  Executive fiats and banning the practice is not going to help melt away the invisible enterprise.  Instead, illuminating the hidden patterns and practices takes technology, particularly advanced artificial intelligence methods, to unearth the invisible enterprise.

The risks of letting the tribal knowledge and the invisible enterprise to continue are manifold.

• The understanding of the processes and their transparency will remain partial, and some process paths will be opaque.
• The tribal knowledge will be difficult to capture and transfer to the other members of the team.
• Assuming the rituals and practices and behaviors of this tribal knowledge are not in line with the company’s stated policies and procedures, it could pose a compliance risk.
• Lack of a holistic picture of the nature of operations will impede efforts to automate and digitally transform the enterprise.

How Skan.AI helps capture the Tribal Knowledge using Artificial Intelligence?

Skan is the next evolution of process mining and leverages computer vision and machine intelligence to observe work at scale to discover processes – both visible and invisible, and written-down and tacit.  Lightweight virtual process agents monitor, observe and record human interactions with digital systems and decipher process paths from the digital traces of work.

Since the monitoring and observation happens unobtrusively and at scale, the millions of interactions tend to capture every nuance and categorize them into process variants alongside attributes such as frequency, cycle time, specific steps, and the start and endpoints.  And this allows for comparison to the standard or reference process variant and unearth the deviations from the expected path.

The illumination of the invisible enterprise not only brings transparency to the tribal knowledge. The outcomes also can become the basis for designing future state process models. The deliverables will drive the full spectrum of requirements for automation, digital transformation, outsourcing, and process re-engineering.