How to start your innovation journey

Welcome to my second column in the series, "Business Innovation for the Future." In my first column, "Why companies need to invest in innovation!", I outlined three key steps a business needs to take to drive significant value in the organization. In this column, we will go through the example of a company that gained insights on future developments by changing the way they analyzed their historical archives.

A major mining company in Canada was looking to innovate the way they analyzed the historical geological documentation that they had been collecting and storing for over 30 years. Fortunately for me, they had just completed a project to digitize these documents and place them in a centralized file share. Unfortunately, these were all in unstructured data formats like Word and PDF documents, so extracting the information was not as simple as running a database query on a structured table.

‍

The challenge

The company wanted to identify the six global projects which they should be developing next. There were over 350,000 documents in unstructured data formats which needs to be analyzed. Keep in mind; all these documents were geological reports completed over the past 30 years of every property the company had purchased or acquired. These reports included maps, tables, and calculations, which, until recent years, only a human could consume and summarize findings. The average length of a report was approximately 25 pages.

Let's put this in perspective. Consider the average Word document has 500 words per page. The company's 25-page report would contain approximately 12,500 words, and we are not talking about easy to read words, remember these are geology terms to be analyzed by geologists. For those of you interested in the type of geological data most companies capture, perform an online search for the "Canadian government open data portal". Continuing the perspective of our analysis, let's consider the number of documents (350,000) and multiply that by the number of words (12,500). We get 4,375,000,000 total words to analyze. A team of trained geologists has to:

• Read each of these documents looking for specific terms
• Highlight the specific terms
• Extract them into a spreadsheet
• Do the math to calculate the probability of finding a location to develop their next mine

How long do you think it would take to complete this process? How long did it take you to read the 1200 words in this column? I won't make you do the homework, here is what the company did.

‍

Original traditional plan

The mining company was planning to hire five intern geology students to complete as much of the above work as possible over the summer. The chief geologist estimated that each student could complete ten documents in a week, which ends up with 50 documents per week. Throughout the 12-week summer break, they would have analyzed only 600 of the 350,000 documents in the file share, which was less than a fraction of 1% of the knowledge this company had collected in 30 years. How will you feel if you were the chief geologist? How would you have presented it to the leadership team?

‍

The modern approach

The company's chief geologist and their team happened to attend one of my presentations at a conference. The topic was about using machine learning for unstructured text analytics, and they saw the potential fit for tacking their business problems.

After a 3-hour discovery session (mostly me learning geology basics), I set out to convert their geological model to machine learning code. I developed the report they needed to make their decision. Our initial estimates for the work were 12 weeks for a team of 3 developers. Still, we were confident the model would be able to read all 350,000 documents in a matter of hours when we were finished building and training the model.

Within the first week of development, we had completed and tested the model on a few sample documents. Once the chief geologist validated the output was exactly what they were looking for, we went to work on the reporting structure. This was the most complicated part of the process, as we were converting unstructured text, images, tables, maps, and charts to a structured format that could be analyzed by the machine learning model. Once we were able to figure out the deconstruction of the information and reconstruction back into the report, the development effort for this took just over three weeks. By week five we had completed everything we needed to replicate the chief geologist's analysis. Now the validation work began. The chief geologist took a week to validate the outcomes and results from our work. They were so impressed that they had some additional insights on how to make the model even better. The original model had 52 variables, and they discovered that if we added six more, the results would be significantly more accurate.

What is significantly more accurate, you ask? The initial run resulted in 164 of the original 350,000 documents as potential candidates for review and a confidence level (probability of mine development success) of 67%. The last run after adding the additional variables resulted in 14 documents with a confidence level of 83%. How confident do you think the chief geologist felt now? In the end, the project took a total of 8 weeks to complete (4 less than our initial estimate), and it was successfully implemented by the business that year. It resulted in their most significant ore discovery ever. It set them up for acquisition, which occurred just over a year after our project.

‍

Conclusion

In summary, through an innovative approach to a complicated business problem, we were able to reduce project effort and increase business insights using the latest in technology techniques. This solution for this company not only reduced their project costs from 5 students, 12 weeks, with less than 1% of the available material covered to 3 developers, eight weeks, with 100% material covered. This solution also increased the potential growth of the company, which resulted in their most massive candidate for mine development ever and set them up for acquisition. If this is the success story you would like for your company, then you need to start investing in innovation now. And I know someone that can get you started!

On average, very few ideas end up as an innovation project, and it has a high risk-to-reward ratio. But that is where excellent innovation management can reduce this risk and increase your probability of success, so invest in an expert to ensure this success.

"Adaptability and constant innovation are key to the survival of any company operating in a competitive market." – Shiv Nadar