Item: Predica
Author: Dawid Detko

Blog

Dawid Detko Digital Advisor

8 MIN READ JULY 07, 2020

A man touching with his finger on a digital search engine

Time and again we search for this one document we urgently need, and it is nowhere to be found. Keeping documents, images, and content in one place has always posed a huge challenge, especially for successful businesses with lots of documentation in every form imaginable.

With the new intelligent search solution and document scanning, AI puts an end to this seemingly unsolvable problem. In 2020, the era of Knowledge Mining finally began.

KEY TAKEAWAYS:

What is Knowledge Mining and what does it have to do with AI search?
What can AI services do to speed up your search experience?
How much will Knowledge Mining cost you?
Which business industries will benefit the most from intelligent search?

The idea of organizing crucial company information in a structured way has been crossing entrepreneurial minds for decades. As more and more data is generated every second, the need for efficient document-processing solutions has increased greatly.

Obviously, rapid development of AI services contributed enormously to the long-awaited Knowledge Mining solutions finally coming true.

Why is AI-driven search so necessary?

Typically, employees search for information manually. They have to copy, paste, or reenter valuable data into other forms. This leads to a number of problems:

Manual and repetitive processing takes plenty of valuable time
There are numerous errors in such content
Combining insights from multiple sources in a short time results in delays and incomplete information
It is impossible to find important links between data
Documents stored only in their physical form are useless when you work remotely.

According to the 2019 Harvard Business Review Analytics Services report, unstructured data poses serious challenges. Traditional processing is time-consuming and ineffective.

With Knowledge Mining, organizations can overcome obstacles that were once impossible to deal with. No text analytics tool has ever been that easy to access.

Also per Harvard Business Review

82% of professionals say that their companies struggle with efficient processing or usage of unstructured data stored in documents, images or audio files.

In addition, 68% of them believe that Knowledge Mining will help their organizations achieve strategic goals over the next 1.5 years.

What are the advantages of Knowledge Mining?

Microsoft shared countless materials on Knowledge Mining, including data repositories and solution accelerators. You can also count on the support of Microsoft partners such as ourselves for further assistance. And what’s best, you can be up-and-running in as little as a week!

Knowledge Mining speeds up the searching and combining different data types. Using AI, it can intelligently scan and analyze texts from Word documents, PDFs, images, videos, and even handwritten notes, for specific phrases. The solution looks for patterns and then extracts the information you need.

This only goes to show how AI can help organizations launch meaningful searches. No wonder everyone’s talking about this new solution.

Azure Cognitive Search – the core of Knowledge Mining

Knowledge Mining works thanks to Azure Cognitive Search. To put it simply, this cloud search service identifies and explores relevant content on a large scale. Your solution won’t be complete without implementing it. Some of its main features include:

OCR – enables conversion of typed, handwritten, and printed texts
Sentiment scoring – helps to identify and extract relevant data
Entity recognition – identifies and categorizes types of words
Language Detection – detects languages and their variants
Key-phrases – searches for key points of a given text.

If you would like to develop the solution even further, it is possible to add tailor-made functionalities, or custom skills.

You can also adopt a brand-new complex machine learning model using Azure Machine Learning. Additionally, you may add more Cognitive Services which will extend and enrich the entire processing pipeline.

How can you connect data to Azure Cognitive Search?

There are two basic approaches. They are used to ingest data and populate indexes in Azure Cognitive Search. We call them Pull Data and Push Data.

Pull Data

You can pull data into the index with Azure Cognitive Search indexer. Supported Azure data sources include:

Blob Storage
Azure Data Lake Storage Gen2 (in preview at the time of writing this article)
Table Storage
Azure Cosmos DB
Azure SQL Database
SQL Server on Azure Virtual Machines
SQL Managed instances on Azure.

Push Data

You can make use of custom apps to push documents directly into a search index programmatically. Applications send data into the index by means of:

A screenshot from our demo presentation

Knowledge Mining on Azure – services and related costs

Although Azure Cognitive Search is the driving force, Knowledge Mining would not be possible without the help of other integral tools. Have a look at the general structure:

Source: Microsoft

Each and every piece of the puzzle plays a vital role. The solution is designed in such a way that it allows you to customize it and you can feel secure at all times.

How does Knowledge Mining work?

Microsoft Azure Cloud hosts the solution which is formed by a collection of several services. You can adjust it to meet your own business needs.

Cognitive Search is the driving force for the solution. Cognitive Services like OCR, Custom Vision or Form Recognizer back it up. They play the role of custom skills in the pipeline.

Additionally, Azure Data Lake stores the unstructured data, so raw data is kept in a centralized place.

End users can carry out request/response operations thanks to a WEB API Service. It enables communication between the users and the search engine.

Azure Web App is the user interface host. You can also adjust and modify it to suit your business needs. Azure Active Directory makes the solution secure at each stage with the help of role-based security mechanisms.

Below you will find an example of cost estimation for these Azure services. These are the components necessary to make the solution work. The figures are based on the previous solutions we implemented.

Estimated Knowledge Mining service costs

The final costs vary on a case-by-case basis and are agreed individually with each client. In short, they will depend on your business needs and the degree of complexity your need.

Dawid Detko Digital Advisor

Want to see the solution in action?

Book a demo

Solutions in practice – business scenarios

At this stage, you definitely know more about the potential of Knowledge Mining, how it works, and how you can use it. Now it is time to find out what types of businesses will benefit most from this solution.

Media & Telecommunication

Businesses may use intelligent search to look through numerous transcripts or phone calls. Such analyses can help to determine possible areas of improvement.

The solution can also look for patterns in employee performance, such as progress against KPIs, and carry out comprehensive assessments. This in turn can help improve contact with customers and choose the most profitable business strategies.

In this case, Knowledge Mining combines search services with Cognitive Services or Video Indexer. If you would like to find out more about phone call analysis, our expert Daniel Krzyczkowski prepared an excellent repository which you can view here.

Finance and insurance

Knowledge Mining has the potential to change the way we calculate credit scores. The AI-driven search can compile and understand the relation between different sources and formats of information.

Seemingly unrelated details may turn out to be critical to the decision-making process. Consequently, FSI experts can make better predictions and act on trends accordingly.

Intelligent search also creates new experiences, services, and solutions for end-users, such as smart banking or intelligent consulting. For example, customers can scan their identity cards themselves and authorize a number of actions.

Legal

You will find the solution really useful when searching through tons of legal documents, especially if they vary in terms of form. You can use Knowledge Mining to analyze and scan legacy paper claims, policies, as well as contracts, invoices, and more. No more valuable documents will be lost. Every detail of a given legal case will be easily accessible.

In addition, the service proves extremely useful during investigations. It can be used to search through different types of evidence, including photos, scans, handwritten documents and more.

Have a look at how real-life evidence is stored in Azure demo:

Knowledge Mining demo screenshot with personal records

Lee Oswald’s records from Azure demo

As you can see below, the solution contains a connection chart. This way, you can examine evidence by looking for links between people and facts, and discover dependencies between them:

Knowledge Mining demo screenshot with connection chart

A connection chart from Azure demo

You can try out the demo yourself here.

Manufacturing and engineering

Your engineering team will save some valuable time if the documents they need are at hand. With intelligent search, they can easily organize technical documentation.

The solution can analyze and scan even handwritten projects. When the time comes, your engineers will quickly find the sought-after details.

AI-driven search comes in handy when new people join the company. They can quickly access service history and find manuals on production line repairs.

Logistics

The service can be helpful when shipping cargo. Even if data sources like the news or RSS services are unstructured, Knowledge Mining is able to analyze the data. It can also find relevant information which may affect the shipping time.

As a result, the solution can predict delays and prepare optimal delivery routes. Time-consuming manual analyses can be substituted by a more accurate, automated process, which can effectively help in critical situations.

Additionally, time and resources previously spent on analyzing data can be redirected towards resolving more complex problems.

Retail

If you have a loyalty program, you can use Knowledge Mining to scan receipts and automatically assign loyalty points. The solution will recognize the form and then scan the receipt content using OCR. This enables you to build a more advanced offering which can increase customer engagement.

Lots of retailers move to online selling channels. There, intelligent search is in its full swing. Customers can use it to check product descriptions, images, or manuals. Chatbots in online stores deal with problems more effectively thanks to Cognitive Search.

All these improvements can make your business far more competitive and visible on the market.

Click to find out more

Relevant information at hand thanks to AI

One thing is clear. The idea of a tool which could search through documents and help employees to quickly get the information they need is nothing new.

The only question was: is there any way to put the idea into practice?

AI has proved once again that only sky is the limit. Not only does Knowledge Mining scan texts in every format imaginable, but it conducts a truly intelligent search. The solution finds relevant patterns and, as a result, helps workers boost their efficiency and avoid repetitive manual document processing.

Knowledge Mining has all the makings of becoming the most powerful solution in modern organizations. The AI-driven search will quickly prove essential in a wide range of business industries.

If you feel like trying Knowledge Mining on your own, here you can request our demo.

Summary

Knowledge Mining is a solution which combines AI services to help organizations speed up and extend their search experience.
Employees will no longer spend their valuable time on laborious copying and pasting of documents kept in different sources. All documentation can be stored in one place.
The AI intelligent search recognizes, scans, and looks for relevant information in texts produced in many different forms, including handwritten pieces, videos, and images.
Azure services power the solution, with Azure Cognitive Search at its core. Businesses may modify and adapt the solution according to their needs.
A number of industries will benefit from applying Knowledge Mining. These include retail, financial, legal, manufacturing, logistic, and media sectors.

Read similar articles