Deriving Value from Knowledge Mining Solutions in Azure

Giving your organization the ability to collect all relevant documents and media in one location with intelligence about those documents provides tremendous value. The level of intelligence that can be provided sets apart a knowledge mining solution with artificial intelligence (AI) compared to traditional document storage in a significant way. The next generation of enterprise search provides the ability to do the following:

• Score sentiment in documents.
• Correlate object images with text from documents.
• Identify personalities in audio, images, and video.
• Detect anomalous behavior in activities.

The preceding abilities, with little or no configuration of the services, and pre-built machine learning (ML) models to support base examples are readily available by coupling Azure Cognitive Services.

By injecting AI into its knowledge mining solution, Ocean Smart is able to take advantage of the skills...

Reviewing a brief history of document collection solutions

The term knowledge mining is much newer than the concept, with a 21st-century enhancement that includes the cloud and AI. In fact, organizations around the world have sought after a centralized management system of their documents and other assets containing company-specific information for decades now. When we consider that document management systems (DMSees) have been around for decades, due to a proliferation of documents and data, it is no surprise that the opportunity has surfaced to align with AI for enhancements. These systems were developed to have a central repository of documents for archival and reference. DMSees align context with the documents themselves by providing tags and hierarchical structure for simpler retrieval and searching. tagging and describing documents is manual in process, as each document needs to be described and tagged individually. As the volume and variety of documents increase dramatically...

Understanding the drawbacks of traditional data collection systems

As previously described, the proliferation of data began in the early 1980s due to the rapid adoption of computer workstations within organizations. Material requirements planning (MRP) and enterprise resource planning (ERP) systems came onto the scene and provided significant business value to organizations needing to centrally manage day-to-day operations. These systems used a database developed to house data flowing through the systems and then archived this when no longer relevant. Word processing and spreadsheet applications also helped organizations to create documents outlining planning, budgeting, financial, and other types of details about the company.

If we examine these systems more closely, we can see that they were severely limited by the compute resources available to run efficiently on the hardware they were deployed to. Also, the limitations of the software compilers and languages these systems were...

Exploring the purpose and benefits of knowledge mining solutions

Ocean Smart is constantly acquiring new digital assets and physical paperwork from customers, partners, suppliers, and internal employees. This proliferation of data has developed a tremendous opportunity for organizations to extract critical information from all those data sources. Documents are delivered via email containing details of products being procured, inventory levels, financial documents, sales-related documents, and much more. Customer call centers are capturing audio details of calls in a digital format used to improve training systems and customer experience. Cameras are capturing video of production-floor processes, the movement of products and employees around different areas in buildings, the exteriors of facilities for security tracking, and other relevant details about business operations. They also gather images of finished goods, raw materials, products being delivered to customers, and other opportunities...

Using cognitive services to develop knowledge mining solutions

When considering which cognitive services to use to enhance your knowledge mining solution, you can assume certain options will be used by default. These are the services that are most commonly used for text-based operations where you may have handwritten documents or notes, and further details can be extrapolated from that text. Let's go through the services that will enhance your document stores and help build a more complete knowledge mining solution in Azure, as follows:

• OCR—OCR capabilities provide organizations with the ability to capture text within documents and surface that text for searching and indexing. These capabilities are extended beyond what can be extracted from digitized text to handwritten and other forms of text to be captured.
• Sentiment analysis—This gives us the ability to gain insight into the tone of the resource being analyzed and helps with determining whether there...

Summary

In this chapter, we discussed some brief histories of DMSees, enterprise search solutions, and the foundation for knowledge mining solutions. We wanted to establish how much value is added to your data when enhancing a standard search index with AI for automation and better insights. So many of these processes were manual for so long and continue to be because companies are frequently unwilling to release their technical debt. When a company invests significant amounts of money into a product, it is extremely difficult for those who decided to invest in it to take a step back and say: "This isn't the right solution going forward." Having been that guy a few times, it was really painful to walk into my boss's office and say: "We need to change direction." However, when you can get back on the right track, things improve much quicker.

We discussed the limitations of DMSs, which are great with structured data and data that can be easily digitized...

Further reading

• KNOWLEDGE MINING: The Next Wave of Artificial Intelligence-Led Transformation (https://azure.microsoft.com/mediahandler/files/resourcefiles/knowledge-mining-the-next-wave-of-artificial-intelligence-led-transformation/HBR-analytic-services.pdf)