Azure Cognitive Services - What Is It?

In the last blog post, we covered the basics of what Artificial Intelligence and Machine learning are. This post provides an overview of the AI services Microsoft offers and how you can make them work for you. 

---

Azure Cognitive Services (ACS) is a suite of application programming interfaces (APIs) that you can leverage in your business applications, including the tools available with Microsoft 365.  ACS comprises Vision, Knowledge, Language, Speech and Search.  With ACS, you can add complex AI capability to your applications with little to no code!  You can even create custom models and train them for your specific needs.

The Vision API provides the ability to gather information from media.  There are several out-of-the-box capabilities. 

• Computer Vision Service you can classify images, perform optical character recognition to pull text from images, perform handwriting recognition, or even determine celebrities or locations the image contains. 
• Face Recognition API finds people in an image, identifies a particular person, determines the emotional state based on facial features, and groups images based on similar faces. 
• Video Indexer scans videos for face detection, object scene and activity detection, and can extract metadata, audio, and perform keyframe analysis. 
• Content Moderate is designed to find explicit or offensive content in media.  It also includes a method for incorporating feedback from people to dynamically adjust the content filters.
• Build and train a Custom Vision Model.  As an example, if you have a MS PowerApp that connects to an inventory database, one of the features of the app is to take a picture of an object and find information like a part number, quantity in inventory, and cost.  Using the Custom Vision API, you can train the model by uploading pictures of an item and manually tagging them with the part number.  By adding diverse pictures of the part (different backgrounds, orientations, and lighting) the accuracy of the model is improved.  Once the model is trained, simply connect the MS PowerApp to ACS and you have an easy-to-use solution.

​The Knowledge API is geared towards creating responsive question and answer applications and knowledge bases from unstructured text.

• QnA Extraction From Unstructured Text enables the creation of a dynamic question and answer platform.  Users can enter a question and the API will find the best suited answer and a confidence score.  Collections of QnAs can then be used to build extensive knowledge bases.  The QnA service supports semantic searching  to interpret the intention of the user’s question and provide better results.

​Natural Language Processing is handled by the Language API.  The API analyzes text and determines sentiment, intent, and semantics with a degree of confidence. 

• Text Analytics is used to extract key phrases, identify named entities like people or places, and evaluate the sentiment implied. 
• Bing Spell Check provides the ability to embed spell checking into your applications. 
• Language Understanding examines the context of a sentence or phrase to determine what action should be taken. 
• Translator Text can be used to determine the language of the supplied text, automatically translate from one language to another, and enable customizable translations.
• Content Moderator API can be used to remove or flag offensive material in images and video (paired with the OCR API and content moderation in the Vision API).

Speech adds robust voice recognition to your apps, translation, and conversion between speech and text.

• ​Speech to Text provides the ability for users to vocally interact with your application.  It can perform automatic speech recognition and transcription, or you can create a customized model to handle unique vocabularies, accents, dialects, and colloquialisms. 
• Text to Speech transcribes written text into computer generated vocalizations.  It also has the ability to customize voice fonts. 
• Speaker Recognition maps audio samples to a model containing know speakers and can attempt to identify or verify who is speaking. 
• Speech Translation, similar to the Language API, can perform real-time translation, automatic speech translation, or customized speech translation.

The last component of ACS is the Search API.
​Using the Bing platform, your application can use the Search API to crawl the web, videos, images, news, local businesses, and named entities like people and locations.  Additionally, it can be leveraged to identify and classify images and find similar images using Visual Search.  If the out-of-the-box APIs are not sufficient, you can create a custom search to crawl your own data sources.  The Autosuggest feature can guide users to the content they are looking for by making suggestions based on previous searches.

The beauty of ACS is these AI components are available to developers without having to understand the details of various machine learning approaches.  ACS also offers the flexibility to create custom models which make sense to your business and your users.  ACS integrates seamlessly with PowerApps and Flow so you can take your Power Platform solutions to the next level.  If you’re interested in hearing more, Pivotal can help!  Contact Us.

Stay tuned for our next post on how you can leverage AI in the Microsoft 365 stack.